WHY THIS LAB
Connect a visible pattern to a careful sentence.
Plots and models compress thousands of observations. That compression can clarify a relationship, but it can also hide changing samples, missingness, unusual cases, and unsupported causal stories. This lab produces a small evidence bundle whose parts can be checked against one another.
Goals
- 01Define and describe the exact analysis sample.
- 02Create a legible figure with intentional encodings and units.
- 03Fit and interpret one simple regression.
- 04Inspect residuals or one surprising observation.
- 05Use an agent as a skeptical reviewer and verify its criticism.
Why the evidence bundle has several parts
A persuasive graph can conceal a fragile sample, while a regression table can conceal an implausible functional form. Keeping the question, sample summary, figure, model, diagnostic, claim, and limitation together makes disagreement productive. A reviewer can point to the exact link in the chain that needs attention.
Your figure and model are two maps of the same observations. Before asking which is “right,” ask what each map preserves, what it suppresses, and which policy journey it is fit to support. A polished map can still be the wrong map.
The agent enters only after students have written their own concern. That sequence prevents the review from becoming a search for authoritative-sounding criticism. Students must compare the generated concern against the code, output, and documentation, then decide whether to verify it, reject it, or leave it unresolved.
Done when. Your evidence bundle includes one question, named sample, labeled figure, simple model, diagnostic check, supported claim, limitation, and reviewed agent concern.
10-27 MINUTES
Build the figure progressively.
Map the outcome and comparison variable to position.
Add at most one grouped encoding such as color or facet.
State units, years, sample, and a factual title.
Compare a raw and transformed scale or inspect overplotting.
health_plot <- ggplot(
health_analysis,
aes(log_income, under5_mortality)
) +
geom_point(
aes(color = income_level),
alpha = 0.45,
size = 1.8
) +
geom_smooth(
method = "lm",
formula = y ~ x,
se = FALSE,
color = "#151515"
) +
labs(
title = "Income and under-five mortality",
subtitle = "Observed country-years, 2000-2022",
x = "Log GDP per capita, PPP",
y = "Deaths per 1,000 live births",
color = "Income level",
caption = "Source: World Bank WDI teaching snapshot. Association is not causation."
) +
theme_minimal()
health_plotFor another policy track, change variables and units, not just the title. The caption should name a limitation specific to the plotted evidence.
Figure audit
- □The title states a factual comparison rather than a conclusion.
- □Both axes include meaningful units.
- □Color or facets carry information rather than decoration.
- □The sample and period are visible.
- □The fitted line is described as a summary, not a causal mechanism.
27-42 MINUTES
Fit, interpret, and inspect a surprise.
fit <- lm(
under5_mortality ~ log_income + income_level,
data = health_analysis
)
summary(fit)
health_model <- health_analysis |>
mutate(
fitted = predict(fit),
residual = resid(fit)
)
health_model |>
arrange(desc(abs(residual))) |>
select(
country, year, under5_mortality,
fitted, residual
) |>
slice_head(n = 8)Choose one coefficient and write its interpretation in the units of the outcome and predictor. Then choose one large residual and inspect the source values, year, dictionary, and missingness before telling a story.
| Component | Required sentence |
|---|---|
| Coefficient | “A one-unit change in [predictor] is associated with an estimated [change and outcome unit], comparing observations with the same included group.” |
| Residual | “For [country-year], observed [outcome] is [amount] above or below the fitted value.” |
| Limitation | “This model does not establish [specific causal or measurement claim] because [specific reason].” |
Do not delete surprises automatically. A large residual may reflect a data error, unusual policy context, omitted variable, or an overly simple model. Diagnose the observation before deciding what it means.
42-53 MINUTES
Ask the agent to try to break the claim.
Conduct the human review first. Write one concern about the code and one concern about the interpretation before opening the agent. Then compare three voices: your initial reading, the agent's criticism, and the observable evidence. None wins by confidence or fluency.
Goal: Critique this small policy-data comparison in two separate passes. Context: Read the policy question, dictionary, analysis script, sample summary, figure, model output, and written claim. Pass 1: Check code behavior, sample construction, missing values, transformation, labels, key, and reproducibility. Pass 2: Check coefficient units, residual interpretation, claim strength, and limitations. Constraints: Do not edit files or propose a more complex model. Do not use causal language. Cite the exact file, code, or output behind every concern. State uncertainty. Done when: Concerns are ranked and each includes an R or source check that can confirm or reject it.
Verification record
| Agent concern | Your check | Result | Change made |
|---|---|---|---|
| Record the exact concern. | Name code, output, or source evidence. | Verified, rejected, or unresolved. | Revise code, claim, limitation, or nothing. |
Accept no criticism you cannot trace. If the agent recommends causal language or invents a variable, record and reject it explicitly.
Counterexample hunt
Use the last two minutes to find one country-year, subgroup, period, or alternative scale that complicates the headline. Write whether it overturns the claim, narrows it, or simply identifies the next question. This is statistical curiosity, not a search for a preferred result.
53-60 MINUTES
Explain what changed your mind.
Prepare a 60-second comparison story:
- 01Question and sample: who, what, and when.
- 02Pattern: what the figure and model summarize.
- 03Surprise: one observation or diagnostic that deserves attention.
- 04Revision: how human or agent review changed the claim.
- 05Limitation: what the evidence still cannot show.
Optional high-ceiling extension
Compare two defensible specifications or samples, such as raw versus logged income, all years versus the latest common year, or pooled data versus facets by income level. Produce a compact table that shows how the sample size and focal coefficient change. Explain why the comparison is informative without selecting the specification solely because it gives a preferred result.
What to keep. Save the question, sample summary, figure, model code, diagnostic output, two-sentence claim, limitation, and verification record. These become the input for Lesson 4's handoff audit.