FIRST DEFINE THE TABLE WE NEED
The policy question determines what one row must represent.
Imagine that a development-policy team is preparing a short briefing on economic resources and child survival. It wants to see where higher income coincides with lower mortality—and which economies do not follow that broad pattern. We therefore ask how national income is associated with under-five mortality across economies from 2000 to 2022. To represent every economy-year once, its draft analysis table should contain one row for each economy and year.
This is a descriptive question, not a causal claim. To compare the two measurements directly, both must belong to the same economy-year observation. That analytical need—not a formatting preference—is why we construct a one-row-per-economy-year table.
Suppose Kenya–2022 appears in two rows of the draft analysis table. What could have produced two rows for the same economy and period? Why could that change the briefing’s table, plot, or conclusion? We use your answers to define the unit and key.
A source file may legitimately contain two Kenya–2022 rows when
one row represents an economy-year-indicator: one for income and
one for mortality. Its key is
iso3c + year + indicator. Our analysis table has the
indicators in separate columns, so one row represents an
economy-year and iso3c + year must be unique.
Two Kenya–2022 rows could come from a legitimate indicator dimension; an unnamed dimension such as sex, unit, or version; a join whose right-side key is not unique; or the same record being imported twice. They are not inherently wrong. They are wrong when they violate the economy-year unit required by this policy question.
If the broken structure remains, Kenya can receive excess
weight, a plot or model can count it more than once, conflicting
values can be treated as separate evidence, and the apparent
association can change. Diagnose the difference before using
distinct().
Before touching a data verb, write the table you need: the population, unit of observation, key, time period, outcome, comparison variable, and grouping variables.
| Decision to make | Health example | Why it matters |
|---|---|---|
| Population | Economy-years with both measures observed and positive income | Defines who can appear in the analysis. |
| Unit | Country-year | Determines what one row means. |
| Key | iso3c + year |
Must identify each row uniquely. |
| Period | 2000–2022 | Sets the comparison window. |
| Outcome | Under-five mortality | Names the quantity to describe. |
| Comparison | GDP per capita, PPP | Names the relationship of interest. |
Together, these decisions form our table plan. The plan gives the columns and rows substantive meaning. If the unit changes from country-year to region-year, the question and interpretation change with it.
A useful map is selective. The table plan records the choices needed for this policy question: who appears, what one row means, which period matters, and which relationship we plan to describe. The build record shows whether the R pipeline followed that plan.
THE GRAMMAR OF DATA CONSTRUCTION
Functions give names to data decisions.
In health_selected <- wdi |> select(...),
wdi is the input object, <- assigns a
name, |> passes the object forward,
select() is the function, and the column names are
its arguments. The syntax is easier to learn when each part has
a visible job.
| Verb | Question before running | Evidence after running |
|---|---|---|
select() |
Which variables and identifiers are necessary? | Names and number of columns. |
filter() |
Which observations leave the population? | Rows, countries, and years before versus after. |
mutate() |
What does the new variable mean and in what unit? | Missingness, range, and a hand-checked example. |
group_by() |
Which observations belong together? | The active grouping variables. |
summarise() |
What will one output row represent? | Expected group count and unit. |
left_join() |
Is the right-side key unique and should every left row match? | Row count, duplicates, and unmatched keys. |
arrange() |
Which order will make the observations inspectable? | Row order changes; rows, values, unit, and key do not. |
pivot_wider() |
Which row dimension should become columns? | The new unit and key are stated and tested. |
Rows, columns, values, order, or unit.
Keep the pipeline small enough to inspect.
Use counts, keys, ranges, or anti-joins.
Leave the reason in code or documentation.
BUILD RECORD
Keep the source and the checks together.
The frozen inputs stay unchanged. The script creates the analysis table from them. The build record reports row counts, the country-year key, year coverage, variable units, missingness, and unmatched joins. A figure or a table created at the end never becomes a new source file.
R can complete a join with the wrong key, duplicated metadata, or unmatched countries. After the syntax runs, compare the expected structure with the observed rows, keys, and matches.
LIVE CODING
Build, delegate, and verify one table together.
Today is a rehearsal for tomorrow's lab. We open the frozen source, define the table, build it in readable steps, verify the saved object in a separate R process, and then ask an agent to audit one draft briefing claim against those artifacts.
Open code/lesson-2-walkthrough.R. Its numbered sections follow the slide sequence. Run one
section with Command+Enter on macOS or Ctrl+Enter on Windows;
use Source when you want to rebuild the complete walkthrough
and its independent verification result.
Open the project and confirm the 4,991-row source.
State the object before changing the source.
Explain the move from 4,991 to 4,296 rows.
Ten minutes away from the screen.
Track value changes and changes in the unit.
Diagnose a row-multiplying metadata join.
Save the artifacts and require independent evidence.
Audit one claim, then locate tomorrow's starter.
source("code/00-setup.R")
wdi <- read_csv(data_file, show_col_types = FALSE)
nrow(wdi)
We continue when the project opens, the source has 4,991 rows,
and everyone can run the first section of
code/lesson-2-walkthrough.R.
source("code/00-setup.R")
wdi <- read_csv(
data_file,
show_col_types = FALSE
)
health_analysis <- wdi |>
select(
iso3c, country, region, income_level_current, year,
gdp_per_capita_ppp, under5_mortality
) |>
filter(
between(year, 2000L, 2022L),
!is.na(gdp_per_capita_ppp),
gdp_per_capita_ppp > 0,
!is.na(under5_mortality)
) |>
mutate(log_income = log(gdp_per_capita_ppp)) |>
arrange(iso3c, year)
stopifnot(!anyDuplicated(health_analysis[c("iso3c", "year")]))
select() makes the analytical scope visible.
filter() defines the period and complete-case sample,
reducing 4,991 source rows to 4,296 analysis rows.
mutate() preserves
the original income measure while adding a logged version whose
interpretation must be explained. The assertions turn two
structural expectations into executable checks.
BOUNDED ASSISTANCE
Let the agent audit a claim you can already evaluate.
Only after the table and its evidence exist, use the agent as a supervised reviewer. The task is deliberately read-only: decide which parts of a draft briefing sentence are supported now, which require Lesson 3 calculations, and which exceed the design.
Goal: Decide whether this draft sentence is ready for a policy briefing: “Across 217 economies from 2000 to 2022, higher income reduced under-five mortality.” Context: Read outputs/02-health-analysis.csv, outputs/02-health-build-record.csv, data/documentation/indicator-dictionary.csv, code/02-build-analysis.R, and code/02-verify-analysis.R. Permission: Read only. Do not edit files or browse the web. Task: Classify each part as supported by the artifacts, not yet calculated, or unsupported by this descriptive design. Proof: Cite the exact file and value behind each factual statement and propose a cautious handoff sentence for Lesson 3.
- □I checked every factual citation against its source.
- □I separated coverage facts from an unestimated relationship.
- □I rejected causal language that the design cannot support.
- □The handoff says what is known now and what Lesson 3 must calculate.
The common path uses the frozen teaching file. Updating an indicator from the public API remains an optional high-ceiling extension.
1. Open code/02-build-analysis.R and click Source.
2. Open code/02-verify-analysis.R and click Source.
The second script reopens the saved source and output, checks
columns, rows, key, period, values, and the logged measure, and
ends with the explicit completion message
RESULT: PASS.
Begin Lab 2 with LAB-2-HANDOUT.pdf and
code/lab-2-starter.R. The handout contains the
prompts and checkpoints. Finish by opening the independent
verifier in RStudio and clicking Source.
SLIDES
Lesson deck.
The slide deck is a 16:9 LaTeX Beamer PDF. It follows one table from its table plan through its build record and then moves into Lab 2.