WHY THIS LAB
Updating data is a research decision.
Public data releases are not append-only. New downloads can add years, revise history, change coverage, and expose metadata differences. The goal is not merely to obtain the newest file. It is to preserve a comparison that makes every consequential change visible.
Goals
- 01Translate a desired update into a bounded agent contract.
- 02Inspect proposed API code before executing it.
- 03Write a new dated output without overwriting the frozen baseline.
- 04Test country-year keys, coverage, missingness, and units.
- 05Separate added observations from historical revisions.
Why the update needs supervision
An agent can quickly discover an endpoint, write a request, and reshape the response. None of those actions establishes that a new release is comparable with the frozen file. The indicator may end earlier than expected, an aggregate may appear beside countries, or the publisher may revise the historical series. Each possibility changes what “updated” means.
The baseline comparison turns an open-ended internet task into an auditable research task. Students know what must remain unique, which metadata must survive, where the new file may be written, and which differences must be reported. The agent accelerates retrieval while the human retains control over comparability and interpretation.
A documented failure counts. If the API is unavailable, the indicator ends in 2022, metadata changed, or the key test fails, preserve the code, error, and diagnosis. A well-documented stop is better evidence than an unverified “updated” file.
0-15 MINUTES
Write the contract before requesting code.
Select one indicator and state what a successful update would add.
Official API only, dated output, preserved code and unit, no overwrite.
Key, years, missingness, revisions, and retrieval date.
A conversational request describes a desire. A work order makes action reviewable. Before sending it, underline the phrase that names the protected file, circle the allowed write target, and box each completion check. If any of those are missing, the harness cannot protect you from an underspecified task.
Goal: Extend World Bank indicator SH.DYN.MORT beyond the frozen 2022 endpoint using the official API. Context: Read data/build_wdi.R, data/README.md, the indicator dictionary, and the frozen long file. The unit is country-year-indicator. Constraints: Do not overwrite any frozen file. Preserve the official code and unit. Write a new retrieval-dated file. Do not assume every country or indicator reaches 2025. Stop if a key duplicates or metadata conflicts. Done when: The new file and reproducible code exist; the retrieval date is recorded; keys are unique; and a comparison reports new country-years, latest available year, missingness, and changed historical values.
Change only the indicator-specific details. Ask the agent to state consequential assumptions before it writes or runs anything.
15-35 MINUTES
Review, run, and stop deliberately.
- 01Review the proposed API URL, indicator code, requested years, and output path.
- 02Confirm that the frozen files are never write targets.
- 03Inspect how JSON fields become country, code, year, and value columns.
- 04Run the smallest request first, such as one indicator and two years.
- 05Expand only after the small request passes structural checks.
- 06Write the full dated output and record the retrieval date.
Keep a workshop receipt
| Agent round | Record |
|---|---|
| Context | Files and official documentation the agent actually read. |
| Permission | Read, plan, edit, run, or network action you allowed. |
| Action | Exact file change, request, or command. |
| Observation | Diff, response status, row count, error, or generated path. |
| Decision | Continue, revise, reject, or stop—and why. |
Stop conditions
| Signal | Response |
|---|---|
| HTTP or parsing error | Preserve the request and exact error; do not switch to an undocumented source. |
| Indicator code absent | Return to the official metadata and dictionary. |
| Duplicate country-year key | Stop and inspect aggregate units, pages, or repeated records. |
| Unit or definition changed | Do not append until comparability is established. |
| Latest year remains 2022 | Record that no later observation is available. |
| Historical values changed | Keep revisions separate from genuinely new years. |
Review the diff. If the agent edits the build script, inspect exactly which lines changed before running them. Look for source URLs, years, filenames, filters, and assertions. A small diff is easier to reason about than a rewritten workflow.
35-52 MINUTES
Ask five version questions.
| Comparison | Evidence to produce |
|---|---|
| New observations | Country-years present only in the updated file. |
| Historical revisions | Earlier country-years whose values differ. |
| Coverage | Observed and missing counts by year. |
| Latest availability | Latest observed year overall and by country. |
| Structure | Row count, types, key uniqueness, code, and unit. |
new_country_years <- updated_indicator |>
anti_join(
baseline_indicator,
by = c("iso3c", "year", "indicator_code")
)
historical_comparison <- baseline_indicator |>
inner_join(
updated_indicator,
by = c("iso3c", "year", "indicator_code"),
suffix = c("_frozen", "_updated")
) |>
filter(
!is.na(value_frozen),
!is.na(value_updated),
value_frozen != value_updated
)
updated_indicator |>
group_by(year) |>
summarise(
observed = sum(!is.na(value)),
missing = sum(is.na(value)),
.groups = "drop"
)The updated object must use the same long-file names for this code to run. If the agent chose a different schema, require it to explain and reconcile the difference first.
- □The updated key is unique.
- □The official code and unit are unchanged or the difference is documented.
- □New years and revised history are reported separately.
- □Latest year is described as “latest available,” not assumed to be 2025.
- □The entire request can be rerun from saved code.
52-60 MINUTES
Share the check that mattered.
Prepare a 45-second report with four parts:
- 01Indicator: code, unit, and frozen endpoint.
- 02Outcome: updated successfully or stopped for a documented reason.
- 03Finding: new years, historical revisions, or unavailable data.
- 04Check: the single test that most changed your confidence.
Optional high-ceiling extension
Write an R function that accepts a frozen long table and an updated long table, tests their keys, and returns three named outputs: added observations, revised historical observations, and coverage by year. Test the function on a second indicator and confirm that a different latest year is handled correctly.
What to keep. Keep the update code, dated output or error record, comparison results, and provenance sentence in your own project. No group file is submitted.