Audit the handoff

Before we begin

ARRIVAL PROMPT

Your code works on your laptop. What would convince you that it is ready for another person?

Today

1. Define reproducibility for this camp.
2. Expose hidden state with a clean run.
3. Organize a readable project.
4. Use Quarto as an optional field notebook.
5. Write provenance, limitations, and an AI-use note.
6. Conduct a four-part audit.

By 5 PM

• Restart R and run the project from the top.
• Explain why a clean render is a test.
• Organize input, transformation, analysis, and output code.
• Write a useful README and AI-use note.
• Rank audit findings by consequence.
• Prepare a project for a cold exchange.

A handoff is an experiment

The question is not “Can the author make it work?”

The question is:

Can another person recover the question, inputs, decisions, and result without relying on the author’s memory?

Someone else’s kitchen

A recipe is not tested because its author can cook from memory.

It is tested when another person can use:

• the listed ingredients,
• the listed equipment,
• the written sequence,
• and the stated result

to produce the dish without whispered instructions.

Reproducibility for Math Camp

Another person can:

• obtain or identify the same inputs,
• run the documented code in order,
• recreate the main figure or table,
• understand the variable definitions,
• see the limitations and AI contributions.

This is a practical standard, not a guarantee that every environment is identical.

Hidden state

Code may work because:

• an object was created in the console,
• a package was loaded earlier,
• the working directory was changed manually,
• a file was edited outside the script,
• chunks ran out of order,
• an old output remained on disk.

Hidden state is invisible help from your past self.

Intentional failure demo

summary(analysis_data)

The command works in the instructor’s current session.

Clean run protocol

1. Save source files.
2. Restart R.
3. Confirm the project root.
4. Run or render from the first line.
5. Stop at the first error.
6. Repair the source, not the environment by hand.
7. Repeat until output is rebuilt.

Project structure communicates

project/
├── README.md
├── data/
│   ├── raw/
│   ├── derived/
│   └── documentation/
├── code/
├── outputs/
└── field-notebook.qmd

Names should help a new reader predict where evidence lives.

The project’s flight recorder

The record of what happened lives across:

• the README and source files,
• data documentation and retrieval dates,
• console or render output,
• the agent work order and change summary,
• the verification result,
• and the AI-use note.

Order the script

One entry point

A reviewer should not need to guess which script runs first.

library(tidyverse)

wdi <- read_csv(
  "data/derived/math-camp-wdi-2000-2022.csv",
  show_col_types = FALSE
)

source("code/01-build-analysis-data.R")
source("code/02-compare-patterns.R")

For this small camp project, one well-ordered script is also acceptable.

What Quarto does here

Quarto combines:

• narrative,
• executable code,
• generated output,
• citations and links,
• a repeatable render step.

The Math Camp website is native HTML, CSS, and JavaScript. Quarto is used for the slides and optional field notebook.

Minimum field notebook

---
title: "Math Camp field notebook"
format:
  html:
    toc: true
    embed-resources: true
execute:
  echo: true
  warning: false
---

A readable notebook

1. Policy question.
2. Data and definitions.
3. First inspection.
4. Analysis sample.
5. Comparison.
6. What this does not show.
7. AI-use note.
8. Verification record.

A clean render is a test

Rendering checks whether:

• objects are created in order,
• paths resolve from the project,
• required packages are loaded,
• figures are generated from code,
• narrative and output remain together.

It does not prove the analysis is substantively correct.

Predict the render failure

mean(health$under5_mortality)

README as an invitation

A useful README answers:

• What question does this project explore?
• What software and packages are needed?
• Where did the data come from?
• Which file should I run?
• What output should appear?
• What is known not to work or not to mean?

Which README is useful?

Data provenance in the handoff

Record:

• publisher and API or download page,
• retrieval date,
• indicator codes and units,
• coverage years,
• transformation script,
• frozen or updated file version,
• known revisions or missingness.

Limitations are part of the result

Useful limitations are specific:

• coverage varies across countries and years,
• aggregate units remain in the teaching file,
• indicator definitions simplify complex policy conditions,
• the comparison is descriptive or associational,
• the simple model does not identify a causal effect.

“More research is needed” is not enough.

AI-use note

I used [Codex / Claude Code] to [specific contribution].
I provided [public project context].
I verified the contribution using [specific checks].
I revised or rejected [important example].
I remain responsible for the analysis and interpretation.

Weak disclosure

I used AI to help with the code.

This does not reveal:

• which tool,
• which contribution,
• which inputs,
• which verification,
• which human decision changed.

Rewrite the disclosure

The four-part audit

What harness evidence can prove

Audit 1: reproducibility

• Is there one clear starting point?
• Are packages and inputs declared?
• Do relative paths work?
• Does the key test run?
• Are outputs recreated rather than copied?
• Does a clean render succeed?

Audit 2: provenance

• Is the publisher named?
• Are indicator codes and units present?
• Are years and retrieval date recorded?
• Is the frozen or updated version clear?
• Can transformations be followed in code?

Audit 3: interpretation

• Is the analysis sample visible?
• Are coefficients described in units?
• Are missingness and unusual cases considered?
• Does the claim avoid unsupported causality?
• Is one consequential limitation stated?

Audit 4: communication

• Does the question appear before the code?
• Are objects and files named clearly?
• Do figures have labels, units, and captions?
• Can a beginner identify the main result?
• Is AI use disclosed plainly?

Rank findings by consequence

Do not bury a broken key beneath formatting advice.

Agent audit contract

Goal: Audit this project for handoff readiness.
Context: Read the README, data documentation, source, and output.
Review separately: reproducibility, provenance, interpretation,
and communication.
Constraints: Do not edit. Rank by consequence. Cite a file and
evidence for every concern. State uncertainty.
Done when: A human-verifiable prioritized checklist is produced.

Human review comes first

Why review before the agent?

• You make your own model of the project.
• You notice what a new reader actually experiences.
• You can compare the agent’s priorities with yours.
• Disagreement becomes evidence for discussion.

Handoff rehearsal

Minimum handoff

• One descriptive or associational question.
• One R script or Quarto notebook.
• Source, years, unit, and indicator definitions.
• One figure or table.
• One supported statement.
• One limitation.
• One AI-use note.
• One successful clean run or documented blocker.

Lab 4 tomorrow

MINI HACKATHON · WED AUG 26 · 4-5 PM

You will:

1. exchange projects across policy tracks,
2. run the handoff cold,
3. compare human and agent audits,
4. repair one high-value issue,
5. give a 90-second show-and-tell.

Nothing is graded or submitted.

Final confidence map