MODEL ≠ AGENT
Know which part is doing what.
A coding agent feels like one intelligent object, but it is a system of parts. Separating those parts makes the work easier to supervise and the failures easier to diagnose.
| Layer | What it contributes | Question to ask |
|---|---|---|
| LLM | Generates language, code, and tool requests from context. | Could this be plausible pattern completion rather than a grounded fact? |
| System instructions | Set durable behavior and boundaries. | What rules is the model following that I cannot see in my prompt? |
| Context | Files, history, documentation, and tool results available now. | Did the agent read the correct file and current version? |
| Tools | Read files, search, edit, run R or shell commands, and browse sources. | Which action actually produced this result? |
| Permissions | Limit which tools, paths, and external actions are allowed. | Did I grant more access than this task needs? |
| Loop | Returns each tool result to the model so it can continue. | What evidence should make the loop stop? |
The model is the language engine; the harness is the workshop; the agent is the engine working inside that workshop. Your prompt is the work order, a diff is the receipt, and tests are the measuring instruments. You remain the investigator.
Codex and Claude Code combine these layers differently, so the same model can behave differently in another product. Do not ask only “Which model is this?” Ask what context, tools, permissions, and feedback the current harness provides.
THE RECURRING LOOP
Predict, delegate, observe, verify, narrate.
Frame an expectation.
State the question, inspect the inputs, and predict one result before asking for help.
Give a bounded contract.
Specify the goal, relevant files, constraints, permissions, and observable definition of done.
Watch the workshop.
Notice which files, commands, edits, outputs, and uncertainties the harness exposes.
Challenge the result.
Run the code, inspect the diff, test keys and ranges, and consult primary documentation.
Make a judgment.
Record what you accepted, revised, rejected, or left unresolved—and why.
A confident explanation is not evidence. The source data, documentation, code behavior, statistical output, and research design are evidence.
PROMPT ANATOMY
Make success observable.
| Part | Question it answers | Example |
|---|---|---|
| Goal | What outcome do I want? | Explain the join or update one indicator. |
| Context | Which files and facts matter? | Read the script and indicator dictionary. |
| Constraints | What must not happen? | Do not overwrite the frozen dataset. |
| Done when | How can I verify completion? | Keys are unique and year coverage is reported. |
Goal: [one concrete outcome] Context: [files, audience, question, relevant definitions] Constraints: [scope, source rules, no-edit boundaries, claim limits] Done when: [tests, output, comparison, or explanation I can inspect] Before acting, state any consequential assumption you need to make.
VERIFICATION MENU
Match the check to the claim.
- □Code claim: run a small example and inspect the object.
- □Data claim: count rows, test keys, inspect ranges, and read source metadata.
- □Statistical claim: reproduce the output and restate it in units.
- □Policy claim: compare the wording with the research design.
- □Source claim: open the primary source and confirm the relevant detail.
When possible, ask the agent to propose checks, then choose and execute the checks yourself.
COMMON FAILURE MODES
Learn to recognize plausible wrongness.
| Failure | Signal | Response |
|---|---|---|
| Invented variable | Code references a column absent from names(). | Inspect the dictionary and exact file. |
| Silent row loss | A filter or model drops observations. | Compare row counts and missingness. |
| Many-to-many join | Rows unexpectedly multiply. | Test keys on both sides before joining. |
| Unsupported cause | Language shifts from “associated” to “caused.” | Return to the design and claim ladder. |
| Hidden state | Code works only after console experiments. | Restart R and run from the top. |
| False citation | A source cannot be opened or does not support the claim. | Use the primary source and verify manually. |
TRANSPARENCY
Describe contribution and verification.
I used [Codex / Claude Code] to [explain, generate, debug, review, or update]. I provided it with [public project context]. I verified its contribution by [tests, source comparison, clean render, or manual review]. I revised or rejected [important example]. I remain responsible for the analysis and interpretation.
Do not send personal data, identifiable student records, credentials, private API keys, confidential research files, or unpublished sensitive material to a coding agent.