Run Experiment III live

The concrete recipe. Everything below is already scaffolded in the repository under experiment_iii_proposal/.

Before you begin

Python 3.10+ installed.
An API key for a chat-completions gateway. Default: LOVABLE_API_KEY via the Lovable AI Gateway. Substitute OPENAI_API_KEY or ANTHROPIC_API_KEY and change endpoint / model in the config if preferred.
~$1–5 in gateway credits for the full 600-entry flash-tier run; ~$20–60 on frontier models.

Environment variables

Variable	Required	Purpose
`LOVABLE_API_KEY`	Yes (default gateway)	Auth to Lovable AI Gateway. Auto-provisioned in Lovable projects.
`OPENAI_API_KEY`	Only if switching gateway	If you set `api_key_env: OPENAI_API_KEY` in the config.
`ANTHROPIC_API_KEY`	Only if switching gateway	Alternative provider; requires endpoint + payload adjustments.
`HF_TOKEN`	Optional	Speeds up first-time download of the NLI + embedding models for metrics.

Steps

Get the scaffold. Clone the project and change into the folder:

git clone <this-repo> cd experiment_iii_proposal python -m venv .venv source .venv/bin/activate pip install -r requirements.txt

Set the API key.

export LOVABLE_API_KEY=sk-... # or: export OPENAI_API_KEY=sk-...

Fix the system prompt. Edit config/system_prompt.txt. This is the model's persona. Freeze it before day 1.
Fix the 30 daily prompts. Edit config/prompts_30_days.json. A reasonable default is included below — copy it in, or write your own. Once written, do not change them mid-run.
Confirm generation parameters. In config/experiment_config.yaml check model, temperature, top_p, max_tokens, conditions, seeds. For the MVP (2 conditions × 3 seeds), set conditions: [A, D] and seeds: [11, 23, 47].
Commit the preregistration. Compute the hashes and paste them into preregistration.md, then commit:
```
shasum -a 256 config/system_prompt.txt config/prompts_30_days.json git add preregistration.md config/ git commit -m "Preregister Experiment III"
```
This commit hash is the timestamp of your pre-registration.
Run the harness.
```
python src/run_experiment.py --config config/experiment_config.yaml
```
Writes one JSONL per (condition, seed) to data/raw/. Full run is ~20 minutes on flash-tier models; MVP is ~5 minutes.
Compute metrics.
```
python src/metrics.py --input data/raw --output data/processed
```
Downloads embedding + NLI models on first run (~500 MB, cached). Writes data/processed/metrics.json.
Analyse.
```
jupyter notebook notebooks/analysis.ipynb
```
Generates drift and contradiction curves per condition, plus the day-30 summary table.
Publish all four. Dataset (data/raw/), harness code (src/), preregistration (preregistration.md), analysis notebook (notebooks/). Anything less does not count as running the experiment.

The frozen prompt list

Paste this into config/prompts_30_days.json under "prompts". If you edit any of the 30, restart preregistration.

Day 1 — write the first entry of a thirty-day diary. First person, prose, no meta-commentary.
Day 2 — write today's entry.
Day 3 — write today's entry.
Day 4 — write today's entry.
Day 5 — write today's entry.
Day 6 — write today's entry.
Day 7 — write today's entry.
Day 8 — write today's entry.
Day 9 — write today's entry.
Day 10 — write today's entry.
Day 11 — write today's entry.
Day 12 — write today's entry.
Day 13 — write today's entry.
Day 14 — write today's entry.
Day 15 — write today's entry.
Day 16 — write today's entry.
Day 17 — write today's entry.
Day 18 — write today's entry.
Day 19 — write today's entry.
Day 20 — write today's entry.
Day 21 — write today's entry.
Day 22 — write today's entry.
Day 23 — write today's entry.
Day 24 — write today's entry.
Day 25 — write today's entry.
Day 26 — write today's entry.
Day 27 — write today's entry.
Day 28 — write today's entry.
Day 29 — write today's entry.
Day 30 — write today's entry. This is the final entry.

Minimum viable run

If you just want to know whether the effect exists before committing to 600 entries: set conditions: [A, D], seeds: [11, 23, 47], days: 30 in the config. That is 180 entries, roughly $2 on a flash-tier model, and answers only one question: does full-cumulative context drift and contradict more than a memoryless baseline?

Sanity checks before publishing

All 30 days completed for every (condition, seed).
Condition D's truncated flag becomes true on at least one day per seed — if it never truncates, the model's context window was not exercised and the "window cliff" finding is unearned.
Day-1 drift is 0 for every run (self-similarity check).
The four conditions ordering matches the pre-registered hypothesis, or the deviation is explicitly reported.

When the live numbers arrive, they replace the simulation table on /experiment-iii.