Record each agent run's settings, and replay one task across crews #1

Merged

jmoran merged 2 commits from feat/crew-replay into main

2026-10-13 22:43:37 +00:00

jmoran commented

2026-10-12 20:48:20 +00:00

Owner

A run now keeps the model, brief and budget its agent started with (run.config, migration 72), so two runs of one task compare on what they actually used, even after the agent changes. Readers without owner facts get null, like cost and turns.

scripts/crew-replay.ts run <spec> --out <dir> gives each arm its own agent and fork of one base, opens the task, waits for the crew and the check, and writes results.json, a blind blind.json and its key.json. grade <dir> asks which side is better and names the arms only after the last answer.

test/crew-replay.test.ts (in GATE): 12 pass with a stand-in Claude. check-list and studio fail on main already (spawn-host.test.ts is unlisted; a page carries <style).

A run now keeps the model, brief and budget its agent started with (`run.config`, migration 72), so two runs of one task compare on what they actually used, even after the agent changes. Readers without owner facts get null, like cost and turns. `scripts/crew-replay.ts run <spec> --out <dir>` gives each arm its own agent and fork of one base, opens the task, waits for the crew and the check, and writes `results.json`, a blind `blind.json` and its `key.json`. `grade <dir>` asks which side is better and names the arms only after the last answer. `test/crew-replay.test.ts` (in GATE): 12 pass with a stand-in Claude. `check-list` and `studio` fail on main already (spawn-host.test.ts is unlisted; a page carries `<style`).

jmoran added 1 commit

2026-10-12 20:48:20 +00:00

Record each agent run's settings, and replay one task across crews 3099ee4ef2

A run now keeps the model, brief and budget its agent had when it
started (run.config, migration 72), so two runs of one task compare on
what they actually used, even after the agent is edited. Readers
without owner facts see it as null, like cost and turns.

scripts/crew-replay.ts is a client of the API: `run` gives each arm of
a spec its own agent and fork of one base, opens the task, waits for
the crew and the check, and writes results, a blind pair file and its
key; `grade` asks a person which side is better and names the arms only
after the last answer.

jmoran added 1 commit

2026-10-13 22:43:26 +00:00

Merge branch 'main' into feat/crew-replay 8f8d1d4623

jmoran merged commit bc05bafaea into main

2026-10-13 22:43:37 +00:00

jmoran referenced this pull request from a commit

2026-10-13 22:43:38 +00:00

Merge pull request 'Record each agent run's settings, and replay one task across crews' (#1) from feat/crew-replay into main

jmoran deleted branch feat/crew-replay

2026-10-13 22:43:38 +00:00