A checkout whose state is elsewhere (BENCH_STATE_DIR, staging) was read as
<checkout>/state, so content review and export found no database and called
a PASS rollout unfinished. One helper, paths.state_dir, used by every lookup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Processes are disposable; rollout state is resumable. When the collector dies,
the next one reconciles each running rollout from durable state instead of
quarantining or losing it:
- execution: a process of an earlier boot on this host is DEAD; a top-level
reaper whose owner (the collector) died still proves its subtree empty
(scopes record the owner that started them); reconcile() pauses a rollout
with a proven-clean crash checkpoint as CONTROLLER_RESTARTED and marks its
running tool calls INTERRUPTED; resume() across a reboot rebases monotonic
clocks by wall time and credits the downtime as a pause
- stageworker.settle_interrupted: an admitted step whose executor is gone ends
as STAGE_EXECUTOR_EXIT (free compile) and its handle is told directly; a
board step without a clean board receipt still quarantines; nothing replays
- collection_runtime.crash_checkpoint: capture the real system prompt, freeze
the session at the last saved message (mid-work phases only); configure_pi
skips a generation ID the ledger already completed
- pi_session.js: checkpoint after every saved message (and compaction); on a
recovered resume drop an unfinished trailing reply and answer open tool
calls with an honest "interrupted by a platform restart" result
- scheduler: a paused rollout's local slot held by a dead holder is released
once every process is proven gone
- pipeline: SIGUSR1 = planned handover (stop claiming, let board steps
finish, shorten in-flight request holds, exit); SIGTERM still drains
- export: bench_recovery closes the cut segment; the dropped unfinished reply
is excluded (UNFINISHED_BEFORE_RESTART)
- tests: test_controller_restart (real Pi on Linux: kill -9 mid-command,
kill -9 mid-reply, SIGUSR1 handover, power loss + reboot), test_settle_
interrupted; three tests of the old quarantine-everything rule now assert
the new outcome while keeping their guarantees (no replay, unrelated PID
untouched, board lease kept)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- broker: a context-window 400 fails its generation (fences nothing) and goes
back to Pi as context_length_exceeded, so Pi compacts and resends
- runner: new generation ID after compaction, wait for Pi's resumed run,
record kept_messages; drop agent/context-mask.js and context_mask_chars
- models.json: contextWindow large so Pi compacts only on a real overflow,
never on an estimate right before a pause (resume would not match)
- export: compactions[] gives what the model actually saw after each one
- prompt: keep facts and acceptance, drop how-to rules and coaching
(SYSTEM.md 89->64 lines, e2e.md 27->20); stage status == stage wait
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reuse execution, stage and review evidence to recover database registration and
SFT publication without replaying completed work. Keep finished futures during
SQLite contention, and hand durable results to startup recovery when draining.
Release proven local leases independently of unresolved board isolation.
Keep UNKNOWN review retries explicit and archived, publish completed receipts
after transient export failures, and prevent a malformed sample from aborting
the entire dataset. Scope routine reconciliation to newly finished attempts.
Add failure-injection coverage and the production acceptance plan.