16 Commits

Author SHA1 Message Date
bench
db5a7a839d Export: read the running checkout's own state wherever it lives
A checkout whose state is elsewhere (BENCH_STATE_DIR, staging) was read as
<checkout>/state, so content review and export found no database and called
a PASS rollout unfinished. One helper, paths.state_dir, used by every lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 16:52:42 +08:00
bench
0da66edde0 Resume rollouts after the collector dies (crash, kill, reboot, planned handover)
Processes are disposable; rollout state is resumable. When the collector dies,
the next one reconciles each running rollout from durable state instead of
quarantining or losing it:

- execution: a process of an earlier boot on this host is DEAD; a top-level
  reaper whose owner (the collector) died still proves its subtree empty
  (scopes record the owner that started them); reconcile() pauses a rollout
  with a proven-clean crash checkpoint as CONTROLLER_RESTARTED and marks its
  running tool calls INTERRUPTED; resume() across a reboot rebases monotonic
  clocks by wall time and credits the downtime as a pause
- stageworker.settle_interrupted: an admitted step whose executor is gone ends
  as STAGE_EXECUTOR_EXIT (free compile) and its handle is told directly; a
  board step without a clean board receipt still quarantines; nothing replays
- collection_runtime.crash_checkpoint: capture the real system prompt, freeze
  the session at the last saved message (mid-work phases only); configure_pi
  skips a generation ID the ledger already completed
- pi_session.js: checkpoint after every saved message (and compaction); on a
  recovered resume drop an unfinished trailing reply and answer open tool
  calls with an honest "interrupted by a platform restart" result
- scheduler: a paused rollout's local slot held by a dead holder is released
  once every process is proven gone
- pipeline: SIGUSR1 = planned handover (stop claiming, let board steps
  finish, shorten in-flight request holds, exit); SIGTERM still drains
- export: bench_recovery closes the cut segment; the dropped unfinished reply
  is excluded (UNFINISHED_BEFORE_RESTART)
- tests: test_controller_restart (real Pi on Linux: kill -9 mid-command,
  kill -9 mid-reply, SIGUSR1 handover, power loss + reboot), test_settle_
  interrupted; three tests of the old quarantine-everything rule now assert
  the new outcome while keeping their guarantees (no replay, unrelated PID
  untouched, board lease kept)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 09:29:26 +08:00
bench
18379401e4 v23: hand context compaction back to Pi, cut process rules from the prompt
- broker: a context-window 400 fails its generation (fences nothing) and goes
  back to Pi as context_length_exceeded, so Pi compacts and resends
- runner: new generation ID after compaction, wait for Pi's resumed run,
  record kept_messages; drop agent/context-mask.js and context_mask_chars
- models.json: contextWindow large so Pi compacts only on a real overflow,
  never on an estimate right before a pause (resume would not match)
- export: compactions[] gives what the model actually saw after each one
- prompt: keep facts and acceptance, drop how-to rules and coaching
  (SYSTEM.md 89->64 lines, e2e.md 27->20); stage status == stage wait

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 17:04:58 +08:00
bench
96d72b2f8c 判分版本固定处跟到 v0.8;导出清单说明改读 RECIPE_VERSION
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:16:48 +08:00
bench
41eb1d2193 agent v17:提示词删掉「评测数据只读可见」;判分 v0.8:板上磁盘满和板子连不上一样免罚
- dev 环境 10-14 起只挂校准集,VM 上 bench/data 为空,agent 已看不到评测数据,提示词与实际不符。
- 用户 10-15 定:板上磁盘满(disk_full)与 board_offline 同等处理,不扣 G2/G3、不扣超时分。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 13:53:34 +08:00
498187569c fix: close rollout finalization and review publication gaps
Reuse execution, stage and review evidence to recover database registration and
SFT publication without replaying completed work. Keep finished futures during
SQLite contention, and hand durable results to startup recovery when draining.
Release proven local leases independently of unresolved board isolation.

Keep UNKNOWN review retries explicit and archived, publish completed receipts
after transient export failures, and prevent a malformed sample from aborting
the entire dataset. Scope routine reconciliation to newly finished attempts.
Add failure-injection coverage and the production acceptance plan.
2026-09-29 16:09:01 +08:00
b398495d54 fix: allow 32768 tokens for content review 2026-09-23 23:29:04 +08:00
Codex
d35c82a8bf fix: integrate managed content review into dataset refresh 2026-09-23 21:30:40 +08:00
Codex
068c56043f Merge managed natural-agent runtime into retained benchmark mainline 2026-09-22 15:35:28 +08:00
bench
d309109f2b pre-grind snapshot 2026-09-19: batches 1-7 adapters/probes/recipes + eval additive changes 2026-09-19 17:52:03 +08:00
bench
ee4c7fce1c reward v0.4: board_offline 不加分只免罚;board-backfill 支持 --rollout 用现成 HBM 补测
- export/reward.py: G3 unverified 且最后一次 07/08 为 board_offline → G2/G3 权重 0、G1 保持 1.0、免 TIMEOUT 罚分,
  返回 unverifiable=board_offline;RECIPE v0.4。实测 r20260916-01 regnet.y.800mf −0.97 → +0.03
- export/to_agentrl.py: 记录 reward_unverifiable
- harness/pipeline.py: board_backfill_rollout:用 rollouts/kk/artifacts 的 HBM 补跑 07/08(lane=backfill)后 verify_rollout
- harness/cli.py: bench board-backfill --rollout JOB_ID[:KK]
- 单测 scripts/test_reward_board_offline.py、scripts/test_rollout_backfill.py;CONTRACT §9 / README 同步
2026-09-16 15:58:24 +08:00
bench
cd220cc81e fix: preserve deployment outcomes and export completed run data 2026-09-16 07:55:44 +08:00
bench
4a0e7d521f fix: run S100P board accuracy through the existing pipeline 2026-09-16 02:17:36 +08:00
bench
50edd8c2a1 sync 2026-09-15_12:21:38 d-robotics@d-roboticsdeMacBook-Pro 2026-09-15 12:21:38 +08:00
bench
7ee7ca7c8f sync 2026-09-15_09:01:15 d-robotics@d-roboticsdeMacBook-Pro 2026-09-15 09:01:15 +08:00
bench
29ef645b58 baseline 2026-09-15: P1-P3 landed; first PASS trajectory; collaboration rules (CONTRACT §12) 2026-09-15 08:10:26 +08:00