4 Commits

Author SHA1 Message Date
bench
375ce4d1f5 stagerun: 编译超时、看门狗判卡死也不扣编译次数
和容器被杀、执行器退出一样算系统的错,每条轨迹合计最多免 3 次。
10-12 17:16 后 DETR 两次编译一小时没日志没 CPU 被看门狗杀掉、
segformer.b5 编译逼近 4 小时上限,原来都会白扣模型一次机会。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:44:24 +08:00
bench
12787a289d fix: 系统失败不占编译次数、连不上可自动续跑、暂停轨迹不被新题抢名额;加健康检查与排空重启脚本
- stagerun: 容器被杀/执行器退出不计入 5 次编译(每条最多免 3 次)
- quota: CONNECT_FAILED / REQUEST_DEADLINE_BEFORE_SEND 证明未发送,允许自动续跑
- pipeline: 有暂停轨迹只差额度时,空出的名额留给它
- scripts/health_check.py + bench_health_alert.sh: 每 10 分钟检查,异常弹本机通知
- scripts/drain_restart.sh: 排空旧调度程序后重启,排空期间用新代码续跑卡住的轨迹

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:38:07 +08:00
f9b3fa6871 test: drive stagerun through the compile-cap refusal path
Covers the branch itself, not only the counter: at cap-1 the compile runs;
at the cap stagerun publishes COMPILE_BUDGET_EXHAUSTED without starting a
stage row or calling the stage. Verified to fail when the check is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 00:19:16 +08:00
4332f43dab feat: cap each trajectory at three started compiles
Compile attempts are now the primary per-trajectory budget: they map to
the real compute cost, unlike turns or tokens, and make the stop reason
obvious. stagerun refuses a 4th 05-compile with COMPILE_BUDGET_EXHAUSTED
before inline calibration or slot acquisition.

- Counted from the stages ledger (05-compile rows that got a slot), so
  resume/recovery cannot reset it; a compile that timed out in the queue
  is free; recompiles after an accuracy miss count.
- Board/eval/review of the last artifact still run after the cap.
- Turn/token/wall budgets, per-phase deadlines, cleanup and API quota
  protection are unchanged as backstops.
- e2e prompt drops "no retry limit"; SYSTEM.md names compiles as the
  primary budget; agent/VERSION 12 -> 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 22:37:15 +08:00