4 Commits

Author SHA1 Message Date
amos
94eaf64261 fix: private grade diagnostics and a public-contract admission check
- bos_verify still runs the original check.py unchanged, but failing
  child processes now copy their stderr tail into grade.stderr; report
  rows carry fail_detail (that line, else the grade reason).
- bos_import rejects envs whose hidden tests use candidate-module names
  the public prompt/template never mention; a repo test applies the same
  check to every admitted task.
- S500_n3_152 v2 publishes process(request): its v1 FAILs came from that
  hidden requirement, not from the models. All Bos tasks carry the new
  wrapper as version 2; 21 tasks recalibrated on trace5 (805 trials).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:27:29 +08:00
amos
f20316fa4d fix: report counts controllers without a receipt instead of failing
A running or killed controller has no receipt yet; it is now an attempt
with status NO_RECEIPT and listed among the exclusions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 00:06:56 +08:00
amos
3e19424391 feat: regrade frozen candidates and report rerun consistency
rollout.regrade checks the original task, image and candidate digests,
grades the frozen candidate again in a fresh grader container and records
whether the verdict matches. rollout.report counts confirmed regrades under
the batch's regrade/ directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:52:29 +08:00
amos
b19cfd4844 feat: Bos-qiao task importer, unprivileged check wrapper and batch report
- rollout.bos_import converts one Bos-qiao env into a task package. Fixture
  scripts are parsed, never run: only quoted cat heredocs, literal
  Path(...).write_text calls or the factory's literal files loop are
  accepted. cheat_exit_zero becomes noop, valid_* become extra PASS
  fixtures, every other cheat stays a FAIL fixture.
- rollout/bos_verify.py runs the original check.py unchanged as UID 65534
  after the same read-isolation probe as frame-decoder; modes are set
  before ownership is given away because the grader lacks CAP_FOWNER.
- rollout.report summarizes a batch of controllers; only PASS/FAIL with
  VERIFIED evidence, clean exit and confirmed cleanup count as complete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:48:25 +08:00