13 Commits

Author SHA1 Message Date
amos
b773033658 Record model requests and payloads as prefix deltas in the native trace
Each model_request/model_payload repeated the full context, so the trace
grew with the square of the request count: batch 6 Qwen runs hit 64 MiB
at ~75 requests and 14 attempts (13 graded PASS) were discarded.
Requests now record only items after the common prefix with the previous
one (schema embedded-rollout.pi/2), and pi_run.expand rebuilds full
payloads before the unchanged broker/feedback checks. Needs a new image
(trace6) and recalibration before use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 10:50:56 +08:00
amos
b71d605d81 Lower the shared-disk floor to 30 GiB at the user's request
The other users of /home left 57 GiB free; the user chose to keep running
and stop only at 30 GiB. Batch start and runtime disk_low now share that floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 13:11:21 +08:00
Codex
c58d5cc345 fix: allow 900 seconds of upstream stream silence 2026-10-17 01:40:40 +08:00
amos
de3755c14b fix: keep the Bos grader away from candidate processes
The Bos wrapper ran check.py and every candidate process as UID 65534, so a
candidate could read the grader copy through /proc/<checker>/cwd and pass
without implementing anything. check.py now runs as root on a 0700 grader
copy; its subprocesses drop to 65534 and only the work copy and mkdtemp
scratch directories are handed over. An audit hook ends the check as
UNVERIFIED if candidate code would run inside the checker.

36 tasks run hidden tests inside the candidate process (tests.py style and
S500_a2_426889b9a968); they carry isolation-blocked.txt and always grade
UNVERIFIED. The importer adds the marker for any extra root *.py. All 74 Bos
tasks get the new verify.py and a version bump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:33:34 +08:00
amos
141f3d06aa fix: cap disk use per attempt and refuse to start on a low output disk
Agent containers wrote to bind-mounted /workspace and /trace with no
size limit, so a runaway Agent could fill the shared /home. Containers
no longer start below 50 GiB free; a running Agent is stopped when the
disk drops below that floor or its attempt directory passes 2 GiB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:22:35 +08:00
amos
c6435255f9 docs: record the user-specified Git remote
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:49:41 +08:00
amos
94eaf64261 fix: private grade diagnostics and a public-contract admission check
- bos_verify still runs the original check.py unchanged, but failing
  child processes now copy their stderr tail into grade.stderr; report
  rows carry fail_detail (that line, else the grade reason).
- bos_import rejects envs whose hidden tests use candidate-module names
  the public prompt/template never mention; a repo test applies the same
  check to every admitted task.
- S500_n3_152 v2 publishes process(request): its v1 FAILs came from that
  hidden requirement, not from the models. All Bos tasks carry the new
  wrapper as version 2; 21 tasks recalibrated on trace5 (805 trials).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:27:29 +08:00
amos
31c862cd3c fix: retry transient upstream failures and classify broken sessions
Two trace4 runs lost their whole session to one upstream HTTP 500 on the
first request (2 of 562 requests), and their controllers still reported
the template's FAIL grade.
- The proxy retries upstream 429/5xx and connection errors up to twice
  (2 s, 5 s) before any byte reaches Pi, records every status in
  upstream_statuses, and never retries while it is closing.
- A controller reports PASS or FAIL only with VERIFIED evidence and a
  clean Agent exit; otherwise the grade stays in task_verdict and the
  status is INFRA_ERROR when the model channel returned non-200, else
  UNVERIFIED.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 02:10:05 +08:00
amos
3e19424391 feat: regrade frozen candidates and report rerun consistency
rollout.regrade checks the original task, image and candidate digests,
grades the frozen candidate again in a fresh grader container and records
whether the verdict matches. rollout.report counts confirmed regrades under
the batch's regrade/ directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:52:29 +08:00
amos
acfa601a56 feat: multi-file task contract, model table and shared I/O helpers
- Tasks may declare up to 16 candidate files; the whole template becomes
  the Agent workspace while only candidate files are frozen and graded.
- Optional fixtures add more cheats or legal alternatives (five trials
  each); optional grader_files ship private tests next to verify.py.
- Tasks without an argv oracle export call_type=verifier ground truth.
- Upstream model routes live in one MODELS table.
- One bounded no-follow reader, one exclusive writer and one SIGTERM
  helper replace per-module copies; drop unused validate_task.

frame-decoder keeps its task digest, calibration receipts and exported
query. Verified on host 36: internal network probe, 20-trial calibration
and a real GLM rollout (PASS, evidence VERIFIED, cleanup confirmed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:25:09 +08:00
amos
e524d9eda5 docs: deployment map, task authoring standard and Qwen trial record
Add the current and target deployment diagrams, the task author standard
and copyable spec template, the five-field data contract, and record the
Qwen Responses trial and pending-create cleanup fix in progress and
acceptance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:06:06 +08:00
amos
d205c44249 fix: cut agent internet egress and loosen grader timeouts
The dedicated Pi network was a plain bridge, so the Agent could reach the
internet and fetch upstream fixes. Create it with --internal, make run
accept only internal networks, and default to --network none.

Verifier limits of 1s per case and 15s per grade turned host load into
false FAIL verdicts. Use 5s per case, 30s compile, 10s isolation probe and
timeout_seconds=60; frame-decoder task version becomes 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:30:00 +08:00
amos
65ceed6c46 docs: establish rollout scope and development contracts 2026-10-12 11:15:44 +08:00