All 161 task packages, including the 21 batch 6 C++17 tasks that passed
gates 1/2, are now in Git. rollout.catalog builds docs/task-catalog.{json,md}
from the task packages, a fresh trace5 calibration (128/128 PASS), every
verified controller receipt on the current task digest, documented holds
and digest-bound difficulty reviews. 123 tasks are usable (6 HARD),
5 are on hold and 33 still need a subprocess-only grader.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each model_request/model_payload repeated the full context, so the trace
grew with the square of the request count: batch 6 Qwen runs hit 64 MiB
at ~75 requests and 14 attempts (13 graded PASS) were discarded.
Requests now record only items after the common prefix with the previous
one (schema embedded-rollout.pi/2), and pi_run.expand rebuilds full
payloads before the unchanged broker/feedback checks. Needs a new image
(trace6) and recalibration before use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The other users of /home left 57 GiB free; the user chose to keep running
and stop only at 30 GiB. Batch start and runtime disk_low now share that floor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bos wrapper ran check.py and every candidate process as UID 65534, so a
candidate could read the grader copy through /proc/<checker>/cwd and pass
without implementing anything. check.py now runs as root on a 0700 grader
copy; its subprocesses drop to 65534 and only the work copy and mkdtemp
scratch directories are handed over. An audit hook ends the check as
UNVERIFIED if candidate code would run inside the checker.
36 tasks run hidden tests inside the candidate process (tests.py style and
S500_a2_426889b9a968); they carry isolation-blocked.txt and always grade
UNVERIFIED. The importer adds the marker for any extra root *.py. All 74 Bos
tasks get the new verify.py and a version bump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent containers wrote to bind-mounted /workspace and /trace with no
size limit, so a runaway Agent could fill the shared /home. Containers
no longer start below 50 GiB free; a running Agent is stopped when the
disk drops below that floor or its attempt directory passes 2 GiB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- bos_verify still runs the original check.py unchanged, but failing
child processes now copy their stderr tail into grade.stderr; report
rows carry fail_detail (that line, else the grade reason).
- bos_import rejects envs whose hidden tests use candidate-module names
the public prompt/template never mention; a repo test applies the same
check to every admitted task.
- S500_n3_152 v2 publishes process(request): its v1 FAILs came from that
hidden requirement, not from the models. All Bos tasks carry the new
wrapper as version 2; 21 tasks recalibrated on trace5 (805 trials).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Output limits were baked into pi.mjs inside the runtime image, so every
change (GLM 32000 -> 131072, Qwen 8192 -> 65536) needed a new image and a
full 785-trial recalibration, and the Qwen baseline ran on an image the
repo could not rebuild.
- MODELS holds max_tokens and its wire field per model; pi_run passes
--model and --max-tokens to the adapter, which has no defaults left.
- Receipts record model_max_tokens and model_sampling=service-default;
evidence requires every payload to carry exactly that limit, which
would have caught the SDK's silent 32000 cap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two trace4 runs lost their whole session to one upstream HTTP 500 on the
first request (2 of 562 requests), and their controllers still reported
the template's FAIL grade.
- The proxy retries upstream 429/5xx and connection errors up to twice
(2 s, 5 s) before any byte reaches Pi, records every status in
upstream_statuses, and never retries while it is closing.
- A controller reports PASS or FAIL only with VERIFIED evidence and a
clean Agent exit; otherwise the grade stays in task_verdict and the
status is INFRA_ERROR when the model channel returned non-200, else
UNVERIFIED.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A running or killed controller has no receipt yet; it is now an attempt
with status NO_RECEIPT and listed among the exclusions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bos tasks with 131072-token reasoning produce ~110 bytes per streamed
token plus a full context per request; the trace3 batch hit 16 MiB
before finishing. Adapter and host share the new limit, and the size
tests now derive their data from MAX_TRACE_BYTES. Runtime trace4 carries
this adapter; all 21 tasks recalibrated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rollout.regrade checks the original task, image and candidate digests,
grades the frozen candidate again in a fresh grader container and records
whether the verdict matches. rollout.report counts confirmed regrades under
the batch's regrade/ directory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- rollout.bos_import converts one Bos-qiao env into a task package. Fixture
scripts are parsed, never run: only quoted cat heredocs, literal
Path(...).write_text calls or the factory's literal files loop are
accepted. cheat_exit_zero becomes noop, valid_* become extra PASS
fixtures, every other cheat stays a FAIL fixture.
- rollout/bos_verify.py runs the original check.py unchanged as UID 65534
after the same read-isolation probe as frame-decoder; modes are set
before ownership is given away because the grader lacks CAP_FOWNER.
- rollout.report summarizes a batch of controllers; only PASS/FAIL with
VERIFIED evidence, clean exit and confirmed cleanup count as complete.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Tasks may declare up to 16 candidate files; the whole template becomes
the Agent workspace while only candidate files are frozen and graded.
- Optional fixtures add more cheats or legal alternatives (five trials
each); optional grader_files ship private tests next to verify.py.
- Tasks without an argv oracle export call_type=verifier ground truth.
- Upstream model routes live in one MODELS table.
- One bounded no-follow reader, one exclusive writer and one SIGTERM
helper replace per-module copies; drop unused validate_task.
frame-decoder keeps its task digest, calibration receipts and exported
query. Verified on host 36: internal network probe, 20-trial calibration
and a real GLM rollout (PASS, evidence VERIFIED, cleanup confirmed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pi controller accepts a fixed Qwen qwen3.8-27b-nvfp4 Responses route next
to GLM-5.3; the broker requires exactly one response.completed per call
and evidence checks the native model identity.
- rollout.export writes the UltraData five-field JSONL from a frozen
dataset.json snapshot; every archive now carries dataset.jsonl.
- A Docker CLI that did not finish cannot confirm container or network
cleanup, and auto-calibration cleanup uncertainty reaches the controller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dedicated Pi network was a plain bridge, so the Agent could reach the
internet and fetch upstream fixes. Create it with --internal, make run
accept only internal networks, and default to --network none.
Verifier limits of 1s per case and 15s per grade turned host load into
false FAIL verdicts. Use 5s per case, 30s compile, 10s isolation probe and
timeout_seconds=60; frame-decoder task version becomes 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>