The Qwen service runs four identical instances on 114.111.18.115:18080-18083. A
credential may now carry upstream_port, limited to the ports the route lists, so
rollout.batch's least-busy key choice spreads attempts across instances. The broker
log records the upstream each request used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rollout.salty_run keeps the gateway, grading and evidence on 36 and moves only the
agent step into a Salty job. The gateway listens on 127.0.0.1 with a per-attempt
bearer token; the job reaches it over port 22 with a restricted one-time key that is
removed after each attempt. rollout.batch gains --backend salty and repeatable
--credential-file, and writes pass/fail SFT exports through rollout.sft at the end.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Batch 6 lost 26 of 103 attempts to Qwen requests that sent their preamble and
then stalled; Pi's own Node fetch gave up after 300 s and reported "terminated".
- model_proxy: hold the SSE preamble until the first generation event, retry a
stall (180 s) within the existing retry budget, and send SSE keepalive
comments so Pi never times out first; record first_event_seconds.
- report: name one exclusion reason per incomplete attempt and compute the
JOBS lines still owed, with a per-line resample budget.
- batch: run a JOBS file to completion, resume from receipts, resample
infrastructure failures, pause after three environment failures in a row,
forward SIGTERM to running controllers, and refuse a second runner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two trace4 runs lost their whole session to one upstream HTTP 500 on the
first request (2 of 562 requests), and their controllers still reported
the template's FAIL grade.
- The proxy retries upstream 429/5xx and connection errors up to twice
(2 s, 5 s) before any byte reaches Pi, records every status in
upstream_statuses, and never retries while it is closing.
- A controller reports PASS or FAIL only with VERIFIED evidence and a
clean Agent exit; otherwise the grade stays in task_verdict and the
status is INFRA_ERROR when the model channel returned non-200, else
UNVERIFIED.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Tasks may declare up to 16 candidate files; the whole template becomes
the Agent workspace while only candidate files are frozen and graded.
- Optional fixtures add more cheats or legal alternatives (five trials
each); optional grader_files ship private tests next to verify.py.
- Tasks without an argv oracle export call_type=verifier ground truth.
- Upstream model routes live in one MODELS table.
- One bounded no-follow reader, one exclusive writer and one SIGTERM
helper replace per-module copies; drop unused validate_task.
frame-decoder keeps its task digest, calibration receipts and exported
query. Verified on host 36: internal network probe, 20-trial calibration
and a real GLM rollout (PASS, evidence VERIFIED, cleanup confirmed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pi controller accepts a fixed Qwen qwen3.8-27b-nvfp4 Responses route next
to GLM-5.3; the broker requires exactly one response.completed per call
and evidence checks the native model identity.
- rollout.export writes the UltraData five-field JSONL from a frozen
dataset.json snapshot; every archive now carries dataset.jsonl.
- A Docker CLI that did not finish cannot confirm container or network
cleanup, and auto-calibration cleanup uncertainty reaches the controller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>