6 Commits

Author SHA1 Message Date
amos
b773033658 Record model requests and payloads as prefix deltas in the native trace
Each model_request/model_payload repeated the full context, so the trace
grew with the square of the request count: batch 6 Qwen runs hit 64 MiB
at ~75 requests and 14 attempts (13 graded PASS) were discarded.
Requests now record only items after the common prefix with the previous
one (schema embedded-rollout.pi/2), and pi_run.expand rebuilds full
payloads before the unchanged broker/feedback checks. Needs a new image
(trace6) and recalibration before use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 10:50:56 +08:00
amos
f0c277a0d1 fix: controller owns model output limits; verify them on the wire
Output limits were baked into pi.mjs inside the runtime image, so every
change (GLM 32000 -> 131072, Qwen 8192 -> 65536) needed a new image and a
full 785-trial recalibration, and the Qwen baseline ran on an image the
repo could not rebuild.
- MODELS holds max_tokens and its wire field per model; pi_run passes
  --model and --max-tokens to the adapter, which has no defaults left.
- Receipts record model_max_tokens and model_sampling=service-default;
  evidence requires every payload to carry exactly that limit, which
  would have caught the SDK's silent 32000 cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:27:29 +08:00
amos
31c862cd3c fix: retry transient upstream failures and classify broken sessions
Two trace4 runs lost their whole session to one upstream HTTP 500 on the
first request (2 of 562 requests), and their controllers still reported
the template's FAIL grade.
- The proxy retries upstream 429/5xx and connection errors up to twice
  (2 s, 5 s) before any byte reaches Pi, records every status in
  upstream_statuses, and never retries while it is closing.
- A controller reports PASS or FAIL only with VERIFIED evidence and a
  clean Agent exit; otherwise the grade stays in task_verdict and the
  status is INFRA_ERROR when the model channel returned non-200, else
  UNVERIFIED.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 02:10:05 +08:00
amos
714f345b63 feat: add Qwen Responses route, five-field task export and pending-create cleanup
- Pi controller accepts a fixed Qwen qwen3.8-27b-nvfp4 Responses route next
  to GLM-5.3; the broker requires exactly one response.completed per call
  and evidence checks the native model identity.
- rollout.export writes the UltraData five-field JSONL from a frozen
  dataset.json snapshot; every archive now carries dataset.jsonl.
- A Docker CLI that did not finish cannot confirm container or network
  cleanup, and auto-calibration cleanup uncertainty reaches the controller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:05:56 +08:00
amos
d205c44249 fix: cut agent internet egress and loosen grader timeouts
The dedicated Pi network was a plain bridge, so the Agent could reach the
internet and fetch upstream fixes. Create it with --internal, make run
accept only internal networks, and default to --network none.

Verifier limits of 1s per case and 15s per grade turned host load into
false FAIL verdicts. Use 5s per case, 30s compile, 10s isolation probe and
timeout_seconds=60; frame-decoder task version becomes 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:30:00 +08:00
amos
563a3745df feat: integrate Pi rollout and protected model proxy 2026-10-12 11:15:44 +08:00