28 Commits

Author SHA1 Message Date
Codex
761078cfcf batch8: replace held-out concurrency tasks and finish 38 trace6 tasks 2026-10-19 13:02:06 +08:00
Codex
6951eb7aa2 batch8: finish isolated watchdog TEE environment I2C and rpmsg checks 2026-10-19 12:51:46 +08:00
Codex
42aa41b98d batch8: preserve full device ABI and clock verification coverage 2026-10-19 12:51:31 +08:00
Codex
ec5e06730f batch8: finish five verifiers and contain calibration scratch 2026-10-19 12:45:55 +08:00
Codex
cf5b1e7315 batch8: isolate lifecycle memory and firmware verifiers 2026-10-19 12:33:04 +08:00
Codex
cdff64dd3f batch8: isolate binary audits and analysis verifiers 2026-10-19 12:29:34 +08:00
Codex
7f6d9244e0 batch8: isolate five callback and protocol verifiers 2026-10-19 12:28:09 +08:00
Codex
6355e46339 batch8: restore first five tasks for trace6 delivery 2026-10-19 12:24:58 +08:00
amos
913d3c367f Add the usable task catalog and admit batch 6 tasks
All 161 task packages, including the 21 batch 6 C++17 tasks that passed
gates 1/2, are now in Git. rollout.catalog builds docs/task-catalog.{json,md}
from the task packages, a fresh trace5 calibration (128/128 PASS), every
verified controller receipt on the current task digest, documented holds
and digest-bound difficulty reviews. 123 tasks are usable (6 HARD),
5 are on hold and 33 still need a subprocess-only grader.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 17:43:15 +08:00
Codex
75408399b6 Add four C++ reliability tasks with verified Qwen screening 2026-10-18 00:26:04 +08:00
Codex
c0a29c489a Record batch5 Qwen evidence and verified cold migration 2026-10-17 12:53:02 +08:00
Codex
d10f062bab Add embedded reliability batch 5 group 4 (five calibrated tasks) 2026-10-17 10:25:22 +08:00
Codex
7559f1a785 Add embedded reliability batch 5 group 3 (five calibrated tasks) 2026-10-17 10:20:18 +08:00
Codex
9e8091320a Add embedded reliability batch 5 group 2 (five calibrated tasks) 2026-10-17 10:18:48 +08:00
Codex
205118d8c4 Add embedded reliability batch 5 group 1 (five calibrated tasks) 2026-10-17 10:17:46 +08:00
Codex
1f58ebdb6b Record Qwen refill results and verified cold migration 2026-10-17 10:04:29 +08:00
Codex
468dbe68a2 Add five symptom-based embedded reliability tasks and Qwen pilots 2026-10-17 03:27:57 +08:00
amos
05bd1e10c3 Re-author three blocked Bos graders so candidates only run in a subprocess
S500_n3_133 v3, S500_n3_102 v4 and S500_a2_426889b9a968 v3 move every
expected value and assertion into check.py; the candidate process only
receives inputs and prints raw observations. isolation-blocked.txt and the
in-process tests.py are removed and each task gains cheat_read_grader.
Formal trace5 calibration on 36 passed with these exact task digests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 01:32:04 +08:00
Codex
fe65c3bf99 Add 20 HWE repair tasks and record blocked GLM pilots 2026-10-17 01:29:59 +08:00
Codex
69521e54e0 Add 15 embedded repair tasks and correct grader-read controls
Complete batch 20261016-C-embedded-2: 45 verified GLM/Pi samples, independent regrades and archives; preserve difficulty misses and Salty blockage.
2026-10-16 14:23:48 +08:00
amos
6d6ae4f257 Add five C11 serial protocol repair pilot tasks 2026-10-15 22:13:48 +08:00
amos
de3755c14b fix: keep the Bos grader away from candidate processes
The Bos wrapper ran check.py and every candidate process as UID 65534, so a
candidate could read the grader copy through /proc/<checker>/cwd and pass
without implementing anything. check.py now runs as root on a 0700 grader
copy; its subprocesses drop to 65534 and only the work copy and mkdtemp
scratch directories are handed over. An audit hook ends the check as
UNVERIFIED if candidate code would run inside the checker.

36 tasks run hidden tests inside the candidate process (tests.py style and
S500_a2_426889b9a968); they carry isolation-blocked.txt and always grade
UNVERIFIED. The importer adds the marker for any extra root *.py. All 74 Bos
tasks get the new verify.py and a version bump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:33:34 +08:00
amos
f1acc8c938 Import Bos direct batch 20261014-A-bos-direct 2026-10-15 00:37:44 +08:00
amos
94eaf64261 fix: private grade diagnostics and a public-contract admission check
- bos_verify still runs the original check.py unchanged, but failing
  child processes now copy their stderr tail into grade.stderr; report
  rows carry fail_detail (that line, else the grade reason).
- bos_import rejects envs whose hidden tests use candidate-module names
  the public prompt/template never mention; a repo test applies the same
  check to every admitted task.
- S500_n3_152 v2 publishes process(request): its v1 FAILs came from that
  hidden requirement, not from the models. All Bos tasks carry the new
  wrapper as version 2; 21 tasks recalibrated on trace5 (805 trials).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:27:29 +08:00
amos
3a200679a3 feat: admit 20 Bos-qiao pilot tasks in three families
Imported with rollout.bos_import from Bos-qiao envs and calibrated on host
36 with the fixed runtime image: 20/20 PASS, 785 trials, every gold and
legal alternative PASS, every template/noop/cheat FAIL, cleanup confirmed.

- linux-device-abi: I2C_RDWR, spidev, GPIO v2, RTC, DMA-BUF, watchdog, evdev
- firmware-image-boot: uImage, FDT, U-Boot env, boot slots, gzip members,
  MTD writer, UBI
- motor-sensor-control: Clarke/Park, SVPWM, PWM dead time, SSI Gray, NTC
  ADC, CAN bit timing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:49:24 +08:00
amos
714f345b63 feat: add Qwen Responses route, five-field task export and pending-create cleanup
- Pi controller accepts a fixed Qwen qwen3.8-27b-nvfp4 Responses route next
  to GLM-5.3; the broker requires exactly one response.completed per call
  and evidence checks the native model identity.
- rollout.export writes the UltraData five-field JSONL from a frozen
  dataset.json snapshot; every archive now carries dataset.jsonl.
- A Docker CLI that did not finish cannot confirm container or network
  cleanup, and auto-calibration cleanup uncertainty reaches the controller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:05:56 +08:00
amos
d205c44249 fix: cut agent internet egress and loosen grader timeouts
The dedicated Pi network was a plain bridge, so the Agent could reach the
internet and fetch upstream fixes. Create it with --internal, make run
accept only internal networks, and default to --network none.

Verifier limits of 1s per case and 15s per grade turned host load into
false FAIL verdicts. Use 5s per case, 30s compile, 10s isolation probe and
timeout_seconds=60; frame-decoder task version becomes 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:30:00 +08:00
amos
e74867a46f feat: add calibrated Docker task harness 2026-10-12 11:15:44 +08:00