wishforge-ide/docs/acceptance-traceability.md
eust-w b47a07e8d2 chore: baseline snapshot of the Codex-delivered code before review
Imports the previously untracked working tree as-is so that the review
changes that follow (requirements -> design -> implementation) can be
reviewed as diffs and reverted if needed. No content changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:18:04 +08:00

25 KiB
Raw Blame History

Requirement → implementation → acceptance

Verification date: 2026-10-17, Asia/Shanghai. Source implementation, local acceptance, and production/customer acceptance are separate claims. This is not a declaration that P0–P5 or the commercial pilot are complete.

Final regression snapshot: root TypeScript checks passed; 295 main tests passed with 16 explicit integration tests skipped; separately enabled PostgreSQL/billing/ MinIO/Vault-protocol tests passed 9/9. IDE suites passed 83 with one optional OCT container test skipped; private gateway sockets passed 11/11. The Web production build passed. See implementation status for scope, final report paths and remaining implementation versus environment gates.

ID Requirement Implemented path Executed evidence / remaining boundary
WF-01 Remote wishes, authorization, idempotency API, contracts, MCP connectors, web Control-plane tests cover auth and state; real Codex/WorkBuddy OAuth client authorization is a separate acceptance case
WF-02 Durable task loop, recovery and fencing Temporal worker, API outbox/leases, run generation Development Temporal is reachable; runtime restoration and stale-worker controls have tests. Kill/replay with real providers is not established by those tests
WF-03 Exact source → validation → build binding Git checkpoints, verifier, integration queue, immutable release Real isolated Git checkpoint/bundle restore, binary file preservation, tamper denial, clean-SHA validation and stale CAS promotion tests pass
WF-04 Managed and BYO deployment Compose/Kubernetes blue-green adapters, signed outbound runner Real local Compose/Caddy acceptance passed, including bad candidate retention and automatic rollback after post-cutover business failure. Managed/customer Kubernetes and customer mTLS Runner acceptance remain pending
WF-05 Version-bound policy and approval API release/evidence policy, runner manifest Runtime binds source, digest, architecture, configuration and protected checks to the candidate identity; same release ID with changed config is rejected. Secret references without a broker fail unconfigured
WF-06 Tenant BYOK and independent evolution budget API credentials, worker role split, evolution domain Unit/integration evidence does not establish real supplier credentials or global data opt-in acceptance
WF-07 Full browser IDE Pinned Theia 1.76.0, gateway, workspace provisioner Independent build and real web → API → gateway → Theia editor/terminal/Socket.IO flow passed; Node/js-debug and Python/debugpy breakpoint, stepping and variable checks pass on macOS; Linux runtime and other extension workflows still require individual acceptance
WF-08 Multi-user editing with workspace ACL Pinned collaboration packages and local OCT fixture Protocol and two-browser editing accepted locally. Isolated OCT routing, bidirectional edits and host save passed. Acknowledged save barriers, saved revision restart and draft comparison recovery also passed locally. Independent production identities and lossless OCT room recovery remain separate; unconfigured instances stay disabled
WF-09 Viewer denial and debugger controller Authenticated gateway, API workspace policy Gateway tests deny viewers and close revoked sockets. Backend native/VS Code DAP gates and real js-debug launch/step/variable checks now pass; takeover of an already-running plugin-host UI session is separate
WF-10 Save barrier and collaborative restart recovery Workspace control-plane records, explicit saved commit/snapshot import and fenced integration lease Two-client saveAll/ack, durable revision restart, conflict refusal, IndexedDB recovery and real API-to-Theia-to-Git snapshot handoff pass. OCT room reconstruction, automatic human Git synchronization and hostile-terminal state protection remain separate
WF-11 Platform RSI Versioned candidate/evaluator/promotion modules Trust-gate tests are distinct from a real repeated evaluation corpus, shadow run and 24-hour canary acceptance
WF-12 Production sandbox isolation Kata provisioner, two role PVCs, scoped secrets, network policy Render/lifecycle tests pass; no actual Kata VM, CNI denial, resource exhaustion, bubblewrap or cluster revocation probes were performed
WF-13 Standard client authentication MCP and OIDC metadata, web login flow Real local Keycloak PKCE login, encrypted session, callback replay denial, logout and a second nonmember account denial passed. Enterprise IdP/HTTPS, real WorkBuddy/Codex OAuth refresh and membership revocation still require live clients
WF-14 Local infrastructure Dedicated wishforge-dev Compose project PostgreSQL, Temporal, Keycloak, MinIO, registry and OCT were observed running locally; this is not production backup, durability or capacity evidence
WF-15 Persistent project Git storage Authenticated smart HTTP Git service on 4504 Real bare-repo create → token clone → deny read-token push → write CAS push → re-clone passed; cross-tenant scope, expiration and repository symlink escape are denied
WF-16 Run restoration across separate execution volumes Git bundle checkpoint + generation-bound provisioner Restore reproduces exact commit and binary contents on a separate directory. Executor file mutation cannot change the restored copy. Inline bundle currently has a 4 MiB limit
WF-17 Plans, quota and usage billing Operator plan assignment, atomic execution reservations, signed resource grants, invoice/CSV/manual settlement Domain, PostgreSQL and browser checks pass. Actual CPU/persistent storage time-series collectors and priced invoice reconciliation remain incomplete; unavailable dimensions remain unknown
WF-18 Workspace lifecycle and tenant routing Persistent lifecycle outbox, save/idle checks, scoped provisioner and gateway map Read-only real IDE idle observation and UI checks pass; real cluster scale-down/up, multi-replica provisioner fencing and production capacity remain unverified
WF-19 Private application access Dedicated gateway, short-lived sessions, current ACL checks, release-bound signed probe proof Real local API/gateway/BFF/browser chain passed; deployment/build identities were explicit fixtures. DNS/TLS and direct-upstream network denial require target acceptance

Executed local deployment evidence

WF_COMPOSE_ACCEPTANCE=true WF_CADDY_BINARY=<verified binary> node --import tsx infra/scripts/compose-acceptance.mjs uses the explicit colima context, a dedicated random Compose project, an isolated Docker config, the local WishForge registry, and a loopback-only Caddy 2.10.2 process. It cleans only its own containers/networks and local fixture tag. Registry fixture blobs remain in the development registry; no unrelated Docker resources are stopped and no cloud resources are purchased.

The 2026-10-17 run built a real Linux/arm64 Node image and used registry digest sha256:e6d60c51c79a20621b17956189355aefd7ef3b99bea1cca89c727f81072d75e1. Assertions passed for:

  • First release, candidate health plus protected business assertion, then Caddy ETag CAS traffic switch.
  • Second immutable release, with the previous candidate kept available.
  • Bad candidate fails business assertions and leaves current traffic unchanged.
  • Candidate healthy directly but failing through the stable route triggers a second CAS restoring the prior proxy configuration; stable business output is checked again.
  • A stale expected previous release and a changed manifest under the same release ID are rejected.
  • Explicit rollback to the first release and independent reconcile readback return verified.

Raw run output is retained locally at /private/tmp/wishforge-compose-acceptance.json. The final rerun after deployment-secret wiring passed all ten checks, including a local KV v2 fixture returning exact version 3 to the broker and a real application container. The app exposes only secretConfigured: true; saved Compose content contains placeholders and public receipts contain no secret value. Final Linux/arm64 image digest: sha256:e0255c67d07163169ff7bb33cdba808a814aa7c9ca052d7a30640e93da9b4c65. Report: /private/tmp/wishforge-compose-final-acceptance.json. Its temporary containers, proxy and fixture files were cleaned. Real Vault/TLS/RBAC and Kubernetes Secret injection remain target-specific acceptance, not established by this fixture. This fixture tests the actual deployment adapter and proxy. It does not exercise a customer-enrolled mTLS Runner, Kubernetes, production traffic, database migration, load capacity, or a vendor model call.

IDE browser evidence

Theia's independent package was built on 2026-10-17. The browser flow is: http://127.0.0.1:3000/ide/:workspaceId/launch → real API workspace authorization → one-time gateway launch → http://127.0.0.1:4400/workspaces/:workspaceId/ → local Theia at 4300. The dedicated local workspace/configuration live under .wishforge/.

Desktop Chrome at 1440 × 1000 showed the README in Monaco, opened the integrated terminal, executed WISHFORGE_TERMINAL_OK, and upgraded Socket.IO to a WebSocket through the gateway. The title was README.md - ide-workspace. No framework overlay was present. The local fixture produced a non-blocking favicon 404 and Node 26 plugin-host deprecation messages; these are not suppressed or represented as a zero-warning production browser acceptance.

The real flow exposed and fixed two integration defects: API authorization uses {data: grant}, and Theia 1.76 requires its own theia-connection-token. The gateway now unwraps the actual API envelope and keeps this upstream process token on the server, separate from user bearer/session cookies. Per-workspace cookie paths support multiple workspaces without replacing each other's browser session. Gateway transport tests use real HTTP/WebSocket sockets and cover single-use launch codes, viewer denial, wrong origins, upstream allowlists, credential stripping, Theia token preservation, and active revocation.

Browser plugin was not available; regular Playwright with the installed Chrome was used. Local browser evidence is saved at /private/tmp/wishforge-theia-real-gateway.json and /private/tmp/wishforge-theia-real-gateway.png. A separate two-browser collaboration flow is recorded below. Read-only room, debugger-controller and lossless room-recovery are not claimed.

Production boundary and remaining acceptance

  • No Kubernetes deployment was applied. An existing kubectl context is not permission to deploy and was not used as acceptance evidence. Kata, CNI, private service routing, separate role PVCs, resource limits, external builder and real mTLS all need the intended test cluster. WF_SANDBOX_ENFORCED=true is an assertion, not proof.
  • Run resources are generation-bound. Before new Jobs, the provisioner deletes only older Jobs matching this namespace, managed-by label and run hash, with foreground completion. Old PVCs remain for diagnosis. Same-generation terminal Jobs return failed and require a new generation; they are not silently recreated.
  • Executor and verifier have independent RWO volumes. The verifier restores a content/digest-checked Git bundle; it does not trust executor filesystem paths. Token generations and API leases independently fence stale evidence.
  • Default networking only permits selected control-plane and DNS flows. Vendor, registry, Git, package, artifact and builder traffic requires an explicit reviewed egress design; source flags do not make that network available.
  • Unscoped OCT room invitations remain disabled. Scoped collaboration uses a separate OCT instance per workspace and gateway edit authorization, including active connection revocation. Production network isolation and multi-identity OIDC acceptance remain gates for broader enablement.
  • Real provider execution, GitHub protected-branch PR/merge flows, customer target enrollment, production secrets, migration compatibility, capacity/SLOs and full platform/PITR recovery drills remain required before claiming the paid pilot complete.

Scoped collaboration protocol evidence

The independent control-hardening run executed WF_OCT_INTEGRATION=1 node --test --test-force-exit apps/ide/collaboration/test/isolated-oct.test.mjs against two actual OCT 0.3.1 containers and the real gateway. It passed encrypted Socket.IO initialization, Yjs host-to-guest and guest-to-host convergence, foreign-workspace cookie and invitation denial, per-instance JWT-key isolation, viewer denial, and revocation disconnection. The authorization service in this protocol test is a fixture. These results establish protocol/routing behavior, not production OIDC or Kata.

The client factory validates the same browser origin and current workspace path, keeps form login relative to that path, and specifies Socket.IO's Engine.IO path explicitly. The default client would otherwise lose the gateway prefix. Theia uses an application policy extension: commands are absent unless its dedicated workspace route is configured. No public OCT fallback is enabled.

Actual two-browser collaboration acceptance

Two isolated Chrome browser contexts (1440 × 1000 each) used the real local Next.js launch, API workspace grant, authenticated gateway and the dedicated wishforge-oct-workspace-90cd0527 instance on 48102. Host room creation, guest login, explicit host Allow, and guest Connected status all completed. Both Engine.IO connections used /workspaces/:id/collaboration/socket.io/.

The guest opened the shared collaboration:/ide-workspace/ide-workspace/README.md through Explorer, rather than editing the old local file: tab. A unique host marker appeared in the guest editor; a unique guest marker appeared in the host editor. Host Save persisted both markers, verified by reading the QA file from disk. Screenshots show Shared/Connected states and peer cursors. The policy also prevents a guest virtual workspace from changing the backend-wide default project for other browser tabs.

Evidence: /private/tmp/wishforge-collaboration-browser.json, /private/tmp/wishforge-collaboration-host.png, and /private/tmp/wishforge-collaboration-guest.png. The test had no failed HTTP responses or page exceptions. Console diagnostics included the known favicon 404, Node 26 plugin deprecations and Theia's INVALID tab diagnostic during the virtual workspace transition; this is not a zero-console-error claim.

Both browser contexts used the local development platform identity, with separate OCT display names. The independent protocol test covers different fixture roles and workspace permissions; neither test substitutes for two real production OIDC users. That earlier host Save was not an acknowledged multi-editor save barrier. The new coordinated snapshot/recovery acceptance below verifies the explicit barrier and recovery commands separately. OCT room restart/rejoin and production isolation are not accepted by this successful flow.

Coordinated snapshots, unconfirmed drafts and DAP control

The actual Theia policy extension now loads both a frontend lifecycle contribution and a Node backend service at /services/wishforge-coordination. It derives actor identity from an Ed25519 gateway assertion, never a caller-supplied actor string. Only the gateway holds the private key. The browser does not receive debug lease tokens; they stay in its authenticated backend connection context.

The consolidated IDE test suite passed 62 cases after the resource-limit additions: coordinator snapshots/restart/fault injection, precise version/dirty/client/disk conflict rejection, lease ownership/renewal/transfer/expiry/revocation, signed actor assertions, gateway relay, real Theia RPC/DAP wire encoding, and malformed-frame denial. Normal plugin RPC remains byte-for-byte preserved.

Real browser acceptance at /private/tmp/wishforge-coordination-ui.json passed:

  • Two connected IDE clients ran saveAll, returned/confirmed document version vectors, and produced saved revision 5. The backend had been restarted after revision 4; the counter and immutable saved content continued from disk.
  • A second client was denied another client's debug lease. Transfer rotated ownership and the prior client was denied reacquisition while the new lease was live. Both windows used the development identity; independent actor cases are additionally covered by the backend tests.
  • An unsaved draft survived navigation/reload in IndexedDB. After the saved file was changed offline, recovery displayed a saved-versus-recovered comparison and an unsaved recovery copy. Reading the actual file proved the newer saved text was not overwritten. Screenshot: /private/tmp/wishforge-coordination-one.png.

Real bundled js-debug acceptance at /private/tmp/wishforge-debug-ui.json passed: F5 without control was rejected by the Node backend (DEBUG_CONTROL_REQUIRED), acquiring the lease allowed a real Node process to pause at entry, call stack and variables rendered, and F10 advanced to line 2 with count: 1. Screenshot: /private/tmp/wishforge-debug-controller.png. The fixture was stopped afterward.

Snapshot limits are 32 MiB/file, 256 MiB total source and 20,000 files by default. Draft+baseline is limited to 2 MiB UTF-8. The final capture rechecks barrier expiry and participant freshness so an overlong scan cannot become a successful commit. Browser diagnostics still include upstream INVALID tab during virtual/diff editor transitions and Node 26 plugin deprecations; no zero-warning claim is made.

Security boundary: the current Theia backend and human terminal share a UID. A separate coordination PVC/directory and mode 0700 do not stop a malicious terminal from modifying that UID's writable state or replacing its backend. These are authenticated cooperative-client and local integrity controls, not hard isolation between hostile editors in one workspace. Workspace snapshots must not be promoted to independently trusted release evidence; the separate Git verifier remains the release trust boundary. Hardening requires direct gateway-to-separate- privilege coordination and managed debugger processes, plus real cluster tests.

The controller lease can transfer; another user's UI taking over an existing plugin-host-owned debug session is not supplied by that token transfer. Production OIDC/CNI variants, existing-session UI takeover, and lossless OCT room reconstruction remain distinct checks. Python debugger local acceptance is below.

Additional executed acceptance — 2026-10-17

  • Pinned Codex SDK/CLI through the actual credential proxy: WF_CODEX_SDK_INTEGRATION=1 npx vitest run tests/integration/codex-sdk.test.ts passed. An injected local upstream sends a shell tool call and final response; the real SDK creates a file in a disposable workspace and exports its Git checkpoint. The proxy records both receipts exactly once (26 synthetic tokens). This is protocol/tool/checkpoint acceptance, not a real model quality test or a complete production sandbox run. See provider-proxy.md for fixes and limits.
  • Independent acceptance profiles: production rejects repository-only tests; operator-installed root-owned scripts are checked by digest, executed separately, and bound to build/release policy. Six gate tests and project-setting UI checks pass. The local installed profile list is intentionally empty; no customer acceptance corpus has been invented. See verification-profiles.md.
  • Python debugger: official fixed Python/debugpy VSIX archives were verified against their recorded SHA-256. The real Theia/browser flow denied F5 without control, then stopped at line 2 with count=1, stepped to line 3 with count=2, and displayed the call stack. Evidence: /private/tmp/wishforge-python-debug-ui.json and /private/tmp/wishforge-python-debug-controller.png. The debug process was stopped and the original Node launch fixture restored. Linux packages are pinned but were not run; upstream extension/Node warnings remain visible.
  • PostgreSQL restore: a real exported-snapshot dump restored into a new, isolated database and matched eight table digests plus schema/RLS/security metadata. The test database was removed. The small local drill took 239 ms; this is not a production RPO/RTO result. See recovery.md for exact evidence and the external assets not included in this backup.
  • Runtime API: health and ready returned 200, metrics returned 401 without its monitoring token and 200 with it. With proxy/evolution production settings absent, they report unconfigured rather than starting paid work. No live OTLP collector or pager delivery was tested.
  • Web console: eight routes passed at 1536/390 px, with additional 320/375/768 checks, search, navigation and detail flows. No page overflow or browser errors. The current frontend production build passes. See ui-design.md for screenshot evidence and the process-local proxy workaround.

Final local acceptance additions

Additional completed local checks in this implementation pass:

  • Saved workspace handoff: real PostgreSQL/API/gateway/Theia to separate Git workspace comparison passed for seven saved files. A missing snapshot refused integration; a newer saved revision invalidated the old accepted input. Evidence: /private/tmp/wishforge-workspace-gate-acceptance.json. The human commits/imports explicitly; no background reset or checkout overwrites the collaborative directory.
  • Language services and extension policy: real TypeScript/Python completions, hovers and diagnostics passed. Installed extensions are limited to 92 pinned builtins and three verified Python extensions. Direct install requests were denied by the backend. Evidence: /private/tmp/wishforge-language-services-ui.json and /private/tmp/wishforge-managed-extensions-ui.json.
  • Private access: actual API signed-task/JIT-probe verification, local HTTP gateway and Next BFF browser launch passed. Anonymous access returned 401, the authorized session redirected to /catalog/start, and the final URL had no code. Build/deployment identity was an explicit fixture, not a deployed OCI workload. Its entities/processes/temporary credentials were removed; audit evidence remains. Reports: /private/tmp/wishforge-private-preview-acceptance.json, /private/tmp/wishforge-private-launch-ui.json and /private/tmp/wishforge-private-preview-cleanup.json.
  • Billing, lifecycle and tenant routing: UI checks covered truthful unknown measurement dimensions, real CSV download, lifecycle controls and a real tenant gateway launch. The read-only IDE idle probe passed; no real Kubernetes workspace was suspended. See billing-entitlements.md and the workspace/gateway runbooks.
  • Control API load: 10 disposable tenants, 20 concurrent streams and 200 actual HTTP submissions produced exactly 20 idempotent runs, all HTTP 202. P95 was 65.88 ms locally. This excludes model execution, IDE concurrency and deployment load; it establishes no production availability/SLO. All fixture tenants were removed. Evidence: /private/tmp/wishforge-api-load-20261017.json.
  • Latest database restore: after the billing migration, exported-snapshot dump/restore plus eight table/security checks passed in 283 ms on a small local fixture. Evidence: .wishforge/backups/restore-fbc2bd0d-b569-414e-a9ff-27f14e27e92c.evidence.json. This is not PITR, five-minute RPO, or a full-platform disaster-recovery result.

Work still required for the complete approved plan

These are implementation or acceptance gaps, not waived requirements:

  • Real Codex/CodeBuddy account execution, WorkBuddy/Codex client OAuth, and the WorkBuddy Cloud export/recovery/cancel contract. CodeBuddy production remains disabled until equivalent scoped credential isolation is implemented/verified.
  • Actual managed and customer Kubernetes/Kata/CNI, customer mTLS enrollment, remote builder/registry egress, production Vault/deployment-secret target checks and migration execution/recovery. Missing secret policy/broker configuration fails closed.
  • Durable cross-process fencing for provisioner side effects still requires implementation and target tests; in-process serialization and Kubernetes CAS do not prove a stale external process can never perform a late mutation.
  • Fully automatic human workspace repository synchronization, durable OCT room reconstruction, and hostile same-workspace terminal isolation with a separate coordinator/debug privilege boundary. A saved snapshot is input provenance, never independent verification evidence.
  • Full paid-pilot operational coverage: real compute/storage usage collectors and priced billing reconciliation, actual cluster workspace idle lifecycle, production tracing/alerts, capacity/failure drills and restoration of all platform assets.
  • Real repeated RSI benchmark/holdout, shadow and 24-hour canary windows. Automatic meta-task generation covers prompt/orchestration bundles and verified tool/platform source proposals. Code rollout/rollback uses signed external action receipts and still requires a real independent build/release controller and end-to-end acceptance. No supplier foundation-model training is claimed.