* ci: whitelist the document-server-backend integration trunk
* feat(storage): DocumentStore HTTP contract + client + reference-server routes (1B PR1) (#975)
Mirrors the runtime line's server-backed delivery for the document store:
a documented /documents HTTP contract (validation/not-found/FUTURE_VERSION
taxonomy; authorization stays a deployment concern via injected hooks with
an any-authenticated default), an HttpDocumentStore that runs the same
validators and migrate-on-read semantics as the browser backend before and
after the wire, and document routes on the shared reference server — one
server, both stores. The backend-agnostic document contract suite now runs
against HTTP → reference server → browser store.
Co-authored-by: Codex <codex@openai.com>
* ci: whitelist the trunk in the storage PG contract workflow too
* feat(storage): PgDocumentStore — Postgres document backend (1B PR2) (#976)
Implements the full DocumentStore contract over document_stages /
document_scenes / document_outlines, reusing the pure adapter split so
every backend shares one aggregate semantics: saveDocument replaces the
aggregate in one transaction under the stage row's FOR UPDATE lock
(vanished scenes deleted, outline row added/removed — omission means
deletion), incremental putStage/putScene/deleteScene take the same
current-version gate as the browser backend, loads read FOR SHARE and
migrate without writing back, listings never touch content, and deletes
cascade by FK. The document contract suite is PG-gated the same way the
runtime one is; local coverage comes from stubbed-queryable unit tests.
Co-authored-by: Codex <codex@openai.com>
* feat(document-store): sealed bootstrap injection seam for the document backend (1B PR3) (#977)
configureDocumentStorage mirrors the runtime seam: module-level bootstrap
injection sealed after first resolution, a factory that receives the app
validators so a server-backed store cannot silently skip them, and
reset-for-tests wiring. A deployment swaps the browser document backend
for HttpDocumentStore without touching app code. Storage README roadmap
updated — the document HTTP/PG backend items are done.
Co-authored-by: Codex <codex@openai.com>
* fix(storage): 1B final-review hardening — error classification, JSON-safety, body limits, parity coverage (#978)
Unversioned legacy incremental writes classify as 400 instead of leaking a
500; document/scene reads assert JSON safety before serialization so a
structured-clone value fails loud (500 NOT_JSON_SAFE naming the path)
instead of silently converting; a shared streaming body reader bounds
request bodies (default 32 MiB, 413 PAYLOAD_TOO_LARGE, option exposed
through every handler factory); the putStage stale/future guards and
corrupt-stamp listing tolerance move into the shared contract suite so all
four backends prove them; both HTTP store errors retain structured details.
Co-authored-by: Codex <codex@openai.com>
* refactor(storage): typed document version/not-found errors replace prose classification (#980)
DocumentVersionError (future | not-current, storedVersion) and
DocumentNotFoundError now carry the classification the server previously
regex-derived from message prose — instanceof matching mirrors the runtime
sibling's RuntimeAppendConflictError pattern, so a wording change can no
longer silently break status mapping. The prose fallback remains, after
the typed checks, solely for third-party DocumentStore implementations.
The HTTP client rematerializes 409s as DocumentVersionError for symmetry,
and http.ts's private validator fields are renamed to match the public
option names all three backends already share.
Co-authored-by: Codex <codex@openai.com>
---------
Co-authored-by: Codex <codex@openai.com>
* ci: run PR checks against the runtime-server-backend integration branch (#939)
* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940)
* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B)
- documented JSON HTTP contract for every RuntimeStore operation
(server-assigned seq, learnerKey derived from auth — never trusted
from the request, machine-readable error codes, idempotency notes)
- HttpRuntimeStore client with injected fetch + auth headers hook;
session reads (including createSession responses) migrate forward
on the runtime line so an older server cannot leak stale envelopes
- conformance test server bridging the contract onto the browser
backend; full runRuntimeStoreContract green over HTTP plus error
mapping cases
* fix(storage): harden HTTP runtime backend after cross-review (#939 Part B)
Cross-review fixes:
- appendRecord gates payloads through assertJsonValue: fail loud on
values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
strings) instead of silently mangling them in transit; payload
domain documented in the contract
- appendRecord/listRecords responses validate via validateRuntimeRecord
(server-assigned seq is no longer trusted verbatim) and listRecords
sorts by seq like listSessions already sorted defensively
- request headers no longer require an ambient Headers constructor
when fetch is injected
- conformance server classifies errors structurally (existence checks,
DSL validators, version stamps) instead of regex-sniffing messages
that interpolate caller-controlled ids
- router preserves empty path segments so empty-key calls keep browser
no-op semantics instead of shifting onto other routes
- stray runtimeDslVersion/seq in request bodies are ignored (store-
assigned wins), matching the browser reference; docs updated
- loopback integration test exercises the real listening server
- docs state the conformance server is test-only; auth-derived
learnerKey is Part D's reference server
* fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B)
- json-value guard rewritten: additionally rejects -0, symbol-keyed and
non-enumerable own properties, and non-index own properties on arrays;
U+2028/U+2029 are accepted again (they round-trip through JSON per
RFC 8259 — rejecting them lost legitimate pasted text)
- conformance server classifies racing duplicate creates as 409 via a
structural post-check, gates payloads and merge learner keys as 400
- segment() rejects '.'/'..' ids the URL layer would fold away
- list responses must be arrays and mergeLearner's moved must be a
finite number, else a typed MALFORMED_RESPONSE error
* fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B)
- json-value guard: NUL is rejected in object keys too, and unpaired
UTF-16 surrogates are rejected in string values and keys (jsonb
refuses both; other JSON stacks corrupt lone surrogates to U+FFFD)
- typed storage errors survive non-object 200 responses instead of
degrading into TypeErrors on .id access
- mergeLearner's moved must be a non-negative integer
- conformance server classifies missing/non-object request bodies as
400 VALIDATION_FAILED
- createSession/appendRecord gate the full init envelope, not only the
payload (stray Date/undefined properties, NUL in ids)
* fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B)
- envelope JSON gate tolerates explicitly-undefined optional anchors
(sceneId: undefined behaves like omission, matching the browser)
- body-carried identifiers reject '.'/'..' so nothing persists that
the URL layer cannot later address; conformance server mirrors it
- mergeLearner keys pass the JSON-domain gate on client and server
- json-value guard v4: rejects enumerable accessors (validation/serialize
TOCTOU), Array subclasses and null-proto arrays, prototype-supplied
array indices; sparse-array test now constructs a genuine hole
* fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B)
- undefined-stripping is limited to the DSL-declared optional anchors
(sceneId/actionIndex/subAnchor); any other undefined member fails the
JSON gate loud instead of being dropped silently
- json-value guard: array and object prototype checks are realm-agnostic
(chain-shape instead of identity), so ordinary values from another
realm are accepted while subclasses stay rejected; shared
isLosslessJsonString predicate exported for SQL key guards
- conformance server merge route validates target-key addressability
* fix(storage): close the toJSON prototype channel in the JSON guard (#939)
toJSON is the single channel through which a prototype can alter JSON
output — prototype properties never serialize and own accessors are
already rejected — so refusing any value with a callable toJSON closes
prototype influence on serialization entirely, including prototypes
crafted to pass the realm-agnostic chain-shape check.
* fix(storage): probe toJSON via descriptor walk, not property read (#939)
Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.
* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)
The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.
* feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941)
* feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C)
- PgRuntimeStore implements RuntimeStore over a minimal injected
Queryable (node-postgres and PGlite both satisfy it) — the package
keeps zero runtime dependencies beyond @openmaic/dsl
- runtime_sessions / runtime_records schema exported as
RUNTIME_PG_SCHEMA with idempotent ensureSchema()
- appends serialize per session via a session-row lock; MAX(seq)+1
and the insert share one transaction, UNIQUE(session_id, seq) plus
bounded retry backstop non-cooperating writers
- envelope semantics mirror the browser backend: version stamping,
validation gates, migrate-on-read, fail-loud on future-stamped rows
- full runRuntimeStoreContract green on PGlite plus PG-specific cases
(concurrent-append seq atomicity, ensureSchema and mergeLearner
idempotence)
* fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C)
Cross-review fixes:
- withTransaction is now required; the BEGIN/COMMIT fallback is removed
(on a pg Pool it spread BEGIN/body/COMMIT across different connections
— no real transaction, leaked idle-in-transaction clients, broken
mergeLearner atomicity; on a shared pinned client concurrent calls
interleaved transactions)
- single-statement deletes no longer wrap in a transaction hook call
- appendRecord gates payloads through assertJsonValue: fail loud on
values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
strings) instead of silently persisting something different
- append retry also covers 40001/40P01; READ COMMITTED assumption
documented at the retry loop
- ensureSchema documented as create-only; redundant stage index dropped
- deterministic interleaving test proves the 23505 retry path; real
PostgreSQL contract lane added (postgres:16 service workflow, pg
driver suite skipped locally without PG_CONTRACT_URL)
* fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C)
- json-value guard rewritten (shared with the HTTP backend): additionally
rejects -0, symbol-keyed and non-enumerable own properties, and
non-index own properties on arrays; accepts U+2028/U+2029
- storage-pg-contract workflow now triggers for the runtime-server-backend
integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1)
when PG_CONTRACT_URL is missing instead of silently skipping
- loadSession distinguishes corrupt non-object rows from absent rows so
getSession fails loud instead of reporting the session missing
* fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C)
- json-value guard (shared): NUL rejected in object keys, unpaired
UTF-16 surrogates rejected in values and keys
- createSession/appendRecord gate the full persisted envelope through
assertJsonValue, not only the payload — stray Date/undefined
properties and NUL in ids fail loud instead of silently diverging
from the returned record or leaking raw 22P05
- real-PG lane covers a genuine 23505 conflict from a second
connection and recovery after an aborted transaction
- retry-set asymmetry and mergeLearner's unbounded lock set documented
* fix(storage): round-4 convergence fixes for the PG backend (#939 Part C)
- NUL/lone-surrogate lookup and delete keys resolve to absent/no-op
instead of leaking 22021 driver errors; setSessionStatus reports the
session missing; mergeLearner from-key moves 0
- mergeLearner destination key passes the JSON-domain gate fail-loud
- envelope JSON gate tolerates explicitly-undefined optional anchors,
matching the browser backend
- json-value guard v4 (shared) + tests for accessors, Array subclasses,
prototype-supplied indices
* fix(storage): round-5 convergence fixes for the PG backend (#939 Part C)
- appendRecord pre-checks the session key like every other lookup path,
so a NUL/lone-surrogate sessionId reports 'no session' instead of
leaking a 22021 driver error
- the queryable-key predicate now structurally reuses the shared
isLosslessJsonString export instead of restating the string rule
- undefined-stripping limited to the DSL-declared optional anchors
- json-value guard: realm-agnostic prototype checks (shared)
* fix(storage): close the toJSON prototype channel in the JSON guard (#939)
Shared guard change with the HTTP backend branch; adds the crafted
null-proto-prototype regression test. Both final-gate reviewers
independently converged on this same channel.
* fix(storage): probe toJSON via descriptor walk, not property read (#939)
Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.
* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)
The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.
* fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943)
The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore
contract while the HTTP and Postgres backends were developed in
parallel against the pre-#926 interface. Implements the method on both
backends (single-statement wipe on PG via the FK cascade; DELETE
/runtime on the HTTP contract, documented as an operator-gated
administrative endpoint), restoring a green typecheck and contract
suite on the integration branch.
* feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944)
* wip(runtime): backend injection seam — pending gate verification
* fix(runtime): cross-review hardening for the storage injection seam
- stage-deletion IndexedDB probe applies only to the default browser
backend; an injected store always receives deleteStageRuntime
- explicit kv argument takes precedence over the configured learner-key
provider, matching the store seam's explicit-beats-global rule
- configured learner-key resolution is latched with in-flight dedup,
mirroring the store singleton; identity changes require app-level
handling
- factory retry semantics documented; seal errors explain the
module-level bootstrap requirement; isRuntimeStorageConfigured()
probe and a test-only reset added
- client-bootstrap-only contract documented (SSR/HMR caveats)
* fix(runtime): snapshot configuration and make the test reset complete
- configureRuntimeStorage copies the option fields so mutating the
caller's object after configuring cannot swap the sealed backend or
identity provider
- resetRuntimeStorageForTests now clears every latched consumer cache
(store singleton, learner-key in-flight promise) via a reset-hook
registry, so a reset-then-reconfigure test actually gets the new
backend
* fix(runtime): reset also clears the default learner-key caches
resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so
a default-path test could leak its anonymous key or KV store into the
next test despite the documented full-reset promise.
* feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945)
* wip(storage): reference server — pending deleteAllRuntime route + gates
* feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D)
- createRuntimeHttpHandler(store, options) wires the documented HTTP
contract onto any injected RuntimeStore over node http
- authenticate is required; every learner-scoped operation verifies the
path/body learnerKey against the authenticated principal (403
FORBIDDEN_LEARNER) — the client-supplied value is never trusted
- mergeLearner requires an explicit authorizeMerge grant and admin
surfaces (stage cascade, DELETE /runtime) require authorizeAdmin;
both default-deny
- runnable reference entry demonstrates a pg Pool withTransaction and
bearer-token authentication, marked as demo-only
- contract suite green through HttpRuntimeStore -> listening reference
handler -> PgRuntimeStore(pglite); security matrix tested (401/403
paths, admin default-deny); threat model documented
* fix(storage): cross-review hardening for the reference server (#939 Part D)
- principal learnerKey is optional: admin/merge-only credentials no
longer fabricate learner identity; learner-scoped routes 403 without
ownership
- full session/record envelopes pass the JSON-domain gate at the
handler, so NUL/lone-surrogate identifiers map to 400 instead of 500
- payload validation follows the injected store's validator map
(options.payloadValidators) instead of imposing DSL defaults
- reference factory accepts authenticate/authorizeMerge/authorizeAdmin/
payloadValidators overrides; docs no longer claim the factory binds
to localhost; demo-impersonation warning hardened
- 500 responses carry a generic message; details go to the server log
- ownership checks precede version checks and unowned sessions read as
404, closing existence/version oracles; cross-learner denial matrix
tested per route
- merge/delete concurrency documented as linearizable-equivalent with
the ownership re-check narrowed to the delete call
* fix(storage): align reference-server semantics with the store contract
- future-stamped sessions read and delete through unchanged; 409
FUTURE_VERSION applies only to guarded writes (status, append, merge)
- check-then-write races reclassify structurally via a post-failure
re-fetch: missing session 404, non-active session 400, never a
message-sniffed or generic 500
* fix(storage): close the reference CLI's pg pool on startup failure
ensureSchema opens connections inside createReferenceRuntimeServer, so
a failed schema init or occupied port left the pool holding database
resources until its idle timeout.
* fix(storage): close the records-route existence oracle
cosarah's review point on #946: the records list answered 200 [] for an
absent session but 404 for another learner's, so the 404 leaked that an
id exists. Absent and foreign sessions now answer identically (404
SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty
list, preserving the store contract's absent-lists-as-empty semantics.
Contract doc records the server MAY/SHOULD and the client MUST.