* feat(storage): KV HTTP contract and clients (#1000 Part A) (#1002)
* feat(storage): KV + Asset HTTP contracts and clients
Adds the client half of the server backends for the two components in the
storage RFC that still had a browser backend only.
Asset contract: the ref is `sha256-<hex>` over the bytes, computed by one
shared rule both backends call, and it *is* the address bytes are written
to (`PUT /assets/{ref}`). The server re-hashes what it received and refuses
a mismatch, so content addressing is enforced end to end rather than
assumed, and uploads become idempotent. `resolve` returns a URL per call,
either an absolute signed URL passed through untouched or a root-relative
proxied path joined to the base origin — the document never stores a raw
URL either way.
KV contract: account scope only, with no scope path segment, query
parameter or body field anywhere on the wire. `HttpAccountKV` is the only
object that can reach it and none of its methods takes a scope, so a
device value is not a request the transport can express; `HttpKVStore`
routes `device` to a local backend it requires at construction. Tests
assert the invariant both ways — `@ts-expect-error` on every scope-passing
call (unused directives fail the build if a scope is ever added) and a
fetch spy proving device traffic does not exist.
Both clients pass the existing `runKVStoreContract` and
`runStorageProviderContract` suites against an in-memory conformance
server, the asset one in both the proxied and signed URL shapes.
* fix(storage): close the review findings on the KV + asset HTTP clients
Cross-review found the device-scope guarantee resting on its weakest layer,
and several places where the contract promised more than the code or the
conformance server could deliver.
deviceStore: the two type-level layers only documented intent — the barrier
that actually kept device values off the wire was the injected store, typed
as `KVStore`, which `HttpAccountKV` satisfies structurally (its methods are
the same methods minus an optional parameter). Injecting the transport as
the device backend type-checked and put device values on the network. The
device scope now requires a branded `LocalKVStore`, the transport declares
itself not to be one, and a runtime check refuses what a cast smuggles past.
resolve URLs: a media `src` carries cookies, never the client's headers
hook, so the proxied shape is only sound for cookie-authenticated
deployments — the contract now says so instead of claiming the content route
sits behind "the same authentication", and the conformance reader uses a
cookie rather than a header no browser could send. The client also validates
what it resolves: http(s) schemes only, and no `//host` or `/\host` slipping
through the relative branch to another origin.
Identifiers: refs and keys arrive percent-decoded, so `..%2F..%2F` was a
traversal waiting for the first backend to join it to a path. Both are now
constrained on both sides, and the contract states the MUST that keeps a
storage layer from being the first to look.
Also: the JSON gate runs before anything reads the value (a `JSON.stringify`
pre-flight ran caller code first and reclassified `{ toJSON: () => undefined }`
as a delete); scope routing fails closed; an upload's Content-Type can no
longer be relabelled by the headers hook; a mutation invalidates in-flight
resolution; unparseable 2xx bodies become typed errors rather than native
SyntaxErrors; signed tokens are validated rather than merely present; and the
contracts settle the per-principal claim model, the media-type allowlist, the
literal prefix rule, and the identifier charset that Part B would otherwise
have had to guess at.
* fix(storage): close the second review round on the KV + asset HTTP clients
The device-scope guard was in one place and needed to be in three. The
transport is exported and structurally a `KVStore`, so `kvPersistStorage(new
HttpAccountKV(...), 'device')` and a plain `KVStore`-typed reference both
compiled without a cast, dropped the scope argument on the floor, and put a
device value on the wire. The transport now takes the scope it was pretending
not to have — typed as `'account'`, refused at runtime for anything else, since
bivariant method parameters mean the type alone cannot hold — and the persist
adapter requires a `LocalKVStore` when paired with `'device'`.
The resolved-URL checks were text tests where only parsing works. The URL
parser strips ASCII tab, LF and CR before parsing, so `/<tab>/evil/x` walked
past the `//` prefix check and loaded cross-origin — in the app-mounted shape,
where a path-only base URL leaves that check as the only guard. The absolute
branch let through `https://` with no host and `https:host/x`, which names one
place standalone and another when a browser resolves it against a same-scheme
page. Both branches are now decided by the parser: control characters refused
outright, relative references validated against a probe origin, absolute ones
required to parse as `scheme://host` with an http(s) scheme — and still
returned byte-for-byte, so signatures survive.
The media-type MUST had no implementation and contradicted itself: the
conformance server stored and served `text/html` verbatim, and a metadata-less
upload became `application/octet-stream`, which cannot be on a renderable
allowlist. The server now implements the allowlist, serves everything else as
an opaque attachment, and the contract scopes its "usable as an img src"
promise to allowlisted types.
The KV key domain was HTTP-only, which made "the same suite proves the backends
equivalent" untrue: `a/b`, an empty key and a 600-byte key all passed in the
browser and would have become unreachable on a server. An audit of the app's
keys (learner key, storage generation, current-scene and migration markers,
persist names) found none in violation, so the rules are lifted to the
primitive and both backends enforce them, with the shared suite asserting it.
Also: the claim model the last round wrote into the contract is now implemented
and covered case by case; malformed percent-escapes are a 400 rather than a
500; the in-flight identity guard has a test that can fail; and the Content-Type
conflict raises a typed error.
* fix(storage): final review round — backend parity, URL probe, credentials
Four small things the final pass found, plus the contract patches Part B would
otherwise inherit.
The `toJSON` delete semantics were documented backwards. `BrowserKVStore.set`
was the trial-serialize the KV contract said no backend does, so
`{ toJSON: () => undefined }` was a silent delete there and a rejection over
HTTP — opposite answers to the same call. The browser backend now validates
scope and key before anything can run caller code, recognizes the delete case by
type, and refuses a value that still fails to serialize. Both cases are in the
shared suite, so the two backends cannot drift apart again.
The probe-origin check could not vouch for a URL naming the probe host itself:
`//asset-url-probe.invalid/x` resolves onto the probe origin and passed, then
resolves somewhere else entirely against a real base. Any authority-bearing
form is refused outright, so the check no longer depends on the probe host
being unguessable.
Cross-origin cookie deployments had no way to send a cookie: `fetch` omits
them cross-origin by default and the headers hook cannot help, since `Cookie`
is a forbidden header name. Both clients accept `credentials` and pass it to
`fetch`; the contract records the CORS obligations that come with it. Rather
than restricting cookie deployments to same-origin, which the asset contract
explicitly does not require.
The conformance server's dot-segment check was reading a path the URL parser
had already normalized, so `/assets/%2e%2e/x` arrived as `/x` and the documented
400 was really post-routing behaviour. It now validates the raw request target,
and the injected transport preserves it instead of handing over a normalized one.
Contract: the prefix rule names backslash (PostgreSQL's default LIKE escape,
and the one an implementer skips because a legal key cannot contain it, though
a prefix can), and acquiring a claim MUST leave the bytes present — a
de-duplicating server that inserts only the claim can race reclamation of the
last previous claim and leave a document resolving to a 404.
Also: base URLs are checked for an http(s) scheme at construction, the ref
domain moved next to the KV key domain and is enforced by both asset backends
with shared suite coverage, the KV client refuses a hook-supplied Content-Type
like the asset one, and CI runs the package typecheck that makes the
`@ts-expect-error` guards mean something.
* fix(storage): probe guard spelling, prefix validator, base url shape
Three one-line-level holes from the closing review.
The probe-host guard was spelled `startsWith('//')`, so the backslash forms
`/\host/x` and `/\/host/x` reached the origin comparison, passed it by naming
the probe host, and were stopped only by that host failing to resolve. It now
refuses any second character that can open an authority — which is also what
makes the comment true rather than aspirational: a path-absolute reference
always inherits the base's origin, so with no authority expressible the
comparison cannot be satisfied cross-origin, whatever the probe host is.
The conformance server validated `?prefix` with the path-segment validator,
rejecting `keys('.')` and `keys('..')` — legitimate prefixes of keys like
`.hidden`, which the client rightly allows. A prefix travels in the query
string and is never a path segment, so it now has its own validator: the key
rules minus the non-empty and dot-segment requirements. The contract says the
same, and the shared suite pins both backends to it.
Base URL validation moved into one shared helper both clients call, since two
copies of a security rule is how they drift. It now also refuses
`https:host/path` — which parses standalone to that host but is resolved
against the current document by `fetch`, sending requests and their credentials
somewhere the operator never wrote — and a base carrying a query or fragment,
which silently swallows the path concatenated onto it.
* fix(storage): close two holes in the shared base-url check
The path-only branch returned its input unexamined, so `//evil.example` and
`/\evil.example` were accepted as paths. A browser reads both as an authority,
which means every request built on such a base — and every credential the
headers hook attaches — leaves the origin. A base is now required to be a
single `/` followed by something that is not `/` or `\`, the same refusal the
resolved-URL check already makes, for the same reason. Control characters are
refused first: the parser strips or re-encodes them, and `/<tab>/evil.example`
becomes `//evil.example` after it does.
The query/fragment check read the parsed `search` and `hash`, which are empty
strings for a base ending in a bare `?` or `#` — so exactly the inputs that
swallow the concatenated path into a query or fragment were the ones that got
through. It reads the raw string now, which also makes the post-parse check
redundant rather than merely unreachable.
* fix(storage): bypass HTTP caches on reads, and require a 404 for not-found
Two mechanisms the clients were missing, on both sides.
Reads are marked `cache: 'no-store'`. The asset resolve request answers two
mutable questions — does this principal hold a claim, and which URL is valid
now — so a cached answer returns an expired signed URL or replays a negatively
cached 404 for an asset written since; the in-flight coalescing that would
otherwise cover this lives in the client and cannot reach the HTTP cache layer.
KV reads have the same problem for a different reason: `account` is by
definition the scope another device may have just written, and nothing here can
invalidate a cached copy when it does. The contracts now require `no-store` on
those responses. Asset content is left cacheable — bytes at a content-addressed
ref never change — and writes are unmarked, having nothing to read from a cache.
The not-found mappings now require the status as well as the code. A proxy or
gateway answering 401, 403 or 500 while echoing the body's error code was being
read as "this does not exist", and `null` is indistinguishable from a real
miss — so an outage reached the caller as data loss.
* fix(storage): reject unusable base urls, and make the harness meet the contract
Two configuration checks and two harness gaps.
A base URL carrying userinfo built a client that threw a native TypeError on
every operation, because Fetch refuses to construct a request from a URL with
embedded credentials — one legible error where the deployment is configured
beats a per-call failure with no explanation. A base with surrounding
whitespace was worse than useless: `new URL` strips it, so validation passed,
and then concatenation moved the space inside the URL where it is no longer
trailing and no longer stripped, breaking every request.
The conformance server now sends `Cache-Control: no-store` on the three reads
the contracts require it on. The requirement landed last round with nothing
asserting it, so the suite would have passed against exactly the implementation
it exists to catch. And `DELETE /kv/entries/{key}` refuses a request body
outright: the query and the header were already checked for a smuggled scope,
leaving the body as the one channel through which a delete could still describe
one.
* fix(storage): allow a device-routing composite to persist device-scoped state
cosarah found kvPersistStorage(httpKVStore, "device") wrongly rejected. An
HttpKVStore is a full dual-scope composite: it routes device operations to the
LocalKVStore it requires at construction and only reaches the network for
account, so device values never leave the machine. The guard tested
isLocalKVStore — "is the whole store local" — and so turned away a store that
is safe for device precisely because it is not fully local.
The guard now turns on the capability that actually matters: does the device
scope stay local. A new DeviceSafeKVStore brand (servesDeviceScopeLocally: true)
marks a store whose device writes never leave the device — held by a fully-local
LocalKVStore (which extends it) and by HttpKVStore, and declared false by
HttpAccountKV, the pure account transport with no local device backend. The
persist device overload requires DeviceSafeKVStore; the account overload is
unchanged.
The security boundary the red team opened this whole line on is untouched:
HttpAccountKV used for device is still doubly refused — its false brand is not
assignable to the true DeviceSafeKVStore requires (type), and the runtime check
rejects the cast that would erase it. The brand is structurally unforgeable, not
a flag the transport could set; and the injected device backend of a composite
still demands the stronger, fully-local LocalKVStore, so composites cannot nest
into a router loop with no local floor.
Regression test (cosarah's ask): kvPersistStorage(httpKVStore, "device")
set/get/remove reach the injected device backend, land device-scoped there, and
make zero HTTP requests. Mutation-checked: widening the brand so HttpAccountKV
is accepted, or neutering the guard, turns the device-out reject test red.
* refactor(storage): scope #1002 down to KV — remove the asset HTTP backend
Per the maintainer decision, the AssetProvider server design moves to the
global-resource-pool RFC (#1007), where AssetRef changes from content-addressed
to an allocated global id and the contract model itself changes. Continuing to
harden the content-addressed asset client here would be work on code that is
being replaced, so the asset half of #1002 is withdrawn and the PR is pure KV.
Removed (all introduced by #1002):
- docs/asset-http-contract.md
- src/asset/http.ts (HttpAssetProvider) and its index / package.json exports
- src/asset/content-ref.ts (assertAssetRef / computeAssetRef / the ref shape)
- test/http-asset-provider.test.ts and the asset half of the conformance server
Reverted to the #858 baseline, untouched:
- src/asset/browser.ts — back to opaque resolve/remove, no assertAssetRef guard.
This directly removes cosarah's latest asset finding: an opaque ref such as
`bucket/path` no longer throws, restoring the DSL AssetRef-opaque contract.
- test/asset-contract.ts, test/asset-browser.test.ts — 0-diff from origin/main.
Kept (all KV): kv-http-contract.md, src/kv/* (HttpKVStore / HttpAccountKV /
the DeviceSafeKVStore brand), the KV tests, and the KV conformance server —
renamed kv-conformance-server.ts and reduced to KV routing over the shared
harness framework (body caps, no-store, scope-channel and dot-segment guards,
auth hooks). base-url.ts stays; KV still uses it.
* fix(storage): treat KV keys as opaque; document account-sync prerequisites
cosarah found the key validation we added for traversal defense breaks
legitimate callers: DSL identifiers (stageId/sceneId/learnerKey) are
unconstrained strings — verified, the DSL only requires typeof === 'string' —
and app callers compose keys like `editor-current-scene:<stageId>`, so a valid
stage id `stage/one` now throws in BrowserKVStore before localStorage.
Fix, matching the asset-opaque precedent: a KV key is an opaque string again.
The `/` and `\` rejection is removed from assertKVKey / assertKVKeyPrefix and
from the conformance server. Traversal is defended by encoding, not rejection —
the HTTP client already percent-encodes the whole key into one path segment, so
a separator never opens a new segment on the wire, and the server stores the
decoded key as a plain map key (bound value), never a path. `a%2F..%2F..%2Fb`
decodes to the single opaque key `a/../../b`, which addresses one entry and
traverses nothing; a raw unencoded `..` path produces extra segments that match
no route (404). The rules that remain are only what encoding cannot cover and
the transport cannot carry (empty, whole-key `.`/`..`, NUL, lone surrogate,
length) — none producible by `prefix + id`.
Tests: the shared suite now round-trips keys with `/`, `\`, `%`, `:`, spaces
across both backends; a `stage/one` key round-trips through the real conformance
server; the traversal input is proven to store as one opaque key and the raw
path to 404. Mutation-checked: re-adding the `/` rejection reddens exactly the
round-trip cases.
Docs (kv-http-contract.md):
- "A key is an opaque string" replaces the identifier-shape section — keys may
contain separators; traversal is an encoding + server-bound-value duty.
- Account settings sync now documents its security prerequisite next to the
account scope: a deployment MUST provide real per-user isolation of stored
rows and encryption at rest before enabling sync; the reference server is not
that posture.
- kvPersistStorage(device) doc corrected from LocalKVStore to DeviceSafeKVStore
(the device-safe brand accepts the HttpKVStore composite; HttpAccountKV stays
double-rejected).
* fix(storage): close the scope/key/prefix contract gaps with a coverage matrix
Three completeness gaps, nailed by an executable matrix so an untested
channel/shape can't slip through the next pass rather than patched one by one.
1. Scope via a GET body was not rejected. `assertNoScopeChannel` covered the
query and header, but the bodyless GET routes never read their body, so
`GET /kv/keys` with `{"scope":"device"}` returned 200 []. Bodyless methods
(GET, DELETE) now reject any request body up front — that is the channel a
scope would hide in — and only `set` reads a body, checking it for a scope
field. The DELETE route's own body check folds into the shared one.
2. The 512-byte key bound wrongly rejected legitimate keys. A DSL id is an
unconstrained string (verified: the DSL requires only typeof === 'string'),
and `editor-current-scene:` + a 500-char stage id is 521 bytes, which
BrowserKVStore rejected before localStorage. Keys are length-unconstrained;
the bound is raised to a generous 8 KiB DoS ceiling and documented as a
resource guard, not part of the key domain — not a derived/hashed key, which
would break the literal-prefix semantics of keys(prefix). Both backends and
the conformance server move in lockstep.
3. The prefix contract text contradicted itself: it claimed a legal key can
never contain a backslash, while the key section (correctly) makes keys
opaque, backslash included. Reworded — a prefix is opaque like a key — and
the shared literal-prefix matrix now covers `\` (PostgreSQL's LIKE escape),
which it had skipped, alongside `%`, `_`, and combinations.
Coverage matrix (CI-executable):
- scope-rejection: {get,set,remove,keys} × {query,header,body}. query/header
run in-process; the body cells use a raw HTTP client (fetch cannot put a body
on a GET — exactly the non-conforming client the finding described) against a
loopback listener, skipped where the sandbox forbids binding.
- key-validity: `/ \ % _ : .` space, BMP + astral Unicode, `%2F`, a 500-char
composed key, and all-at-once — each round-trips (set/get, keys(prefix)
match, remove) on both browser and HTTP backends via the shared suite.
- prefix-charset: `%`, `_`, `\`, and combinations match literally, both backends.
- Boundaries still rejected: whole-key `.`/`..`, NUL, lone surrogate, empty,
and the DoS ceiling.
Mutation-checked: removing the bodyless-body rejection reddens the three GET/
DELETE body cells; lowering the ceiling back to 512 reddens the long-key
round-trip on both backends.
Bumps @openmaic/storage to 0.1.2 (src/kv/* changed).
* fix(storage): bound the KV key DoS ceiling by encoded size, not UTF-8 bytes
Follow-up to the 8 KiB ceiling: measuring it in UTF-8 bytes ignored
percent-encoding expansion and reopened a backend-parity break. A non-ASCII key
is small in bytes but large on the wire — `€` is 3 UTF-8 bytes and `%E2%82%AC`,
9, encoded — so `'€'.repeat(2730)` (8190 bytes, under the old cap) encoded to
~24.6 KB, which the Node HTTP server rejects with 431 before routing while
`BrowserKVStore`, which never builds a URL, accepted it.
The ceiling is now measured on `encodeURIComponent(key).length` — the size the
key occupies as a request target — at 4096, far above any real id (a 500-char id
under `editor-current-scene:` encodes to ~523) and well below Node's default
16 KiB `maxHeaderSize` (empirically the 431 trigger on Node 22), so no key that
passes validation can reach a 431. Both backends and the conformance server use
the same encoded-size limit, so a key is accepted iff its encoded form fits,
whichever backend it reaches — parity restored. `MAX_KV_KEY_BYTES` becomes
`MAX_KV_KEY_ENCODED_LENGTH`.
Tests, against a real node:http server for the 431 boundary:
- the max-legal non-ASCII key (455×€, encoded 4095) round-trips over real HTTP
with no 431, and the browser accepts the same key — parity on accept;
- one code point over (456×€, encoded 4104) is rejected by both backends at
validation, before any request — the exact parity case a byte bound missed
(1368 bytes, would have passed, then 431'd over HTTP);
- a raw client bypassing client validation with an over-ceiling (under 16 KiB)
encoded key gets a clean 400, not a 431.
Shared suite's key-domain matrix now bounds by encoded size on both backends.
Mutation-checked: reverting the ceiling to UTF-8 bytes reddens the three
encoded-length parity tests. Bumps @openmaic/storage to 0.1.3 (src/kv/* changed).
* fix(storage): make the KV key domain truly opaque; keep transport limits out
The recurring key-limit findings (512 → 8 KiB → 4096, each patch exposing the
next) shared one root cause: transport-driven rules kept being added to the key
domain — the browser primitive included — breaking the "opaque unconstrained
string key" contract. This is the structural fix, per cosarah: "keep HTTP
transport limits out of the browser primitive and preserve the existing string
key domain."
Approach (a), the minimal one: a KV key is any string, validated nowhere. The
browser primitive is a Map/Storage that stores any key; the HTTP backend carries
the key as a URL path segment and inherits that transport's limits, documented
as HTTP-deployment concerns that never constrain the key domain. Every real key
(a `prefix:id` from an unconstrained DSL string) round-trips on both backends;
only pathological keys hit a transport limit, and none of those are keys a caller
produces. (b) — keys in the request body for accept-parity — was unnecessary:
verified that no real key needs it.
Removed from the key domain (browser + HTTP + conformance server):
- the encoded-size / length ceiling entirely — there is no key-length limit;
- the NUL exclusion — NUL percent-encodes to %00, is preserved end to end, and
is a legitimate Map/Storage key (a false "cannot represent" premise);
- the `.`/`..`/empty/`/`/`\` rejections — all opaque, all round-trip.
`assertKVKey` / `assertKVKeyPrefix` / `MAX_KV_KEY_*` are gone.
Kept, at the transport layer only (HttpAccountKV), never reaching the primitive:
- an unpaired UTF-16 surrogate genuinely has no percent-encoding, so the HTTP
client surfaces a clear transport error (KEY_NOT_ENCODABLE) while the browser
stores the same key — verified: encodeURIComponent throws on it.
Scope completeness (P2): the "reject scope in every channel" guarantee had two
holes — a scope path segment (`/kv/device/keys`) 404'd instead of 400, and only
`x-scope` was checked among header spellings. The server now rejects a scope
path segment and the whole prohibited header set (scope, x-scope, kv-scope,
x-kv-scope). The coverage matrix is {get,set,remove,keys} × {path, query, each
header spelling, body}.
Coverage matrix reworked to verify arbitrary opaque keys (empty, NUL, control
chars, separators, long, Unicode) round-trip on both backends — not to verify a
limit. A real node:http test proves NUL and long keys round-trip end to end, and
that a lone surrogate is a transport limit (browser accepts, HTTP client errors).
Mutation-checked: re-adding a NUL rejection, removing the path-segment check, or
dropping a header spelling each reddens the matching cells.
Cleanup: package.json description updated (KV HTTP delivered, not "later");
network-server bind/skip/close lifecycle deduped into one helper. Bumps
@openmaic/storage to 0.1.4 (src/kv/* changed).
* fix(storage): reject whole-key "."/".." in the HTTP client — they alias entries
Follow-up to making the key domain opaque: removing the `.`/`..` key-domain
rejection was right (the browser stores them fine), but the HTTP client then let
them reach `fetch`, where URL path normalization silently rewrites the target
before it is sent. `encodeURIComponent('.')` is `.`, so `/kv/entries/.` collapses
to `/kv/entries/` (the empty key) and `/kv/entries/..` walks up a level — so
`get`/`set`/`remove` on a whole-key `.` or `..` silently read, overwrite, or
delete a *different* entry, with no error. A data-correctness bug.
Fix, same pattern as the unpaired-surrogate limit already handled: a whole-key
`.`/`..` is something the URL-path transport structurally cannot carry, so the
client's path-segment encoder refuses it with `KEY_NOT_ENCODABLE` before a
request is built. Split the encoder in two: `encodeKeyPathSegment` (key → path
segment, rejects whole-key `.`/`..`) and `encodeComponent` (the `keys()` prefix →
query value, where `.`/`..` are legitimate and untouched). A key that merely
*contains* a dot (`a.b`, `prefix:id`) is unaffected and round-trips. The browser
primitive still stores `.`/`..` opaquely.
The HTTP transport's limits are now complete — three cases, all fail-loud at the
transport layer, all storable on the browser: (1) an unpaired surrogate (no
percent-encoding), (2) a whole-key `.`/`..` (URL normalization aliases it), and
(3) an over-long key (deployment request-target ceiling). The contract lists all
three as HTTP-deployment limits, not key-domain constraints.
Tests: whole-key `.`/`..` — browser stores and round-trips (distinct from the
empty key); HTTP client throws `KEY_NOT_ENCODABLE` and builds no request; and a
real-server test asserts the empty-key entry is never read, overwritten, or
deleted by a `.`/`..` operation. A dot-containing key (`a.b`) round-trips.
Mutation-checked: dropping the whole-key check reddens the alias tests.
Bumps @openmaic/storage to 0.1.5 (src/kv/* changed).
* fix(storage): reconcile the settings/profile persist consumer with the KV API
The integration merge (#1002 KV into main) surfaced two things the merge itself
could not resolve mechanically:
- `lib/store/kv-persist.ts` (added on main by #1001) called
`kvPersistStorage(kv, scope)` with a *variable* scope, which #1002's overloads
deliberately reject — `'device'` must be paired with a device-safe store, so
the overloads take the scope as a literal. Branch on the scope: `'account'`
takes any `KVStore`; `'device'` narrows the backend to `DeviceSafeKVStore` via
its brand (the adapter re-checks at runtime) and otherwise fails loud. The app
wires only `'account'` today; the `'device'` arm keeps the generic helper
honest. `ControllableKV` in the persist test wraps a local `BrowserKVStore`,
so it is genuinely device-safe and now declares the brand.
- `packages/@openmaic/storage` version bumped 0.1.5 → 0.1.6: the merge changed
storage inputs (README + a pg-schema test carried from main) relative to the
previous integration tip, so the version-bump guard requires an increase above
#1002's 0.1.5. Still ahead of main's 0.1.1.
Verified: root `tsc --noEmit` clean, the persist test green (40 passed), and the
storage package suite green on Node 22.
* feat(persistence): one-command server-backed stack — embedded API + compose postgres profile
docker compose --profile server-persistence up --build runs exactly two
containers: the app (persistence HTTP server embedded at /api/persistence,
enabled only when DATABASE_URL is set — unset keeps today's browser-only
behavior byte-for-byte) and PostgreSQL 16. Client bootstrap configures both
storage seams against the same-origin route when NEXT_PUBLIC_PERSISTENCE=1,
riding the existing lazy migrations so prior browser data reaches the
server per-course on first open. Dev auth (bearer token + client-asserted
learner header) lives in one overridable module and is loudly documented
as development-grade. Verified end-to-end against the real stack:
document PUT/GET and runtime session create land in Postgres; bad tokens 401.
Co-authored-by: Codex <codex@openai.com>
* fix(persistence): review round — structural bootstrap ordering, retryable init, honest failure surfaces
The bootstrap side-effect import moves into the two storage seam entry
modules, so any module graph that can resolve a store necessarily
evaluates it first — no client-entry ordering luck, and the two configure
calls are all-or-nothing. A failed server init clears its cached promise
(next request retries instead of permanent 500s). listStages rethrows
backend failures and the dashboard surfaces a persistence-unavailable
error instead of rendering an empty list that reads as data loss. Docs
state the build-time nature of NEXT_PUBLIC_PERSISTENCE, the real token
boundary (any visitor can extract it and impersonate any learner —
localhost/trusted-network only), and the correct postgres password
rotation. Dev-token compare is timing-safe; the handler singleton lives
on globalThis so HMR cannot leak pools.
Co-authored-by: Codex <codex@openai.com>
* fix(persistence): review response — no-confidentiality wording + adapter round-trip coverage
The token docs now state plainly that the NEXT_PUBLIC token is compiled
into the public bundle and provides no confidentiality or user isolation
(its only role is keeping unrelated scanners off a trusted network). The
Fetch/Node adapter — the route's most bug-prone code — gains a round-trip
integration test: PUT body streaming, 201 with multi-value headers, 204
empty body, and encoded path segments reaching the handler un-decoded.
---------
Co-authored-by: Codex <codex@openai.com>
* ci: whitelist the document-server-backend integration trunk
* feat(storage): DocumentStore HTTP contract + client + reference-server routes (1B PR1) (#975)
Mirrors the runtime line's server-backed delivery for the document store:
a documented /documents HTTP contract (validation/not-found/FUTURE_VERSION
taxonomy; authorization stays a deployment concern via injected hooks with
an any-authenticated default), an HttpDocumentStore that runs the same
validators and migrate-on-read semantics as the browser backend before and
after the wire, and document routes on the shared reference server — one
server, both stores. The backend-agnostic document contract suite now runs
against HTTP → reference server → browser store.
Co-authored-by: Codex <codex@openai.com>
* ci: whitelist the trunk in the storage PG contract workflow too
* feat(storage): PgDocumentStore — Postgres document backend (1B PR2) (#976)
Implements the full DocumentStore contract over document_stages /
document_scenes / document_outlines, reusing the pure adapter split so
every backend shares one aggregate semantics: saveDocument replaces the
aggregate in one transaction under the stage row's FOR UPDATE lock
(vanished scenes deleted, outline row added/removed — omission means
deletion), incremental putStage/putScene/deleteScene take the same
current-version gate as the browser backend, loads read FOR SHARE and
migrate without writing back, listings never touch content, and deletes
cascade by FK. The document contract suite is PG-gated the same way the
runtime one is; local coverage comes from stubbed-queryable unit tests.
Co-authored-by: Codex <codex@openai.com>
* feat(document-store): sealed bootstrap injection seam for the document backend (1B PR3) (#977)
configureDocumentStorage mirrors the runtime seam: module-level bootstrap
injection sealed after first resolution, a factory that receives the app
validators so a server-backed store cannot silently skip them, and
reset-for-tests wiring. A deployment swaps the browser document backend
for HttpDocumentStore without touching app code. Storage README roadmap
updated — the document HTTP/PG backend items are done.
Co-authored-by: Codex <codex@openai.com>
* fix(storage): 1B final-review hardening — error classification, JSON-safety, body limits, parity coverage (#978)
Unversioned legacy incremental writes classify as 400 instead of leaking a
500; document/scene reads assert JSON safety before serialization so a
structured-clone value fails loud (500 NOT_JSON_SAFE naming the path)
instead of silently converting; a shared streaming body reader bounds
request bodies (default 32 MiB, 413 PAYLOAD_TOO_LARGE, option exposed
through every handler factory); the putStage stale/future guards and
corrupt-stamp listing tolerance move into the shared contract suite so all
four backends prove them; both HTTP store errors retain structured details.
Co-authored-by: Codex <codex@openai.com>
* refactor(storage): typed document version/not-found errors replace prose classification (#980)
DocumentVersionError (future | not-current, storedVersion) and
DocumentNotFoundError now carry the classification the server previously
regex-derived from message prose — instanceof matching mirrors the runtime
sibling's RuntimeAppendConflictError pattern, so a wording change can no
longer silently break status mapping. The prose fallback remains, after
the typed checks, solely for third-party DocumentStore implementations.
The HTTP client rematerializes 409s as DocumentVersionError for symmetry,
and http.ts's private validator fields are renamed to match the public
option names all three backends already share.
Co-authored-by: Codex <codex@openai.com>
---------
Co-authored-by: Codex <codex@openai.com>
* feat(runtime): persist quiz attempts in RuntimeStore
* fix(runtime): coalesce quiz draft snapshots
* fix(runtime): recover concurrent quiz attempts
* fix(runtime): handle quiz completion races
* fix(runtime): commit quiz review atomically
* fix(runtime): dedupe concurrent quiz writes
* fix(runtime): drop stale quiz drafts
* fix: harden quiz runtime review recovery
* fix: serialize quiz attempt identity
* feat: read quiz state from runtime store
* fix: persist quiz retries before resetting
* fix: preserve authoritative quiz outcomes
* fix: preserve legacy quiz retries during cutover
* fix: reconcile legacy quiz snapshots safely
* fix: drain rollover quiz write queues
* fix: drain completed quiz retry queues
* fix: reuse concurrent quiz retries
* fix(quiz): preserve drafts across abrupt reloads
* fix(quiz): recover empty retry sessions
* fix(chat): abort stalled runtime state reads
* fix(quiz): expose queued phases to readers
* fix(quiz): retain concurrent writer tails
* fix(quiz): canonicalize retry branches
* fix(quiz): keep retry rollovers monotonic
* fix(quiz): validate skipped retry siblings
* fix(quiz): close read cutover races
* test(classroom): cover legacy quiz summaries
* fix(quiz): reset async consumers on scene changes
* fix(quiz): close scene transition windows
* test(pbl): cover launch freshness guards
* fix(quiz): close cutover concurrency gaps
* fix(quiz): harden retry and context freshness
* fix(quiz): reject malformed legacy answers
* fix(quiz): validate legacy answer values
* test(quiz): cover legacy multi-answer migration
* fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover
Main's three call sites built static storeState blocks that runAgentLoopFn
never consumed (it always rebuilds fresh state via getStoreState); the quiz
read cutover replaced that callback with the async two-phase RuntimeStore
read, leaving the template field with zero consumers. Drop it.
* feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions
The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError
semantics existed only in the browser backend; server-backed deployments would
silently accept conflicting appends and leave completed sessions active. Forward
the options over the wire, detect conflicts atomically under the PG transaction,
map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error
client-side so quiz retry logic works across every backend.
Co-authored-by: Codex <codex@openai.com>
* fix(chat): Pi single requests build storeState via the async runtime quiz read
Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the
request template straight to /api/chat/pi, which rejects bodies without
storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read
with the scene-transition guard) and call it on the Pi path too.
* ci: whitelist the runtime-data-cutover integration trunk for PR checks
* feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956)
* feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869)
The fourth and last runtime family. Consumed-discussion facts become
append-only 'playback' records folded into a set at read (at-least-once
appends, no conflict machinery); the resume cursor is device-scoped
last-write-wins KV per the amended #779/#869 split. sessionStorage keeps
same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie
playbackState machinery is retired, with a one-time lazy migration of any
legacy row (cursor half + facts half) before deletion, and stage deletion
now clears both the KV cursor and any unmigrated legacy row.
Co-authored-by: Codex <codex@openai.com>
* test(runtime): include playbackState in the stage-delete db mock
---------
Co-authored-by: Codex <codex@openai.com>
* fix(playback): persist discussion facts on every consumption path (#957)
* fix(playback): persist discussion facts on every consumption path (final-review P0+P1s)
- The engine now publishes a progress snapshot the moment a discussion is
consumed (join / skip / unselected-agent auto-skip). onProgress otherwise
fires before the discussion action executes and a discussion is the scene's
last action, so the fact never reached persistence.
- Reads fold records across ALL playback sessions in the learner partition
(mergeLearner deliberately preserves same-kind sessions from both keys).
- Legacy migration appends only not-yet-durable facts, so an interrupted
migration resumes instead of dropping the tail.
- recordConsumedDiscussion reports durability; the component drops failed ids
from its observed set so a later progress tick retries (at-least-once).
* test(e2e): live verification of the playback persistence chain
Seeds a deterministic stage straight into the Dexie DB, starts the lecture
via the canvas overlay, and asserts the full chain: discussion auto-skip
appends a discussionConsumed record to maic-runtime, the device cursor lands
in KV, and both survive a fresh browsing context (empty sessionStorage).
* refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959)
Product ruling on #869's fourth family: playback learner state is front-end
ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined
discussions' content already lives in chat runtime records, and no replay
export / analytics consumer exists — so durable facts bought nothing over
in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the
RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half
with real UX value), the engine's consumption-time progress snapshot (cursor
freshness), and the legacy Dexie retirement (cursor half migrates, row
deletes, consumed ids are dropped).
* fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966)
* fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair)
Review findings on #955: didActiveSceneRemainUnchanged compared the active
scene by object identity, so a store update reallocating the scene during
the async quiz read dropped the learner's graded answers from that turn's
request — the scene id is the real boundary. The records route now
classifies a malformed sessionTransition as a validation failure instead of
letting the store's throw surface as a 500.
* fix(playback): superseded-engine cursor guard + migration write-window recheck
Second-vendor review of the #959 shrink (requested after the cross-review
noted it had single-vendor coverage) found: an engine orphaned by a scene
switch during async lecture resume could pass the idle-only recheck, be
resurrected, and publish its old scene's progress over the new scene's
debounced cursor — the resume continuation now requires identity with the
installed engine, and onProgress drops snapshots from superseded engines.
The legacy cursor migration also rechecks KV immediately before its write
so a concurrent tab's newer cursor cannot be overwritten and orphaned by
the legacy-row delete.
Co-authored-by: Codex <codex@openai.com>
---------
Co-authored-by: Codex <codex@openai.com>
* fix(review): approval follow-up P3 nits (#967)
* release: v0.3.1
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard
- The records route's sessionTransition guard now requires an ISO
updatedAt (isIsoTimestamp), matching the sibling PATCH /status route
- Dead vi.mock factories for the deleted playback-storage module dropped
- A corrupt legacy playback timestamp falls back to 'now' instead of
wedging migration into a permanent re-throw that disabled resume
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* ci: run PR checks against the runtime-server-backend integration branch (#939)
* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940)
* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B)
- documented JSON HTTP contract for every RuntimeStore operation
(server-assigned seq, learnerKey derived from auth — never trusted
from the request, machine-readable error codes, idempotency notes)
- HttpRuntimeStore client with injected fetch + auth headers hook;
session reads (including createSession responses) migrate forward
on the runtime line so an older server cannot leak stale envelopes
- conformance test server bridging the contract onto the browser
backend; full runRuntimeStoreContract green over HTTP plus error
mapping cases
* fix(storage): harden HTTP runtime backend after cross-review (#939 Part B)
Cross-review fixes:
- appendRecord gates payloads through assertJsonValue: fail loud on
values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
strings) instead of silently mangling them in transit; payload
domain documented in the contract
- appendRecord/listRecords responses validate via validateRuntimeRecord
(server-assigned seq is no longer trusted verbatim) and listRecords
sorts by seq like listSessions already sorted defensively
- request headers no longer require an ambient Headers constructor
when fetch is injected
- conformance server classifies errors structurally (existence checks,
DSL validators, version stamps) instead of regex-sniffing messages
that interpolate caller-controlled ids
- router preserves empty path segments so empty-key calls keep browser
no-op semantics instead of shifting onto other routes
- stray runtimeDslVersion/seq in request bodies are ignored (store-
assigned wins), matching the browser reference; docs updated
- loopback integration test exercises the real listening server
- docs state the conformance server is test-only; auth-derived
learnerKey is Part D's reference server
* fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B)
- json-value guard rewritten: additionally rejects -0, symbol-keyed and
non-enumerable own properties, and non-index own properties on arrays;
U+2028/U+2029 are accepted again (they round-trip through JSON per
RFC 8259 — rejecting them lost legitimate pasted text)
- conformance server classifies racing duplicate creates as 409 via a
structural post-check, gates payloads and merge learner keys as 400
- segment() rejects '.'/'..' ids the URL layer would fold away
- list responses must be arrays and mergeLearner's moved must be a
finite number, else a typed MALFORMED_RESPONSE error
* fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B)
- json-value guard: NUL is rejected in object keys too, and unpaired
UTF-16 surrogates are rejected in string values and keys (jsonb
refuses both; other JSON stacks corrupt lone surrogates to U+FFFD)
- typed storage errors survive non-object 200 responses instead of
degrading into TypeErrors on .id access
- mergeLearner's moved must be a non-negative integer
- conformance server classifies missing/non-object request bodies as
400 VALIDATION_FAILED
- createSession/appendRecord gate the full init envelope, not only the
payload (stray Date/undefined properties, NUL in ids)
* fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B)
- envelope JSON gate tolerates explicitly-undefined optional anchors
(sceneId: undefined behaves like omission, matching the browser)
- body-carried identifiers reject '.'/'..' so nothing persists that
the URL layer cannot later address; conformance server mirrors it
- mergeLearner keys pass the JSON-domain gate on client and server
- json-value guard v4: rejects enumerable accessors (validation/serialize
TOCTOU), Array subclasses and null-proto arrays, prototype-supplied
array indices; sparse-array test now constructs a genuine hole
* fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B)
- undefined-stripping is limited to the DSL-declared optional anchors
(sceneId/actionIndex/subAnchor); any other undefined member fails the
JSON gate loud instead of being dropped silently
- json-value guard: array and object prototype checks are realm-agnostic
(chain-shape instead of identity), so ordinary values from another
realm are accepted while subclasses stay rejected; shared
isLosslessJsonString predicate exported for SQL key guards
- conformance server merge route validates target-key addressability
* fix(storage): close the toJSON prototype channel in the JSON guard (#939)
toJSON is the single channel through which a prototype can alter JSON
output — prototype properties never serialize and own accessors are
already rejected — so refusing any value with a callable toJSON closes
prototype influence on serialization entirely, including prototypes
crafted to pass the realm-agnostic chain-shape check.
* fix(storage): probe toJSON via descriptor walk, not property read (#939)
Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.
* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)
The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.
* feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941)
* feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C)
- PgRuntimeStore implements RuntimeStore over a minimal injected
Queryable (node-postgres and PGlite both satisfy it) — the package
keeps zero runtime dependencies beyond @openmaic/dsl
- runtime_sessions / runtime_records schema exported as
RUNTIME_PG_SCHEMA with idempotent ensureSchema()
- appends serialize per session via a session-row lock; MAX(seq)+1
and the insert share one transaction, UNIQUE(session_id, seq) plus
bounded retry backstop non-cooperating writers
- envelope semantics mirror the browser backend: version stamping,
validation gates, migrate-on-read, fail-loud on future-stamped rows
- full runRuntimeStoreContract green on PGlite plus PG-specific cases
(concurrent-append seq atomicity, ensureSchema and mergeLearner
idempotence)
* fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C)
Cross-review fixes:
- withTransaction is now required; the BEGIN/COMMIT fallback is removed
(on a pg Pool it spread BEGIN/body/COMMIT across different connections
— no real transaction, leaked idle-in-transaction clients, broken
mergeLearner atomicity; on a shared pinned client concurrent calls
interleaved transactions)
- single-statement deletes no longer wrap in a transaction hook call
- appendRecord gates payloads through assertJsonValue: fail loud on
values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
strings) instead of silently persisting something different
- append retry also covers 40001/40P01; READ COMMITTED assumption
documented at the retry loop
- ensureSchema documented as create-only; redundant stage index dropped
- deterministic interleaving test proves the 23505 retry path; real
PostgreSQL contract lane added (postgres:16 service workflow, pg
driver suite skipped locally without PG_CONTRACT_URL)
* fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C)
- json-value guard rewritten (shared with the HTTP backend): additionally
rejects -0, symbol-keyed and non-enumerable own properties, and
non-index own properties on arrays; accepts U+2028/U+2029
- storage-pg-contract workflow now triggers for the runtime-server-backend
integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1)
when PG_CONTRACT_URL is missing instead of silently skipping
- loadSession distinguishes corrupt non-object rows from absent rows so
getSession fails loud instead of reporting the session missing
* fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C)
- json-value guard (shared): NUL rejected in object keys, unpaired
UTF-16 surrogates rejected in values and keys
- createSession/appendRecord gate the full persisted envelope through
assertJsonValue, not only the payload — stray Date/undefined
properties and NUL in ids fail loud instead of silently diverging
from the returned record or leaking raw 22P05
- real-PG lane covers a genuine 23505 conflict from a second
connection and recovery after an aborted transaction
- retry-set asymmetry and mergeLearner's unbounded lock set documented
* fix(storage): round-4 convergence fixes for the PG backend (#939 Part C)
- NUL/lone-surrogate lookup and delete keys resolve to absent/no-op
instead of leaking 22021 driver errors; setSessionStatus reports the
session missing; mergeLearner from-key moves 0
- mergeLearner destination key passes the JSON-domain gate fail-loud
- envelope JSON gate tolerates explicitly-undefined optional anchors,
matching the browser backend
- json-value guard v4 (shared) + tests for accessors, Array subclasses,
prototype-supplied indices
* fix(storage): round-5 convergence fixes for the PG backend (#939 Part C)
- appendRecord pre-checks the session key like every other lookup path,
so a NUL/lone-surrogate sessionId reports 'no session' instead of
leaking a 22021 driver error
- the queryable-key predicate now structurally reuses the shared
isLosslessJsonString export instead of restating the string rule
- undefined-stripping limited to the DSL-declared optional anchors
- json-value guard: realm-agnostic prototype checks (shared)
* fix(storage): close the toJSON prototype channel in the JSON guard (#939)
Shared guard change with the HTTP backend branch; adds the crafted
null-proto-prototype regression test. Both final-gate reviewers
independently converged on this same channel.
* fix(storage): probe toJSON via descriptor walk, not property read (#939)
Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.
* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)
The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.
* fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943)
The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore
contract while the HTTP and Postgres backends were developed in
parallel against the pre-#926 interface. Implements the method on both
backends (single-statement wipe on PG via the FK cascade; DELETE
/runtime on the HTTP contract, documented as an operator-gated
administrative endpoint), restoring a green typecheck and contract
suite on the integration branch.
* feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944)
* wip(runtime): backend injection seam — pending gate verification
* fix(runtime): cross-review hardening for the storage injection seam
- stage-deletion IndexedDB probe applies only to the default browser
backend; an injected store always receives deleteStageRuntime
- explicit kv argument takes precedence over the configured learner-key
provider, matching the store seam's explicit-beats-global rule
- configured learner-key resolution is latched with in-flight dedup,
mirroring the store singleton; identity changes require app-level
handling
- factory retry semantics documented; seal errors explain the
module-level bootstrap requirement; isRuntimeStorageConfigured()
probe and a test-only reset added
- client-bootstrap-only contract documented (SSR/HMR caveats)
* fix(runtime): snapshot configuration and make the test reset complete
- configureRuntimeStorage copies the option fields so mutating the
caller's object after configuring cannot swap the sealed backend or
identity provider
- resetRuntimeStorageForTests now clears every latched consumer cache
(store singleton, learner-key in-flight promise) via a reset-hook
registry, so a reset-then-reconfigure test actually gets the new
backend
* fix(runtime): reset also clears the default learner-key caches
resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so
a default-path test could leak its anonymous key or KV store into the
next test despite the documented full-reset promise.
* feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945)
* wip(storage): reference server — pending deleteAllRuntime route + gates
* feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D)
- createRuntimeHttpHandler(store, options) wires the documented HTTP
contract onto any injected RuntimeStore over node http
- authenticate is required; every learner-scoped operation verifies the
path/body learnerKey against the authenticated principal (403
FORBIDDEN_LEARNER) — the client-supplied value is never trusted
- mergeLearner requires an explicit authorizeMerge grant and admin
surfaces (stage cascade, DELETE /runtime) require authorizeAdmin;
both default-deny
- runnable reference entry demonstrates a pg Pool withTransaction and
bearer-token authentication, marked as demo-only
- contract suite green through HttpRuntimeStore -> listening reference
handler -> PgRuntimeStore(pglite); security matrix tested (401/403
paths, admin default-deny); threat model documented
* fix(storage): cross-review hardening for the reference server (#939 Part D)
- principal learnerKey is optional: admin/merge-only credentials no
longer fabricate learner identity; learner-scoped routes 403 without
ownership
- full session/record envelopes pass the JSON-domain gate at the
handler, so NUL/lone-surrogate identifiers map to 400 instead of 500
- payload validation follows the injected store's validator map
(options.payloadValidators) instead of imposing DSL defaults
- reference factory accepts authenticate/authorizeMerge/authorizeAdmin/
payloadValidators overrides; docs no longer claim the factory binds
to localhost; demo-impersonation warning hardened
- 500 responses carry a generic message; details go to the server log
- ownership checks precede version checks and unowned sessions read as
404, closing existence/version oracles; cross-learner denial matrix
tested per route
- merge/delete concurrency documented as linearizable-equivalent with
the ownership re-check narrowed to the delete call
* fix(storage): align reference-server semantics with the store contract
- future-stamped sessions read and delete through unchanged; 409
FUTURE_VERSION applies only to guarded writes (status, append, merge)
- check-then-write races reclassify structurally via a post-failure
re-fetch: missing session 404, non-active session 400, never a
message-sniffed or generic 500
* fix(storage): close the reference CLI's pg pool on startup failure
ensureSchema opens connections inside createReferenceRuntimeServer, so
a failed schema init or occupied port left the pool holding database
resources until its idle timeout.
* fix(storage): close the records-route existence oracle
cosarah's review point on #946: the records list answered 200 [] for an
absent session but 404 for another learner's, so the 404 leaked that an
id exists. Absent and foreign sessions now answer identically (404
SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty
list, preserving the store contract's absent-lists-as-empty semantics.
Contract doc records the server MAY/SHOULD and the client MUST.