* feat(storage): KV HTTP contract and clients (#1000 Part A) (#1002) * feat(storage): KV + Asset HTTP contracts and clients Adds the client half of the server backends for the two components in the storage RFC that still had a browser backend only. Asset contract: the ref is `sha256-<hex>` over the bytes, computed by one shared rule both backends call, and it *is* the address bytes are written to (`PUT /assets/{ref}`). The server re-hashes what it received and refuses a mismatch, so content addressing is enforced end to end rather than assumed, and uploads become idempotent. `resolve` returns a URL per call, either an absolute signed URL passed through untouched or a root-relative proxied path joined to the base origin — the document never stores a raw URL either way. KV contract: account scope only, with no scope path segment, query parameter or body field anywhere on the wire. `HttpAccountKV` is the only object that can reach it and none of its methods takes a scope, so a device value is not a request the transport can express; `HttpKVStore` routes `device` to a local backend it requires at construction. Tests assert the invariant both ways — `@ts-expect-error` on every scope-passing call (unused directives fail the build if a scope is ever added) and a fetch spy proving device traffic does not exist. Both clients pass the existing `runKVStoreContract` and `runStorageProviderContract` suites against an in-memory conformance server, the asset one in both the proxied and signed URL shapes. * fix(storage): close the review findings on the KV + asset HTTP clients Cross-review found the device-scope guarantee resting on its weakest layer, and several places where the contract promised more than the code or the conformance server could deliver. deviceStore: the two type-level layers only documented intent — the barrier that actually kept device values off the wire was the injected store, typed as `KVStore`, which `HttpAccountKV` satisfies structurally (its methods are the same methods minus an optional parameter). Injecting the transport as the device backend type-checked and put device values on the network. The device scope now requires a branded `LocalKVStore`, the transport declares itself not to be one, and a runtime check refuses what a cast smuggles past. resolve URLs: a media `src` carries cookies, never the client's headers hook, so the proxied shape is only sound for cookie-authenticated deployments — the contract now says so instead of claiming the content route sits behind "the same authentication", and the conformance reader uses a cookie rather than a header no browser could send. The client also validates what it resolves: http(s) schemes only, and no `//host` or `/\host` slipping through the relative branch to another origin. Identifiers: refs and keys arrive percent-decoded, so `..%2F..%2F` was a traversal waiting for the first backend to join it to a path. Both are now constrained on both sides, and the contract states the MUST that keeps a storage layer from being the first to look. Also: the JSON gate runs before anything reads the value (a `JSON.stringify` pre-flight ran caller code first and reclassified `{ toJSON: () => undefined }` as a delete); scope routing fails closed; an upload's Content-Type can no longer be relabelled by the headers hook; a mutation invalidates in-flight resolution; unparseable 2xx bodies become typed errors rather than native SyntaxErrors; signed tokens are validated rather than merely present; and the contracts settle the per-principal claim model, the media-type allowlist, the literal prefix rule, and the identifier charset that Part B would otherwise have had to guess at. * fix(storage): close the second review round on the KV + asset HTTP clients The device-scope guard was in one place and needed to be in three. The transport is exported and structurally a `KVStore`, so `kvPersistStorage(new HttpAccountKV(...), 'device')` and a plain `KVStore`-typed reference both compiled without a cast, dropped the scope argument on the floor, and put a device value on the wire. The transport now takes the scope it was pretending not to have — typed as `'account'`, refused at runtime for anything else, since bivariant method parameters mean the type alone cannot hold — and the persist adapter requires a `LocalKVStore` when paired with `'device'`. The resolved-URL checks were text tests where only parsing works. The URL parser strips ASCII tab, LF and CR before parsing, so `/<tab>/evil/x` walked past the `//` prefix check and loaded cross-origin — in the app-mounted shape, where a path-only base URL leaves that check as the only guard. The absolute branch let through `https://` with no host and `https:host/x`, which names one place standalone and another when a browser resolves it against a same-scheme page. Both branches are now decided by the parser: control characters refused outright, relative references validated against a probe origin, absolute ones required to parse as `scheme://host` with an http(s) scheme — and still returned byte-for-byte, so signatures survive. The media-type MUST had no implementation and contradicted itself: the conformance server stored and served `text/html` verbatim, and a metadata-less upload became `application/octet-stream`, which cannot be on a renderable allowlist. The server now implements the allowlist, serves everything else as an opaque attachment, and the contract scopes its "usable as an img src" promise to allowlisted types. The KV key domain was HTTP-only, which made "the same suite proves the backends equivalent" untrue: `a/b`, an empty key and a 600-byte key all passed in the browser and would have become unreachable on a server. An audit of the app's keys (learner key, storage generation, current-scene and migration markers, persist names) found none in violation, so the rules are lifted to the primitive and both backends enforce them, with the shared suite asserting it. Also: the claim model the last round wrote into the contract is now implemented and covered case by case; malformed percent-escapes are a 400 rather than a 500; the in-flight identity guard has a test that can fail; and the Content-Type conflict raises a typed error. * fix(storage): final review round — backend parity, URL probe, credentials Four small things the final pass found, plus the contract patches Part B would otherwise inherit. The `toJSON` delete semantics were documented backwards. `BrowserKVStore.set` was the trial-serialize the KV contract said no backend does, so `{ toJSON: () => undefined }` was a silent delete there and a rejection over HTTP — opposite answers to the same call. The browser backend now validates scope and key before anything can run caller code, recognizes the delete case by type, and refuses a value that still fails to serialize. Both cases are in the shared suite, so the two backends cannot drift apart again. The probe-origin check could not vouch for a URL naming the probe host itself: `//asset-url-probe.invalid/x` resolves onto the probe origin and passed, then resolves somewhere else entirely against a real base. Any authority-bearing form is refused outright, so the check no longer depends on the probe host being unguessable. Cross-origin cookie deployments had no way to send a cookie: `fetch` omits them cross-origin by default and the headers hook cannot help, since `Cookie` is a forbidden header name. Both clients accept `credentials` and pass it to `fetch`; the contract records the CORS obligations that come with it. Rather than restricting cookie deployments to same-origin, which the asset contract explicitly does not require. The conformance server's dot-segment check was reading a path the URL parser had already normalized, so `/assets/%2e%2e/x` arrived as `/x` and the documented 400 was really post-routing behaviour. It now validates the raw request target, and the injected transport preserves it instead of handing over a normalized one. Contract: the prefix rule names backslash (PostgreSQL's default LIKE escape, and the one an implementer skips because a legal key cannot contain it, though a prefix can), and acquiring a claim MUST leave the bytes present — a de-duplicating server that inserts only the claim can race reclamation of the last previous claim and leave a document resolving to a 404. Also: base URLs are checked for an http(s) scheme at construction, the ref domain moved next to the KV key domain and is enforced by both asset backends with shared suite coverage, the KV client refuses a hook-supplied Content-Type like the asset one, and CI runs the package typecheck that makes the `@ts-expect-error` guards mean something. * fix(storage): probe guard spelling, prefix validator, base url shape Three one-line-level holes from the closing review. The probe-host guard was spelled `startsWith('//')`, so the backslash forms `/\host/x` and `/\/host/x` reached the origin comparison, passed it by naming the probe host, and were stopped only by that host failing to resolve. It now refuses any second character that can open an authority — which is also what makes the comment true rather than aspirational: a path-absolute reference always inherits the base's origin, so with no authority expressible the comparison cannot be satisfied cross-origin, whatever the probe host is. The conformance server validated `?prefix` with the path-segment validator, rejecting `keys('.')` and `keys('..')` — legitimate prefixes of keys like `.hidden`, which the client rightly allows. A prefix travels in the query string and is never a path segment, so it now has its own validator: the key rules minus the non-empty and dot-segment requirements. The contract says the same, and the shared suite pins both backends to it. Base URL validation moved into one shared helper both clients call, since two copies of a security rule is how they drift. It now also refuses `https:host/path` — which parses standalone to that host but is resolved against the current document by `fetch`, sending requests and their credentials somewhere the operator never wrote — and a base carrying a query or fragment, which silently swallows the path concatenated onto it. * fix(storage): close two holes in the shared base-url check The path-only branch returned its input unexamined, so `//evil.example` and `/\evil.example` were accepted as paths. A browser reads both as an authority, which means every request built on such a base — and every credential the headers hook attaches — leaves the origin. A base is now required to be a single `/` followed by something that is not `/` or `\`, the same refusal the resolved-URL check already makes, for the same reason. Control characters are refused first: the parser strips or re-encodes them, and `/<tab>/evil.example` becomes `//evil.example` after it does. The query/fragment check read the parsed `search` and `hash`, which are empty strings for a base ending in a bare `?` or `#` — so exactly the inputs that swallow the concatenated path into a query or fragment were the ones that got through. It reads the raw string now, which also makes the post-parse check redundant rather than merely unreachable. * fix(storage): bypass HTTP caches on reads, and require a 404 for not-found Two mechanisms the clients were missing, on both sides. Reads are marked `cache: 'no-store'`. The asset resolve request answers two mutable questions — does this principal hold a claim, and which URL is valid now — so a cached answer returns an expired signed URL or replays a negatively cached 404 for an asset written since; the in-flight coalescing that would otherwise cover this lives in the client and cannot reach the HTTP cache layer. KV reads have the same problem for a different reason: `account` is by definition the scope another device may have just written, and nothing here can invalidate a cached copy when it does. The contracts now require `no-store` on those responses. Asset content is left cacheable — bytes at a content-addressed ref never change — and writes are unmarked, having nothing to read from a cache. The not-found mappings now require the status as well as the code. A proxy or gateway answering 401, 403 or 500 while echoing the body's error code was being read as "this does not exist", and `null` is indistinguishable from a real miss — so an outage reached the caller as data loss. * fix(storage): reject unusable base urls, and make the harness meet the contract Two configuration checks and two harness gaps. A base URL carrying userinfo built a client that threw a native TypeError on every operation, because Fetch refuses to construct a request from a URL with embedded credentials — one legible error where the deployment is configured beats a per-call failure with no explanation. A base with surrounding whitespace was worse than useless: `new URL` strips it, so validation passed, and then concatenation moved the space inside the URL where it is no longer trailing and no longer stripped, breaking every request. The conformance server now sends `Cache-Control: no-store` on the three reads the contracts require it on. The requirement landed last round with nothing asserting it, so the suite would have passed against exactly the implementation it exists to catch. And `DELETE /kv/entries/{key}` refuses a request body outright: the query and the header were already checked for a smuggled scope, leaving the body as the one channel through which a delete could still describe one. * fix(storage): allow a device-routing composite to persist device-scoped state cosarah found kvPersistStorage(httpKVStore, "device") wrongly rejected. An HttpKVStore is a full dual-scope composite: it routes device operations to the LocalKVStore it requires at construction and only reaches the network for account, so device values never leave the machine. The guard tested isLocalKVStore — "is the whole store local" — and so turned away a store that is safe for device precisely because it is not fully local. The guard now turns on the capability that actually matters: does the device scope stay local. A new DeviceSafeKVStore brand (servesDeviceScopeLocally: true) marks a store whose device writes never leave the device — held by a fully-local LocalKVStore (which extends it) and by HttpKVStore, and declared false by HttpAccountKV, the pure account transport with no local device backend. The persist device overload requires DeviceSafeKVStore; the account overload is unchanged. The security boundary the red team opened this whole line on is untouched: HttpAccountKV used for device is still doubly refused — its false brand is not assignable to the true DeviceSafeKVStore requires (type), and the runtime check rejects the cast that would erase it. The brand is structurally unforgeable, not a flag the transport could set; and the injected device backend of a composite still demands the stronger, fully-local LocalKVStore, so composites cannot nest into a router loop with no local floor. Regression test (cosarah's ask): kvPersistStorage(httpKVStore, "device") set/get/remove reach the injected device backend, land device-scoped there, and make zero HTTP requests. Mutation-checked: widening the brand so HttpAccountKV is accepted, or neutering the guard, turns the device-out reject test red. * refactor(storage): scope #1002 down to KV — remove the asset HTTP backend Per the maintainer decision, the AssetProvider server design moves to the global-resource-pool RFC (#1007), where AssetRef changes from content-addressed to an allocated global id and the contract model itself changes. Continuing to harden the content-addressed asset client here would be work on code that is being replaced, so the asset half of #1002 is withdrawn and the PR is pure KV. Removed (all introduced by #1002): - docs/asset-http-contract.md - src/asset/http.ts (HttpAssetProvider) and its index / package.json exports - src/asset/content-ref.ts (assertAssetRef / computeAssetRef / the ref shape) - test/http-asset-provider.test.ts and the asset half of the conformance server Reverted to the #858 baseline, untouched: - src/asset/browser.ts — back to opaque resolve/remove, no assertAssetRef guard. This directly removes cosarah's latest asset finding: an opaque ref such as `bucket/path` no longer throws, restoring the DSL AssetRef-opaque contract. - test/asset-contract.ts, test/asset-browser.test.ts — 0-diff from origin/main. Kept (all KV): kv-http-contract.md, src/kv/* (HttpKVStore / HttpAccountKV / the DeviceSafeKVStore brand), the KV tests, and the KV conformance server — renamed kv-conformance-server.ts and reduced to KV routing over the shared harness framework (body caps, no-store, scope-channel and dot-segment guards, auth hooks). base-url.ts stays; KV still uses it. * fix(storage): treat KV keys as opaque; document account-sync prerequisites cosarah found the key validation we added for traversal defense breaks legitimate callers: DSL identifiers (stageId/sceneId/learnerKey) are unconstrained strings — verified, the DSL only requires typeof === 'string' — and app callers compose keys like `editor-current-scene:<stageId>`, so a valid stage id `stage/one` now throws in BrowserKVStore before localStorage. Fix, matching the asset-opaque precedent: a KV key is an opaque string again. The `/` and `\` rejection is removed from assertKVKey / assertKVKeyPrefix and from the conformance server. Traversal is defended by encoding, not rejection — the HTTP client already percent-encodes the whole key into one path segment, so a separator never opens a new segment on the wire, and the server stores the decoded key as a plain map key (bound value), never a path. `a%2F..%2F..%2Fb` decodes to the single opaque key `a/../../b`, which addresses one entry and traverses nothing; a raw unencoded `..` path produces extra segments that match no route (404). The rules that remain are only what encoding cannot cover and the transport cannot carry (empty, whole-key `.`/`..`, NUL, lone surrogate, length) — none producible by `prefix + id`. Tests: the shared suite now round-trips keys with `/`, `\`, `%`, `:`, spaces across both backends; a `stage/one` key round-trips through the real conformance server; the traversal input is proven to store as one opaque key and the raw path to 404. Mutation-checked: re-adding the `/` rejection reddens exactly the round-trip cases. Docs (kv-http-contract.md): - "A key is an opaque string" replaces the identifier-shape section — keys may contain separators; traversal is an encoding + server-bound-value duty. - Account settings sync now documents its security prerequisite next to the account scope: a deployment MUST provide real per-user isolation of stored rows and encryption at rest before enabling sync; the reference server is not that posture. - kvPersistStorage(device) doc corrected from LocalKVStore to DeviceSafeKVStore (the device-safe brand accepts the HttpKVStore composite; HttpAccountKV stays double-rejected). * fix(storage): close the scope/key/prefix contract gaps with a coverage matrix Three completeness gaps, nailed by an executable matrix so an untested channel/shape can't slip through the next pass rather than patched one by one. 1. Scope via a GET body was not rejected. `assertNoScopeChannel` covered the query and header, but the bodyless GET routes never read their body, so `GET /kv/keys` with `{"scope":"device"}` returned 200 []. Bodyless methods (GET, DELETE) now reject any request body up front — that is the channel a scope would hide in — and only `set` reads a body, checking it for a scope field. The DELETE route's own body check folds into the shared one. 2. The 512-byte key bound wrongly rejected legitimate keys. A DSL id is an unconstrained string (verified: the DSL requires only typeof === 'string'), and `editor-current-scene:` + a 500-char stage id is 521 bytes, which BrowserKVStore rejected before localStorage. Keys are length-unconstrained; the bound is raised to a generous 8 KiB DoS ceiling and documented as a resource guard, not part of the key domain — not a derived/hashed key, which would break the literal-prefix semantics of keys(prefix). Both backends and the conformance server move in lockstep. 3. The prefix contract text contradicted itself: it claimed a legal key can never contain a backslash, while the key section (correctly) makes keys opaque, backslash included. Reworded — a prefix is opaque like a key — and the shared literal-prefix matrix now covers `\` (PostgreSQL's LIKE escape), which it had skipped, alongside `%`, `_`, and combinations. Coverage matrix (CI-executable): - scope-rejection: {get,set,remove,keys} × {query,header,body}. query/header run in-process; the body cells use a raw HTTP client (fetch cannot put a body on a GET — exactly the non-conforming client the finding described) against a loopback listener, skipped where the sandbox forbids binding. - key-validity: `/ \ % _ : .` space, BMP + astral Unicode, `%2F`, a 500-char composed key, and all-at-once — each round-trips (set/get, keys(prefix) match, remove) on both browser and HTTP backends via the shared suite. - prefix-charset: `%`, `_`, `\`, and combinations match literally, both backends. - Boundaries still rejected: whole-key `.`/`..`, NUL, lone surrogate, empty, and the DoS ceiling. Mutation-checked: removing the bodyless-body rejection reddens the three GET/ DELETE body cells; lowering the ceiling back to 512 reddens the long-key round-trip on both backends. Bumps @openmaic/storage to 0.1.2 (src/kv/* changed). * fix(storage): bound the KV key DoS ceiling by encoded size, not UTF-8 bytes Follow-up to the 8 KiB ceiling: measuring it in UTF-8 bytes ignored percent-encoding expansion and reopened a backend-parity break. A non-ASCII key is small in bytes but large on the wire — `€` is 3 UTF-8 bytes and `%E2%82%AC`, 9, encoded — so `'€'.repeat(2730)` (8190 bytes, under the old cap) encoded to ~24.6 KB, which the Node HTTP server rejects with 431 before routing while `BrowserKVStore`, which never builds a URL, accepted it. The ceiling is now measured on `encodeURIComponent(key).length` — the size the key occupies as a request target — at 4096, far above any real id (a 500-char id under `editor-current-scene:` encodes to ~523) and well below Node's default 16 KiB `maxHeaderSize` (empirically the 431 trigger on Node 22), so no key that passes validation can reach a 431. Both backends and the conformance server use the same encoded-size limit, so a key is accepted iff its encoded form fits, whichever backend it reaches — parity restored. `MAX_KV_KEY_BYTES` becomes `MAX_KV_KEY_ENCODED_LENGTH`. Tests, against a real node:http server for the 431 boundary: - the max-legal non-ASCII key (455×€, encoded 4095) round-trips over real HTTP with no 431, and the browser accepts the same key — parity on accept; - one code point over (456×€, encoded 4104) is rejected by both backends at validation, before any request — the exact parity case a byte bound missed (1368 bytes, would have passed, then 431'd over HTTP); - a raw client bypassing client validation with an over-ceiling (under 16 KiB) encoded key gets a clean 400, not a 431. Shared suite's key-domain matrix now bounds by encoded size on both backends. Mutation-checked: reverting the ceiling to UTF-8 bytes reddens the three encoded-length parity tests. Bumps @openmaic/storage to 0.1.3 (src/kv/* changed). * fix(storage): make the KV key domain truly opaque; keep transport limits out The recurring key-limit findings (512 → 8 KiB → 4096, each patch exposing the next) shared one root cause: transport-driven rules kept being added to the key domain — the browser primitive included — breaking the "opaque unconstrained string key" contract. This is the structural fix, per cosarah: "keep HTTP transport limits out of the browser primitive and preserve the existing string key domain." Approach (a), the minimal one: a KV key is any string, validated nowhere. The browser primitive is a Map/Storage that stores any key; the HTTP backend carries the key as a URL path segment and inherits that transport's limits, documented as HTTP-deployment concerns that never constrain the key domain. Every real key (a `prefix:id` from an unconstrained DSL string) round-trips on both backends; only pathological keys hit a transport limit, and none of those are keys a caller produces. (b) — keys in the request body for accept-parity — was unnecessary: verified that no real key needs it. Removed from the key domain (browser + HTTP + conformance server): - the encoded-size / length ceiling entirely — there is no key-length limit; - the NUL exclusion — NUL percent-encodes to %00, is preserved end to end, and is a legitimate Map/Storage key (a false "cannot represent" premise); - the `.`/`..`/empty/`/`/`\` rejections — all opaque, all round-trip. `assertKVKey` / `assertKVKeyPrefix` / `MAX_KV_KEY_*` are gone. Kept, at the transport layer only (HttpAccountKV), never reaching the primitive: - an unpaired UTF-16 surrogate genuinely has no percent-encoding, so the HTTP client surfaces a clear transport error (KEY_NOT_ENCODABLE) while the browser stores the same key — verified: encodeURIComponent throws on it. Scope completeness (P2): the "reject scope in every channel" guarantee had two holes — a scope path segment (`/kv/device/keys`) 404'd instead of 400, and only `x-scope` was checked among header spellings. The server now rejects a scope path segment and the whole prohibited header set (scope, x-scope, kv-scope, x-kv-scope). The coverage matrix is {get,set,remove,keys} × {path, query, each header spelling, body}. Coverage matrix reworked to verify arbitrary opaque keys (empty, NUL, control chars, separators, long, Unicode) round-trip on both backends — not to verify a limit. A real node:http test proves NUL and long keys round-trip end to end, and that a lone surrogate is a transport limit (browser accepts, HTTP client errors). Mutation-checked: re-adding a NUL rejection, removing the path-segment check, or dropping a header spelling each reddens the matching cells. Cleanup: package.json description updated (KV HTTP delivered, not "later"); network-server bind/skip/close lifecycle deduped into one helper. Bumps @openmaic/storage to 0.1.4 (src/kv/* changed). * fix(storage): reject whole-key "."/".." in the HTTP client — they alias entries Follow-up to making the key domain opaque: removing the `.`/`..` key-domain rejection was right (the browser stores them fine), but the HTTP client then let them reach `fetch`, where URL path normalization silently rewrites the target before it is sent. `encodeURIComponent('.')` is `.`, so `/kv/entries/.` collapses to `/kv/entries/` (the empty key) and `/kv/entries/..` walks up a level — so `get`/`set`/`remove` on a whole-key `.` or `..` silently read, overwrite, or delete a *different* entry, with no error. A data-correctness bug. Fix, same pattern as the unpaired-surrogate limit already handled: a whole-key `.`/`..` is something the URL-path transport structurally cannot carry, so the client's path-segment encoder refuses it with `KEY_NOT_ENCODABLE` before a request is built. Split the encoder in two: `encodeKeyPathSegment` (key → path segment, rejects whole-key `.`/`..`) and `encodeComponent` (the `keys()` prefix → query value, where `.`/`..` are legitimate and untouched). A key that merely *contains* a dot (`a.b`, `prefix:id`) is unaffected and round-trips. The browser primitive still stores `.`/`..` opaquely. The HTTP transport's limits are now complete — three cases, all fail-loud at the transport layer, all storable on the browser: (1) an unpaired surrogate (no percent-encoding), (2) a whole-key `.`/`..` (URL normalization aliases it), and (3) an over-long key (deployment request-target ceiling). The contract lists all three as HTTP-deployment limits, not key-domain constraints. Tests: whole-key `.`/`..` — browser stores and round-trips (distinct from the empty key); HTTP client throws `KEY_NOT_ENCODABLE` and builds no request; and a real-server test asserts the empty-key entry is never read, overwritten, or deleted by a `.`/`..` operation. A dot-containing key (`a.b`) round-trips. Mutation-checked: dropping the whole-key check reddens the alias tests. Bumps @openmaic/storage to 0.1.5 (src/kv/* changed). * fix(storage): reconcile the settings/profile persist consumer with the KV API The integration merge (#1002 KV into main) surfaced two things the merge itself could not resolve mechanically: - `lib/store/kv-persist.ts` (added on main by #1001) called `kvPersistStorage(kv, scope)` with a *variable* scope, which #1002's overloads deliberately reject — `'device'` must be paired with a device-safe store, so the overloads take the scope as a literal. Branch on the scope: `'account'` takes any `KVStore`; `'device'` narrows the backend to `DeviceSafeKVStore` via its brand (the adapter re-checks at runtime) and otherwise fails loud. The app wires only `'account'` today; the `'device'` arm keeps the generic helper honest. `ControllableKV` in the persist test wraps a local `BrowserKVStore`, so it is genuinely device-safe and now declares the brand. - `packages/@openmaic/storage` version bumped 0.1.5 → 0.1.6: the merge changed storage inputs (README + a pg-schema test carried from main) relative to the previous integration tip, so the version-bump guard requires an increase above #1002's 0.1.5. Still ahead of main's 0.1.1. Verified: root `tsc --noEmit` clean, the persist test green (40 passed), and the storage package suite green on Node 22.
17 KiB
KVStore HTTP contract
This contract exposes the KVStore operations — get / set / remove / keys(prefix?) — over JSON HTTP for the account scope only. All paths below are relative to a deployment-defined base URL. Path segments and query parameter values are percent-encoded UTF-8 strings. Request and response bodies use application/json; successful operations with no return value respond with 204 No Content.
The conformance server in this package is test-only. It implements no authentication or authorization model beyond what the contract's response codes require; deriving the principal from an authenticated session is the reference server's job.
The scope is not on the wire
KVStore distinguishes two scopes. account is user data a server-backed deployment syncs across devices (provider and model configuration, profile); device is machine-local state (theme, locale, layout) that must never leave the machine. That is a property of the primitive, not a deployment setting.
This contract therefore has no scope at all: no scope path segment, no scope query parameter, no scope header, no scope body field. There is nothing to set to device, so there is no configuration or client bug that could ship a device value to a server — a request carrying a device value is not a request this contract can express. Servers MUST reject a request that invents a scope in any of those channels with 400 VALIDATION_FAILED rather than ignoring it, so a client attempting to describe a scope fails loud instead of silently having its intent discarded. "Any channel" is exhaustive, and each of the four is a place a real client or proxy might reach for a scope:
- Path segment. The contract's segment after
kvisentriesorkeys; a scope name there (/kv/device/keys,/kv/account/entries/{key}) is an attempt to route by scope and MUST be400, not a404route miss. (A key nameddeviceis unaffected — that sits afterentries, at/kv/entries/device.) - Query parameter. A
?scope=…MUST be rejected. - Header. There is no single canonical spelling, so the server MUST reject the whole prohibited set — at least
scope,x-scope,kv-scope, andx-kv-scope. Checking onlyx-scopeleaves the others open. - Body.
get,remove, andkeysare bodyless: a server that routes them without reading the body would letGET /kv/keyswith{"scope":"device"}succeed, the guarantee failing silently. So a bodyless method (GET,DELETE) MUST reject any request body outright, and onlysetreads a body, where ascopefield is rejected.
Every one of {get, set, remove, keys} × {path, query, header spellings, body} is closed, and the shared suite pins each cell.
The client half mirrors this in three layers, only the last of which actually carries the weight:
HttpAccountKVis the only object in@openmaic/storagethat can reach this contract, and its scope parameter admits'account'alone — a literal'device'at a call site is a type error. It also refuses one at runtime, because the type is not enough: TypeScript compares method parameters bivariantly, so this transport stands in for the widerKVStorewherever one is expected, and a caller holding it that way can pass any scope. Silently dropping that argument is how a device value reaches a server, so the transport takes the scope and fails loud on anything butaccount.HttpKVStore— the fullKVStore— is the one place the two scopes are told apart. An unrecognized scope is refused rather than folded into the account path: guessing would send a value the caller believed was device-local to a server.HttpKVStorerequires aLocalKVStorefor thedevicescope — a store branded as keeping its values on this machine.KVStorealone is not enough, because a networked store satisfies it structurally:HttpAccountKVis aKVStorewith one optional parameter fewer, so aKVStore-typed parameter would accept the very transport this design exists to exclude. The brand makes that a type error, and a runtime check refuses a remote store that a cast smuggled past the types.
There is no default and no fallback, so a deployment cannot end up with device values that have nowhere local to go.
The same pairing rule reaches the zustand adapter, which is where a store and a scope are chosen as separate arguments and therefore the easiest place to pair them wrongly: kvPersistStorage(store, 'device') requires a DeviceSafeKVStore at the type level and checks the brand at runtime. DeviceSafeKVStore is the capability that actually matters — "a device write stays on this machine" — and it is broader than LocalKVStore: it admits both a fully-local backend and the HttpKVStore composite, which routes device to its injected local backend while sending only account to a server. What it excludes is the pure HttpAccountKV transport, which has no local device backend and would put a device value on the wire; that store declares the capability false and is double-rejected.
Account settings are only as isolated as the deployment makes them
The account scope is where a server-backed deployment syncs a user's settings across devices, and the contract's principal derivation keeps one account from addressing another's keys over the wire. That is a routing guarantee, not a storage one. Before enabling settings sync, a deployment MUST provide real per-user isolation of the stored rows (the (principal, key) partition enforced end to end, not merely by a query filter a bug could drop) and encryption at rest for the values, which hold user configuration and profile data. The reference server is a reference: it demonstrates the principal derivation, not a production isolation or encryption posture, and MUST NOT be exposed with real user data as-is.
A key is a truly opaque, unconstrained string
A KV key is any string — of any length, containing any character, including NUL, separators, and things that look like structure. The KVStore primitive imposes no key-domain rules whatsoever: callers compose keys from unconstrained DSL identifiers (the DSL constrains ids only to be strings), and the browser primitive is a Map / Storage, which stores any string key. A rule in the key domain — a length cap, a / ban, a NUL ban — would be a transport concern reaching back into the primitive, and would make a key the browser can perfectly well store unreadable. There is no such rule, and no key validation. A server likewise MUST NOT reject a key for its content; it stores the decoded key as an opaque value — a bound parameter or a derived, constrained storage id — never as a path component or an unescaped fragment of a query, so a key like a/../../b (arriving percent-encoded as a%2F..%2F..%2Fb) is one opaque entry that traverses nothing.
Validity is a property of the key; reachability is a property of the transport. The two are kept separate, and this is the parity guarantee: a key is valid on every backend identically (all accept it), even where it is not reachable over a particular transport.
- The browser backend uses no transport and imposes nothing — every key round-trips.
- The HTTP backend carries a key as a URL path segment, so it inherits exactly three limits of that transport. Each is an HTTP-deployment concern, not a key-domain constraint, each makes the affected key fail loud rather than silently misbehave, and the browser backend stores every one of them fine:
- An unpaired UTF-16 surrogate has no percent-encoding —
encodeURIComponentthrows — so the client refuses it withKEY_NOT_ENCODABLEbefore building a request. - A whole-key
.or..is normalized away by URL path parsing before the request is sent (/kv/entries/.collapses to the empty-key segment,/kv/entries/..walks up a level), so sending it would silently alias a different entry. The client refuses these two whole keys withKEY_NOT_ENCODABLEas well. This is only the whole key: a key that merely contains a dot (a.b,prefix:id) is an ordinary segment and round-trips, and a./..prefix in akeys()query is unaffected (it is a query value, not a path segment). - An extremely long key may exceed a deployment's request-target / header size ceiling (Node's default is ~16 KiB, above which the server answers
431before routing). No length is invalid; a long key is simply not reachable past that transport bound.
- An unpaired UTF-16 surrogate has no percent-encoding —
None of these arise for the keys real callers produce (prefix:id from a DSL string). All are the URL transport's, not the key's; a deployment that must accept literally any key over HTTP would carry the key outside the URL path. This contract takes the simpler route and fails loud on the three cases above, and the browser backend is unaffected by all of them.
Endpoints
| Method | Path | Purpose | Success |
|---|---|---|---|
GET |
/kv/entries/{key} |
Read one value. | 200 with { "value": <json> }, or 404 KEY_NOT_FOUND |
PUT |
/kv/entries/{key} |
Write one value from { "value": <json> }. Replaces any existing value. |
204 |
DELETE |
/kv/entries/{key} |
Delete one value. Idempotent; deleting an absent key succeeds. | 204 |
GET |
/kv/keys |
List the principal's keys. Optional ?prefix={prefix} returns only keys starting with it. |
200 with string[] |
Both read routes MUST be served with Cache-Control: no-store, and MUST NOT be cached by any intermediary. account is by definition the scope another device may have just written, and neither the client nor the server has a way to invalidate a cached copy when that happens — a cached read serves precisely the stale state this scope exists to move past. The client sends its reads with cache: 'no-store' for the same reason. Writes carry no such requirement, having nothing to read from a cache.
The value travels in an envelope rather than as the body itself. KVStore.get cannot distinguish a stored null from an absent key — both are null to a caller — but the wire still has to be unambiguous: a bare null body would be indistinguishable from an empty body or a stored null, leaving the framing to depend on Content-Length. The envelope makes "there is a value, and it is null" a statement the response can make. get maps 404 KEY_NOT_FOUND back to null, matching KVStore.get; the client MUST use the machine-readable code, not the status alone, so a 404 ROUTE_NOT_FOUND stays an error instead of masquerading as a missing key.
GET /kv/keys returns every matching key. The listing is not paginated, matching GET /documents in the DocumentStore HTTP contract; the primitive is sized for small configuration values, and a deployment that outgrows an unbounded listing needs a change to KVStore itself, not a second listing shape here.
The prefix is a literal, byte-for-byte comparison, not a pattern. A prefix is as opaque as a key: it too travels in the query string rather than as a path segment, so it may be empty (that is what "list everything" means), may be . or .. (legitimate prefixes of keys such as .hidden), and may contain any character a key may — %, _, \, /, and the rest. Only the transport-fatal characters and the DoS ceiling constrain it.
Because both the prefix and the keys it matches are opaque, a server MUST escape every metacharacter of its own query language before applying the comparison. In the obvious SQL translation, key LIKE prefix || '%', that means % and _, and the backslash — PostgreSQL's default LIKE escape character. Backslash is the one an implementer is most likely to skip, precisely because it is easy to assume a key could never contain one; it can — a key is opaque — and so can a prefix, and both reach the query. Escape all three, or the literal comparison silently becomes a pattern match. The result order is unspecified, but the listing MUST NOT repeat a key.
Value domain
Values travel through JSON, so this backend accepts only plain JSON values that survive serialization without changing meaning. It MUST fail loud before sending values such as Map, Set, Date, non-finite numbers, negative zero, nested undefined, bigint, sparse arrays, symbol-keyed properties, non-enumerable properties, strings containing U+0000, class instances, and circular references. U+2028 and U+2029 are valid JSON string contents and MUST be accepted. Keys are held to the same rule, because a key becomes a URL path segment.
This is intentionally narrower than BrowserKVStore, whose JSON.stringify would quietly rewrite a Date into a string or drop a nested undefined. Where the browser backend is silently lossy, the HTTP backend refuses.
Both backends agree on one deliberate exception: set(key, value) where value has no JSON representation at all (undefined, a function, a symbol) is a removal, not a write. Storing such a value would produce an entry that throws on read, so the key is deleted instead.
Every backend decides that by inspecting the value, never by trial-serializing it, and this is a rule about all of them rather than an HTTP detail. A JSON.stringify pre-flight runs caller code — toJSON, getters — before any validation has looked at anything, which reclassifies values as deletes ({ toJSON: () => undefined } stringifies to undefined) and lets a stateful accessor show the probe one value and the serializer another. Two backends probing and validating in different orders would disagree about whether a write was a delete, which is precisely the divergence the shared suite exists to prevent. Scope and key are validated first, then the delete case is recognized by type, and only then is anything serialized. A value that still fails to serialize is refused rather than quietly deleted.
Principal derivation
The principal is derived server-side from the authenticated session and never appears in a path, query parameter, or body. This is the same non-negotiable that governs learnerKey in the RuntimeStore HTTP contract: a client-submitted principal is not proof of identity, and trusting one turns every route here into a lateral-authorization vulnerability that lets one account read or overwrite another's configuration.
Every route requires an authenticated principal; there is no anonymous KV. Because keys are caller-chosen and this store holds user configuration, a deployment MUST scope every operation — including GET /kv/keys — to the derived principal, and MUST NOT expose a cross-principal listing on this contract.
Errors
Every non-2xx response has this machine-readable JSON shape:
{
"error": {
"code": "VALIDATION_FAILED",
"message": "@openmaic/storage: kv write body must carry \"value\"",
"details": []
}
}
details is optional.
| Condition | HTTP status | Error code | Client behavior |
|---|---|---|---|
Malformed JSON, a scope in any channel, a missing value member, or a non-JSON value (a key is opaque and never a validation failure) |
400 |
VALIDATION_FAILED |
Throw HttpKVStoreError with the server message |
| A key cannot be percent-encoded for the URL transport (an unpaired surrogate) — a transport limit, not a rejection by the server | — | KEY_NOT_ENCODABLE |
Client-side only; throw HttpKVStoreError before any request. The browser backend stores the key |
| Request body exceeds the deployment's size bound | 413 |
PAYLOAD_TOO_LARGE |
Throw HttpKVStoreError |
| Request target exceeds the deployment's size ceiling | 431 |
(transport) | An HTTP-transport limit for a pathologically long key, not a key-domain rejection |
| No entry is stored under the key | 404 |
KEY_NOT_FOUND |
get returns null |
| Route does not exist | 404 |
ROUTE_NOT_FOUND |
Throw HttpKVStoreError |
| Missing or invalid credential | 401 |
UNAUTHENTICATED |
Throw HttpKVStoreError |
| Principal may not perform the operation | 403 |
FORBIDDEN_KV |
Throw HttpKVStoreError |
| Unexpected server failure | 500 |
INTERNAL_ERROR |
Throw HttpKVStoreError; the handler does not expose internal details |
Only KEY_NOT_FOUND becomes null. Status alone is not sufficient — ROUTE_NOT_FOUND shares its status and means a broken deployment, and a 401 or 403 must never be reported as a missing key.
A response the client cannot interpret — a body that is not JSON despite a 2xx status, a get body without a value member, a keys body that is not an array of strings — raises MALFORMED_RESPONSE, a client-side code with no server counterpart. It is a typed storage error like any other: the client never lets a native SyntaxError escape in its place.
Retry and atomicity guarantees
Every operation is safely retryable. PUT is a whole-value replacement and therefore idempotent for the same body; DELETE is idempotent because deleting an absent key succeeds; both GET routes are reads. The contract offers no compare-and-swap: a concurrent write is last-writer-wins per key, which suits the small independent configuration values this primitive exists for. Callers needing an atomic read-modify-write should not model that state as KV.