wyuc 091989c48b
feat(storage): promote KV HTTP contract + clients to main (Part A of #1000) (#1020)
* feat(storage): KV HTTP contract and clients (#1000 Part A) (#1002)

* feat(storage): KV + Asset HTTP contracts and clients

Adds the client half of the server backends for the two components in the
storage RFC that still had a browser backend only.

Asset contract: the ref is `sha256-<hex>` over the bytes, computed by one
shared rule both backends call, and it *is* the address bytes are written
to (`PUT /assets/{ref}`). The server re-hashes what it received and refuses
a mismatch, so content addressing is enforced end to end rather than
assumed, and uploads become idempotent. `resolve` returns a URL per call,
either an absolute signed URL passed through untouched or a root-relative
proxied path joined to the base origin — the document never stores a raw
URL either way.

KV contract: account scope only, with no scope path segment, query
parameter or body field anywhere on the wire. `HttpAccountKV` is the only
object that can reach it and none of its methods takes a scope, so a
device value is not a request the transport can express; `HttpKVStore`
routes `device` to a local backend it requires at construction. Tests
assert the invariant both ways — `@ts-expect-error` on every scope-passing
call (unused directives fail the build if a scope is ever added) and a
fetch spy proving device traffic does not exist.

Both clients pass the existing `runKVStoreContract` and
`runStorageProviderContract` suites against an in-memory conformance
server, the asset one in both the proxied and signed URL shapes.

* fix(storage): close the review findings on the KV + asset HTTP clients

Cross-review found the device-scope guarantee resting on its weakest layer,
and several places where the contract promised more than the code or the
conformance server could deliver.

deviceStore: the two type-level layers only documented intent — the barrier
that actually kept device values off the wire was the injected store, typed
as `KVStore`, which `HttpAccountKV` satisfies structurally (its methods are
the same methods minus an optional parameter). Injecting the transport as
the device backend type-checked and put device values on the network. The
device scope now requires a branded `LocalKVStore`, the transport declares
itself not to be one, and a runtime check refuses what a cast smuggles past.

resolve URLs: a media `src` carries cookies, never the client's headers
hook, so the proxied shape is only sound for cookie-authenticated
deployments — the contract now says so instead of claiming the content route
sits behind "the same authentication", and the conformance reader uses a
cookie rather than a header no browser could send. The client also validates
what it resolves: http(s) schemes only, and no `//host` or `/\host` slipping
through the relative branch to another origin.

Identifiers: refs and keys arrive percent-decoded, so `..%2F..%2F` was a
traversal waiting for the first backend to join it to a path. Both are now
constrained on both sides, and the contract states the MUST that keeps a
storage layer from being the first to look.

Also: the JSON gate runs before anything reads the value (a `JSON.stringify`
pre-flight ran caller code first and reclassified `{ toJSON: () => undefined }`
as a delete); scope routing fails closed; an upload's Content-Type can no
longer be relabelled by the headers hook; a mutation invalidates in-flight
resolution; unparseable 2xx bodies become typed errors rather than native
SyntaxErrors; signed tokens are validated rather than merely present; and the
contracts settle the per-principal claim model, the media-type allowlist, the
literal prefix rule, and the identifier charset that Part B would otherwise
have had to guess at.

* fix(storage): close the second review round on the KV + asset HTTP clients

The device-scope guard was in one place and needed to be in three. The
transport is exported and structurally a `KVStore`, so `kvPersistStorage(new
HttpAccountKV(...), 'device')` and a plain `KVStore`-typed reference both
compiled without a cast, dropped the scope argument on the floor, and put a
device value on the wire. The transport now takes the scope it was pretending
not to have — typed as `'account'`, refused at runtime for anything else, since
bivariant method parameters mean the type alone cannot hold — and the persist
adapter requires a `LocalKVStore` when paired with `'device'`.

The resolved-URL checks were text tests where only parsing works. The URL
parser strips ASCII tab, LF and CR before parsing, so `/<tab>/evil/x` walked
past the `//` prefix check and loaded cross-origin — in the app-mounted shape,
where a path-only base URL leaves that check as the only guard. The absolute
branch let through `https://` with no host and `https:host/x`, which names one
place standalone and another when a browser resolves it against a same-scheme
page. Both branches are now decided by the parser: control characters refused
outright, relative references validated against a probe origin, absolute ones
required to parse as `scheme://host` with an http(s) scheme — and still
returned byte-for-byte, so signatures survive.

The media-type MUST had no implementation and contradicted itself: the
conformance server stored and served `text/html` verbatim, and a metadata-less
upload became `application/octet-stream`, which cannot be on a renderable
allowlist. The server now implements the allowlist, serves everything else as
an opaque attachment, and the contract scopes its "usable as an img src"
promise to allowlisted types.

The KV key domain was HTTP-only, which made "the same suite proves the backends
equivalent" untrue: `a/b`, an empty key and a 600-byte key all passed in the
browser and would have become unreachable on a server. An audit of the app's
keys (learner key, storage generation, current-scene and migration markers,
persist names) found none in violation, so the rules are lifted to the
primitive and both backends enforce them, with the shared suite asserting it.

Also: the claim model the last round wrote into the contract is now implemented
and covered case by case; malformed percent-escapes are a 400 rather than a
500; the in-flight identity guard has a test that can fail; and the Content-Type
conflict raises a typed error.

* fix(storage): final review round — backend parity, URL probe, credentials

Four small things the final pass found, plus the contract patches Part B would
otherwise inherit.

The `toJSON` delete semantics were documented backwards. `BrowserKVStore.set`
was the trial-serialize the KV contract said no backend does, so
`{ toJSON: () => undefined }` was a silent delete there and a rejection over
HTTP — opposite answers to the same call. The browser backend now validates
scope and key before anything can run caller code, recognizes the delete case by
type, and refuses a value that still fails to serialize. Both cases are in the
shared suite, so the two backends cannot drift apart again.

The probe-origin check could not vouch for a URL naming the probe host itself:
`//asset-url-probe.invalid/x` resolves onto the probe origin and passed, then
resolves somewhere else entirely against a real base. Any authority-bearing
form is refused outright, so the check no longer depends on the probe host
being unguessable.

Cross-origin cookie deployments had no way to send a cookie: `fetch` omits
them cross-origin by default and the headers hook cannot help, since `Cookie`
is a forbidden header name. Both clients accept `credentials` and pass it to
`fetch`; the contract records the CORS obligations that come with it. Rather
than restricting cookie deployments to same-origin, which the asset contract
explicitly does not require.

The conformance server's dot-segment check was reading a path the URL parser
had already normalized, so `/assets/%2e%2e/x` arrived as `/x` and the documented
400 was really post-routing behaviour. It now validates the raw request target,
and the injected transport preserves it instead of handing over a normalized one.

Contract: the prefix rule names backslash (PostgreSQL's default LIKE escape,
and the one an implementer skips because a legal key cannot contain it, though
a prefix can), and acquiring a claim MUST leave the bytes present — a
de-duplicating server that inserts only the claim can race reclamation of the
last previous claim and leave a document resolving to a 404.

Also: base URLs are checked for an http(s) scheme at construction, the ref
domain moved next to the KV key domain and is enforced by both asset backends
with shared suite coverage, the KV client refuses a hook-supplied Content-Type
like the asset one, and CI runs the package typecheck that makes the
`@ts-expect-error` guards mean something.

* fix(storage): probe guard spelling, prefix validator, base url shape

Three one-line-level holes from the closing review.

The probe-host guard was spelled `startsWith('//')`, so the backslash forms
`/\host/x` and `/\/host/x` reached the origin comparison, passed it by naming
the probe host, and were stopped only by that host failing to resolve. It now
refuses any second character that can open an authority — which is also what
makes the comment true rather than aspirational: a path-absolute reference
always inherits the base's origin, so with no authority expressible the
comparison cannot be satisfied cross-origin, whatever the probe host is.

The conformance server validated `?prefix` with the path-segment validator,
rejecting `keys('.')` and `keys('..')` — legitimate prefixes of keys like
`.hidden`, which the client rightly allows. A prefix travels in the query
string and is never a path segment, so it now has its own validator: the key
rules minus the non-empty and dot-segment requirements. The contract says the
same, and the shared suite pins both backends to it.

Base URL validation moved into one shared helper both clients call, since two
copies of a security rule is how they drift. It now also refuses
`https:host/path` — which parses standalone to that host but is resolved
against the current document by `fetch`, sending requests and their credentials
somewhere the operator never wrote — and a base carrying a query or fragment,
which silently swallows the path concatenated onto it.

* fix(storage): close two holes in the shared base-url check

The path-only branch returned its input unexamined, so `//evil.example` and
`/\evil.example` were accepted as paths. A browser reads both as an authority,
which means every request built on such a base — and every credential the
headers hook attaches — leaves the origin. A base is now required to be a
single `/` followed by something that is not `/` or `\`, the same refusal the
resolved-URL check already makes, for the same reason. Control characters are
refused first: the parser strips or re-encodes them, and `/<tab>/evil.example`
becomes `//evil.example` after it does.

The query/fragment check read the parsed `search` and `hash`, which are empty
strings for a base ending in a bare `?` or `#` — so exactly the inputs that
swallow the concatenated path into a query or fragment were the ones that got
through. It reads the raw string now, which also makes the post-parse check
redundant rather than merely unreachable.

* fix(storage): bypass HTTP caches on reads, and require a 404 for not-found

Two mechanisms the clients were missing, on both sides.

Reads are marked `cache: 'no-store'`. The asset resolve request answers two
mutable questions — does this principal hold a claim, and which URL is valid
now — so a cached answer returns an expired signed URL or replays a negatively
cached 404 for an asset written since; the in-flight coalescing that would
otherwise cover this lives in the client and cannot reach the HTTP cache layer.
KV reads have the same problem for a different reason: `account` is by
definition the scope another device may have just written, and nothing here can
invalidate a cached copy when it does. The contracts now require `no-store` on
those responses. Asset content is left cacheable — bytes at a content-addressed
ref never change — and writes are unmarked, having nothing to read from a cache.

The not-found mappings now require the status as well as the code. A proxy or
gateway answering 401, 403 or 500 while echoing the body's error code was being
read as "this does not exist", and `null` is indistinguishable from a real
miss — so an outage reached the caller as data loss.

* fix(storage): reject unusable base urls, and make the harness meet the contract

Two configuration checks and two harness gaps.

A base URL carrying userinfo built a client that threw a native TypeError on
every operation, because Fetch refuses to construct a request from a URL with
embedded credentials — one legible error where the deployment is configured
beats a per-call failure with no explanation. A base with surrounding
whitespace was worse than useless: `new URL` strips it, so validation passed,
and then concatenation moved the space inside the URL where it is no longer
trailing and no longer stripped, breaking every request.

The conformance server now sends `Cache-Control: no-store` on the three reads
the contracts require it on. The requirement landed last round with nothing
asserting it, so the suite would have passed against exactly the implementation
it exists to catch. And `DELETE /kv/entries/{key}` refuses a request body
outright: the query and the header were already checked for a smuggled scope,
leaving the body as the one channel through which a delete could still describe
one.

* fix(storage): allow a device-routing composite to persist device-scoped state

cosarah found kvPersistStorage(httpKVStore, "device") wrongly rejected. An
HttpKVStore is a full dual-scope composite: it routes device operations to the
LocalKVStore it requires at construction and only reaches the network for
account, so device values never leave the machine. The guard tested
isLocalKVStore — "is the whole store local" — and so turned away a store that
is safe for device precisely because it is not fully local.

The guard now turns on the capability that actually matters: does the device
scope stay local. A new DeviceSafeKVStore brand (servesDeviceScopeLocally: true)
marks a store whose device writes never leave the device — held by a fully-local
LocalKVStore (which extends it) and by HttpKVStore, and declared false by
HttpAccountKV, the pure account transport with no local device backend. The
persist device overload requires DeviceSafeKVStore; the account overload is
unchanged.

The security boundary the red team opened this whole line on is untouched:
HttpAccountKV used for device is still doubly refused — its false brand is not
assignable to the true DeviceSafeKVStore requires (type), and the runtime check
rejects the cast that would erase it. The brand is structurally unforgeable, not
a flag the transport could set; and the injected device backend of a composite
still demands the stronger, fully-local LocalKVStore, so composites cannot nest
into a router loop with no local floor.

Regression test (cosarah's ask): kvPersistStorage(httpKVStore, "device")
set/get/remove reach the injected device backend, land device-scoped there, and
make zero HTTP requests. Mutation-checked: widening the brand so HttpAccountKV
is accepted, or neutering the guard, turns the device-out reject test red.

* refactor(storage): scope #1002 down to KV — remove the asset HTTP backend

Per the maintainer decision, the AssetProvider server design moves to the
global-resource-pool RFC (#1007), where AssetRef changes from content-addressed
to an allocated global id and the contract model itself changes. Continuing to
harden the content-addressed asset client here would be work on code that is
being replaced, so the asset half of #1002 is withdrawn and the PR is pure KV.

Removed (all introduced by #1002):
- docs/asset-http-contract.md
- src/asset/http.ts (HttpAssetProvider) and its index / package.json exports
- src/asset/content-ref.ts (assertAssetRef / computeAssetRef / the ref shape)
- test/http-asset-provider.test.ts and the asset half of the conformance server

Reverted to the #858 baseline, untouched:
- src/asset/browser.ts — back to opaque resolve/remove, no assertAssetRef guard.
  This directly removes cosarah's latest asset finding: an opaque ref such as
  `bucket/path` no longer throws, restoring the DSL AssetRef-opaque contract.
- test/asset-contract.ts, test/asset-browser.test.ts — 0-diff from origin/main.

Kept (all KV): kv-http-contract.md, src/kv/* (HttpKVStore / HttpAccountKV /
the DeviceSafeKVStore brand), the KV tests, and the KV conformance server —
renamed kv-conformance-server.ts and reduced to KV routing over the shared
harness framework (body caps, no-store, scope-channel and dot-segment guards,
auth hooks). base-url.ts stays; KV still uses it.

* fix(storage): treat KV keys as opaque; document account-sync prerequisites

cosarah found the key validation we added for traversal defense breaks
legitimate callers: DSL identifiers (stageId/sceneId/learnerKey) are
unconstrained strings — verified, the DSL only requires typeof === 'string' —
and app callers compose keys like `editor-current-scene:<stageId>`, so a valid
stage id `stage/one` now throws in BrowserKVStore before localStorage.

Fix, matching the asset-opaque precedent: a KV key is an opaque string again.
The `/` and `\` rejection is removed from assertKVKey / assertKVKeyPrefix and
from the conformance server. Traversal is defended by encoding, not rejection —
the HTTP client already percent-encodes the whole key into one path segment, so
a separator never opens a new segment on the wire, and the server stores the
decoded key as a plain map key (bound value), never a path. `a%2F..%2F..%2Fb`
decodes to the single opaque key `a/../../b`, which addresses one entry and
traverses nothing; a raw unencoded `..` path produces extra segments that match
no route (404). The rules that remain are only what encoding cannot cover and
the transport cannot carry (empty, whole-key `.`/`..`, NUL, lone surrogate,
length) — none producible by `prefix + id`.

Tests: the shared suite now round-trips keys with `/`, `\`, `%`, `:`, spaces
across both backends; a `stage/one` key round-trips through the real conformance
server; the traversal input is proven to store as one opaque key and the raw
path to 404. Mutation-checked: re-adding the `/` rejection reddens exactly the
round-trip cases.

Docs (kv-http-contract.md):
- "A key is an opaque string" replaces the identifier-shape section — keys may
  contain separators; traversal is an encoding + server-bound-value duty.
- Account settings sync now documents its security prerequisite next to the
  account scope: a deployment MUST provide real per-user isolation of stored
  rows and encryption at rest before enabling sync; the reference server is not
  that posture.
- kvPersistStorage(device) doc corrected from LocalKVStore to DeviceSafeKVStore
  (the device-safe brand accepts the HttpKVStore composite; HttpAccountKV stays
  double-rejected).

* fix(storage): close the scope/key/prefix contract gaps with a coverage matrix

Three completeness gaps, nailed by an executable matrix so an untested
channel/shape can't slip through the next pass rather than patched one by one.

1. Scope via a GET body was not rejected. `assertNoScopeChannel` covered the
   query and header, but the bodyless GET routes never read their body, so
   `GET /kv/keys` with `{"scope":"device"}` returned 200 []. Bodyless methods
   (GET, DELETE) now reject any request body up front — that is the channel a
   scope would hide in — and only `set` reads a body, checking it for a scope
   field. The DELETE route's own body check folds into the shared one.

2. The 512-byte key bound wrongly rejected legitimate keys. A DSL id is an
   unconstrained string (verified: the DSL requires only typeof === 'string'),
   and `editor-current-scene:` + a 500-char stage id is 521 bytes, which
   BrowserKVStore rejected before localStorage. Keys are length-unconstrained;
   the bound is raised to a generous 8 KiB DoS ceiling and documented as a
   resource guard, not part of the key domain — not a derived/hashed key, which
   would break the literal-prefix semantics of keys(prefix). Both backends and
   the conformance server move in lockstep.

3. The prefix contract text contradicted itself: it claimed a legal key can
   never contain a backslash, while the key section (correctly) makes keys
   opaque, backslash included. Reworded — a prefix is opaque like a key — and
   the shared literal-prefix matrix now covers `\` (PostgreSQL's LIKE escape),
   which it had skipped, alongside `%`, `_`, and combinations.

Coverage matrix (CI-executable):
- scope-rejection: {get,set,remove,keys} × {query,header,body}. query/header
  run in-process; the body cells use a raw HTTP client (fetch cannot put a body
  on a GET — exactly the non-conforming client the finding described) against a
  loopback listener, skipped where the sandbox forbids binding.
- key-validity: `/ \ % _ : .` space, BMP + astral Unicode, `%2F`, a 500-char
  composed key, and all-at-once — each round-trips (set/get, keys(prefix)
  match, remove) on both browser and HTTP backends via the shared suite.
- prefix-charset: `%`, `_`, `\`, and combinations match literally, both backends.
- Boundaries still rejected: whole-key `.`/`..`, NUL, lone surrogate, empty,
  and the DoS ceiling.

Mutation-checked: removing the bodyless-body rejection reddens the three GET/
DELETE body cells; lowering the ceiling back to 512 reddens the long-key
round-trip on both backends.

Bumps @openmaic/storage to 0.1.2 (src/kv/* changed).

* fix(storage): bound the KV key DoS ceiling by encoded size, not UTF-8 bytes

Follow-up to the 8 KiB ceiling: measuring it in UTF-8 bytes ignored
percent-encoding expansion and reopened a backend-parity break. A non-ASCII key
is small in bytes but large on the wire — `€` is 3 UTF-8 bytes and `%E2%82%AC`,
9, encoded — so `'€'.repeat(2730)` (8190 bytes, under the old cap) encoded to
~24.6 KB, which the Node HTTP server rejects with 431 before routing while
`BrowserKVStore`, which never builds a URL, accepted it.

The ceiling is now measured on `encodeURIComponent(key).length` — the size the
key occupies as a request target — at 4096, far above any real id (a 500-char id
under `editor-current-scene:` encodes to ~523) and well below Node's default
16 KiB `maxHeaderSize` (empirically the 431 trigger on Node 22), so no key that
passes validation can reach a 431. Both backends and the conformance server use
the same encoded-size limit, so a key is accepted iff its encoded form fits,
whichever backend it reaches — parity restored. `MAX_KV_KEY_BYTES` becomes
`MAX_KV_KEY_ENCODED_LENGTH`.

Tests, against a real node:http server for the 431 boundary:
- the max-legal non-ASCII key (455×€, encoded 4095) round-trips over real HTTP
  with no 431, and the browser accepts the same key — parity on accept;
- one code point over (456×€, encoded 4104) is rejected by both backends at
  validation, before any request — the exact parity case a byte bound missed
  (1368 bytes, would have passed, then 431'd over HTTP);
- a raw client bypassing client validation with an over-ceiling (under 16 KiB)
  encoded key gets a clean 400, not a 431.
Shared suite's key-domain matrix now bounds by encoded size on both backends.

Mutation-checked: reverting the ceiling to UTF-8 bytes reddens the three
encoded-length parity tests. Bumps @openmaic/storage to 0.1.3 (src/kv/* changed).

* fix(storage): make the KV key domain truly opaque; keep transport limits out

The recurring key-limit findings (512 → 8 KiB → 4096, each patch exposing the
next) shared one root cause: transport-driven rules kept being added to the key
domain — the browser primitive included — breaking the "opaque unconstrained
string key" contract. This is the structural fix, per cosarah: "keep HTTP
transport limits out of the browser primitive and preserve the existing string
key domain."

Approach (a), the minimal one: a KV key is any string, validated nowhere. The
browser primitive is a Map/Storage that stores any key; the HTTP backend carries
the key as a URL path segment and inherits that transport's limits, documented
as HTTP-deployment concerns that never constrain the key domain. Every real key
(a `prefix:id` from an unconstrained DSL string) round-trips on both backends;
only pathological keys hit a transport limit, and none of those are keys a caller
produces. (b) — keys in the request body for accept-parity — was unnecessary:
verified that no real key needs it.

Removed from the key domain (browser + HTTP + conformance server):
- the encoded-size / length ceiling entirely — there is no key-length limit;
- the NUL exclusion — NUL percent-encodes to %00, is preserved end to end, and
  is a legitimate Map/Storage key (a false "cannot represent" premise);
- the `.`/`..`/empty/`/`/`\` rejections — all opaque, all round-trip.
`assertKVKey` / `assertKVKeyPrefix` / `MAX_KV_KEY_*` are gone.

Kept, at the transport layer only (HttpAccountKV), never reaching the primitive:
- an unpaired UTF-16 surrogate genuinely has no percent-encoding, so the HTTP
  client surfaces a clear transport error (KEY_NOT_ENCODABLE) while the browser
  stores the same key — verified: encodeURIComponent throws on it.

Scope completeness (P2): the "reject scope in every channel" guarantee had two
holes — a scope path segment (`/kv/device/keys`) 404'd instead of 400, and only
`x-scope` was checked among header spellings. The server now rejects a scope
path segment and the whole prohibited header set (scope, x-scope, kv-scope,
x-kv-scope). The coverage matrix is {get,set,remove,keys} × {path, query, each
header spelling, body}.

Coverage matrix reworked to verify arbitrary opaque keys (empty, NUL, control
chars, separators, long, Unicode) round-trip on both backends — not to verify a
limit. A real node:http test proves NUL and long keys round-trip end to end, and
that a lone surrogate is a transport limit (browser accepts, HTTP client errors).
Mutation-checked: re-adding a NUL rejection, removing the path-segment check, or
dropping a header spelling each reddens the matching cells.

Cleanup: package.json description updated (KV HTTP delivered, not "later");
network-server bind/skip/close lifecycle deduped into one helper. Bumps
@openmaic/storage to 0.1.4 (src/kv/* changed).

* fix(storage): reject whole-key "."/".." in the HTTP client — they alias entries

Follow-up to making the key domain opaque: removing the `.`/`..` key-domain
rejection was right (the browser stores them fine), but the HTTP client then let
them reach `fetch`, where URL path normalization silently rewrites the target
before it is sent. `encodeURIComponent('.')` is `.`, so `/kv/entries/.` collapses
to `/kv/entries/` (the empty key) and `/kv/entries/..` walks up a level — so
`get`/`set`/`remove` on a whole-key `.` or `..` silently read, overwrite, or
delete a *different* entry, with no error. A data-correctness bug.

Fix, same pattern as the unpaired-surrogate limit already handled: a whole-key
`.`/`..` is something the URL-path transport structurally cannot carry, so the
client's path-segment encoder refuses it with `KEY_NOT_ENCODABLE` before a
request is built. Split the encoder in two: `encodeKeyPathSegment` (key → path
segment, rejects whole-key `.`/`..`) and `encodeComponent` (the `keys()` prefix →
query value, where `.`/`..` are legitimate and untouched). A key that merely
*contains* a dot (`a.b`, `prefix:id`) is unaffected and round-trips. The browser
primitive still stores `.`/`..` opaquely.

The HTTP transport's limits are now complete — three cases, all fail-loud at the
transport layer, all storable on the browser: (1) an unpaired surrogate (no
percent-encoding), (2) a whole-key `.`/`..` (URL normalization aliases it), and
(3) an over-long key (deployment request-target ceiling). The contract lists all
three as HTTP-deployment limits, not key-domain constraints.

Tests: whole-key `.`/`..` — browser stores and round-trips (distinct from the
empty key); HTTP client throws `KEY_NOT_ENCODABLE` and builds no request; and a
real-server test asserts the empty-key entry is never read, overwritten, or
deleted by a `.`/`..` operation. A dot-containing key (`a.b`) round-trips.
Mutation-checked: dropping the whole-key check reddens the alias tests.

Bumps @openmaic/storage to 0.1.5 (src/kv/* changed).

* fix(storage): reconcile the settings/profile persist consumer with the KV API

The integration merge (#1002 KV into main) surfaced two things the merge itself
could not resolve mechanically:

- `lib/store/kv-persist.ts` (added on main by #1001) called
  `kvPersistStorage(kv, scope)` with a *variable* scope, which #1002's overloads
  deliberately reject — `'device'` must be paired with a device-safe store, so
  the overloads take the scope as a literal. Branch on the scope: `'account'`
  takes any `KVStore`; `'device'` narrows the backend to `DeviceSafeKVStore` via
  its brand (the adapter re-checks at runtime) and otherwise fails loud. The app
  wires only `'account'` today; the `'device'` arm keeps the generic helper
  honest. `ControllableKV` in the persist test wraps a local `BrowserKVStore`,
  so it is genuinely device-safe and now declares the brand.

- `packages/@openmaic/storage` version bumped 0.1.5 → 0.1.6: the merge changed
  storage inputs (README + a pg-schema test carried from main) relative to the
  previous integration tip, so the version-bump guard requires an increase above
  #1002's 0.1.5. Still ahead of main's 0.1.1.

Verified: root `tsc --noEmit` clean, the persist test green (40 passed), and the
storage package suite green on Node 22.
2026-07-30 22:31:17 -04:00

17 KiB
Raw Permalink Blame History

KVStore HTTP contract

This contract exposes the KVStore operations — get / set / remove / keys(prefix?) — over JSON HTTP for the account scope only. All paths below are relative to a deployment-defined base URL. Path segments and query parameter values are percent-encoded UTF-8 strings. Request and response bodies use application/json; successful operations with no return value respond with 204 No Content.

The conformance server in this package is test-only. It implements no authentication or authorization model beyond what the contract's response codes require; deriving the principal from an authenticated session is the reference server's job.

The scope is not on the wire

KVStore distinguishes two scopes. account is user data a server-backed deployment syncs across devices (provider and model configuration, profile); device is machine-local state (theme, locale, layout) that must never leave the machine. That is a property of the primitive, not a deployment setting.

This contract therefore has no scope at all: no scope path segment, no scope query parameter, no scope header, no scope body field. There is nothing to set to device, so there is no configuration or client bug that could ship a device value to a server — a request carrying a device value is not a request this contract can express. Servers MUST reject a request that invents a scope in any of those channels with 400 VALIDATION_FAILED rather than ignoring it, so a client attempting to describe a scope fails loud instead of silently having its intent discarded. "Any channel" is exhaustive, and each of the four is a place a real client or proxy might reach for a scope:

  • Path segment. The contract's segment after kv is entries or keys; a scope name there (/kv/device/keys, /kv/account/entries/{key}) is an attempt to route by scope and MUST be 400, not a 404 route miss. (A key named device is unaffected — that sits after entries, at /kv/entries/device.)
  • Query parameter. A ?scope=… MUST be rejected.
  • Header. There is no single canonical spelling, so the server MUST reject the whole prohibited set — at least scope, x-scope, kv-scope, and x-kv-scope. Checking only x-scope leaves the others open.
  • Body. get, remove, and keys are bodyless: a server that routes them without reading the body would let GET /kv/keys with {"scope":"device"} succeed, the guarantee failing silently. So a bodyless method (GET, DELETE) MUST reject any request body outright, and only set reads a body, where a scope field is rejected.

Every one of {get, set, remove, keys} × {path, query, header spellings, body} is closed, and the shared suite pins each cell.

The client half mirrors this in three layers, only the last of which actually carries the weight:

  1. HttpAccountKV is the only object in @openmaic/storage that can reach this contract, and its scope parameter admits 'account' alone — a literal 'device' at a call site is a type error. It also refuses one at runtime, because the type is not enough: TypeScript compares method parameters bivariantly, so this transport stands in for the wider KVStore wherever one is expected, and a caller holding it that way can pass any scope. Silently dropping that argument is how a device value reaches a server, so the transport takes the scope and fails loud on anything but account.
  2. HttpKVStore — the full KVStore — is the one place the two scopes are told apart. An unrecognized scope is refused rather than folded into the account path: guessing would send a value the caller believed was device-local to a server.
  3. HttpKVStore requires a LocalKVStore for the device scope — a store branded as keeping its values on this machine. KVStore alone is not enough, because a networked store satisfies it structurally: HttpAccountKV is a KVStore with one optional parameter fewer, so a KVStore-typed parameter would accept the very transport this design exists to exclude. The brand makes that a type error, and a runtime check refuses a remote store that a cast smuggled past the types.

There is no default and no fallback, so a deployment cannot end up with device values that have nowhere local to go.

The same pairing rule reaches the zustand adapter, which is where a store and a scope are chosen as separate arguments and therefore the easiest place to pair them wrongly: kvPersistStorage(store, 'device') requires a DeviceSafeKVStore at the type level and checks the brand at runtime. DeviceSafeKVStore is the capability that actually matters — "a device write stays on this machine" — and it is broader than LocalKVStore: it admits both a fully-local backend and the HttpKVStore composite, which routes device to its injected local backend while sending only account to a server. What it excludes is the pure HttpAccountKV transport, which has no local device backend and would put a device value on the wire; that store declares the capability false and is double-rejected.

Account settings are only as isolated as the deployment makes them

The account scope is where a server-backed deployment syncs a user's settings across devices, and the contract's principal derivation keeps one account from addressing another's keys over the wire. That is a routing guarantee, not a storage one. Before enabling settings sync, a deployment MUST provide real per-user isolation of the stored rows (the (principal, key) partition enforced end to end, not merely by a query filter a bug could drop) and encryption at rest for the values, which hold user configuration and profile data. The reference server is a reference: it demonstrates the principal derivation, not a production isolation or encryption posture, and MUST NOT be exposed with real user data as-is.

A key is a truly opaque, unconstrained string

A KV key is any string — of any length, containing any character, including NUL, separators, and things that look like structure. The KVStore primitive imposes no key-domain rules whatsoever: callers compose keys from unconstrained DSL identifiers (the DSL constrains ids only to be strings), and the browser primitive is a Map / Storage, which stores any string key. A rule in the key domain — a length cap, a / ban, a NUL ban — would be a transport concern reaching back into the primitive, and would make a key the browser can perfectly well store unreadable. There is no such rule, and no key validation. A server likewise MUST NOT reject a key for its content; it stores the decoded key as an opaque value — a bound parameter or a derived, constrained storage id — never as a path component or an unescaped fragment of a query, so a key like a/../../b (arriving percent-encoded as a%2F..%2F..%2Fb) is one opaque entry that traverses nothing.

Validity is a property of the key; reachability is a property of the transport. The two are kept separate, and this is the parity guarantee: a key is valid on every backend identically (all accept it), even where it is not reachable over a particular transport.

  • The browser backend uses no transport and imposes nothing — every key round-trips.
  • The HTTP backend carries a key as a URL path segment, so it inherits exactly three limits of that transport. Each is an HTTP-deployment concern, not a key-domain constraint, each makes the affected key fail loud rather than silently misbehave, and the browser backend stores every one of them fine:
    1. An unpaired UTF-16 surrogate has no percent-encoding — encodeURIComponent throws — so the client refuses it with KEY_NOT_ENCODABLE before building a request.
    2. A whole-key . or .. is normalized away by URL path parsing before the request is sent (/kv/entries/. collapses to the empty-key segment, /kv/entries/.. walks up a level), so sending it would silently alias a different entry. The client refuses these two whole keys with KEY_NOT_ENCODABLE as well. This is only the whole key: a key that merely contains a dot (a.b, prefix:id) is an ordinary segment and round-trips, and a ./.. prefix in a keys() query is unaffected (it is a query value, not a path segment).
    3. An extremely long key may exceed a deployment's request-target / header size ceiling (Node's default is ~16 KiB, above which the server answers 431 before routing). No length is invalid; a long key is simply not reachable past that transport bound.

None of these arise for the keys real callers produce (prefix:id from a DSL string). All are the URL transport's, not the key's; a deployment that must accept literally any key over HTTP would carry the key outside the URL path. This contract takes the simpler route and fails loud on the three cases above, and the browser backend is unaffected by all of them.

Endpoints

Method Path Purpose Success
GET /kv/entries/{key} Read one value. 200 with { "value": <json> }, or 404 KEY_NOT_FOUND
PUT /kv/entries/{key} Write one value from { "value": <json> }. Replaces any existing value. 204
DELETE /kv/entries/{key} Delete one value. Idempotent; deleting an absent key succeeds. 204
GET /kv/keys List the principal's keys. Optional ?prefix={prefix} returns only keys starting with it. 200 with string[]

Both read routes MUST be served with Cache-Control: no-store, and MUST NOT be cached by any intermediary. account is by definition the scope another device may have just written, and neither the client nor the server has a way to invalidate a cached copy when that happens — a cached read serves precisely the stale state this scope exists to move past. The client sends its reads with cache: 'no-store' for the same reason. Writes carry no such requirement, having nothing to read from a cache.

The value travels in an envelope rather than as the body itself. KVStore.get cannot distinguish a stored null from an absent key — both are null to a caller — but the wire still has to be unambiguous: a bare null body would be indistinguishable from an empty body or a stored null, leaving the framing to depend on Content-Length. The envelope makes "there is a value, and it is null" a statement the response can make. get maps 404 KEY_NOT_FOUND back to null, matching KVStore.get; the client MUST use the machine-readable code, not the status alone, so a 404 ROUTE_NOT_FOUND stays an error instead of masquerading as a missing key.

GET /kv/keys returns every matching key. The listing is not paginated, matching GET /documents in the DocumentStore HTTP contract; the primitive is sized for small configuration values, and a deployment that outgrows an unbounded listing needs a change to KVStore itself, not a second listing shape here.

The prefix is a literal, byte-for-byte comparison, not a pattern. A prefix is as opaque as a key: it too travels in the query string rather than as a path segment, so it may be empty (that is what "list everything" means), may be . or .. (legitimate prefixes of keys such as .hidden), and may contain any character a key may — %, _, \, /, and the rest. Only the transport-fatal characters and the DoS ceiling constrain it.

Because both the prefix and the keys it matches are opaque, a server MUST escape every metacharacter of its own query language before applying the comparison. In the obvious SQL translation, key LIKE prefix || '%', that means % and _, and the backslash — PostgreSQL's default LIKE escape character. Backslash is the one an implementer is most likely to skip, precisely because it is easy to assume a key could never contain one; it can — a key is opaque — and so can a prefix, and both reach the query. Escape all three, or the literal comparison silently becomes a pattern match. The result order is unspecified, but the listing MUST NOT repeat a key.

Value domain

Values travel through JSON, so this backend accepts only plain JSON values that survive serialization without changing meaning. It MUST fail loud before sending values such as Map, Set, Date, non-finite numbers, negative zero, nested undefined, bigint, sparse arrays, symbol-keyed properties, non-enumerable properties, strings containing U+0000, class instances, and circular references. U+2028 and U+2029 are valid JSON string contents and MUST be accepted. Keys are held to the same rule, because a key becomes a URL path segment.

This is intentionally narrower than BrowserKVStore, whose JSON.stringify would quietly rewrite a Date into a string or drop a nested undefined. Where the browser backend is silently lossy, the HTTP backend refuses.

Both backends agree on one deliberate exception: set(key, value) where value has no JSON representation at all (undefined, a function, a symbol) is a removal, not a write. Storing such a value would produce an entry that throws on read, so the key is deleted instead.

Every backend decides that by inspecting the value, never by trial-serializing it, and this is a rule about all of them rather than an HTTP detail. A JSON.stringify pre-flight runs caller code — toJSON, getters — before any validation has looked at anything, which reclassifies values as deletes ({ toJSON: () => undefined } stringifies to undefined) and lets a stateful accessor show the probe one value and the serializer another. Two backends probing and validating in different orders would disagree about whether a write was a delete, which is precisely the divergence the shared suite exists to prevent. Scope and key are validated first, then the delete case is recognized by type, and only then is anything serialized. A value that still fails to serialize is refused rather than quietly deleted.

Principal derivation

The principal is derived server-side from the authenticated session and never appears in a path, query parameter, or body. This is the same non-negotiable that governs learnerKey in the RuntimeStore HTTP contract: a client-submitted principal is not proof of identity, and trusting one turns every route here into a lateral-authorization vulnerability that lets one account read or overwrite another's configuration.

Every route requires an authenticated principal; there is no anonymous KV. Because keys are caller-chosen and this store holds user configuration, a deployment MUST scope every operation — including GET /kv/keys — to the derived principal, and MUST NOT expose a cross-principal listing on this contract.

Errors

Every non-2xx response has this machine-readable JSON shape:

{
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "@openmaic/storage: kv write body must carry \"value\"",
    "details": []
  }
}

details is optional.

Condition HTTP status Error code Client behavior
Malformed JSON, a scope in any channel, a missing value member, or a non-JSON value (a key is opaque and never a validation failure) 400 VALIDATION_FAILED Throw HttpKVStoreError with the server message
A key cannot be percent-encoded for the URL transport (an unpaired surrogate) — a transport limit, not a rejection by the server — KEY_NOT_ENCODABLE Client-side only; throw HttpKVStoreError before any request. The browser backend stores the key
Request body exceeds the deployment's size bound 413 PAYLOAD_TOO_LARGE Throw HttpKVStoreError
Request target exceeds the deployment's size ceiling 431 (transport) An HTTP-transport limit for a pathologically long key, not a key-domain rejection
No entry is stored under the key 404 KEY_NOT_FOUND get returns null
Route does not exist 404 ROUTE_NOT_FOUND Throw HttpKVStoreError
Missing or invalid credential 401 UNAUTHENTICATED Throw HttpKVStoreError
Principal may not perform the operation 403 FORBIDDEN_KV Throw HttpKVStoreError
Unexpected server failure 500 INTERNAL_ERROR Throw HttpKVStoreError; the handler does not expose internal details

Only KEY_NOT_FOUND becomes null. Status alone is not sufficient — ROUTE_NOT_FOUND shares its status and means a broken deployment, and a 401 or 403 must never be reported as a missing key.

A response the client cannot interpret — a body that is not JSON despite a 2xx status, a get body without a value member, a keys body that is not an array of strings — raises MALFORMED_RESPONSE, a client-side code with no server counterpart. It is a typed storage error like any other: the client never lets a native SyntaxError escape in its place.

Retry and atomicity guarantees

Every operation is safely retryable. PUT is a whole-value replacement and therefore idempotent for the same body; DELETE is idempotent because deleting an absent key succeeds; both GET routes are reads. The contract offers no compare-and-swap: a concurrent write is last-writer-wins per key, which suits the small independent configuration values this primitive exists for. Callers needing an atomic read-modify-write should not model that state as KV.