wyuc 1c5078842a
feat(storage): RuntimeStore — sessions + append-only records, browser backend (#869 Part B) (#880)
* feat(storage): RuntimeStore contract — interface + backend-agnostic suite (#869)

* feat(storage): BrowserRuntimeStore — IndexedDB backend for the runtime layer (#869)

* test(storage): runtime version-skew coverage; run the storage suite in CI (#869)

Backend-specific tests seed raw session rows (mirroring the document
backend's reStampStage) to cover the version-skew surface the public API
cannot express: a stale stamp fails loud while the runtime ladder is empty,
a future stamp reads through unchanged but rejects appendRecord and
setSessionStatus, and a stamp-stripped row hits the no-unversioned-epoch
guard. Also covers the injected payloadValidators override (the contract
case deferred to the backend test, as document-contract does for scene
validators).

The @openmaic/storage suite — including the pre-existing DocumentStore /
KV / asset contracts — never ran in CI at all; add the missing step after
the dsl one. README gains the RuntimeStore section.

* fix(storage): harden RuntimeStore writes and listings per cross-review (#869)

- mergeLearner now takes the same gates as every other write: rejects
  empty learner keys, self-merges return 0 without a transaction,
  future-stamped rows throw (mutating a newer client's data), and every
  re-keyed envelope is validated before put — any throw aborts the whole
  merge atomically instead of contaminating the target partition.
- listSessions sorts by the instant a timestamp denotes (Date.parse),
  not by string order, which mis-ranks numeric zone offsets; and it
  tolerates corrupt rows by omission (the listDocuments precedent) so
  one poison row cannot make a partition unenumerable — direct reads
  stay fail-loud.
- appendRecord validates the ACTUAL completed record (with the assigned
  seq) inside the transaction, on top of the pre-flight, and derives seq
  from a key cursor's primaryKey instead of deserializing the previous
  record's payload.
- Docstrings now enumerate the corrupt-stamp throw paths and no longer
  claim every query is partition-scoped (mergeLearner is the one
  deliberate cross-stage sweep).
- The stale-row test comment no longer promises an impossible one-line
  flip; the IDBKeyRange shim moves to test/setup.ts alongside the other
  guarded shims.

* fix(storage): mergeLearner migrates stale sessions before re-keying (#869)

The re-key previously spread the RAW stored row, unlike setSessionStatus
and appendRecord which migrate a stale session in place before mutating
it. Once the runtime ladder gains a real step, that would either write an
old runtimeDslVersion into the target partition or fail validation when a
migration adds required fields. Build the updated row from
migrateSession(row) instead (order: future-guard, migrate, re-key,
validate, put).

Observable today (empty ladder): a below-epoch stale row makes the merge
abort with the ladder's no-migration-path error — pinned by a new backend
test with nothing moved.

* fix(storage): reads gate stored sessions through envelope validation (#869)

Reads previously ran only version resolution, so a row with a valid
runtimeDslVersion but corrupt other fields (a non-ISO createdAt, a bogus
status) came back as-is — contradicting the documented corrupt-row
semantics, which covered only version corruption. Extend the same policy
to the whole envelope: getSession validates the migrated row and fails
loud as a stored-row integrity error; listSessions omits rows whose
envelope validation fails, exactly like version-resolution failures.

setSessionStatus and appendRecord need no change: the status update
validates the full post-update envelope before writing (catching corrupt
non-status fields), and appendRecord's own writes are already validated.
2026-07-09 00:03:51 +08:00
..

@openmaic/storage

The MAIC pluggable persistence layer: small, swappable-backend primitives for persisting app state, depending only on @openmaic/dsl.

The DSL owns what persists (document / runtime shape + validation + migration + the asset StorageProvider interface). This package owns where / how it persists — the primitives and their backends. The pluggable seam is the backend, not the database driver: a browser backend (zero server, the clone-and-run default) and, later, an HTTP backend whose server owns a database.

Dependency arrow (acyclic)

@openmaic/storage -> @openmaic/dsl

No dependency on React, zustand, or any host app. Backends take their Storage / IDBFactory by injection, so the package is app-agnostic and testable without a browser.

What's in here

Export Role Browser backend
KVStore small device / account-scoped values not owned by the DSL BrowserKVStore over localStorage
StorageProvider (from @openmaic/dsl) the asset seam: put(blob) → ref, resolve(ref) → url, remove(ref) BrowserAssetProvider over IndexedDB + object URLs
kvPersistStorage adapt a KVStore into a zustand persist storage —
DocumentStore persist the DSL document aggregate (stage + scenes + embedded agents / quiz / actions + an outline snapshot) BrowserDocumentStore over IndexedDB (normalized stages / scenes / outlines)
RuntimeStore persist what a learner produces while taking a course — sessions + append-only records (chat, quiz attempts, playback facts) BrowserRuntimeStore over IndexedDB (sessions / records)
  • Scopes. account values are user data a server-backed deployment syncs across devices; device values (theme, locale, layout) never leave the device — every backend honours that, so the scope is part of the primitive, not the backend choice.
  • Content-addressed assets. BrowserAssetProvider refs are sha256-<hex>, so identical bytes de-duplicate to one stored asset. A document embeds only the stable ref; the provider resolves it to a URL at render time (a raw URL would bake in a provider + expiry and break portability).
  • Document normalization. The DSL document is a portable embedded aggregate; DocumentStore normalizes it into per-entity rows so scene-level writes (putScene) stay cheap, and reassembles it on read. Each document is stamped with a dslVersion; reads run the DSL migration ladder forward, and writes are validated against the DSL gate (validateStage / validateScene) so schema drift fails loud. The outline is an opaque, app-owned snapshot carried alongside — persisted verbatim, neither validated nor migrated.
  • Generic over scene type. DocumentStore<TScene> defaults to the DSL Scene (universal slide / quiz). An app that widens Scene with its own kinds (interactive / pbl, content the DSL does not own) parameterizes the store over its scene union and injects a matching validateScene, so those scenes persist and the gate stays fail-loud for the app's shapes.
  • Runtime layer. RuntimeStore is partitioned by (stageId, learnerKey): a stage has many sessions — one or more per learner — so every listing is partition-scoped (there is deliberately no global listing; single-session operations are id-keyed, and mergeLearner is the one deliberate cross-stage sweep). Sessions are born stamped: the store writes runtimeDslVersion itself at createSession, and the runtime line has no unversioned epoch, so an unstamped row fails loud instead of being lifted like a legacy document. Records are append-only ordered facts under an active session; the store assigns the per-session monotonic seq on append — the sole replay ordering key, never timestamps. Record payloads are gated per kind by injectable validators, defaulting to the DSL skeleton guards for chat / quizAttempt (playback and app-defined kinds carry app-owned payloads). mergeLearner re-keys an anonymous learner's sessions to a signed-in key across all stages; deleteLearnerRuntime cascades one learner's sessions + records on one stage, and deleteStageRuntime clears a whole stage — the hook a document deletion cascades through.

Backend equivalence

Each primitive has one implementation-agnostic contract suite (test/kv-contract.ts, test/asset-contract.ts, test/document-contract.ts, test/runtime-contract.ts). Every backend is proven by running the same suite against it, so a new backend (the coming HTTP one) cannot silently diverge from the primitive's semantics.

Roadmap

  • KVStore + browser backend; zustand persist adapter
  • StorageProvider (in @openmaic/dsl) + browser BrowserAssetProvider
  • implementation-agnostic contract suites
  • DocumentStore (aggregate ↔ normalized adapter, migrate-on-read via the DSL migration registry, validation gate) + browser backend
  • RuntimeStore (sessions + append-only records, runtime version line, per-kind payload gate) + browser backend
  • wire the app's zustand stores + ad-hoc localStorage through KVStore
  • HTTP backend + reference server + one HTTP contract

License

MIT