wyuc 0a10af11cf
feat(storage): opt-in indirect asset byte egress (#1007) (#1100)
* feat(storage): add opt-in indirect asset byte egress

A deployment can now opt an asset byte GET into a 302 to a short-lived
signed URL when the byte layer can sign. It is off by default, and
byte-for-byte unchanged when off. The byte layer gains an optional
signReadUrl capability; S3AssetByteStore implements it through the new
optional @aws-sdk/s3-request-presigner peer, resolved lazily exactly
like the client SDK, with a signer seam mirroring the commands seam so
tests can bind doubles. The signed URL pins the contract's response
headers -- the media type after the renderable allowlist, the fixed
disposition, and the no-store cache posture -- via S3 response-header
overrides. PgAssetStore mints the URL from the same ownership-checked
transactional read a direct resolve performs, so authorization still
runs per read before any URL exists, and a store whose byte layer
cannot sign falls back to direct bytes. The contract documents the
shape and the disclosure tradeoff: a Location into a hash-keyed byte
layer names the content hash, so deployments that need the
no-disclosure property keep direct egress.

Refs #1007

* feat(app): wire ASSET_BYTE_EGRESS into the persistence route

ASSET_BYTE_EGRESS=redirect opts asset byte GETs into the storage
package's indirect egress; unset, direct, or an unrecognized value
keeps the default byte-for-byte behavior, with a warning for the
unrecognized case. The lazy byte-store wrapper now forwards
signReadUrl and answers undefined when the resolved layer has no
signer, so the PostgreSQL byte column degrades to direct bytes and
the S3 layer signs without the wrapper knowing which it holds. The
collector path is untouched: it builds its byte store through
createAssetByteStore as before and only ever calls delete.

Refs #1007

* fix(storage): let the packaged client survive redirect egress

Cross-review round 1 on the indirect byte egress change found two
problems.

A byte response that followed a redirect carries the pinned content
type but no X-Asset-Revision, and HttpAssetStore required both from
the same response, so every cold resolve under redirect egress failed
MALFORMED_RESPONSE. The client now detects the shape once -- a
redirected byte response with no revision -- latches it, and probes
HEAD before each cold GET from then on, taking the label from the
probe. The probe predates the download on purpose: a replacement in
between labels newer bytes with an older revision, which the next
revalidation detects and corrects, while the reverse order could pin
stale bytes under a fresh revision with no signal to repair it. The
probe doubles as the miss check, so a redirect-mode miss costs no
download. The direct path is byte-for-byte unchanged; the one double
download per client lifetime on first contact is documented in the
contract, which gains the client-side paragraph this behavior
implements.

signedUrlTtlSeconds also accepted lifetimes beyond the seven-day
SigV4 presigning maximum, minting redirects the object store would
reject. The option is now capped at construction.

Refs #1007

* fix(storage): cap signed URL lifetime below the reclamation grace

A signed URL whose lifetime exceeds the collector grace period can
outlive its object: the last reference goes, the grace elapses, the
collector deletes the object, and the still-valid URL errors at the
object store. The previous ceiling -- the seven-day SigV4 maximum --
made that window days wide against the one-hour default grace. The
cap is now fifteen minutes, and the contract text states both the
ceiling and the rule for deployments that shorten their grace.

Refs #1007

* fix(storage): decline signing when the optional presigner is absent

An unresolvable @aws-sdk/s3-request-presigner made signReadUrl throw,
which the registry surfaced as a failed asset read -- a 500 for a
deployment whose only fault is a missing optional peer. The contract's
answer for a byte layer that cannot sign is the capability fallback:
return undefined and let the caller serve the bytes directly. Signing
errors from a resolved signer still fail loud; only the missing
capability declines.

Refs #1007

* fix(storage): answer redirect egress as a descriptor to asking clients

Following a 302 is not header-neutral: the platform fetch forwards the
original request's headers to the object store's origin, stripping only
Authorization, so a deployment whose credential travels in a custom
header would hand it to the object store -- and the preflight the
forwarded custom headers provoke commonly fails there besides. This
replaces the latch-and-probe client shape with an explicit one: the
client sends X-Asset-Egress: descriptor on every byte GET, a
redirect-egress server answers 200 with a JSON { url, revision } body
instead of a Location, and the client fetches the signed URL with no
deployment headers at all. The revision comes from the descriptor, so
the probe-first ordering and its one wasted download are gone too. The
302 remains the answer for consumers that did not ask; the descriptor
response is marked by its own header so a JSON-media asset can never
parse as one.

Refs #1007

* fix(storage): negotiate the descriptor through Accept, not a custom header

A custom X-Asset-Egress request header is not CORS-safelisted, so every
byte GET to a cross-origin server became a preflighted request -- a
regression for direct-egress deployments that never opted into
anything, against servers with no reason to allow the header. The
negotiation now rides Accept with a vendor media type, which is
safelisted and costs no preflight, and the descriptor answer is
identified by that media type as its Content-Type rather than by a
marker header.

Refs #1007

* fix(storage): sign outside the registry transaction and reserve the descriptor type

Two review findings on the indirect egress path.

resolveIndirect awaited the signer inside the coordinated read, but a
signer on refreshable credentials can wait on the network, and a
database connection plus the blob row's FOR SHARE lock would be held
across that delay. The read now closes first -- the lock still puts
the hash, label and revision in one snapshot -- and signing runs
afterwards, which cannot observe anything the read did not.

The descriptor media type is also reserved from renderableTypes: the
client identifies a descriptor answer by that exact Content-Type, and
an allowlisted asset served with it would be misread as a descriptor
and fail MALFORMED_RESPONSE. The handler now rejects the configuration
at construction, and the contract states the reservation.

Refs #1007

* fix(storage): ship the signing deps and negotiate without ambiguity

Three review findings on the indirect egress path.

The app now declares @aws-sdk/client-s3 and
@aws-sdk/s3-request-presigner as runtime dependencies, and the
standalone build force-includes them for the persistence route: both
are reached through deliberately untraced dynamic imports, so the
shipped image could not resolve them, and a deployment opting into S3
or redirect egress would have found the capability silently absent.

The descriptor negotiation matches exactly on both sides: the client
compares the Content-Type essence, so a longer media type that merely
begins with the reserved value still serves as bytes, and the server
parses Accept media ranges, so a descriptor range sent with q=0
selects the redirect instead of the descriptor it explicitly rejects.

Refs #1007

* fix(app): refuse redirect egress when the collection grace is shorter

A signed URL must never outlive its object: with the handler's
60-second default lifetime, a deployment that sets
ASSET_COLLECTION_GRACE_MS below ten minutes lets the collector delete
an object while a URL minted against it is still valid. The route now
refuses the combination at handler initialization, naming both
variables -- a deterministic misconfiguration, loud at startup rather
than silent at read time.

Refs #1007

* fix(app): ship the signing SDKs without loading them

Two findings on the standalone deployment of redirect egress.

The outputFileTracingIncludes globs did not follow pnpm's scoped
symlinks, so the standalone image carried neither AWS package and both
S3 mode and redirect egress could not resolve their SDK at runtime.
The packages are now server-external and referenced from literal --
but never called -- dynamic import thunks in the persistence byte-store
wiring, which gets them traced into the image while module resolution
still happens only on first S3 use. Verified against a local
standalone build: node_modules/@aws-sdk now contains both packages,
and the route tests that pin the never-resolve-unless-configured
discipline still pass.

Media type comparisons are also case-insensitive now, as HTTP
requires, on both the Accept parse and the descriptor recognition.

Refs #1007

* fix(app): treat an empty collection grace as unset

Number('') is 0, so a present-but-empty ASSET_COLLECTION_GRACE_MS
failed the redirect-egress coordination check and took persistence
down with it, while the collector's own parsing reads the same value
as the one-hour default. The check now trims and treats empty as
unset, matching durationEnv.

Refs #1007

* fix(storage): accept bytes too, omit signing on the known-PG wrapper, state the full tradeoff

Three review findings.

The client's descriptor request now advertises both representations --
the vendor descriptor type preferred, */* accepted -- so a strict
content-negotiating layer can never answer 406 to a client that does
in fact consume plain bytes.

The lazy byte-store wrapper no longer advertises signReadUrl when no
bucket is configured: the layer is known to be PostgreSQL at
construction, and carrying the method made resolveIndirect take its
ownership query and blob-row lock just to decline, then repeat them
in resolve, on every cold GET. With a bucket configured the lazy
validation is preserved and the wrapper still declines when the
resolved layer cannot sign.

And the contract's disclosure tradeoff now states the second edge:
object-store GETs carry ETag and Last-Modified, response overrides
cannot strip them, and because deduplicated PUTs rewrite the
hash-keyed object, Last-Modified moves when another principal
re-uploads the same bytes -- shared-object write timing, beyond byte
equality. Deployments for whom that signal is sensitive must keep
direct egress.

Refs #1007

* fix(app): degrade on bad grace, share the ttl invariant, cover the real boundary

The latest review round, four findings.

A misconfigured ASSET_COLLECTION_GRACE_MS now degrades redirect
egress to direct with a warning instead of failing the shared
handler's initialization -- the asset backend is optional, and its
misconfiguration must never take document and runtime traffic down.

The package exports assertSignedUrlTtlWithinGrace so a deployment
wiring the handler and the collector separately validates the
invariant once at its own boundary: the handler caps the lifetime,
but only the deployment knows both numbers.

The contract states that nosniff does not survive the redirect --
response-header overrides cannot set it -- and why the client-minted
blob makes that safe for conforming clients.

And the route gains a real-boundary test: the Fetch<->Node adapter,
the real createStorageHttpHandler, and the egress wiring, exercised
with an actual byte GET through the composed stack instead of mocks.

Refs #1007

* fix(storage): make the unsafe egress configuration unrepresentable

Three findings on the indirect egress path, all resolved by moving a
rule into a place where it cannot be bypassed rather than by adding a
check a consumer has to remember.

The grace/TTL invariant now lives in the option shape. byteEgress takes
'direct' or { mode: 'redirect', collectionGraceMs, signedUrlTtlSeconds? },
so enabling indirect egress without declaring the reclamation window it
lives inside no longer type-checks, and the handler validates both
numbers at construction. The flat signedUrlTtlSeconds option, its
cross-field "requires byteEgress redirect" check, and the standalone
900-second cap as the only guard all go away; the ceiling stays, but
alongside the ratio rather than in place of it. This is what the earlier
exported assertSignedUrlTtlWithinGrace could not do -- it protected only
consumers who called it -- and it costs nothing in compatibility because
the option is new in this branch. The app resolves the grace through one
shared parser with the collector, so the number the handler checks is the
number the collector runs with.

A signed URL whose object is gone is now a miss. The client maps a 404
from the byte fetch to ASSET_NOT_FOUND, matching the direct path: the
entry was owned and readable when the URL was minted, so a reclaimed
object is the same physical state the direct read reports as a miss, and
what a read means must not depend on the deployment's egress setting.
Signing still does not probe for the object -- that would restore the
round trip this feature removes and make the mint's price vary with prior
presence -- so the contract instead requires the byte layer to answer 404
for an absent object, and states that S3 needs s3:ListBucket to do so.

Route coverage now spans the whole egress matrix through the real
composed handler rather than only the direct case: 302 for a consumer
that did not ask, descriptor for one that did, direct bytes when the
byte layer declines to sign, and direct bytes when a short grace degraded
the configured mode.

Refs #1007

* feat(storage): forward byte egress options through reference server

* fix(storage): fail closed on redirects and unconfirmed 404s in indirect byte egress

A signed-fetch 404 now maps to a miss only when the object store's XML
body declares NoSuchKey; every other 404 from the signed fetch fails loud,
so a wrong bucket, access point, or endpoint can no longer make a live
asset read as absent. The descriptor byte GET is sent with redirect:
'manual' and any 3xx answer is treated as an error, so a server that
ignores the descriptor negotiation can never forward the deployment's
custom credential headers to a redirect target.

The ASSET_BYTE_EGRESS switch is now documented in .env.example and the
server-persistence deployment section, together with the object-store CORS
and s3:ListBucket prerequisites it requires, and the asset descriptor
media type is exported from the package root and the asset/http subpath.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-08-14 23:57:21 +08:00

117 lines
13 KiB
Markdown

# RuntimeStore, DocumentStore, and AssetStore reference server
The `@openmaic/storage/server` subpath exports a Node-only HTTP request handler implementing the [RuntimeStore HTTP contract](./runtime-http-contract.md), plus the [DocumentStore HTTP contract](./document-http-contract.md) and [AssetStore HTTP contract](./asset-http-contract.md) when their stores are supplied. It accepts injected store implementations; the runnable `@openmaic/storage/server/reference` composition creates and initializes `PgRuntimeStore`, accepts optional host-created document and asset stores, and demonstrates the required node-postgres checkout/transaction/release pattern.
The OpenMAIC application also mounts these same composed handlers as an
app-integrated Next.js route at `/api/persistence`. That embedded route is the
deployment form used by the repository's `server-persistence` Compose profile;
it changes the Fetch/Node request boundary only, not either HTTP contract.
This module is a reference, not a production authentication service. **The example bearer authentication is fully impersonatable.** A production host must supply its own authenticated identity and authorization policy. It must also terminate TLS, bound request sizes and timeouts, rate-limit abusive clients, keep database credentials outside the process image, and expose the service only through an appropriate application gateway.
## Deployment
Build the package and run the compiled entrypoint in a host that provides `pg`:
```sh
DATABASE_URL=postgres://user:password@host/database PORT=3000 \
node packages/@openmaic/storage/dist/server/reference.js
```
The executable `main()` binds to `127.0.0.1`; `createReferenceRuntimeServer()` only creates and returns an unbound Node `Server`. The default bearer token payload is used directly as the demo `learnerKey`, self-merge is the only allowed merge, and admin operations are denied. Supplying `documentStore` adds the DocumentStore routes to that same server; any authenticated principal is allowed by default, or `authorizeDocuments` can enforce deployment policy. The factory also accepts `authenticate`, `authorizeMerge`, `authorizeAdmin`, and validator overrides. Replace the policy hooks before exposing a deployment:
- `authenticate(req)` must validate a real credential and derive canonical learner and asset partition keys from server-controlled identity state.
- `authorizeDocuments(principal, req)` must establish that the principal may access the requested author document operation. The default permits every authenticated principal.
- `authorizeAssets(principal, req)` must establish that the principal may access the requested asset operation. The default permits every authenticated principal carrying an asset key; registry ownership remains mandatory on every operation.
- `authorizeMerge(principal, fromKey, toKey)` must explicitly establish that the principal may migrate the complete source partition into the destination identity. Default denial is intentional.
- `authorizeAdmin(principal)` must require a separately protected administrative role. Default denial is intentional.
The handler's `payloadValidators` option has the same whole-table replacement semantics as the `BrowserRuntimeStore` and `PgRuntimeStore` constructor option, and defaults to the DSL `chat` / `quizAttempt` skeleton table. Whatever you pass to the store, pass the same thing to the handler. `createReferenceRuntimeServer()` applies its `payloadValidators` override to both automatically. Its `maxBodyBytes` option applies to runtime and document routes and defaults to 32 MiB; oversized bodies receive `413 PAYLOAD_TOO_LARGE`.
The package has no PostgreSQL driver runtime dependency. A host injects its `Pool` (or another compatible `Queryable`) and owns driver lifecycle. Every transactional operation must check out a fresh connection, issue `BEGIN`, run all callback queries on that same connection, issue `COMMIT` or `ROLLBACK`, and release it in `finally`.
## Document endpoints
Every document route requires an authenticated principal. By default,
`authorizeDocuments` allows any authenticated principal; production deployments
must replace that policy when documents are tenant-, role-, or author-scoped.
| Method | Path | Purpose |
| --- | --- | --- |
| `PUT` | `/documents/{stageId}` | Save a complete document |
| `GET` | `/documents/{stageId}` | Load a complete document |
| `GET` | `/documents` | List document summaries |
| `DELETE` | `/documents/{stageId}` | Delete a document and its children |
| `PUT` | `/documents/{stageId}/stage` | Replace stage metadata |
| `PUT` | `/documents/{stageId}/scenes/{sceneId}` | Upsert one scene |
| `GET` | `/documents/{stageId}/scenes/{sceneId}` | Read one scene |
| `DELETE` | `/documents/{stageId}/scenes/{sceneId}` | Delete one scene |
The reference composition applies its `validateScene` and `validateStage`
overrides to the HTTP handler; the host must configure the injected document
store with the same validators. Keep those validators in sync when composing
the lower-level handler yourself. Full response, validation, version, and retry
semantics are specified in the
[DocumentStore HTTP contract](./document-http-contract.md).
## Asset endpoints
Every asset route requires an authenticated principal with an asset partition key. Supplying `assetStore` adds these routes to the composed server; `authorizeAssets` can apply an additional deployment policy before any registry entry is read. Supplying `byteEgress` forwards the option unchanged to the composed asset handler: `{ mode: 'redirect', collectionGraceMs }` opts into indirect byte egress for `GET /assets/{id}/content`, and omitting it keeps direct egress. The full tradeoff is specified under [Indirect byte egress](./asset-http-contract.md) in the asset HTTP contract.
| Method | Path | Purpose |
| --- | --- | --- |
| `POST` | `/assets` | Allocate a new id and store bytes |
| `GET` | `/assets/{id}/content` | Read bytes and identity headers |
| `HEAD` | `/assets/{id}/content` | Read identity headers without bytes |
| `PUT` | `/assets/{id}/content` | Replace bytes behind an existing id |
| `DELETE` | `/assets/{id}` | Remove an entry; absent and foreign ids are no-ops |
Writes use bounded `multipart/form-data`; reads are private and uncached. The full media-type allowlist, size limits, response headers, and client snapshot rules are specified in the [AssetStore HTTP contract](./asset-http-contract.md).
## Endpoint authorization matrix
The matrix treats learner, merge, and admin credentials as separate capabilities. An admin-only or merge-only credential does not implicitly own a learner partition; a deployment may combine capabilities, but every applicable check still has to pass.
| Method and path | No credential | Owning learner | Other learner | Merge-authorized | Admin-authorized |
| --- | --- | --- | --- | --- | --- |
| `POST /runtime/sessions` | Deny (`401`) | Allow | Deny (`403`) | Deny | Deny |
| `GET /runtime/sessions/{sessionId}` | Deny (`401`) | Allow | Not found (`404`) | Deny (`403`) | Deny (`403`) |
| `PATCH /runtime/sessions/{sessionId}/status` | Deny (`401`) | Allow | Not found (`404`) | Deny (`403`) | Deny (`403`) |
| `DELETE /runtime/sessions/{sessionId}` | Deny (`401`) | Allow | Not found (`404`) | Deny (`403`) | Deny (`403`) |
| `GET /runtime/stages/{stageId}/learners/{learnerKey}/sessions` | Deny (`401`) | Allow | Deny (`403`) | Deny | Deny |
| `POST /runtime/sessions/{sessionId}/records` | Deny (`401`) | Allow | Not found (`404`) | Deny (`403`) | Deny (`403`) |
| `GET /runtime/sessions/{sessionId}/records` | Deny (`401`) | Allow | Not found (`404`) | Deny (`403`) | Deny (`403`) |
| `POST /runtime/learners/merge` | Deny (`401`) | Deny by default | Deny by default | Allow | Deny |
| `DELETE /runtime/stages/{stageId}/learners/{learnerKey}` | Deny (`401`) | Allow | Deny (`403`) | Deny | Deny |
| `DELETE /runtime/stages/{stageId}` | Deny (`401`) | Deny (`403`) | Deny (`403`) | Deny (`403`) | Allow |
| `DELETE /runtime` | Deny (`401`) | Deny (`403`) | Deny (`403`) | Deny (`403`) | Allow |
| `POST /assets` | Deny (`401`) | Allow in own partition | Allow in own partition | Deny without asset key | Deny without asset key |
| `GET /assets/{id}/content` | Deny (`401`) | Allow for own id | Not found (`404`) | Deny without asset key | Deny without asset key |
| `HEAD /assets/{id}/content` | Deny (`401`) | Allow for own id | Not found (`404`) | Deny without asset key | Deny without asset key |
| `PUT /assets/{id}/content` | Deny (`401`) | Allow for own id | Not found (`404`) | Deny without asset key | Deny without asset key |
| `DELETE /assets/{id}` | Deny (`401`) | Allow | Allow as no-op | Deny without asset key | Deny without asset key |
## Threat model
`learnerKey` is an opaque partition key, never a credential. An attacker can alter path segments and JSON bodies, so trusting a submitted key enables lateral movement: reading another learner's sessions or records, writing records into their sessions, changing status, or deleting their data. The handler authenticates every contract operation and compares stored or submitted learner ownership before touching learner-scoped data. Direct stage / learner partition mismatches return `403 FORBIDDEN_LEARNER`; missing credentials return `401 UNAUTHENTICATED`.
Session-scoped routes deliberately conceal whether another learner's session ID exists. A credential with a different `learnerKey` receives the same `404 SESSION_NOT_FOUND` as an absent session, and ownership is checked before future-version classification so version metadata cannot disclose existence. A principal with no `learnerKey` is different: it lacks the learner capability entirely and receives `403 FORBIDDEN_LEARNER` on every learner-scoped route.
An asset id is an identifier, not a bearer token. Every asset operation rechecks the authenticated principal's ownership, including every byte read. An id belonging to another principal and an id that was never allocated are indistinguishable on every route: reads and replacements return the same fixed `404 ASSET_NOT_FOUND`, while deletion succeeds as the same no-op.
Merge is a privilege-escalation boundary because it rewrites every source session across every stage. Merely owning either key is insufficient in a real identity system: the authorization hook must verify the account-linking or identity-upgrade proof for both the source and destination. The default is deny.
Stage cascade deletion and whole-runtime deletion are admin-plane capabilities. If exposed to ordinary learners they can erase every partition on one stage or across the entire runtime store, so both routes are controlled by the separate admin authorization hook and denied by default. Production systems should isolate admin credentials, audit decisions, protect against confused-deputy use, and avoid deriving admin authority from a learner-controlled claim.
Authentication failures and authorization denials should be logged without recording bearer credentials or sensitive request payloads. Operators should monitor repeated cross-learner denials, merge attempts, and admin-plane calls as possible account-enumeration or privilege-escalation signals.
### Concurrency semantics during merge-time deletion
`mergeLearner` and `deleteSession` may race. If deletion verifies ownership and a merge changes ownership immediately afterward, the resulting deletion is equivalent to the legal serial order in which delete happens before merge. The handler therefore re-reads and re-checks ownership immediately adjacent to the delete call to shrink the race window; this is treated as a linearizable ordering case, not as an API or schema defect. A store-level conditional delete such as delete-if-owner could further harden this boundary in the future.
### Version and concurrent-write classification
Reads return future-stamped sessions and their records without applying a version gate, while every delete endpoint remains version-agnostic so it can clean up data written by a newer client. Only the three mutating operations that can rewrite session-owned data apply the future-version guard: status updates, record appends, and learner merges return `409 FUTURE_VERSION` rather than mutating a future-stamped session.
Status updates and record appends authorize and validate the session before entering the backing store's write transaction. If the store then rejects the write because a concurrent operation deleted or completed the session, the handler re-fetches the session instead of classifying the failure from driver or error-message text. A now-absent session returns `404 SESSION_NOT_FOUND`; a session whose current status is no longer `active` returns `400 VALIDATION_FAILED` and reports that status. If the structured re-fetch still finds an active, current-version session, or if the re-fetch itself fails, the original failure remains an undisclosed `500 INTERNAL_ERROR`.