6.2 KiB
Configurable final delivery directory
The continuous-processing settings now include a required delivery root in the same connection/bucket as intake. The UI previews:
<delivery root>/<project name>__<project ID>/<capture name>/<part name>/
clips/, calibration/ and package/evidence manifests are direct children of each part directory. Every part creates an independent Run using the same workflow instance, with its own annotation, quality review and acceptance sampling. A single part also retains its directory. Flat legacy sources omit the part layer. Run IDs are not added to the delivery path.
Part-index delivery mode
The package node selects the delivery shape with the PART_INDEX_DELIVERY hyperparameter (default false).
false— the directory form above: sliced media per kept slice, calibration copies, package/evidence manifests.true— one JSON document per part. The part directory holdspart.jsonplus the shared completionmanifest.json. No sliced media, PTS, IMU, pose or calibration bytes are delivered; downstream packaging reads the index and resolves media from the pinned materialization revision.
The index carries both split levels in one document: the task-level split (slices[].taskSplit) and the action-level split (slices[].segments). Every reviewed interval is classified exactly once — kept intervals in slices[], discarded intervals in discardedSlices[] with their discardEvidence. Discarded intervals are now visible to the customer by design; earlier directory deliveries omitted discard reasons entirely.
clips/ is absent in this mode, so consumers must branch on the completion manifest kind (part-index versus directory) instead of assuming a directory layout. The index carries the per-slice poseDelivery include/exclude decision, but never pose frame data itself — frames stay in the pinned materialization revision for downstream resolution.
Migrating an already delivered part
An already published directory delivery keeps its clips/, calibration and evidence files. scripts/part_index_migration.py produces the candidate part.json for such a part from a local copy of the delivered directory plus the accepted materialization revision. It reuses the package operator's own builder, so a migrated index is byte-identical to what an online part-index package would have produced for the same evidence — apart from a lineage.migratedFrom provenance marker.
The tool never writes object storage, and the worker publication path cannot publish that candidate into a destination whose completion manifest already exists: rewriting a customer's completion marker in place is refused (a same-owner, same-bytes retry of an existing part-index manifest is the one accepted idempotent case). A part that was already delivered in the directory form therefore gains the index through an explicit re-delivery of that part, never through an in-place patch of the delivered directory. If the part already has a part.json with different bytes, the candidate is refused as well.
Safety and persistence
- API validates disjoint input/output prefixes (including other workflow listeners), rejects reserved system prefixes and probes write/delete permission before enabling intake. The project directory name is server-owned and reused across subsequent configuration updates.
- New admissions freeze connection, bucket and complete target prefix on the Capture record. Existing snapshots are not changed by settings edits or project renames.
- Older tasks with a frozen no-part target are not silently moved. Publication rejects ancestor/descendant overlap with another task's frozen delivery location; resolve the old binding explicitly before publishing into an overlapping location.
- An old task without a delivery snapshot fails publication explicitly. It is not silently migrated and cannot fall back to the Run directory. No existing output is moved automatically.
- The package operator still prepares and validates files in its internal workspace. The worker publishes the
delivery.directory.indexartifact, or thepart.jsonartifact in part-index mode, into the frozen destination. Other intermediate artifacts keep their existing storage policy. .delivery-owner.jsonreserves the destination using conditional creation. Every member also uses conditional creation and metadata SHA-256/size verification. Retrying identical publication is idempotent; changed content or a different owner is rejected, not overwritten. Both delivery modes share one owner marker per destination, so a part cannot be delivered twice by different attempts.- A part-index publication refuses to adopt a destination that already holds files without an owner marker. Additive migration into an existing legacy directory therefore requires that directory to carry its owner marker; otherwise the destination is rejected rather than claimed.
- Files can be visible during publication, but
manifest.jsonis written last and is the authoritative completion marker. The business task cannot complete until publication succeeds. This is not a claim of atomic BOS directory rename. packages-manifest.jsonis published after data members.manifest.jsonindexes every published member with size and SHA-256. The directory form additionally publishes package and evidence manifests; the part-index form publishes the index alone. Neither output is a ZIP.- Temporary upload objects stay under
system/runs/<run>/delivery-staging/and successful uploads remove them. Interrupted staging is covered by normal Run cleanup. - Project and workflow deletion protect configured/frozen final delivery destinations. Deleting source business records is not permission to delete delivered data.
- Capture records show the frozen destination and an “open delivery directory” link after delivery.
Release boundary
API and worker must be deployed together. The earlier part-layout change additionally needs both updated directory/normalizer operator images. Do not deploy only the frontend or start a new scanner with old operators that ignore its explicit file selection. Part-index mode additionally requires the worker that understands the part.json routing block: an older worker would upload the index as an ordinary run artifact instead of honoring the frozen destination.