🔀 Integrate delivery v4 with layered frontend while preserving legacy materialization

This commit is contained in:
d-robotics 2026-10-22 00:24:05 +08:00
commit 3549ad6216
68 changed files with 3749 additions and 178 deletions

View File

@ -0,0 +1,33 @@
# 最终分类交付与内部追溯分离
Status: done
2026-10-21 用户要求将两轮总 JSON 核查的问题一起本地修复,并明确选择独立 `tags` 列表、管理员维护。依据为马迪 10-09 至 10-14 的格式对话与本会话补充。原始核查证据在主仓 `local-data/diagnostics/delivery-json-audit-20261021/`。
## 验收条件
- 字典词条独立 tags,不从 category/aliases 猜补。选择字典后 ID、类型、版本、名称、编码、tags 经服务器校验并冻结,切片/动作修订、物化、审查及最终 JSON 保持一致;字典后改不改写已确认结果。
- 未写动作描述仍保留切片基础分类和全部区间。显式无效与舍弃独立,不从空文本推断;舍弃原因是分类事实,保留。纯留空、混合、全无效及全舍弃均可表达。
- 新分类交付使用显式 v4 协议及默认关闭的 DELIVERY_JSON_V4(要求 PART_INDEX_DELIVERY)。客户主 JSON 保持 capture/part、slices/taskSplit/segments 的约定形态,只含定位、最终分类、字典快照、边界及必要源引用;动作 contentKind 使用马迪约定的 annotated/invalid_marked/gap_placeholder。
- v4 全舍弃 part 仍输出分类 JSON。旧目录/v1/v2/v3 的 no_deliverables 行为、历史文件和在途任务兼容保留,不静默重写旧批次。
- 交付只取最终验收通过的固定物化修订与摘要;返工正式修正须重新通过验收。批注、人员、轮次变化不能改分类,审核者不能通过标注接口修改业务结果。
- 全部已提交追溯留内部 JSON,按 capture/part/结果版本关联:审核人、批注、轮次、退回/复审及正式处理记录。客户主 JSON 不带审核历史、lineage、抽检过程或内部路径引用,客户下载不包含内部追溯文件。
- 不可变批次的选择/身份/SHA/范围/条件发布/重试规则保留;版本必须显式识别,拒绝混搭和不一致证据。历史缺少字典标签不得按最新字典猜补,保留已有名称事实和未知字段边界。
- 舍弃 reasonNote/at 与 note/discardedAt 兼容映射;原始 meta/标定路径基准显式且一致。规范、真实入口样例与校验器枚举一致。
## 验证
定向 Go/Python/前端回归,隔离本机 Mongo 与真实 router 的保存→返工→复审→验收→原生打包→批次及下载,检查最终分类/标签和完整内部追溯;覆盖失败重试、重启/旧版本读取、字典更新、未授权篡改、全舍弃、缺失历史证据。前端与 Go 全量门禁按既有基线对照;真实浏览器检查字典 tags 管理和选择,Web/Go 构建及生产包启动。
不上传镜像、不部署、不写公司数据、不补写历史交付、不 push/PR。本地完成后以新分支提交。
## 本地完成证据(2026-10-21)
- 独立分支 `codex/delivery-classification-20261021`,基于 `bc00467d`。本次只做本地代码、合成数据验证及提交;未 push、提 PR、上传镜像、部署、启用工作流或改写历史交付。
- Go 与前端全量门禁无新增失败,保留原有 70 / 13 个已知失败,基线文件未修改;类型、格式、构建、生成源码一致性及生产页面启动通过。
- 定向 Python 37 项通过(含真实视频物化→v4 的 5 项验证);一个旧版 Go 物化 opt-in 用例未单独启用,新旧真实媒体链路已由独立用例覆盖。
- 本机 27018 随机独立 Mongo 库的三轮退回和 API 重启、原生 v4 的正式接口批次发布/失败恢复/重试/下载、停用与移除词条后的直接及面板撤销重做、v5/v6 标签防伪造均通过。测试库执行后清理。
- 静态复核的 Standards / Spec 两轴发现均已修正并关闭;保留无快照历史的名称事实,不以当前 tags 猜补。未确认临时名称不能绕过最终确认/提交。
- 完整日志、分离的客户/内部样例和浏览器截图保留在本工作树 `local-data/verification/delivery-classification-20261021/`。客户样例子目录 `customer-samples/partNNNN/` 每份只含 `part.json`;`customer-samples/internal/` 不属于客户包。
- 新协议默认关闭,线上尚未验证/启用;后续正式发布需要覆盖 API/processor、Worker、package 镜像及内部 artifact 存储,按现有发布授权流程执行。未把旧镜像 release 摘要伪装成新源码摘要。

View File

@ -0,0 +1,21 @@
Implement approved local v4 contract only. Freeze source ref metadata in revision; require complete customer v4 original source fields. Derive tags from own dictionary or parent frozen dictionary without overwriting selection provenance. Explicit timestamps + local/part seconds, half-open time bases and original PTS resolution. Consumer roundtrip uses only exported JSON + synthetic original source. Legacy formats remain unchanged. Commit local only, no deploy.
Status: resolved
Acceptance:
- Effective tags and action/slice origin are explicit; independent selections and invalid flags remain intact.
- part → slices → segments hierarchy and discard-reason-only semantics remain intact.
- Frozen original BOS references survive metadata-only reuse; no connection credentials or workspace paths enter customer output.
- Action local/part times and hardware PTS boundaries are complete and continuous.
- Missing tags, source, timestamps, source files or conflicting identities reject v4 package/batch publication.
- Independent reader resolves every synthetic original camera frame from JSON alone in physical and source-index modes.
- Immutable retry/recovery, separate private audit, legacy formats and local Mongo publication remain verified.
Compatibility: v4 default stays off; no production changes, retrospective tag guessing, old delivery replacement, downstream recutting or source writes.
Verification:
- 73 relevant Python tests passed, including physical/source-index independent real-media consumption, legacy v1/v2/v3 and 60-second materialization.
- Native Python v4 output passed Go batch/API publication, failure recovery and download tests with an isolated local Mongo 27018 database.
- Go full baseline gate: the same 70 named known failures, no new failures and no baseline edits.
- Operator repository generation check and git diff whitespace check passed.
- Local evidence: `local-data/verification/delivery-consumer-20261021/`; final customer samples under `final-samples/{physical,source-index}/customer/part0001/part.json`.

View File

@ -2,6 +2,14 @@
以下词汇描述人工标注与审核中的业务对象。
**分类标签**:管理员为字典词条明确维护的 tags 集合,用于按已确认类别归组;它不同于词条分类 category、别名 aliases 或动作描述文本。确认后的词条身份、版本和标签共同构成当次分类快照,字典后续更新不改写该快照。
**最终分类结果**:最终验收通过的固定版本中,视频各区间的基础分类、动作细分、显式无效及舍弃判断。未写动作描述不撤销基础分类;审核批注和返工过程不属于分类结果。
**动作有效标签**:动作独立选择描述字典词条时,使用当次选择冻结的标签;未独立选择时,继承所属切片冻结的分类标签。是否填写自由描述不决定继承,无效判断与标签分别保留。
**原始区间定位**:以原始采集及 part 的文件身份和硬件时间戳定位人工确认区间;切片内相对时间与原始 part 相对时间是不同坐标,不能混用。下游按定位和分类处理原始数据,不依赖平台内部审核文件。
## 标注对象
**Capture(源采集)**:一次源采集所关联的双目视频、时间戳与传感器数据,是本流程组织人工处理和审核的对象。

View File

@ -5,6 +5,7 @@ import hashlib
import bisect
import csv
import json
import math
import pathlib
import re
import shutil
@ -752,9 +753,11 @@ def _part_index_discarded_slice(
"reason": str(evidence.get("reason") or ""),
"reasonLabelId": str(evidence.get("reasonLabelId") or ""),
"reasonName": str(evidence.get("reasonName") or ""),
"note": str(evidence.get("note") or ""),
"note": str(evidence.get("note") or evidence.get("reasonNote") or ""),
"reasonVersion": evidence.get("reasonVersion"),
"dictionary": evidence.get("dictionary"),
"discardedByUserId": str(evidence.get("discardedByUserId") or ""),
"discardedAt": str(evidence.get("discardedAt") or ""),
"discardedAt": str(evidence.get("discardedAt") or evidence.get("at") or ""),
},
},
"segments": [],
@ -843,13 +846,184 @@ def _description_status(actions: list[dict[str, Any]]) -> str:
return "empty_confirmed"
def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
"""Use the established immutable directory publisher for the two JSON files.
def _v4_tags(selection: Any, kind: str) -> list[str]:
if not isinstance(selection, dict) or selection.get("kind") != kind or not str(selection.get("id") or "").strip() or type(selection.get("version")) is not int or selection["version"] < 1 or not str(selection.get("name") or "").strip():
raise RuntimeError(f"v4 missing frozen {kind} dictionary; confirm historical classification before delivery")
tags = selection.get("tags")
if not isinstance(tags, list) or not tags or any(not isinstance(tag, str) or not tag.strip() or tag != tag.strip() for tag in tags) or len(set(tags)) != len(tags):
raise RuntimeError(f"v4 missing frozen {kind} tags; confirm classification before delivery")
return list(tags)
def _v4_original_sources(manifest, capture_id, part_id, capture_prefix):
source = manifest.get("originalSource")
if not isinstance(source, dict) or source.get("provider") != "BOS" or not str(source.get("bucket") or "").strip():
raise RuntimeError("v4 requires frozen original BOS source selection; workspace paths cannot identify original data")
prefix = str(source.get("prefix") or "")
part_prefix = str(source.get("partPrefix") or "")
if prefix != capture_prefix or part_prefix != prefix + "/" + part_id or source.get("captureUnit") != capture_id + "/" + part_id:
raise RuntimeError("v4 original source identity disagrees with accepted capture/part")
if not prefix or str(pathlib.PurePosixPath(prefix)) != prefix or pathlib.PurePosixPath(prefix).is_absolute() or ".." in pathlib.PurePosixPath(prefix).parts or any(c in prefix for c in ("\\", "\x00")):
raise RuntimeError("unsafe v4 original source prefix")
keys = source.get("objectKeys")
if not isinstance(keys, list) or not keys or any(not isinstance(key, str) for key in keys) or len(set(keys)) != len(keys):
raise RuntimeError("v4 requires an unambiguous original object selection")
files = {}
cap, part = re.escape(capture_id), re.escape(part_id)
patterns = {
"leftVideo": rf"{part}/{cap}_camera_left_{part}(?:_[a-z0-9_]+)?\.(?:mp4|mov|mkv)",
"rightVideo": rf"{part}/{cap}_camera_right_{part}(?:_[a-z0-9_]+)?\.(?:mp4|mov|mkv)",
"leftPts": rf"{part}/{cap}_camera_left_{part}(?:_[a-z0-9_]+)?_pts\.csv",
"rightPts": rf"{part}/{cap}_camera_right_{part}(?:_[a-z0-9_]+)?_pts\.csv",
"imu": rf"{part}/{cap}_imu_{part}\.csv",
"cameraCalibration": rf"{cap}_calibration_camera\.yaml",
"imuCalibration": rf"{cap}_calibration_imu\.yaml",
"metadata": rf"meta_{cap}\.json",
}
for key in keys:
if not key.startswith(prefix + "/") or str(pathlib.PurePosixPath(key)) != key or ".." in pathlib.PurePosixPath(key).parts or any(c in key for c in ("\\", "\x00")):
raise RuntimeError("unsafe v4 original source object")
relative = key[len(prefix) + 1:]
if "/" in relative and not relative.startswith(part_id + "/"):
raise RuntimeError("v4 original selection contains another part")
for role, pattern in patterns.items():
if re.fullmatch(pattern, relative, re.IGNORECASE):
if role in files: raise RuntimeError(f"ambiguous v4 original {role} selection")
files[role] = {"path": relative}
missing = sorted(set(patterns) - files.keys())
if missing: raise RuntimeError("v4 missing original files: " + ", ".join(missing))
return {"provider": "BOS", "bucket": source["bucket"], "prefix": prefix, "partPrefix": part_prefix,
"snapshotFingerprint": str(source.get("snapshotFingerprint") or ""), "files": files}
def _v4_number(value, label):
if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value):
raise RuntimeError(f"v4 missing or invalid {label}")
return float(value)
def _v4_timestamp(value, label):
if type(value) is not int or value <= 0: raise RuntimeError(f"v4 missing or invalid {label}")
return value
def _v4_consumer_payload(payload, manifest, source_root):
"""Project immutable accepted facts into a self-contained consumer contract.
No dictionary lookup, source media reads, transcoding or BOS writes here.
Hardware PTS are authoritative; media seconds are explicitly named and never
used to manufacture missing hardware timestamps.
"""
if not isinstance(manifest, dict): raise RuntimeError("v4 requires accepted source manifest")
original = _v4_original_sources(manifest, payload["capture"]["captureId"], payload["partId"], payload["capture"]["prefix"])
payload["originalSource"] = original
payload["consumerContract"] = "droboticflow.classification-consumer/v1"
payload["timeBases"] = {"taskSplitSeconds": "original_left_video", "segmentSeconds": "slice_left_video",
"segmentPartSeconds": "original_left_video", "timestampUs": "original_hardware_pts",
"interval": "half-open", "frameRanges": "zero-based-half-open"}
payload["pathBases"] = {"sources": "original_capture", "sourceMetadata": "original_capture", "calibration": "original_capture"}
# Workspace sources are private transport references, not the original files.
payload["sources"] = {kind: original["files"][kind] for kind in ("leftVideo", "rightVideo", "leftPts", "rightPts", "imu")}
payload["sourceMetadata"]["path"] = original["files"]["metadata"]["path"]
for item in payload["calibration"]:
# Bind by actual calibration kind, not input ordering.
kind = "imuCalibration" if "imu" in item["name"].lower() else "cameraCalibration" if "camera" in item["name"].lower() else ""
if not kind: raise RuntimeError("v4 unknown calibration kind")
item["path"] = original["files"][kind]["path"]
rows = {str(row["sliceId"]): row for row in manifest["items"]}
if len(rows) != len(manifest["items"]): raise RuntimeError("v4 duplicate slice identities")
for item in payload["slices"] + payload["discardedSlices"]:
split = item["taskSplit"]
start, end = (_v4_number(split[k], k) for k in ("startSecond", "endSecond"))
first, last = (_v4_timestamp(split[k], k) for k in ("startTimestampUs", "endTimestampUs"))
if start < 0 or end <= start or last <= first: raise RuntimeError("v4 invalid accepted slice range")
if item["classification"]["deliveryDisposition"] == "discard":
item["sliceName"] = ""
evidence = item["classification"]["discardEvidence"]
selected = evidence.get("dictionary")
if selected is not None:
if (not isinstance(selected, dict) or selected.get("kind") != "discard_reason" or
not selected.get("id") or not selected.get("name") or
type(selected.get("version")) is not int or selected["version"] < 1):
raise RuntimeError("v4 invalid frozen discard dictionary")
for alias, field in (("reasonLabelId", "id"), ("reasonName", "name"), ("reasonVersion", "version")):
if evidence.get(alias) not in (None, "", 0, selected[field]):
raise RuntimeError("v4 discard reason disagrees with frozen dictionary")
evidence[alias] = selected[field]
continue
parent_tags = _v4_tags(item["classification"].get("dictionary"), "slice")
item["classification"]["tags"] = parent_tags
row = rows[item["sliceId"]]
sync = json.loads((source_root / row["directory"] / "synchronized-slice.json").read_text())
seconds, stamps = sync.get("boundarySeconds", []), sync.get("boundaryTimestampUs", [])
if seconds or stamps:
if not isinstance(seconds, list) or not isinstance(stamps, list) or len(seconds) < 2 or len(seconds) != len(stamps):
raise RuntimeError("v4 invalid hardware boundary mapping")
seconds = [_v4_number(value, "boundary") for value in seconds]
stamps = [_v4_timestamp(value, "boundary timestamp") for value in stamps]
if (abs(seconds[0]) > 1e-7 or abs(seconds[-1] - (end-start)) > 1e-7 or
stamps[0] != first or stamps[-1] != last or
any(a >= b for a, b in zip(seconds, seconds[1:])) or
any(a >= b for a, b in zip(stamps, stamps[1:]))):
raise RuntimeError("v4 invalid hardware boundary mapping")
def timestamp(second):
if abs(second) < 1e-7: return first
if abs(second - (end-start)) < 1e-7: return last
hits = [stamps[i] for i, value in enumerate(seconds) if abs(_v4_number(value,"boundary") - second) < 1e-7]
if len(hits) != 1: raise RuntimeError("v4 action boundary has no original PTS mapping")
return _v4_timestamp(hits[0], "boundary timestamp")
actions = []
for action in item["segments"]:
a = _v4_number(action.get("clipStartMs"), "action clipStartMs") / 1000
b = _v4_number(action.get("clipEndMs"), "action clipEndMs") / 1000
if a < -1e-7 or b <= a or b > end-start+1e-7: raise RuntimeError("v4 action escapes its parent slice")
ta, tb = (_v4_timestamp(action.get(k), k) for k in ("startTimestampUs", "endTimestampUs"))
if not first <= ta < tb <= last: raise RuntimeError("v4 action timestamps escape its parent slice")
if seconds and (timestamp(a) != ta or timestamp(b) != tb): raise RuntimeError("v4 action seconds disagree with original PTS")
action.update(startSecond=a, endSecond=b, partStartSecond=start+a, partEndSecond=start+b)
own = action.get("dictionary")
action["tags"] = _v4_tags(own, "action") if own is not None else list(parent_tags)
action["tagSource"] = "action" if own is not None else "slice"
actions.append(action)
actions.sort(key=lambda a: a["startSecond"])
complete, previous = [], 0.0
def gap(a,b):
ta,tb=timestamp(a),timestamp(b)
return {"id":f"{item['sliceId']}:gap:{ta}:{tb}", "description":"", "isInvalid":False,
"contentKind":"gap_placeholder", "startSecond":a,"endSecond":b,
"clipStartMs":a*1000,"clipEndMs":b*1000,"partStartSecond":start+a,"partEndSecond":start+b,
"startTimestampUs":ta,"endTimestampUs":tb,"tags":list(parent_tags),"tagSource":"slice"}
for action in actions:
if action["startSecond"] < previous-1e-7: raise RuntimeError("v4 overlapping actions")
if action["startSecond"] > previous+1e-7: complete.append(gap(previous,action["startSecond"]))
complete.append(action);previous=action["endSecond"]
if previous < end-start-1e-7: complete.append(gap(previous,end-start))
if not complete: raise RuntimeError("v4 kept slice has no classified action coverage")
previous_stamp = first
for action in complete:
if action["startTimestampUs"] != previous_stamp or action["endTimestampUs"] <= previous_stamp:
raise RuntimeError("v4 action PTS coverage is discontinuous")
previous_stamp = action["endTimestampUs"]
if previous_stamp != last: raise RuntimeError("v4 action PTS coverage is incomplete")
item["segments"] = complete
counts = {kind:sum(a["contentKind"]==kind for a in complete) for kind in ("annotated","gap_placeholder","invalid_marked")}
item["classification"]["contentCounts"] = counts
item["classification"]["descriptionStatus"] = _description_status(complete)
if not counts["annotated"] and counts["invalid_marked"] and counts["gap_placeholder"]: item["classification"]["descriptionStatus"]="invalid_and_blank"
item["synchronization"]["rightBoundarySeconds"] = sync.get("rightBoundarySeconds", [])
def _emit_part_index_directory(run_id, node_id, output, node_data, helpers, payload, internal_audit=None, revision_manifest=None, source_root=None):
"""Use the established immutable publisher for versioned customer JSON.
V3 retains its historical sibling review file; v4 stores that audit in a
separate internal artifact before publishing the customer directory.
Legacy single-file publishers deliberately accept only v1/v2. A directory
artifact preserves their ownership/conditional-publication protocol without
teaching an old worker to misinterpret a new single-file schema.
"""
v4 = internal_audit is not None
capture = dict(payload["capture"])
prefix = pathlib.PurePosixPath(str(capture.get("prefix") or ""))
part_id = prefix.name
@ -860,7 +1034,7 @@ def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
if prefix.is_absolute() or ".." in prefix.parts or len(prefix.parts) < 3:
raise RuntimeError("unsafe v3 capture identity prefix")
capture.update(captureId=capture_id, prefix=str(prefix.parent))
payload.update(schemaVersion="droboticflow.delivery.part-index/v3", capture=capture, partId=part_id)
payload.update(schemaVersion="droboticflow.delivery.part-index/v4" if v4 else "droboticflow.delivery.part-index/v3", capture=capture, partId=part_id)
payload["pathBases"] = {"sourceMetadata": "original_capture", "calibration": "original_capture", "sources": "workspace"}
payload["sourceMetadata"] = {"path": f"meta_{capture_id}.json"}
history = {
@ -869,7 +1043,8 @@ def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
"qualityReviews": payload["acceptance"].pop("qualityReviews", []),
}
payload["reviewHistory"] = {"path": "review-history.json", "pathBase": "part_package"}
payload["acceptance"]["finalAcceptance"].pop("confirmedByUserId", None)
if not v4:
payload["acceptance"]["finalAcceptance"].pop("confirmedByUserId", None)
for item in payload["calibration"]:
if pathlib.PurePosixPath(item["name"]).name != item["name"] or "\\" in item["name"]:
raise RuntimeError("unsafe calibration filename")
@ -889,6 +1064,54 @@ def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
classification["contentCounts"] = counts
if not counts["described"] and counts["invalid_marked"] and counts["gap_placeholder"]:
classification["descriptionStatus"] = "invalid_and_blank"
if v4:
if not isinstance(internal_audit.get("trace"), list):
raise RuntimeError("DELIVERY_JSON_V4 requires complete internal trace input")
if _output(node_data, helpers, "deliveryAudit") is None:
raise RuntimeError("DELIVERY_JSON_V4 requires an internal deliveryAudit output")
audit = {
"schemaVersion": "droboticflow.delivery.review-history/v2",
"captureId": capture_id, "partId": part_id, "revision": payload["revision"],
"acceptance": payload.pop("acceptance"), "lineage": payload.pop("lineage"),
"qualityReviews": internal_audit.get("qualityReviews", []),
"trace": internal_audit.get("trace", []),
"acceptedMaterializations": internal_audit.get("acceptedMaterializations", []),
}
audit["acceptance"]["finalAcceptance"] = internal_audit.get("finalAcceptance", {})
payload.pop("reviewHistory", None)
payload["capture"] = {"captureId": capture_id, "prefix": str(prefix.parent)}
for item in payload["slices"] + payload["discardedSlices"]:
classification = item["classification"]
# Blank descriptions retain the independently selected base class.
if classification.get("dictionary") is None:
classification.pop("dictionary", None)
evidence = classification.get("discardEvidence", {})
evidence.pop("discardedAt", None)
if evidence.get("dictionary") is None:
evidence.pop("dictionary", None)
counts = classification.get("contentCounts")
if counts is not None:
counts["annotated"] = counts.pop("described")
for action in item["segments"]:
if action["contentKind"] == "described": action["contentKind"] = "annotated"
for key in ("source", "confidence", "phrases", "labelId"):
action.pop(key, None)
if action.get("dictionary") is None: action.pop("dictionary", None)
audit["materializedSources"] = payload.get("sources", {})
_v4_consumer_payload(payload, revision_manifest, source_root)
body = (json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2) + "\n").encode("utf8")
audit["partSha256"] = hashlib.sha256(body).hexdigest()
if len(json.dumps(audit, ensure_ascii=False, indent=2).encode("utf8")) > 8 * 1024 * 1024:
raise RuntimeError("internal delivery audit exceeds 8 MiB; refusing incomplete export")
staging = pathlib.Path(tempfile.mkdtemp(prefix="part-index-v4-", dir=output))
(staging / "part.json").write_bytes(body)
# A sibling artifact stays in the internal run, never in the customer tree.
audit_artifact = helpers["write_json_artifact"](run_id, node_id, output, "review-history.json", audit, output=_output(node_data, helpers, "deliveryAudit"))
directory = helpers["build_virtual_artifact"](
run_id, node_id, "delivery.directory.index", str(staging), "file", 0,
data={"schemaVersion": "droboticflow.directory-index/v1", "presentation": "collection", "datasetType": "directory", "dataSource": "FILESYSTEM", "rootPath": str(staging), "sliceCount": len(payload["slices"]), "classificationOnly": True},
output=_output(node_data, helpers, "deliveryDirectory"))
return [directory, audit_artifact]
staging = pathlib.Path(tempfile.mkdtemp(prefix="part-index-v3-", dir=output))
for name, document in (("part.json", payload), ("review-history.json", history)):
(staging / name).write_text(json.dumps(document, ensure_ascii=False, sort_keys=True, indent=2) + "\n", encoding="utf8")
@ -944,6 +1167,7 @@ def _part_index_slice(
"clipStartMs": _optional_float(action.get("clipStartMs")),
"clipEndMs": _optional_float(action.get("clipEndMs")),
"labelId": str(action.get("descriptionTemplateId") or ""),
"dictionary": action.get("dictionary"),
"phrases": [str(item) for item in action.get("phrases", [])] if isinstance(action.get("phrases"), list) else [],
"source": str(action.get("source") or ""),
"confidence": _optional_float(action.get("confidence")),
@ -976,6 +1200,7 @@ def _part_index_slice(
"deliveryDisposition": declared,
"taskLabel": str(synchronization.get("taskLabel") or ""),
"taskLabelId": str(synchronization.get("taskLabelId") or ""),
"dictionary": synchronization.get("dictionary"),
"descriptionStatus": _description_status(actions),
},
"segments": actions,
@ -999,9 +1224,11 @@ def _part_index_slice(
"reason": str(evidence.get("reason") or ""),
"reasonLabelId": str(evidence.get("reasonLabelId") or ""),
"reasonName": str(evidence.get("reasonName") or ""),
"note": str(evidence.get("note") or ""),
"note": str(evidence.get("note") or evidence.get("reasonNote") or ""),
"reasonVersion": evidence.get("reasonVersion"),
"dictionary": evidence.get("dictionary"),
"discardedByUserId": str(evidence.get("discardedByUserId") or ""),
"discardedAt": str(evidence.get("discardedAt") or ""),
"discardedAt": str(evidence.get("discardedAt") or evidence.get("at") or ""),
}
# Pose frames deliberately stay out of the index: the customer contract for
# this mode is the split information, and embedded per-frame pose would
@ -1651,6 +1878,12 @@ def _run_accepted_slice_directory(
raise RuntimeError("acceptedSliceDirectory does not prove gap-free full-video coverage")
summary_artifact = next((item for item in inputs if _port(item) == "acceptanceSummary"), None)
summary_payload = summary_artifact.get("data") if isinstance((summary_artifact or {}).get("data"), dict) else {}
if summary_artifact is not None and not isinstance(summary_payload.get("finalAcceptance"), dict):
summary_path = _materialize_input_file(summary_artifact, output / "_acceptance", helpers)
if summary_path.stat().st_size > 8 * 1024 * 1024:
raise RuntimeError("acceptance summary exceeds 8 MiB")
summary_payload = json.loads(summary_path.read_text(encoding="utf8"))
if not isinstance(summary_payload, dict): raise RuntimeError("invalid acceptance summary reference")
final_acceptance = summary_payload.get("finalAcceptance") if isinstance(summary_payload.get("finalAcceptance"), dict) else {}
capture_record = _part_index_capture_record(summary_payload, revision_manifest)
all_slices = revision_manifest.get("items", [])
@ -1669,7 +1902,11 @@ def _run_accepted_slice_directory(
discarded_slices.append(row)
else:
kept_slices.append(row)
if disposition_contract:
params = helpers["hyperparams"](node_data) if callable(helpers.get("hyperparams")) else {}
classification_v4 = _bool(params, "DELIVERY_JSON_V4", False)
if classification_v4 and not coverage_required:
raise RuntimeError("DELIVERY_JSON_V4 requires verified full-video coverage")
if disposition_contract or classification_v4:
pins = summary_payload.get("acceptedMaterializations", [])
expected_sha = _sha256_file(revision_manifest_path)
if summary_payload.get("decision") != "approved" or final_acceptance.get("confirmed") is not True or not any(
@ -1679,7 +1916,7 @@ def _run_accepted_slice_directory(
raise RuntimeError("slice dispositions are not pinned to the accepted manifest revision")
if int(coverage.get("sliceCount", -1)) != len(all_slices):
raise RuntimeError("review coverage slice count mismatch")
if disposition_contract and not kept_slices:
if disposition_contract and not kept_slices and not _bool(params, "DELIVERY_JSON_V4", False):
if any(p.get("key") == "PROCESSING_PROFILE" and p.get("value") == "raw_human" for p in node_data.get("hyperparams", [])) and (summary_payload.get("processingProfile") != "raw_human" or final_acceptance.get("processingProfile") != "raw_human"):
raise RuntimeError("raw-human package requires matching accepted task profile")
# Internal evidence only: deliberately no deliveryDirectory output and
@ -1697,7 +1934,6 @@ def _run_accepted_slice_directory(
payloads = _payload_tree(_payloads_with_full_pose_manifests(inputs, output, helpers))
headpose = _first_payload(payloads, "droboticflow.headpose/v1")
handpose = _first_payload(payloads, "droboticflow.wilor.hand_pose_result/v1")
params = helpers["hyperparams"](node_data) if callable(helpers.get("hyperparams")) else {}
if _bool(params, "PART_INDEX_DELIVERY", False):
if source_index and not _bool(params, "SOURCE_INDEX_DELIVERY", False):
raise RuntimeError("source-index JSON delivery requires explicit SOURCE_INDEX_DELIVERY approval; use directory delivery to preserve physical discard exclusion")
@ -1715,8 +1951,10 @@ def _run_accepted_slice_directory(
for item in payload["slices"] + payload["discardedSlices"]:
item["sourceFrames"] = frames[item["sliceId"]]
body = (json.dumps(payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + "\n").encode("utf8")
if _bool(params, "DELIVERY_JSON_V4", False):
return _emit_part_index_directory(run_id, node_id, output, node_data, helpers, payload, summary_payload, revision_manifest, source_root)
if _bool(params, "DELIVERY_JSON_V3", False):
return _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload)
return _emit_part_index_directory(run_id, node_id, output, node_data, helpers, payload)
return _emit_part_index(run_id, node_id, output, node_data, helpers, body)
if source_index:
source_root = _source_index_physical_export(source_root, revision_manifest, kept_slices, output)
@ -1953,6 +2191,8 @@ def _run_multiport(
def run(run_id: str, node_id: str, output_dir: Any, node_data: dict[str, Any], inputs: list[dict[str, Any]], helpers: HelperMap) -> list[dict[str, Any]]:
params = helpers["hyperparams"](node_data) if callable(helpers.get("hyperparams")) else {}
if _bool(params, "DELIVERY_JSON_V4", False) and (not _bool(params, "PART_INDEX_DELIVERY", False) or _bool(params, "DELIVERY_JSON_V3", False)):
raise RuntimeError("DELIVERY_JSON_V4 requires PART_INDEX_DELIVERY and cannot combine with DELIVERY_JSON_V3")
if _bool(params, "DELIVERY_JSON_V3", False) and not _bool(params, "PART_INDEX_DELIVERY", False):
raise RuntimeError("DELIVERY_JSON_V3 requires PART_INDEX_DELIVERY")
# A part-index delivery intentionally has no directory port; it must still

View File

@ -43,3 +43,19 @@ status: accepted
路径基准显式写入:总索引相对本批次项目根,采集清单相对采集交付目录,原始 meta 和标定文件相对原始采集目录;已固定 workspace 来源另标基准。采集清单记录原始 `meta_<captureId>.json` 的 SHA/大小,expected parts 来自原始目录完整分页列举,不用已验收 part 数推定交齐。最终验收结论及最大 acceptedAt 的聚合仅覆盖本批次选择的修订。
总索引生成由管理员显式预览、按预览 snapshotId 条件发布,不随普通 GET/下载自动写对象存储。旧已完成 part 目录不能原地补发 v3,历史迁移仍需另行规划新交付目标。已有调用者和在途任务默认沿用旧协议;回退 API/开关不删除新批次,也不能把新格式任务改成旧格式后重用已完成目录。
## 2026-10-21 修订:分类结果与内部追溯独立
用户明确交付价值为最终人工确认的分类及标签,而非过程记录。新 v4 主 JSON 只包含最终验收通过的分类、区间及必要源定位;审核批注、人员、轮次、返工和抽检过程不进入主 JSON,也不默认进入客户批次。完整正式追溯保存为内部 JSON,并与固定结果修订关联。返工经过重新验收的分类修正应进入新交付版本,单纯修改批注不能改变分类。
独立字典 tags 由管理员维护,与 ID/版本/名称一起在确认时冻结;category 与 aliases 不当作 tags。未写动作描述不等于没有动作,也不撤销切片已有分类。历史没有标签快照时不反查最新字典拼造历史。全舍弃也属于人工分类结果,因此 v4 仍交付该 part 的分类 JSON;这显式替代 v4 路径的旧 no_deliverables 规则,旧媒体目录和 v1–v3 路径继续保持原行为。
采用独立 v4 协议与默认关闭的开关,避免同一 v3 schema 悄悄改变字段、枚举或文件集合。保持已约定的 capture/part、slices/taskSplit/segments 层级及不可变批次身份,动作有描述使用约定的 annotated。保留旧版本读取与发布恢复,不原地修补历史完成目录。单独写内部追溯会增加一份内部产物,但避免客户为读取分类被迫接触人员、批注和平台流程字段。
## 2026-10-21 补充:动作继承及原始数据消费
用户确认切片与动作是父子分类。动作独立选择词条时使用该快照的标签,否则继承父切片冻结标签;不按描述是否为空决定,不伪造动作词条。v4 显式输出有效 tags 和 action/slice 来源,保留独立无效判断;舍弃仅保留原因分类及定位,不带旧动作或审核批注。缺标签不反查最新词条猜补。
下游需要仅凭总 JSON 和原始数据定位、切分、按类归组。新 v4 使用任务原始输入的 BOS 选择冻结八类源文件及 bucket/prefix,主文件不再依赖平台 workspace。动作同时明确切片内秒数、原始 part 秒数与原始硬件时间戳;用原始左右 PTS 各自解析硬件区间,避免用左视频秒数误切另一相机。输入快照指纹不是单个 BOS 对象内容版本的保证。实际切割和分类打包仍由下游执行。
缺少可靠来源、分类标签或时间证据时,选择拒绝新 v4 交付,而非“尽量输出”后让客户承受丢分类或错切的风险。发布同时验证消费契约、来源归属和文件存在。已交付结果不原地改写;历史只按证据补齐并产生新批次。这个要求适用于尚未发布的 v4,不改变 v1–v3;旧本地 v4 候选须重新生成。实现和本地消费验证见 `docs/delivery-classification-v4.zh-CN.md`。

View File

@ -0,0 +1,47 @@
# 最终分类总 JSON v4
本地实现对应 [ADR-0005 的 2026-10-21 修订](adr/0005-part-index-delivery-shape.md)。历史 v1/v2/v3 和已交付文件保持原状。开关默认关闭;本文不授权启用工作流或发布。
`DELIVERY_JSON_V4=true` 要求 `PART_INDEX_DELIVERY=true`,与 `DELIVERY_JSON_V3` 互斥;源索引修订另需 `SOURCE_INDEX_DELIVERY=true`。新工作流须有内部 `deliveryAudit` JSON 输出,以及正常的 `deliveryDirectory` 输出。配套 Worker 负责固定审核追溯,大于 inline 限额的验收摘要走现有内部 artifact 存储,打包节点读取固定引用;必须先验证内部存储可用。旧 Worker/旧算子不能启用新协议。
客户格式保持 `capture`、`partId`、`slices[].taskSplit`、`slices[].segments[]` 和 `discardedSlices`。按马迪约定的采集目录/part 组织文件:每个 part 一个总分类 JSON;批次索引 `delivery_index.json` 和采集索引 `annotation_manifest.json` 只负责目录定位、文件校验及汇总,不额外携带审核流水。正式批次目录不可变。
- `sliceName` 是已确认的基础分类名称;`classification.dictionary` 是可获得的字典快照。动作描述为空也保留这个分类,不代表无动作。
- 字典快照固定 `id/kind/version/code/name/tags`。`tags` 是独立分类标签,与 `category`、别名和描述原义不混用。新选择由服务端从项目字典校验,客户不能伪造标签;历史缺失快照不按当前词条补填。
- 每个动作显式导出有效 `tags` 和 `tagSource`。独立选择动作词条用该快照标签,来源为 `action`;没有独立选择则继承本切片冻结标签,来源为 `slice`,不伪造动作 `dictionary`。自由填写文字也沿用这个判定;有文字不代表选择了独立词条,没有文字不代表没有动作。独立词条缺 tags 时拒绝交付,不偷偷退回父标签。
- 动作 `contentKind` 为 `annotated`(有描述)、`invalid_marked`(明确无效)、`gap_placeholder`(保留的留空区间)。`descriptionStatus` 描述文字覆盖情况,不能用于推断整个切片是否有动作。
- `isInvalid` 与标签独立;无效动作可保留继承标签,但仍必须按无效判断排除有效动作处理。历史动作未覆盖的留空范围保留为继承父标签的 `gap_placeholder`,不能把该段视频漏掉。
- `discardedSlices[].classification.discardEvidence` 保留业务舍弃原因、原因词条/版本/标签和补充说明。全舍弃 part 也输出总分类 JSON。审核退回原因不放在这里。
- 舍弃原因有冻结字典时,旧兼容字段 `reasonLabelId/reasonVersion/reasonName` 从同一快照补齐,不导出空名称或版本 0;已有非空值若与快照冲突即拒绝交付。手填原因仍按原有规则保留,不猜造字典选择。
- `originalSource` 冻结 BOS bucket、原采集 prefix、partPrefix、输入快照指纹和所选原始文件路径。来自任务创建时的输入引用,不查询当前目录猜原文件,不输出连接 ID、凭据或签名 URL。八类文件为左右视频、左右 PTS、IMU、相机标定、IMU 标定和采集 meta。`sources`、`sourceMetadata.path` 与标定路径统一相对原采集目录,`pathBases` 显式写为 `original_capture`;原 workspace 路径只留内部追溯。
内部 `droboticflow.delivery.review-history/v2` JSON 按 capture/part/revision 关联,以 `partSha256` 固定原生主文件字节;包含最终验收、抽检、审核人/批注/轮次/确认项、已有正式退回基线、处理记录和阶段事件。该 artifact 留在内部 Run 存储,不进入最终交付列表、不可变客户批次或客户下载。分类选择的服务端快照另外保存在当前任务的恢复记录中,使字典改名、更新或停用后的撤销仍可恢复原版本,客户端不能伪造历史标签。旧记录没有的追溯不制造补写。追溯超出限制时拒绝交付,不静默截断。
最终分类取通过验收的固定物化修订;沿用既有“未抽中”验收策略的任务,内部保留明确豁免,不能伪造人工通过日期。返工正式修改分类后必须重新送检、验收;查看或处理意见本身不修改分类。相同名称而字典版本/标签不同的切片不能自动合并成一个分类。
时间按两级明确表达,全部区间采用左闭右开:
| 字段 | 坐标 |
| --- | --- |
| `taskSplit.startSecond/endSecond` | 原始 part 左相机视频秒数 |
| `segments[].startSecond/endSecond`、`clipStartMs/clipEndMs` | 本切片左相机内秒数/毫秒数 |
| `segments[].partStartSecond/partEndSecond` | 原始 part 左相机视频秒数 |
| `startTimestampUs/endTimestampUs` | 原始硬件时间戳,非墙钟时间、非从秒数估算 |
`timeBases` 描述以上坐标及帧区间基准。下游从原始左右 PTS 分别定位同一硬件时间窗,不能把左相机秒数直接当右相机秒数。原始输入快照指纹标识当时选择,不冒充每个 BOS 对象的版本号或内容 SHA;原始数据应保留,当前发布时仅以 HEAD 校验所引用文件存在,不证明同名文件从未被外部覆盖。
新 v4 要求完整的冻结分类、非空标签、原始来源和可靠时间边界;任一缺失或动作重叠、时间戳不连续即停止交付。批次发布再核对原始 bucket/part 归属、文件存在性和消费契约,不能只凭主文件的 SHA 放行。元数据修订复用原始引用,不追随后来改变的采集记录。未发布的旧本地 v4 样例须重新生成;正式 v1/v2/v3 及其恢复路径保持兼容。历史缺证据应先核实补齐,已交付文件不覆盖,修正交付另建不可变批次。
平台只输出分类索引,不按标签执行 BOS 数据切割、搬运或集中打包。内部审核文件不在客户目录内,也不从主 JSON 引用。
本地可用 `python -m tests.backend.export_delivery_v4_fixture <全新输出目录>` 生成明确标记的合成样例。目录中 `partNNNN/part.json` 是客户主文件;`internal/partNNNN/review-history.json` 是分离的内部示例,不属于客户包。不要把整个验证输出目录直接交付给客户。
独立消费验证:设置 `FANYU_DELIVERY_CONSUMER_FIXTURE_DIR` 为全新绝对目录,先运行 Go `TestDeliveryV4ConsumerReplicaFixture`,再以同一环境变量运行 `tests/backend/test_delivery_v4_consumer.py`。合成真实双目视频分别经过物理切片、源索引和元数据复用;独立进程仅读取导出 JSON 与原始文件,核对每帧覆盖、非零 part 起点、标签继承、无效与舍弃。独立读取器 `tests/backend/consume_delivery_v4.py` 只生成处理计划,不执行切割或写源数据。
## 2026-10-22 组合候选与默认关闭阶段
用户确认先与前端分层版本合并验证,`DELIVERY_JSON_V4` 继续保持关闭,启用另行制定计划。新代码的标签编辑、字典快照保存和内部物化元数据并非全部受该打包参数控制;关闭参数只保证不主动选择 v4 交付路径,不代表 API 行为与旧版本逐字节一致。
整采集的多 part 输入可能无法证明单 part 的原始来源。普通物理切片、源索引及元数据复用会省略不可靠的新增 `originalSource` 并保留诊断,不因这项可选证据阻塞旧任务;跨 Run/组织错误与存储失败仍拒绝,已有冻结来源不追随新目录。真正生成或发布 v4 时,缺原始来源、标签或时间证据仍被打包/批次门禁拒绝,不能据此输出不完整 JSON。
回退旧 API 不是无损写入保证:新版本保存分类后,旧版本重建草稿数组可能丢弃字典快照。回退前应暂停相关写操作或准备能保留新字段的兼容回退版本;只保留恢复历史不等于能自动恢复所有后续修改。已交付不可变文件不因此原地迁移或覆盖。

View File

@ -31,6 +31,7 @@
["npm", "run", "test:browser:annotation-pools"],
["npm", "run", "test:browser:slice-input"],
["npm", "run", "test:browser:annotation-persistence"],
["go", "test", "./internal/api", "-run", "TestDurableAnnotationOutbox|TestSliceRecovery|TestReviewCannotMutateAnnotation|TestReviewDecision|Test.*ReviewDraft|TestReviewConfirmations|TestRouteSnapshot", "-count=1"]
["go", "test", "./internal/deliveryindex", "-count=1"],
["go", "test", "./internal/api", "-run", "TestDurableAnnotationOutbox|TestSliceRecovery|TestReviewCannotMutateAnnotation|TestReviewDecision|Test.*ReviewDraft|TestReviewConfirmations|TestRouteSnapshot|Test.*Dictionary|TestProjectSliceNames|TestLegacyMaterializationWithoutPartScopedOriginalSelection|TestOptionalOriginalSource|TestAnnotationOriginalSource|TestClassificationBatch", "-count=1"]
]
}

View File

@ -0,0 +1,31 @@
# 前端分层与交付 v4 组合候选
用户选择:先整合并验证,v4 保持关闭;启用另行确认。新个人分支 `codex/dashboard-delivery-v4-20261022` 从 `2149b862` 合并 v4 分支 `e66c7726`,原两个分支保留。
三处文本冲突已处理:切片字典接口保留 v4 的服务端冻结/历史恢复语义,工作台和标注页保留分层协调与字典选择两侧导入。不能以文本合并成功代替业务验证。
## 已核对的发布范围
2026-10-22 的只读 Deployment 快照显示 Fanyu / droboticflow-fanyu 的 API 与处理器均为 `3221ac65`,Worker 为 `963b530f`;它们均为组合候选的祖先。此事实只说明源码历史没有遗漏,不证明所有语义兼容,也不能代替发布前新快照。
本次只准备候选;没有上传镜像、部署、迁移、启用工作流或改写客户交付。后续若发布默认关闭版本,计划限 API 与处理器,Worker、独立打包算子和工作流继续使用现有版本。v4 需要配套升级 Worker/算子、验证内部审计存储及配置 deliveryAudit 输出,不能只切一个开关。
候选未新增 Store/cmd 迁移或索引修改。内置算子目录新增默认关闭参数和审计输出,目录查询沿现有代码可能更新内置算子定义;这不是现有工作流自动升级,也不能说镜像更新绝不触发数据库元数据写入。发布前仍须审查实际配置、在途任务和数据影响。
## 修复的组合兼容问题
合成多 part 输入引用实测使物理切片、源索引在 v4 关闭时也因新增来源证据检查失败。现将不可信的可选来源证据省略并记录诊断,普通媒体处理继续;跨任务/组织及存储错误仍拒绝。真实 v4 的缺来源、文件归属、时间和标签门禁继续有效。已有冻结来源不被后来目录变化改写。
v4 默认关闭不等于旧版本写入安全:新字段开始保存后,旧 API 的数组重建可能丢失字典快照。后续上线必须明确暂停相关写入或使用兼容回退版本,不能承诺直接回退现网镜像后继续编辑无损。
## 本地证据
- 类型与分层检查、全量前端基线门禁通过;13 项已有失败,14 项基线未修改。
- v4、字典、批次、历史 part-index 的 Go 定向回归通过;新来源兼容回归覆盖物理/源索引两次修订、元数据复用、身份隔离和不伪造来源。
- 新合成真实双目媒体经过 Go 物理切片/源索引,再由独立 Python 消费客户 JSON 与原始文件,确认标签继承、留空、无效、舍弃与硬件时间窗;Python 共 43 通过、1 项旧夹具依赖场景跳过,未将跳过计入通过。
- 组合版完整标注 19 场景、质检/验收草稿 34 场景和审核权限浏览器回归通过。
- 修复前的来源错误、开发态字典夹具依赖优化导致的页面未渲染尝试均保留,不能冒充产品失败或已通过验证。
- 字典标签的生产组件构建及浏览器测试通过;共享保存 24 项、发布工具 37 项、生成源码一致性、Go 编译、前端构建与生产启动通过。
- 完整算子发布 manifest 检查未通过:annotation_gate 的 sourceTreeSha256 漂移在原始 `3221ac65` 工作树逐项复现,期望值/实际值完全相同,属于已有问题;新 v4 package 源码也尚未绑定新算子镜像发布。不得伪造 release.json 摘要使检查通过。它阻断算子升级/v4 启用,不将本轮限定的 API/处理器默认关闭候选冒充为算子可发布版本。
证据存于本工作树忽略目录 `local-data/verification/combined-v4-20261022/`。本记录不是部署批准,也不保证回退旧写入路径无损。

View File

@ -2,6 +2,7 @@ package api
import (
"fmt"
"reflect"
"sort"
"strings"
@ -19,7 +20,7 @@ func consolidateInvalidActionSlices(input []domain.SliceSegment, actions []domai
eligible := map[string]int{}
for i := 1; i < len(ordered)-1; i++ {
l, c, r := ordered[i-1], ordered[i], ordered[i+1]
if l.IsDiscarded() || c.IsDiscarded() || r.IsDiscarded() || strings.TrimSpace(c.Name) == "" || l.Name != c.Name || r.Name != c.Name || l.End != c.Start || c.End != r.Start {
if l.IsDiscarded() || c.IsDiscarded() || r.IsDiscarded() || strings.TrimSpace(c.Name) == "" || l.Name != c.Name || r.Name != c.Name || !reflect.DeepEqual(l.Dictionary, c.Dictionary) || !reflect.DeepEqual(r.Dictionary, c.Dictionary) || l.End != c.Start || c.End != r.Start {
continue
}
cursor, has, valid := c.Start, false, true

View File

@ -14,6 +14,7 @@ import (
)
type annotationLabelInput struct {
Tags []string `json:"tags"`
RequiresNote *bool `json:"requiresNote"`
Kind string `json:"kind"`
Code string `json:"code"`
@ -46,7 +47,7 @@ func annotationLabelRevision(item domain.AnnotationLabelRecord) domain.Annotatio
return domain.AnnotationLabelRevision{
RequiresNote: item.RequiresNote, Version: item.Version, Code: item.Code, Name: item.Name, Category: item.Category,
Description: item.Description, Example: item.Example, Color: item.Color,
Aliases: append([]string(nil), item.Aliases...), Status: item.Status,
Tags: append([]string(nil), item.Tags...), Aliases: append([]string(nil), item.Aliases...), Status: item.Status,
UpdatedByUserID: item.UpdatedByUserID, UpdatedAt: item.UpdatedAt,
}
}
@ -79,6 +80,19 @@ func normalizeAnnotationLabelInput(input annotationLabelInput) annotationLabelIn
seen[value] = struct{}{}
aliases = append(aliases, value)
}
if input.Tags != nil {
tags := []string{}
seenTags := map[string]bool{}
for _, tag := range input.Tags {
tag = strings.TrimSpace(tag)
if tag != "" && !seenTags[tag] {
tags = append(tags, tag)
seenTags[tag] = true
}
}
input.Tags = tags
}
input.Aliases = aliases
return input
}
@ -97,6 +111,15 @@ func validateAnnotationLabelInput(input annotationLabelInput) []validationIssue
if input.Status != domain.AnnotationLabelActive && input.Status != domain.AnnotationLabelInactive {
issues = append(issues, validationIssue{Field: "status", Message: "标签状态必须是 active 或 inactive"})
}
if len(input.Tags) > 32 {
issues = append(issues, validationIssue{Field: "tags", Message: "每个词条最多 32 个标签"})
}
for _, tag := range input.Tags {
if len([]rune(tag)) > 80 {
issues = append(issues, validationIssue{Field: "tags", Message: "标签最多 80 字"})
break
}
}
return issues
}
@ -165,7 +188,7 @@ func (server *server) handleAnnotationLabelCreate(writer http.ResponseWriter, re
ID: domain.NewID("annotation-label"), OrganizationID: organizationID,
Code: payload.Code, Name: payload.Name, Category: payload.Category,
Description: payload.Description, Example: payload.Example, Color: payload.Color,
Aliases: payload.Aliases, Status: payload.Status, Version: 1,
Tags: payload.Tags, Aliases: payload.Aliases, Status: payload.Status, Version: 1,
CreatedByUserID: actorID, UpdatedByUserID: actorID, CreatedAt: now, UpdatedAt: now,
}
item.History = []domain.AnnotationLabelRevision{annotationLabelRevision(item)}
@ -225,6 +248,9 @@ func (server *server) handleAnnotationLabelPatch(writer http.ResponseWriter, req
item.Code, item.Name, item.Category = payload.Code, payload.Name, payload.Category
item.Description, item.Example, item.Color = payload.Description, payload.Example, payload.Color
item.Aliases, item.Status = payload.Aliases, payload.Status
if payload.Tags != nil {
item.Tags = payload.Tags
}
item.Version++
item.UpdatedByUserID, item.UpdatedAt = principal.User.ID, nowISO()
item.History = append(item.History, annotationLabelRevision(*item))
@ -267,13 +293,16 @@ func (server *server) handleProjectAnnotationLabelsGet(writer http.ResponseWrite
if selection == nil {
selection = &domain.ProjectAnnotationLabelsRecord{ID: "project-annotation-labels-" + project.ID, OrganizationID: project.OrganizationID, ProjectID: project.ID, LabelIDs: []string{}}
}
if projectDictionaryKind(request) != "action" {
{
available, e := server.store.ListAnnotationLabels(request.Context(), project.OrganizationID)
if e != nil {
renderError(writer, 500, e)
return
}
ids := selection.SliceLabelIDs
ids := selection.LabelIDs
if projectDictionaryKind(request) == "slice" {
ids = selection.SliceLabelIDs
}
if projectDictionaryKind(request) == "discard_reason" {
ids = selection.DiscardReasonLabelIDs
}
@ -290,14 +319,20 @@ func (server *server) handleProjectAnnotationLabelsGet(writer http.ResponseWrite
items = append(items, label)
}
}
response := map[string]any{"projectId": project.ID, "labelIds": ids, "items": items}
response := struct {
domain.ProjectAnnotationLabelsRecord
Items []domain.AnnotationLabelRecord `json:"items"`
AllowManual *bool `json:"allowManual,omitempty"`
}{ProjectAnnotationLabelsRecord: *selection, Items: items}
response.LabelIDs = ids
if projectDictionaryKind(request) == "discard_reason" {
response["allowManual"] = selection.OrganizationID == project.OrganizationID && len(ids) == 0
manual := selection.OrganizationID == project.OrganizationID && len(ids) == 0
response.AllowManual = &manual
}
renderJSON(writer, http.StatusOK, response)
return
}
renderJSON(writer, http.StatusOK, selection)
}
func (server *server) handleProjectAnnotationLabelsPut(writer http.ResponseWriter, request *http.Request) {
@ -399,7 +434,7 @@ func (server *server) projectAnnotationLabelDefinitions(ctx context.Context, org
if !exists || item.Status != domain.AnnotationLabelActive || dictionaryKind(item.Kind) != "action" {
continue
}
labels = append(labels, domain.SemanticLabelDefinition{ID: item.Code, Name: item.Name, Description: item.Description, Aliases: item.Aliases, Color: item.Color})
labels = append(labels, domain.SemanticLabelDefinition{Dictionary: item.Selection(), ID: item.Code, Name: item.Name, Description: item.Description, Aliases: item.Aliases, Color: item.Color})
}
return labels, nil
}

View File

@ -72,6 +72,15 @@ func (s *server) tryAnnotationMetadataRevision(ctx context.Context, task domain.
if e != nil {
return nil, nil
}
if manifest["originalSource"] == nil {
if record, err := s.store.GetCaptureRecordByRunID(ctx, task.RunID); err != nil {
return nil, err
} else if original, err := annotationRevisionOriginalSource(run, record); err != nil {
return nil, err
} else if original != nil {
manifest["originalSource"] = original
}
}
entries := listValue(manifest["items"])
if len(entries) != len(slices) {
return nil, nil
@ -134,6 +143,7 @@ func (s *server) tryAnnotationMetadataRevision(ctx context.Context, task domain.
}
entry["sourceSliceId"], entry["sliceName"], entry["deliveryDisposition"] = slice.ID, slice.Name, sliceDisposition(*slice)
synchronized["sliceName"], synchronized["deliveryDisposition"] = slice.Name, sliceDisposition(*slice)
entry["dictionary"], synchronized["dictionary"] = slice.Dictionary, slice.Dictionary
delete(entry, "discardEvidence")
delete(synchronized, "discardEvidence")
if slice.IsDiscarded() {
@ -146,9 +156,9 @@ func (s *server) tryAnnotationMetadataRevision(ctx context.Context, task domain.
if slice.IsDiscarded() || segment.End <= slice.Start || segment.Start >= slice.End {
continue
}
actions = append(actions, map[string]any{"id": segment.ID, "status": effectiveActionStatus(segment), "isInvalid": segment.IsInvalid, "description": segment.Description, "descriptionTemplateId": segment.LabelID, "phrases": segment.Phrases, "source": segment.Source, "confidence": segment.Confidence, "clipStartMs": (segment.Start - slice.Start) * 1000, "clipEndMs": (segment.End - slice.Start) * 1000, "startTimestampUs": segment.StartTimestampUS, "endTimestampUs": segment.EndTimestampUS})
actions = append(actions, map[string]any{"id": segment.ID, "status": effectiveActionStatus(segment), "isInvalid": segment.IsInvalid, "description": segment.Description, "descriptionTemplateId": segment.LabelID, "dictionary": segment.Dictionary, "phrases": segment.Phrases, "source": segment.Source, "confidence": segment.Confidence, "clipStartMs": (segment.Start - slice.Start) * 1000, "clipEndMs": (segment.End - slice.Start) * 1000, "startTimestampUs": segment.StartTimestampUS, "endTimestampUs": segment.EndTimestampUS})
}
annotation := map[string]any{"schemaVersion": "droboticflow.timestamped-description/v3", "boundaryContract": "hardware-frame-v1", "sliceId": entry["sliceId"], "sliceName": slice.Name, "window": map[string]any{"startTimestampUs": slice.StartTimestampUS, "endTimestampUs": slice.EndTimestampUS, "semantics": "half-open"}, "descriptions": actions, "actions": actions}
annotation := map[string]any{"schemaVersion": "droboticflow.timestamped-description/v3", "boundaryContract": "hardware-frame-v1", "sliceId": entry["sliceId"], "sliceName": slice.Name, "dictionary": slice.Dictionary, "window": map[string]any{"startTimestampUs": slice.StartTimestampUS, "endTimestampUs": slice.EndTimestampUS, "semantics": "half-open"}, "descriptions": actions, "actions": actions}
if e = os.MkdirAll(filepath.Join(stage, oldDir), 0755); e != nil {
return nil, e
}

View File

@ -1831,6 +1831,7 @@ func (server *server) handleCombinedAnnotationDraftPut(writer http.ResponseWrite
return
}
template = annotationTemplateWithTaskSnapshot(template, task)
issues = append(issues, server.canonicalActionDictionaries(request.Context(), humanItem, payload.Segments)...)
issues = append(issues, validateDraftFanyuV5CombinedSemanticSegments(payload.Segments, payload.SliceSegments, template)...)
if len(issues) > 0 {
renderValidationError(writer, issues...)
@ -1977,6 +1978,7 @@ func (server *server) handleAnnotationConfirmationCreate(writer http.ResponseWri
return
}
template = annotationTemplateWithTaskSnapshot(template, task)
issues = append(issues, server.canonicalActionDictionaries(request.Context(), item, payload.Segments)...)
issues = append(issues, validateDraftFanyuV5CombinedSemanticSegments(payload.Segments, slices, template)...)
if len(issues) > 0 {
renderValidationError(writer, issues...)
@ -2496,6 +2498,10 @@ func (server *server) handleCombinedAnnotationSubmission(writer http.ResponseWri
return
}
template = annotationTemplateWithTaskSnapshot(template, task)
if issues := server.canonicalActionDictionaries(request.Context(), humanItem, payload.Segments); len(issues) > 0 {
renderValidationError(writer, issues...)
return
}
if issues := validateFanyuV5CombinedSemanticSegmentsWithRequireDesc(payload.Segments, payload.SliceSegments, template, !continuousActions); len(issues) > 0 {
renderValidationError(writer, issues...)
return
@ -3104,13 +3110,18 @@ func (server *server) handleAnnotationRevisionCreate(writer http.ResponseWriter,
renderJSON(writer, http.StatusForbidden, map[string]string{"error": "annotator assignment is required"})
return
}
if _, claimed := server.requireLegacyHumanItemClaim(writer, request, *task, "semantic_annotation"); !claimed {
humanItem, claimed := server.requireLegacyHumanItemClaim(writer, request, *task, "semantic_annotation")
if !claimed {
return
}
var payload annotationRevisionInput
if !decodeJSON(writer, request, &payload) {
return
}
if issues := server.canonicalActionDictionaries(request.Context(), humanItem, payload.Segments); len(issues) > 0 {
renderValidationError(writer, issues...)
return
}
if issues := validateAnnotationSegments(payload.Segments, *task); len(issues) > 0 {
renderValidationError(writer, issues...)
return

View File

@ -0,0 +1,79 @@
package api
import (
"encoding/json"
"fmt"
"log"
"path"
"sort"
"strings"
"droboticflow/internal/domain"
)
// Freeze only credential-free facts from the run's original input, never the
// current workflow configuration or a later source-directory listing.
func annotationOriginalSource(run *domain.RunRecord, capture *domain.CaptureRecord) (map[string]any, error) {
if run == nil || capture == nil || len(run.Inputs) == 0 {
return nil, nil
}
if capture.RunID != run.ID || capture.OrganizationID != run.OrganizationID {
return nil, fmt.Errorf("original capture scope mismatch")
}
root := capture.SourcePrefix
if root == "" {
root = capture.Prefix
}
var result map[string]any
for _, input := range run.Inputs {
ref := input.Ref
if ref == nil || ref.Provider != domain.StorageProviderBOS || ref.Kind != domain.ArtifactRefKindDirectory || ref.Bucket != capture.Bucket || ref.Prefix != root {
continue
}
if ref.CaptureUnit != "" && ref.CaptureUnit != capture.CaptureID {
continue
}
if len(ref.ObjectKeys) == 0 {
continue
} // Older non-selected directory inputs need explicit recovery.
if root == "" || path.Clean(root) != root || strings.HasPrefix(root, "/") || strings.Contains(root, "\\") {
return nil, fmt.Errorf("unsafe original capture prefix")
}
keys := append([]string(nil), ref.ObjectKeys...)
sort.Strings(keys)
for i, key := range keys {
if !strings.HasPrefix(key, root+"/") || path.Clean(key) != key || strings.Contains(key, "\\") || i > 0 && key == keys[i-1] {
return nil, fmt.Errorf("unsafe original capture object selection")
}
relative := strings.TrimPrefix(key, root+"/")
if strings.Contains(relative, "/") && !strings.HasPrefix(key, capture.Prefix+"/") {
return nil, fmt.Errorf("original capture selection includes another part")
}
}
candidate := map[string]any{"provider": string(ref.Provider), "bucket": ref.Bucket, "prefix": root, "partPrefix": capture.Prefix, "captureUnit": capture.CaptureID, "objectKeys": keys, "snapshotFingerprint": ref.VersionID}
if result != nil {
a, _ := json.Marshal(result)
b, _ := json.Marshal(candidate)
if string(a) != string(b) {
return nil, fmt.Errorf("ambiguous original capture inputs")
}
}
result = candidate
}
return result, nil
}
// Source provenance is optional for legacy materialization, but mandatory at
// v4 export. Never invent a part selection from an ambiguous capture input.
// Tenant/run violations are still fatal; storage failures remain with callers.
func annotationRevisionOriginalSource(run *domain.RunRecord, capture *domain.CaptureRecord) (map[string]any, error) {
source, err := annotationOriginalSource(run, capture)
if err == nil {
return source, nil
}
if run != nil && capture != nil && (capture.RunID != run.ID || capture.OrganizationID != run.OrganizationID) {
return nil, err
}
log.Printf("[api] original_source_evidence_unavailable reason=%s", err)
return nil, nil
}

View File

@ -0,0 +1,238 @@
package api
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"droboticflow/internal/domain"
)
func consumerSourceRef() *domain.ArtifactReference {
root := "incoming/capture-A"
keys := []string{root + "/meta_capture-A.json", root + "/capture-A_calibration_camera.yaml", root + "/capture-A_calibration_imu.yaml", root + "/part0001/capture-A_imu_part0001.csv"}
for _, side := range []string{"left", "right"} {
keys = append(keys, root+"/part0001/capture-A_camera_"+side+"_part0001.mp4", root+"/part0001/capture-A_camera_"+side+"_part0001_pts.csv")
}
return &domain.ArtifactReference{Provider: domain.StorageProviderBOS, Kind: domain.ArtifactRefKindDirectory, ConnectionID: "PRIVATE_CONNECTION", Bucket: "synthetic-original", Prefix: root, CaptureUnit: "capture-A/part0001", ObjectKeys: keys, VersionID: "frozen-input-version"}
}
func TestAnnotationOriginalSourceUsesFrozenInputSelection(t *testing.T) {
ref := consumerSourceRef()
run := &domain.RunRecord{ID: "run", OrganizationID: "org", Inputs: map[string]domain.ArtifactPayload{"raw": {Ref: ref}}}
capture := &domain.CaptureRecord{RunID: "run", OrganizationID: "org", Bucket: ref.Bucket, Prefix: ref.Prefix + "/part0001", SourcePrefix: ref.Prefix, CaptureID: ref.CaptureUnit, SourceKeys: []string{"later-current-directory-must-not-be-used"}}
source, err := annotationOriginalSource(run, capture)
if err != nil {
t.Fatal(err)
}
if len(source["objectKeys"].([]string)) != 8 {
t.Fatal("frozen source selection lost")
}
if _, ok := source["connectionId"]; ok {
t.Fatal("private connection leaked")
}
ref.ObjectKeys = append(ref.ObjectKeys, ref.Prefix+"/part0002/foreign.mp4")
if _, err = annotationOriginalSource(run, capture); err == nil {
t.Fatal("foreign part accepted")
}
ref.ObjectKeys = ref.ObjectKeys[:len(ref.ObjectKeys)-1]
capture.OrganizationID = "other"
if _, err = annotationOriginalSource(run, capture); err == nil {
t.Fatal("foreign organization accepted")
}
}
// Retain real synthetic source data for the independent Python consumer test.
func TestDeliveryV4ConsumerReplicaFixture(t *testing.T) {
parent := os.Getenv("FANYU_DELIVERY_CONSUMER_FIXTURE_DIR")
if parent == "" {
t.Skip("opt-in retained synthetic fixture")
}
if !filepath.IsAbs(parent) {
t.Fatal("absolute local output required")
}
for _, sourceIndex := range []bool{false, true} {
mode := "physical"
if sourceIndex {
mode = "source-index"
}
t.Run(mode, func(t *testing.T) {
f, temp := annotationProcessorFixture(t)
dir := filepath.Join(parent, mode)
root := filepath.Join(dir, "workspace")
if err := os.MkdirAll(dir, 0700); err != nil {
t.Fatal(err)
}
if err := copyAnnotationTree(f.ctx, temp, root); err != nil {
t.Fatal(err)
}
run, _ := f.store.GetRun(f.ctx, f.task.RunID)
for i := range run.NodeExecutions {
for j := range run.NodeExecutions[i].InputBindings {
for k := range run.NodeExecutions[i].InputBindings[j].Artifacts {
a := &run.NodeExecutions[i].InputBindings[j].Artifacts[k]
a.Path = filepath.Join(root, filepath.Base(a.Path))
}
}
}
ref := consumerSourceRef()
run.Inputs = map[string]domain.ArtifactPayload{"raw": {Ref: ref}}
if err := f.store.UpsertRun(f.ctx, *run); err != nil {
t.Fatal(err)
}
capture := domain.CaptureRecord{ID: "consumer-capture", RunID: run.ID, OrganizationID: run.OrganizationID, Bucket: ref.Bucket, Prefix: ref.Prefix + "/part0001", SourcePrefix: ref.Prefix, CaptureID: ref.CaptureUnit}
if err := f.store.UpsertCaptureRecord(f.ctx, capture); err != nil {
t.Fatal(err)
}
base := &domain.DictionarySelection{ID: "base", Kind: "slice", Version: 2, Code: "operate", Name: "物品操作", Tags: []string{"商品", "手部"}}
reason := &domain.DictionarySelection{ID: "discard", Kind: "discard_reason", Version: 1, Code: "occluded", Name: "遮挡", Tags: []string{"不可用"}}
action := &domain.DictionarySelection{ID: "pick", Kind: "action", Version: 3, Code: "pick", Name: "拿取", Tags: []string{"抓取"}}
input := combinedAnnotationSubmissionInput{SliceSegments: []domain.SliceSegment{{ID: "first", Name: base.Name, Dictionary: base, Start: 0, End: 1}, {ID: "discard", Start: 1, End: 3, DeliveryDisposition: "discard", DiscardReason: "遮挡", DiscardReasonDictionary: reason}, {ID: "last", Name: base.Name, Dictionary: base, Start: 3, End: 4}}, Segments: []domain.AnnotationSegment{{ID: "blank", Start: 0, End: .5}, {ID: "pick", Start: .5, End: 1, Description: "拿起杯子", Dictionary: action}, {ID: "invalid", Start: 3, End: 3.5, IsInvalid: true}, {ID: "free-text", Start: 3.5, End: 4, Description: "人工补充描述"}}}
s := &server{store: f.store, workspaceMountPath: root, annotationSourceIndex: sourceIndex}
result, err := s.materializeCombinedSliceSubmission(f.ctx, f.task, f.item, input, 1)
if err != nil {
t.Fatal(err)
}
if !sourceIndex {
before, err := os.ReadFile(filepath.Join(result.Artifact.Path, "revision-manifest.json"))
if err != nil {
t.Fatal(err)
}
var first map[string]any
if err = json.Unmarshal(before, &first); err != nil {
t.Fatal(err)
}
// Metadata-only reuse must keep the original source, even if discovery changes.
capture.Bucket = "later-bucket"
if err = f.store.UpsertCaptureRecord(f.ctx, capture); err != nil {
t.Fatal(err)
}
run, _ = f.store.GetRun(f.ctx, f.task.RunID)
result, err = s.materializeCombinedSliceSubmission(f.ctx, f.task, f.item, input, 2)
if err != nil {
t.Fatal(err)
}
if result.Payload["metadataOnly"] != true {
t.Fatal("metadata-only recovery not exercised")
}
after, err := os.ReadFile(filepath.Join(result.Artifact.Path, "revision-manifest.json"))
if err != nil {
t.Fatal(err)
}
var second map[string]any
if err = json.Unmarshal(after, &second); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(first["originalSource"], second["originalSource"]) {
t.Fatal("metadata revision followed mutable source")
}
}
original := filepath.Join(dir, "original")
for _, key := range ref.ObjectKeys {
relative := key[len(ref.Prefix)+1:]
dst := filepath.Join(original, filepath.FromSlash(relative))
if err = os.MkdirAll(filepath.Dir(dst), 0700); err != nil {
t.Fatal(err)
}
var body []byte
switch {
case filepath.Ext(dst) == ".yaml":
body = []byte("synthetic: true\n")
case filepath.Ext(dst) == ".json":
body = []byte("{\"synthetic\":true,\"partId\":\"part0001\"}\n")
default:
name := "imu.csv"
for _, side := range []string{"left", "right"} {
if strings.Contains(relative, "camera_"+side) {
name = side + ".mp4"
if strings.HasSuffix(relative, "_pts.csv") {
name = side + "_pts.csv"
}
}
}
body, err = os.ReadFile(filepath.Join(root, name))
if err != nil {
t.Fatal(err)
}
}
if err = os.WriteFile(dst, body, 0600); err != nil {
t.Fatal(err)
}
}
body, _ := json.MarshalIndent(map[string]any{"artifact": result.Artifact, "original": original, "workspace": root}, "", " ")
if err = os.WriteFile(filepath.Join(dir, "context.json"), body, 0600); err != nil {
t.Fatal(err)
}
})
}
}
// Legacy capture inputs can select an entire multi-part directory. They are
// insufficient evidence for v4, but must not block existing annotation work.
func TestLegacyMaterializationWithoutPartScopedOriginalSelection(t *testing.T) {
for _, sourceIndex := range []bool{false, true} {
name := "physical"
if sourceIndex {
name = "source-index"
}
t.Run(name, func(t *testing.T) {
f, root := annotationProcessorFixture(t)
run, err := f.store.GetRun(f.ctx, f.task.RunID)
if err != nil {
t.Fatal(err)
}
ref := consumerSourceRef()
ref.ObjectKeys = append(ref.ObjectKeys, ref.Prefix+"/part0002/other.mp4")
run.Inputs = map[string]domain.ArtifactPayload{"legacy-capture": {Ref: ref}}
if err = f.store.UpsertRun(f.ctx, *run); err != nil {
t.Fatal(err)
}
capture := domain.CaptureRecord{ID: "legacy-capture", RunID: run.ID, OrganizationID: run.OrganizationID, Bucket: ref.Bucket, Prefix: ref.Prefix + "/part0001", SourcePrefix: ref.Prefix, CaptureID: ref.CaptureUnit}
if err = f.store.UpsertCaptureRecord(f.ctx, capture); err != nil {
t.Fatal(err)
}
input := combinedAnnotationSubmissionInput{SliceSegments: []domain.SliceSegment{{ID: "whole", Name: "历史名称", Start: 0, End: 4}}, Segments: []domain.AnnotationSegment{{ID: "old", Start: 0, End: 4, Description: "历史描述"}}}
s := &server{store: f.store, workspaceMountPath: root, annotationSourceIndex: sourceIndex}
for revision := 1; revision <= 2; revision++ {
result, err := s.materializeCombinedSliceSubmission(f.ctx, f.task, f.item, input, revision)
if err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(filepath.Join(result.Artifact.Path, "revision-manifest.json"))
if err != nil {
t.Fatal(err)
}
var manifest map[string]any
if err = json.Unmarshal(data, &manifest); err != nil {
t.Fatal(err)
}
if manifest["originalSource"] != nil {
t.Fatal("unreliable source must not be exported as v4 evidence")
}
if len(listValue(manifest["items"])) != 1 {
t.Fatal("legacy materialization lost its slice")
}
if !sourceIndex && revision == 2 && result.Payload["metadataOnly"] != true {
t.Fatal("metadata-only reuse was not exercised")
}
}
})
}
}
func TestOptionalOriginalSourceStillRejectsIdentityMismatch(t *testing.T) {
ref := consumerSourceRef()
run := &domain.RunRecord{ID: "run", OrganizationID: "org", Inputs: map[string]domain.ArtifactPayload{"raw": {Ref: ref}}}
capture := &domain.CaptureRecord{RunID: "other-run", OrganizationID: "org"}
if _, err := annotationRevisionOriginalSource(run, capture); err == nil {
t.Fatal("run mismatch accepted")
}
capture.RunID = run.ID
capture.OrganizationID = "other-org"
if _, err := annotationRevisionOriginalSource(run, capture); err == nil {
t.Fatal("organization mismatch accepted")
}
}

View File

@ -85,10 +85,10 @@ func (s *server) materializeSourceIndexRevision(ctx context.Context, t domain.An
if segment.Start < slice.Start || segment.End > slice.End {
return nil, fmt.Errorf("动作不能跨越切片边界")
}
actions = append(actions, map[string]any{"id": segment.ID, "status": effectiveActionStatus(segment), "isInvalid": segment.IsInvalid, "description": segment.Description, "descriptionTemplateId": segment.LabelID, "phrases": segment.Phrases, "source": segment.Source, "confidence": segment.Confidence, "clipStartMs": (segment.Start - slice.Start) * 1000, "clipEndMs": (segment.End - slice.Start) * 1000, "startTimestampUs": segment.StartTimestampUS, "endTimestampUs": segment.EndTimestampUS})
actions = append(actions, map[string]any{"id": segment.ID, "status": effectiveActionStatus(segment), "isInvalid": segment.IsInvalid, "description": segment.Description, "descriptionTemplateId": segment.LabelID, "dictionary": segment.Dictionary, "phrases": segment.Phrases, "source": segment.Source, "confidence": segment.Confidence, "clipStartMs": (segment.Start - slice.Start) * 1000, "clipEndMs": (segment.End - slice.Start) * 1000, "startTimestampUs": segment.StartTimestampUS, "endTimestampUs": segment.EndTimestampUS})
}
window := map[string]any{"startTimestampUs": slice.StartTimestampUS, "endTimestampUs": slice.EndTimestampUS, "semantics": "half-open"}
annotation := map[string]any{"schemaVersion": "droboticflow.timestamped-description/v3", "boundaryContract": "hardware-frame-v1", "sliceId": id, "sliceName": slice.Name, "window": window, "actions": actions, "descriptions": actions}
annotation := map[string]any{"schemaVersion": "droboticflow.timestamped-description/v3", "boundaryContract": "hardware-frame-v1", "sliceId": id, "sliceName": slice.Name, "dictionary": slice.Dictionary, "window": window, "actions": actions, "descriptions": actions}
seconds, rightSeconds, stamps := []float64{}, []float64{}, []int64{}
for j := a; j <= b; j++ {
seconds = append(seconds, grid.Seconds[j]-slice.Start)
@ -96,8 +96,8 @@ func (s *server) materializeSourceIndexRevision(ctx context.Context, t domain.An
stamps = append(stamps, grid.Timestamps[j])
}
frames := map[string]any{"left": []int{leftStart, leftEnd}, "right": []int{rightStart, rightEnd}}
sync := map[string]any{"schemaVersion": "droboticflow.synchronized-slice/v1", "sourceIndexSchema": sourceIndexSchema, "boundaryContract": "hardware-frame-v1", "collectionId": cert.CollectionID, "sliceIndex": i + 1, "sliceCount": len(slices), "sliceName": slice.Name, "window": window, "boundarySeconds": seconds, "rightBoundarySeconds": rightSeconds, "boundaryTimestampUs": stamps, "sourceFrames": frames, "sourceMediaOffsets": map[string]any{"left": slice.Start, "right": cert.Timeline.RightSeconds[a]}, "deliveryDisposition": sliceDisposition(*slice), "left": map[string]any{"pts": map[string]any{"rowCount": leftEnd - leftStart}}, "right": map[string]any{"pts": map[string]any{"rowCount": rightEnd - rightStart}}, "imu": map[string]any{"sampleCount": cert.IMURowsBeforeBoundary[b] - cert.IMURowsBeforeBoundary[a]}, "commonRange": map[string]any{"startTimestampUs": grid.Timestamps[0], "endTimestampUs": grid.Timestamps[len(grid.Timestamps)-1], "startSecond": grid.Seconds[0], "endSecond": grid.Seconds[len(grid.Seconds)-1]}, "coverage": map[string]any{"mode": "full_common_range", "contiguous": true}}
entry := map[string]any{"sliceId": id, "sourceSliceId": slice.ID, "sliceName": slice.Name, "startSecond": slice.Start, "endSecond": slice.End, "startTimestampUs": slice.StartTimestampUS, "endTimestampUs": slice.EndTimestampUS, "directory": filepath.ToSlash(filepath.Join("slices", id)), "sourceFrames": frames, "deliveryDisposition": sliceDisposition(*slice)}
sync := map[string]any{"schemaVersion": "droboticflow.synchronized-slice/v1", "sourceIndexSchema": sourceIndexSchema, "boundaryContract": "hardware-frame-v1", "collectionId": cert.CollectionID, "sliceIndex": i + 1, "sliceCount": len(slices), "sliceName": slice.Name, "dictionary": slice.Dictionary, "window": window, "boundarySeconds": seconds, "rightBoundarySeconds": rightSeconds, "boundaryTimestampUs": stamps, "sourceFrames": frames, "sourceMediaOffsets": map[string]any{"left": slice.Start, "right": cert.Timeline.RightSeconds[a]}, "deliveryDisposition": sliceDisposition(*slice), "left": map[string]any{"pts": map[string]any{"rowCount": leftEnd - leftStart}}, "right": map[string]any{"pts": map[string]any{"rowCount": rightEnd - rightStart}}, "imu": map[string]any{"sampleCount": cert.IMURowsBeforeBoundary[b] - cert.IMURowsBeforeBoundary[a]}, "commonRange": map[string]any{"startTimestampUs": grid.Timestamps[0], "endTimestampUs": grid.Timestamps[len(grid.Timestamps)-1], "startSecond": grid.Seconds[0], "endSecond": grid.Seconds[len(grid.Seconds)-1]}, "coverage": map[string]any{"mode": "full_common_range", "contiguous": true}}
entry := map[string]any{"sliceId": id, "sourceSliceId": slice.ID, "sliceName": slice.Name, "dictionary": slice.Dictionary, "startSecond": slice.Start, "endSecond": slice.End, "startTimestampUs": slice.StartTimestampUS, "endTimestampUs": slice.EndTimestampUS, "directory": filepath.ToSlash(filepath.Join("slices", id)), "sourceFrames": frames, "deliveryDisposition": sliceDisposition(*slice)}
if slice.IsDiscarded() {
entry["discardEvidence"] = discardReasonEvidence(*slice)
sync["discardEvidence"] = discardReasonEvidence(*slice)
@ -126,12 +126,20 @@ func (s *server) materializeSourceIndexRevision(ctx context.Context, t domain.An
coverage := map[string]any{"mode": "full_common_range", "contiguous": true, "gapMicroseconds": 0, "overlapMicroseconds": 0, "startSecond": grid.Seconds[0], "endSecond": grid.Seconds[previous], "durationSeconds": grid.Seconds[previous] - grid.Seconds[0], "startTimestampUs": grid.Timestamps[0], "endTimestampUs": grid.Timestamps[previous], "ptsToMediaScale": cert.Timeline.Scale, "sliceCount": len(slices), "reconstruction": map[string]any{"leftFrameCount": grid.LeftFrames[previous] - grid.LeftFrames[0], "rightFrameCount": grid.RightFrames[previous] - grid.RightFrames[0], "imuRowCount": cert.IMURowsBeforeBoundary[previous] - cert.IMURowsBeforeBoundary[0]}}
selection := map[string]any{"keptSliceCount": kept, "discardedSliceCount": discarded, "keptDurationSeconds": keepDuration, "discardedDurationSeconds": discardDuration}
capture := map[string]any{}
var originalSource map[string]any
if record, e := s.store.GetCaptureRecordByRunID(ctx, t.RunID); e != nil {
return nil, e
} else if record != nil {
originalSource, e = annotationRevisionOriginalSource(run, record)
if e != nil {
return nil, e
}
capture = map[string]any{"captureRecordId": record.ID, "captureId": record.CaptureID, "prefix": record.Prefix, "fingerprint": record.Fingerprint, "contractRevision": record.ContractRevision}
}
manifest := map[string]any{"schemaVersion": sourceIndexSchema, "revision": revision, "taskId": t.ID, "humanItemId": h.ID, "runId": t.RunID, "collectionId": cert.CollectionID, "capture": capture, "sourceFingerprint": fingerprint, "sources": cert.Sources, "sourceDurations": map[string]any{"left": cert.LeftDuration, "right": cert.RightDuration}, "status": "materialized", "sliceCount": len(items), "coverage": coverage, "deliverySelection": selection, "items": items}
if originalSource != nil {
manifest["originalSource"] = originalSource
}
manifest["sourceFrameCounts"] = map[string]any{"left": len(cert.LeftPTS), "right": len(cert.RightPTS)}
encoded, e := json.Marshal(manifest)
if e != nil || len(encoded)+1 > maxRunArtifactManifestBytes {

View File

@ -7860,6 +7860,12 @@
"type": "boolean",
"value": "false",
"description": "原生 v3:part.json 与独立审核历史,通过目录发布;须同时启用 PART_INDEX_DELIVERY"
},
{
"key": "DELIVERY_JSON_V4",
"type": "boolean",
"value": "false",
"description": "最终分类总 JSON;审核返工追溯单独保存在内部,须启用 PART_INDEX_DELIVERY"
}
],
"id": "operator-builtin-package",
@ -8170,6 +8176,16 @@
"role": "summary",
"transmission": "oss",
"type": "json"
},
{
"id": "deliveryAudit",
"name": "deliveryAudit",
"type": "json",
"container": "single",
"role": "internal-audit",
"payloadMode": "inline",
"transmission": "bus",
"description": "内部审核与返工追溯,不属于客户交付文件"
}
],
"publicationStatus": "built",

View File

@ -0,0 +1,125 @@
package api
import (
"encoding/json"
"fmt"
"net/http"
"strings"
"droboticflow/internal/deliveryindex"
"droboticflow/internal/domain"
)
// Read only the audit produced by the same completed, tenant-scoped package.
// Customer JSON deliberately contains no private run pointers.
func (s *server) classificationPackageAudit(r *http.Request, packages []domain.DeliveryPackageRecord, target deliveryDownloadTarget, partBytes []byte) ([]byte, string, error) {
var part struct {
Capture struct {
ID string `json:"captureId"`
} `json:"capture"`
PartID string `json:"partId"`
Revision int `json:"revision"`
}
if json.Unmarshal(partBytes, &part) != nil {
return nil, "", fmt.Errorf("invalid classification part")
}
var matched []byte
sourceRun := ""
for _, pkg := range packages {
run, err := s.store.GetRun(r.Context(), pkg.ChildRunID)
if err != nil {
return nil, "", err
}
if run == nil || run.OrganizationID != s.requestOrganizationID(r) || run.ProjectID != pkg.ProjectID || run.Status != domain.RunStatusCompleted {
continue
}
ownsTarget := false
for _, a := range run.Artifacts {
if !deliveryPackageArtifactAllowed(&pkg, a.ID) || a.RunID != run.ID || a.Payload == nil || a.Payload.Ref == nil {
continue
}
ref := a.Payload.Ref
if ref.ConnectionID == target.connectionID && ref.Bucket == target.bucket && strings.Trim(ref.Prefix, "/") == target.prefix && ref.ManifestKey == target.prefix+"/manifest.json" {
ownsTarget = true
}
}
if !ownsTarget {
continue
}
for _, a := range run.Artifacts {
if a.RunID != run.ID || !domain.IsInternalDeliveryAudit(a) {
continue
}
data, readErr := s.readClassificationAudit(r, run, a)
if readErr != nil {
return nil, "", readErr
}
raw, err := json.Marshal(data)
if err != nil || len(raw) > 8<<20 {
return nil, "", fmt.Errorf("内部追溯不可读或超过 8 MiB")
}
var audit struct {
Schema string `json:"schemaVersion"`
CaptureID string `json:"captureId"`
PartID string `json:"partId"`
Revision int `json:"revision"`
PartSHA string `json:"partSha256"`
Lineage struct {
RunID string `json:"sourceRunId"`
} `json:"lineage"`
}
if json.Unmarshal(raw, &audit) != nil || audit.Schema != "droboticflow.delivery.review-history/v2" || audit.CaptureID != part.Capture.ID || audit.PartID != part.PartID || audit.Revision != part.Revision || audit.PartSHA != deliveryindex.Hash(partBytes) || audit.Lineage.RunID != pkg.ParentRunID {
return nil, "", fmt.Errorf("内部追溯与固定验收结果不一致")
}
if matched != nil {
return nil, "", fmt.Errorf("交付分类对应多个内部追溯")
}
matched, sourceRun = raw, audit.Lineage.RunID
}
}
if matched == nil || sourceRun == "" {
return nil, "", fmt.Errorf("缺少同一正式交付任务的内部验收证据")
}
return matched, sourceRun, nil
}
func (s *server) readClassificationAudit(r *http.Request, run *domain.RunRecord, a domain.ExecutionArtifact) (map[string]any, error) {
if a.Payload != nil && a.Payload.Ref != nil {
ref := a.Payload.Ref
if ref.Kind != domain.ArtifactRefKindObject || ref.SizeBytes <= 0 || ref.SizeBytes > 8<<20 || len(ref.SHA256) != 64 {
return nil, fmt.Errorf("内部追溯引用缺少完整校验")
}
connection, err := s.resolveArtifactStorageConnection(r.Context(), run, a, ref.Bucket)
if err != nil || connection == nil || connection.ID != ref.ConnectionID {
return nil, fmt.Errorf("内部追溯连接不可访问")
}
browser, cleanup, err := storageConnectionBrowserResolver(r.Context(), s.storageConnectionWithResolvedEndpoint(connection))
if err != nil {
return nil, err
}
if cleanup != nil {
defer cleanup()
}
raw, _, err := browser.ReadObjectPreview(r.Context(), ref.Bucket, ref.Key, 8<<20+1)
if err != nil || int64(len(raw)) != ref.SizeBytes || deliveryindex.Hash(raw) != ref.SHA256 {
return nil, fmt.Errorf("内部追溯文件校验失败")
}
var data map[string]any
if json.Unmarshal(raw, &data) != nil {
return nil, fmt.Errorf("内部追溯 JSON 无效")
}
return data, nil
}
if a.Payload != nil && a.Payload.Mode == domain.ArtifactTransferModeInline {
raw, err := json.Marshal(a.Payload.Inline)
if err != nil || len(raw) > 8<<20 {
return nil, fmt.Errorf("内部追溯过大")
}
var data map[string]any
if json.Unmarshal(raw, &data) != nil {
return nil, fmt.Errorf("内部追溯 JSON 无效")
}
return data, nil
}
return artifactStructuredData(&a), nil
}

View File

@ -99,7 +99,7 @@ func (s *server) planIndexBatch(r *http.Request, q indexBatchRequest) (*batchPla
if e != nil {
return nil, e
}
if len(files) != 3 {
if len(files) != 2 && len(files) != 3 {
return nil, fmt.Errorf("原生 v3 part 包必须只含 part.json 与独立审核历史")
}
var partBytes, historyBytes []byte
@ -124,14 +124,24 @@ func (s *server) planIndexBatch(r *http.Request, q indexBatchRequest) (*batchPla
historyBytes = raw
}
}
if len(partBytes) == 0 || len(historyBytes) == 0 {
if len(partBytes) == 0 {
return nil, fmt.Errorf("选择的包尚未由原生 v3 算子交付;旧格式必须另行重交,不会覆盖旧文件")
}
var part struct {
Schema string `json:"schemaVersion"`
Capture struct {
ID string `json:"captureId"`
Prefix string `json:"prefix"`
} `json:"capture"`
OriginalSource struct {
Provider string `json:"provider"`
Bucket string `json:"bucket"`
Prefix string `json:"prefix"`
PartPrefix string `json:"partPrefix"`
Files map[string]struct {
Path string `json:"path"`
} `json:"files"`
} `json:"originalSource"`
PartID string `json:"partId"`
Lineage struct {
RunID string `json:"sourceRunId"`
@ -140,16 +150,53 @@ func (s *server) planIndexBatch(r *http.Request, q indexBatchRequest) (*batchPla
if json.Unmarshal(partBytes, &part) != nil || target.prefix != root+"/"+part.Capture.ID+"/"+part.PartID {
return nil, fmt.Errorf("part 身份与冻结交付目标不一致")
}
switch part.Schema {
case deliveryindex.ClassificationPartSchema:
if len(files) != 2 || len(historyBytes) != 0 {
return nil, fmt.Errorf("分类交付包中包含额外文件")
}
historyBytes, part.Lineage.RunID, e = s.classificationPackageAudit(r, selected, target, partBytes)
if e != nil {
return nil, e
}
case deliveryindex.PartSchema:
if len(files) != 3 || len(historyBytes) == 0 {
return nil, fmt.Errorf("旧 v3 交付缺少审核历史")
}
default:
return nil, fmt.Errorf("不支持的交付格式;历史交付不会被覆盖")
}
record, e := s.store.GetCaptureRecordByRunID(r.Context(), part.Lineage.RunID)
if e != nil || record == nil || record.OrganizationID != project.OrganizationID || record.ProjectID != project.ID || record.Prefix != part.Capture.Prefix+"/"+part.PartID || record.Delivery == nil || record.Delivery.Prefix != target.prefix || record.Delivery.ConnectionID != t.connectionID || record.Delivery.Bucket != t.bucket {
return nil, fmt.Errorf("part 来源与采集记录不一致")
}
if part.Schema == deliveryindex.ClassificationPartSchema && (part.OriginalSource.Provider != "BOS" || part.OriginalSource.Bucket != record.Bucket || part.OriginalSource.Prefix != part.Capture.Prefix || part.OriginalSource.PartPrefix != record.Prefix) {
return nil, fmt.Errorf("分类 JSON 原始存储来源与采集记录不一致,停止发布")
}
sourceRuns = append(sourceRuns, part.Lineage.RunID)
sourceBrowser, closeSource, e := s.deliveryReadBrowser(r.Context(), project.OrganizationID, record.ConnectionID, record.Bucket, part.Capture.Prefix, false)
if e != nil {
return nil, e
}
expected, meta, e := indexBatchSourceEvidence(r.Context(), sourceBrowser, record.Bucket, part.Capture.Prefix, part.Capture.ID)
if e == nil && part.Schema == deliveryindex.ClassificationPartSchema {
// HEAD only: prove the frozen references still resolve without reading
// video bodies, recutting media or expanding the original selection.
for _, role := range []string{"leftVideo", "rightVideo", "leftPts", "rightPts", "imu", "cameraCalibration", "imuCalibration", "metadata"} {
rel := part.OriginalSource.Files[role].Path
media := role == "leftVideo" || role == "rightVideo" || role == "leftPts" || role == "rightPts" || role == "imu"
if rel == "" || path.Clean(rel) != rel || strings.ContainsAny(rel, "\\\x00") || strings.HasPrefix(rel, "/") || strings.HasPrefix(rel, "../") || rel == ".." || media && !strings.HasPrefix(rel, part.PartID+"/") || !media && path.Base(rel) != rel {
e = fmt.Errorf("分类 JSON 原始文件路径无效:%s", role)
break
}
key := part.Capture.Prefix + "/" + rel
info, err := sourceBrowser.GetObjectSummary(r.Context(), record.Bucket, key)
if err != nil || info.Key != key || info.Kind != "file" || info.SizeBytes <= 0 {
e = fmt.Errorf("分类 JSON 原始文件不存在或不可读:%s", role)
break
}
}
}
if closeSource != nil {
closeSource()
}
@ -425,14 +472,14 @@ func readIndexBatchDownloadManifest(ctx context.Context, b objectStorageBrowser,
} `json:"project"`
Entries []deliveryindex.Entry `json:"entries"`
}
if json.Unmarshal(raw, &index) != nil || index.Schema != 3 || index.Kind != "part-index" || index.ID != id || index.Project.Root != t.prefix || index.Count != len(index.Entries) || index.Count < 1 || index.Count > 400 {
if json.Unmarshal(raw, &index) != nil || (index.Schema != 3 && index.Schema != 4) || index.Kind != "part-index" || index.ID != id || index.Project.Root != t.prefix || index.Count != len(index.Entries) || index.Count < 1 || index.Count > 400 {
return nil, fmt.Errorf("批次总索引身份或成员数量无效")
}
files := []deliveryDownloadFile{}
var size int64
for _, entry := range index.Entries {
sha, shaErr := hex.DecodeString(entry.SHA256)
if !validDeliveryDownloadPath(entry.Path) || entry.Path == "delivery_index.json" || strings.HasPrefix(entry.Path, ".") || entry.Key != t.prefix+"/"+entry.Path || entry.SizeBytes < 0 || entry.SizeBytes > 8<<20 || shaErr != nil || len(sha) != 32 {
if (index.Schema == 4 && !validClassificationBatchPath(entry.Path)) || !validDeliveryDownloadPath(entry.Path) || entry.Path == "delivery_index.json" || strings.HasPrefix(entry.Path, ".") || entry.Key != t.prefix+"/"+entry.Path || entry.SizeBytes < 0 || entry.SizeBytes > 8<<20 || shaErr != nil || len(sha) != 32 {
return nil, fmt.Errorf("批次成员校验信息无效")
}
size += entry.SizeBytes
@ -444,3 +491,15 @@ func readIndexBatchDownloadManifest(ctx context.Context, b objectStorageBrowser,
files = append(files, deliveryDownloadFile{RelativePath: t.relative + "/delivery_index.json", SizeBytes: int64(len(raw)), SHA256: deliveryindex.Hash(raw), key: t.prefix + "/delivery_index.json", bucket: t.bucket, connectionID: t.connectionID})
return files, nil
}
func validClassificationBatchPath(value string) bool {
parts := strings.Split(value, "/")
if len(parts) != 2 {
return false
}
if parts[1] == "annotation_manifest.json" {
return true
}
name := strings.TrimSuffix(parts[1], ".json")
return strings.HasSuffix(parts[1], ".json") && len(name) >= 8 && strings.HasPrefix(name, "part") && strings.Trim(name[4:], "0123456789") == ""
}

View File

@ -25,6 +25,14 @@ type indexTestBrowser struct {
ambiguous bool
}
func (b *indexTestBrowser) GetObjectSummary(_ context.Context, bucket, key string) (domain.ObjectStorageObjectSummary, error) {
body, ok := b.objects[key]
if !ok {
return domain.ObjectStorageObjectSummary{}, os.ErrNotExist
}
return domain.ObjectStorageObjectSummary{Bucket: bucket, Key: key, Kind: "file", SizeBytes: int64(len(body))}, nil
}
func (b *indexTestBrowser) PutObjectConditional(_ context.Context, _ string, key string, body []byte, _ string, match, none string, _ map[string]string) (domain.ObjectStorageObjectSummary, error) {
if match != "" || none != "*" {
return domain.ObjectStorageObjectSummary{}, fmt.Errorf("unconditional write")
@ -108,6 +116,10 @@ func seedIndexFixture(t *testing.T, ds store.Store) (*indexTestBrowser, *authent
must(ds.UpsertProject(ctx, domain.ProjectRecord{ID: "project-A", OrganizationID: org, Name: "项目A"}))
must(ds.UpsertStorageConnection(ctx, domain.StorageConnectionRecord{ID: "index-storage", OrganizationID: org, Provider: domain.StorageProviderBOS, Bucket: "test-bucket", Endpoint: "https://bj.bcebos.com"}))
dir := os.Getenv("DELIVERY_V3_FIXTURE_DIR")
v4 := os.Getenv("DELIVERY_CLASSIFICATION_TEST") == "1"
if v4 {
dir = os.Getenv("DELIVERY_V4_FIXTURE_DIR")
}
for _, part := range []string{"part0001", "part0002"} {
prefix := root + "/capture-A/" + part
sourceRun := "source-run-" + part
@ -119,19 +131,57 @@ func seedIndexFixture(t *testing.T, ds store.Store) (*indexTestBrowser, *authent
var e error
body, e = os.ReadFile(filepath.Join(dir, part, "part.json"))
must(e)
history, e = os.ReadFile(filepath.Join(dir, part, "review-history.json"))
historyPath := filepath.Join(dir, part, "review-history.json")
if v4 {
historyPath = filepath.Join(dir, "internal", part, "review-history.json")
}
history, e = os.ReadFile(historyPath)
must(e)
}
if v4 && dir == "" {
var doc map[string]any
must(json.Unmarshal(body, &doc))
acceptance := doc["acceptance"]
delete(doc, "acceptance")
delete(doc, "lineage")
doc["schemaVersion"] = deliveryindex.ClassificationPartSchema
enrichConsumerFixture(doc)
body, _ = json.Marshal(doc)
history, _ = json.Marshal(map[string]any{"schemaVersion": "droboticflow.delivery.review-history/v2", "captureId": "capture-A", "partId": part, "revision": 1, "partSha256": deliveryindex.Hash(body), "acceptance": acceptance, "lineage": map[string]any{"sourceRunId": sourceRun}, "qualityReviews": []any{map[string]any{"comment": "PRIVATE_REWORK"}}, "trace": []any{}})
}
b.objects[prefix+"/.delivery-owner.json"] = []byte(`{"owner":"synthetic-source"}`)
b.objects[prefix+"/part.json"] = body
b.objects[prefix+"/review-history.json"] = history
entries := []deliveryindex.Entry{{Path: "part.json", Key: prefix + "/part.json", SizeBytes: int64(len(body)), SHA256: deliveryindex.Hash(body)}, {Path: "review-history.json", Key: prefix + "/review-history.json", SizeBytes: int64(len(history)), SHA256: deliveryindex.Hash(history)}}
b.objects[prefix+"/manifest.json"], _ = json.Marshal(map[string]any{"schemaVersion": 1, "kind": "directory", "fileCount": 2, "totalBytes": len(body) + len(history), "entries": entries})
if v4 {
delete(b.objects, prefix+"/review-history.json")
entries = entries[:1]
b.objects[prefix+"/manifest.json"], _ = json.Marshal(map[string]any{"schemaVersion": 1, "kind": "directory", "fileCount": 1, "totalBytes": len(body), "entries": entries})
}
b.objects["incoming/capture-A/"+part+"/left.mp4"] = []byte("synthetic")
if v4 {
var doc struct {
Original struct {
Files map[string]struct {
Path string `json:"path"`
} `json:"files"`
} `json:"originalSource"`
}
must(json.Unmarshal(body, &doc))
for _, file := range doc.Original.Files {
b.objects["incoming/capture-A/"+file.Path] = []byte("synthetic original")
}
}
run := domain.RunRecord{ID: runID, OrganizationID: org, ProjectID: "project-A", Status: domain.RunStatusCompleted, Artifacts: []domain.ExecutionArtifact{{ID: artifactID, RunID: runID, Name: "delivery.directory.index", Payload: &domain.ArtifactPayload{Mode: domain.ArtifactTransferModeRef, Ref: &domain.ArtifactReference{Kind: domain.ArtifactRefKindDirectory, Provider: domain.StorageProviderBOS, ConnectionID: "index-storage", Bucket: "test-bucket", Prefix: prefix, ManifestKey: prefix + "/manifest.json"}}}}}
if v4 {
var audit map[string]any
must(json.Unmarshal(history, &audit))
run.Artifacts = append(run.Artifacts, domain.ExecutionArtifact{ID: "audit-" + part, RunID: runID, PortID: "deliveryAudit", Type: domain.PortTypeJSON, Data: audit})
}
run.NodeExecutions = []domain.NodeExecutionRecord{{NodeID: "package", Artifacts: run.Artifacts}}
must(ds.UpsertRun(ctx, run))
must(ds.UpsertDeliveryPackage(ctx, domain.DeliveryPackageRecord{ID: "package-" + part, OrganizationID: org, ProjectID: "project-A", ChildRunID: runID, Status: domain.DeliveryPackageAccepted, Mode: domain.RunModeProduction, FinalExportArtifactIDs: []string{artifactID}}))
must(ds.UpsertDeliveryPackage(ctx, domain.DeliveryPackageRecord{ID: "package-" + part, OrganizationID: org, ProjectID: "project-A", ParentRunID: sourceRun, ChildRunID: runID, Status: domain.DeliveryPackageAccepted, Mode: domain.RunModeProduction, FinalExportArtifactIDs: []string{artifactID}}))
must(ds.UpsertCaptureRecord(ctx, domain.CaptureRecord{ID: "record-" + part, OrganizationID: org, ProjectID: "project-A", RunID: sourceRun, CaptureID: "capture-A/" + part, ConnectionID: "index-storage", Bucket: "test-bucket", Prefix: "incoming/capture-A/" + part, SourcePrefix: "incoming/capture-A", Delivery: &domain.CaptureDeliveryTarget{ConnectionID: "index-storage", Bucket: "test-bucket", Prefix: prefix}}))
}
b.objects["incoming/capture-A/meta_capture-A.json"] = []byte(`{"synthetic":true}`)
@ -207,6 +257,16 @@ func exerciseIndexBatch(t *testing.T, ds store.Store) {
if download.Code != 200 || !strings.Contains(download.Body.String(), "delivery_index.json") {
t.Fatal(download.Code, download.Body.String())
}
if os.Getenv("DELIVERY_CLASSIFICATION_TEST") == "1" {
for _, file := range files {
if strings.Contains(file.RelativePath, "history") || bytes.Contains(b.objects[file.key], []byte("PRIVATE_REWORK")) {
t.Fatal("private audit delivered")
}
}
if strings.Contains(download.Body.String(), "history") {
t.Fatal("download exposes private audit")
}
}
// A later partial snapshot does not alter the old full snapshot.
first := indexCall(h, p, "GET", "/api/delivery-index-batches?projectId=project-A&packageId=package-part0001", nil)
var partial struct {
@ -299,3 +359,98 @@ func TestIndexBatchDelegationKeepsSourceAndDeliveryScopesSeparate(t *testing.T)
t.Fatal("sibling batch prefix escaped delivery grant", w.Code, w.Body.String())
}
}
func TestClassificationBatchHTTPPublicationRecoveryAndDownload(t *testing.T) {
t.Setenv("DELIVERY_CLASSIFICATION_TEST", "1")
exerciseIndexBatch(t, store.NewMemoryStore())
}
func TestClassificationBatchRejectsMissingOrChangedOriginalSource(t *testing.T) {
t.Setenv("DELIVERY_CLASSIFICATION_TEST", "1")
t.Setenv("DELIVERY_V4_FIXTURE_DIR", "")
for _, failure := range []string{"bucket", "missing-file"} {
t.Run(failure, func(t *testing.T) {
ds := store.NewMemoryStore()
b, p, _ := seedIndexFixture(t, ds)
if failure == "bucket" {
record, err := ds.GetCaptureRecordByRunID(t.Context(), "source-run-part0001")
if err != nil {
t.Fatal(err)
}
record.Bucket = "changed-bucket"
if err = ds.UpsertCaptureRecord(t.Context(), *record); err != nil {
t.Fatal(err)
}
} else {
delete(b.objects, "incoming/capture-A/part0001/leftVideo")
}
w := indexCall(NewHandler(ds, nil), p, "GET", "/api/delivery-index-batches?projectId=project-A&packageId=package-part0001", nil)
if w.Code != 409 || len(b.puts) != 0 || !strings.Contains(w.Body.String(), "原始") {
t.Fatalf("incomplete source accepted: %d %s", w.Code, w.Body.String())
}
})
}
}
func enrichConsumerFixture(p map[string]any) {
part := p["partId"].(string)
p["consumerContract"] = "droboticflow.classification-consumer/v1"
p["timeBases"] = map[string]string{"taskSplitSeconds": "original_left_video", "segmentSeconds": "slice_left_video", "segmentPartSeconds": "original_left_video", "timestampUs": "original_hardware_pts", "interval": "half-open", "frameRanges": "zero-based-half-open"}
p["pathBases"] = map[string]string{"sources": "original_capture", "sourceMetadata": "original_capture", "calibration": "original_capture"}
files := map[string]any{}
sources := map[string]any{}
for _, role := range []string{"leftVideo", "rightVideo", "leftPts", "rightPts", "imu", "cameraCalibration", "imuCalibration", "metadata"} {
name := part + "/" + role
if role == "cameraCalibration" || role == "imuCalibration" {
name = role + ".yaml"
}
if role == "metadata" {
name = "meta_capture-A.json"
}
files[role] = map[string]any{"path": name}
if role == "leftVideo" || role == "rightVideo" || role == "leftPts" || role == "rightPts" || role == "imu" {
sources[role] = files[role]
}
}
p["sources"] = sources
p["sourceMetadata"] = files["metadata"]
p["originalSource"] = map[string]any{"provider": "BOS", "bucket": "test-bucket", "prefix": "incoming/capture-A", "partPrefix": "incoming/capture-A/" + part, "files": files}
for _, kind := range []string{"slices", "discardedSlices"} {
for _, raw := range p[kind].([]any) {
item := raw.(map[string]any)
split := item["taskSplit"].(map[string]any)
number := func(v any) float64 {
switch n := v.(type) {
case float64:
return n
case int:
return float64(n)
default:
panic("invalid fixture time")
}
}
start, end := number(split["startSecond"]), number(split["endSecond"])
split["startTimestampUs"] = int64(1e6 + start*1e6)
split["endTimestampUs"] = int64(1e6 + end*1e6)
if kind == "discardedSlices" {
continue
}
class := item["classification"].(map[string]any)
class["dictionary"] = map[string]any{"id": "base", "kind": "slice", "version": 1, "name": "操作", "tags": []string{"操作"}}
class["tags"] = []string{"操作"}
actions := item["segments"].([]any)
for i, raw := range actions {
a := raw.(map[string]any)
low, high := float64(i)*(end-start)/float64(len(actions)), float64(i+1)*(end-start)/float64(len(actions))
a["startSecond"] = low
a["endSecond"] = high
a["partStartSecond"] = start + low
a["partEndSecond"] = start + high
a["startTimestampUs"] = int64(1e6 + (start+low)*1e6)
a["endTimestampUs"] = int64(1e6 + (start+high)*1e6)
a["tags"] = []string{"操作"}
a["tagSource"] = "slice"
}
}
}
}

View File

@ -114,7 +114,7 @@ func (server *server) handleDeliveryPackageContents(writer http.ResponseWriter,
artifacts := make([]domain.ExecutionArtifact, 0, len(allowed))
var manifest map[string]any
for _, artifact := range run.Artifacts {
if _, exists := allowed[artifact.ID]; !exists || artifact.RunID != run.ID {
if _, exists := allowed[artifact.ID]; !exists || artifact.RunID != run.ID || domain.IsInternalDeliveryAudit(artifact) {
continue
}
copy := artifact

View File

@ -0,0 +1,80 @@
package api
import (
"encoding/json"
"fmt"
"droboticflow/internal/domain"
)
const dictionarySelectionHistoryKey = "dictionarySelectionHistory"
// Only server-resolved selections enter this item-scoped recovery ledger.
// It allows undo after a shared word changes or leaves the project, without
// treating the client's undo snapshot as trustworthy dictionary metadata.
func rememberDictionarySelections(item *domain.HumanItemRecord, selections ...*domain.DictionarySelection) []validationIssue {
if item == nil {
return nil
}
history := decodeMaterializationValue[domain.DictionarySelection](item.Subject.Payload[dictionarySelectionHistoryKey])
seen := map[string]bool{}
key := func(d domain.DictionarySelection) string { return fmt.Sprintf("%s/%s/%d", d.Kind, d.ID, d.Version) }
for _, d := range history {
seen[key(d)] = true
}
for _, d := range selections {
if d != nil && !seen[key(*d)] {
copy := *d
copy.Tags = append([]string{}, d.Tags...)
history = append(history, copy)
seen[key(copy)] = true
}
}
body, err := json.Marshal(history)
if err != nil || len(body) > 1<<20 {
return []validationIssue{{Field: "dictionary", Message: "分类恢复记录超过限制,请拆分任务后重试"}}
}
if item.Subject.Payload == nil {
item.Subject.Payload = map[string]any{}
}
if len(history) > 0 {
item.Subject.Payload[dictionarySelectionHistoryKey] = history
}
return nil
}
func savedDictionarySelection(item *domain.HumanItemRecord, requested *domain.DictionarySelection, kind string) *domain.DictionarySelection {
if item == nil || requested == nil {
return nil
}
for _, saved := range decodeMaterializationValue[domain.DictionarySelection](item.Subject.Payload[dictionarySelectionHistoryKey]) {
if saved.Kind == kind && saved.ID == requested.ID && saved.Version == requested.Version {
copy := saved
return &copy
}
}
return nil
}
func sliceDictionarySelections(slices []domain.SliceSegment) []*domain.DictionarySelection {
selections := []*domain.DictionarySelection{}
for _, slice := range slices {
selections = append(selections, slice.Dictionary, slice.DiscardReasonDictionary)
}
return selections
}
func actionDictionarySelections(actions []domain.AnnotationSegment) []*domain.DictionarySelection {
selections := []*domain.DictionarySelection{}
for _, action := range actions {
selections = append(selections, action.Dictionary)
}
return selections
}
func copyClassificationRecoveryState(target, source *domain.HumanItemRecord) {
for _, key := range []string{dictionarySelectionHistoryKey, "unconfirmedSliceNames"} {
if value, ok := source.Subject.Payload[key]; ok {
target.Subject.Payload[key] = value
} else {
delete(target.Subject.Payload, key)
}
}
}

View File

@ -0,0 +1,106 @@
package api
import (
"context"
"fmt"
"strings"
"droboticflow/internal/domain"
)
// Always resolve new selections on the server. Existing saved selections are
// copied from the workspace/revision, so dictionary maintenance cannot rewrite
// a submitted classification and a client cannot forge tags or another tenant.
func (s *server) projectDictionarySelections(ctx context.Context, item *domain.HumanItemRecord, kind string) (map[string]domain.AnnotationLabelRecord, error) {
selection, err := s.store.GetProjectAnnotationLabels(ctx, item.ProjectID)
if err != nil {
return nil, err
}
result := map[string]domain.AnnotationLabelRecord{}
if selection == nil {
return result, nil
}
if selection.OrganizationID != item.OrganizationID {
return nil, fmt.Errorf("项目字典不属于当前组织")
}
ids := selection.LabelIDs
if kind == "slice" {
ids = selection.SliceLabelIDs
}
for _, id := range ids {
label, err := s.store.GetAnnotationLabel(ctx, id)
if err != nil {
return nil, err
}
if label != nil && label.OrganizationID == item.OrganizationID && dictionaryKind(label.Kind) == kind && label.Status == domain.AnnotationLabelActive {
result[id] = *label
}
}
return result, nil
}
func sameDictionaryVersion(a, b *domain.DictionarySelection) bool {
return a != nil && b != nil && a.ID == b.ID && a.Version == b.Version
}
func (s *server) canonicalActionDictionaries(ctx context.Context, item *domain.HumanItemRecord, segments []domain.AnnotationSegment, mode ...string) []validationIssue {
previous := decodeMaterializationValue[domain.AnnotationSegment](annotationWorkspacePayload(item)["segments"])
if len(previous) == 0 && item.LegacyTaskID != "" {
revisions, err := s.store.ListAnnotationRevisions(ctx, item.OrganizationID, item.LegacyTaskID)
if err != nil {
return []validationIssue{{Field: "segments", Message: "无法读取已保存的动作分类"}}
}
for _, rev := range revisions {
if rev.Revision == item.CurrentRevision {
previous = rev.Segments
}
}
}
if issues := rememberDictionarySelections(item, actionDictionarySelections(previous)...); len(issues) > 0 {
return issues
}
old := map[string]domain.AnnotationSegment{}
for _, seg := range previous {
old[seg.ID] = seg
}
var allowed map[string]domain.AnnotationLabelRecord
for i := range segments {
seg := &segments[i]
p := old[seg.ID]
if seg.ClearDictionary {
seg.Dictionary = nil
seg.ClearDictionary = false
continue
}
if len(mode) > 0 && mode[0] == "restore" && !seg.IsInvalid && strings.TrimSpace(seg.Description) != "" {
if saved := savedDictionarySelection(item, seg.Dictionary, "action"); saved != nil {
seg.Dictionary = saved
continue
}
}
if seg.IsInvalid || strings.TrimSpace(seg.Description) == "" {
seg.Dictionary = nil
continue
}
if p.Dictionary != nil && (sameDictionaryVersion(p.Dictionary, seg.Dictionary) || (seg.Dictionary == nil && seg.Description == p.Description && !p.IsInvalid)) {
seg.Dictionary = p.Dictionary
continue
}
if seg.Dictionary == nil {
continue
} // Legacy/free text is never guessed into a category.
if allowed == nil {
var err error
allowed, err = s.projectDictionarySelections(ctx, item, "action")
if err != nil {
return []validationIssue{{Field: "segments", Message: "无法读取动作字典,请重试"}}
}
}
label, ok := allowed[seg.Dictionary.ID]
if !ok || label.Version != seg.Dictionary.Version {
return []validationIssue{{Field: fmt.Sprintf("segments.%d.dictionary", i), Code: "dictionary_selection_stale", Message: "动作词条已变化,请从项目字典重新选择"}}
}
seg.Dictionary = label.Selection()
}
return rememberDictionarySelections(item, actionDictionarySelections(segments)...)
}

View File

@ -0,0 +1,411 @@
package api
import (
"encoding/json"
"net/http"
"reflect"
"testing"
"droboticflow/internal/domain"
)
func TestDictionarySelectionsPersistThroughHTTPAndDictionaryMaintenance(t *testing.T) {
f := newCombinedDraftFixture(t)
labels := []domain.AnnotationLabelRecord{
{ID: "base", OrganizationID: f.item.OrganizationID, Kind: "slice", Code: "base", Name: "物品操作", Tags: []string{"商品"}, Version: 2, Status: domain.AnnotationLabelActive},
{ID: "action", OrganizationID: f.item.OrganizationID, Kind: "action", Code: "pick", Name: "拿取", Tags: []string{"抓取"}, Version: 3, Status: domain.AnnotationLabelActive},
}
for _, label := range labels {
if err := f.store.UpsertAnnotationLabel(f.ctx, label); err != nil {
t.Fatal(err)
}
}
selection := domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID, LabelIDs: []string{"action"}, SliceLabelIDs: []string{"base"}}
if err := f.store.UpsertProjectAnnotationLabels(f.ctx, selection); err != nil {
t.Fatal(err)
}
if err := f.store.UpsertProjectSliceLabels(f.ctx, selection); err != nil {
t.Fatal(err)
}
forgedBase := *labels[0].Selection()
forgedBase.Tags = []string{"FORGED"}
forgedAction := *labels[1].Selection()
forgedAction.Tags = []string{"FORGED"}
payload := combinedAnnotationDraftInput{Version: f.item.Version, SliceSegments: []domain.SliceSegment{{ID: "s", Name: "物品操作", Start: 0, End: 10, Dictionary: &forgedBase}}, Segments: []domain.AnnotationSegment{{ID: "a", Start: 1, End: 2, Description: "拿起杯子", Dictionary: &forgedAction}}}
endpoint := "/api/plugins/annotation-qa/tasks/" + f.task.ID + "/drafts"
save := func(actor domain.UserRecord) *domain.HumanItemRecord {
t.Helper()
r := f.request(http.MethodPut, endpoint, actor, payload)
if r.Code != 200 {
t.Fatalf("save: %d %s", r.Code, r.Body.String())
}
item, err := f.store.GetHumanItem(f.ctx, f.item.ID)
if err != nil {
t.Fatal(err)
}
return item
}
saved := save(f.annotator)
draft := annotationWorkspacePayload(saved)
slices := decodeMaterializationValue[domain.SliceSegment](draft["sliceSegments"])
actions := decodeMaterializationValue[domain.AnnotationSegment](draft["segments"])
if !reflect.DeepEqual(slices[0].Dictionary, labels[0].Selection()) || !reflect.DeepEqual(actions[0].Dictionary, labels[1].Selection()) {
t.Fatal("server trusted forged tags")
}
// Empty action text never clears the independent base classification.
payload.Version = saved.Version
payload.SliceSegments = slices
payload.Segments = nil
saved = save(f.annotator)
slices = decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(saved)["sliceSegments"])
if slices[0].Dictionary == nil || slices[0].Dictionary.Tags[0] != "商品" {
t.Fatal("blank description lost base class")
}
for i := range labels {
labels[i].Version++
labels[i].Name += "新版"
labels[i].Tags = []string{"新标签"}
labels[i].Status = domain.AnnotationLabelInactive
f.store.UpsertAnnotationLabel(f.ctx, labels[i])
}
f.store.UpsertProjectSliceLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID})
payload.Version = saved.Version
payload.SliceSegments = slices
saved = save(f.annotator)
// A fresh handler recovers the frozen snapshot, independent of active dictionaries.
f.handler = NewHandler(f.store, nil)
payload.Version = saved.Version
saved = save(f.annotator)
slices = decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(saved)["sliceSegments"])
if slices[0].Dictionary.Version != 2 || slices[0].Dictionary.Name != "物品操作" {
t.Fatal("historical classification rewritten")
}
payload.Version = saved.Version
payload.Segments = []domain.AnnotationSegment{{ID: "new-action", Start: 1, End: 2, Description: "拿取", Dictionary: &forgedAction}}
if r := f.request(http.MethodPut, endpoint, f.annotator, payload); r.Code != 400 {
t.Fatalf("stale/new action accepted %d %s", r.Code, r.Body.String())
}
if r := f.request(http.MethodPut, endpoint, f.reviewer, payload); r.Code != 403 {
t.Fatal("reviewer can change classification")
}
}
func TestDictionaryTagsHaveIndependentRevisionAndAdminAuthorization(t *testing.T) {
f := newCombinedDraftFixture(t)
body := map[string]any{"kind": "slice", "name": "分拣", "code": "sort", "category": "business-category", "aliases": []string{"别名"}, "tags": []string{" 商品 ", "商品", "分拣"}}
r := f.request("POST", "/api/annotation-labels", f.annotator, body)
if r.Code != 403 {
t.Fatal("annotator maintained dictionary")
}
r = f.request("POST", "/api/annotation-labels", f.admin, body)
if r.Code != 201 {
t.Fatalf("%d %s", r.Code, r.Body.String())
}
var label domain.AnnotationLabelRecord
if err := json.Unmarshal(r.Body.Bytes(), &label); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(label.Tags, []string{"商品", "分拣"}) || label.Category != "business-category" || len(label.Aliases) != 1 {
t.Fatal("tags confused with other fields")
}
body["tags"] = []string{}
r = f.request("PATCH", "/api/annotation-labels/"+label.ID, f.admin, body)
if r.Code != 200 {
t.Fatalf("%d %s", r.Code, r.Body.String())
}
var updated domain.AnnotationLabelRecord
json.Unmarshal(r.Body.Bytes(), &updated)
if updated.Version != 2 || len(updated.Tags) != 0 || !reflect.DeepEqual(updated.History[0].Tags, label.Tags) {
t.Fatal("tag history lost")
}
}
func TestLegacyClassificationDoesNotGuessCurrentTagsAtConfirmation(t *testing.T) {
f := newCombinedDraftFixture(t)
legacy := domain.SliceSegment{ID: "legacy", Name: "物品操作", Start: 0, End: 10}
f.item.Subject.Payload["annotationWorkspace"] = map[string]any{"sliceSegments": []domain.SliceSegment{legacy}}
f.store.UpsertHumanItem(f.ctx, f.item)
f.store.UpsertAnnotationLabel(f.ctx, domain.AnnotationLabelRecord{ID: "current", OrganizationID: f.item.OrganizationID, Kind: "slice", Code: "base", Name: legacy.Name, Tags: []string{"现在新增的标签"}, Version: 9, Status: domain.AnnotationLabelActive})
f.store.UpsertProjectSliceLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID, SliceLabelIDs: []string{"current"}})
server := &server{store: f.store}
slices := []domain.SliceSegment{legacy}
if issues := server.validateProjectSliceNames(f.ctx, &f.item, slices, "confirm"); len(issues) > 0 || slices[0].Dictionary != nil {
t.Fatal("legacy tags guessed at confirm")
}
slices[0].Dictionary = &domain.DictionarySelection{ID: "current", Version: 9}
if issues := server.validateProjectSliceNames(f.ctx, &f.item, slices, "confirm"); len(issues) > 0 || slices[0].Dictionary == nil || slices[0].Dictionary.Tags[0] != "现在新增的标签" {
t.Fatal("explicit new selection did not freeze current tags")
}
}
func TestSliceEditApplicationCanonicalizesAgainstStoredDictionaryState(t *testing.T) {
for _, foreign := range []bool{false, true} {
t.Run(map[bool]string{false: "forged-tags", true: "foreign-word"}[foreign], func(t *testing.T) {
f := continuousFixture(t)
selection, _ := f.store.GetProjectAnnotationLabels(f.ctx, f.task.ProjectID)
label, _ := f.store.GetAnnotationLabel(f.ctx, selection.SliceLabelIDs[0])
label.Tags = []string{"服务端标签"}
label.Version = 2
f.store.UpsertAnnotationLabel(f.ctx, *label)
before := []domain.SliceSegment{{ID: "s", Name: label.Name, Start: 0, End: 10}}
item := f.item
item.Subject.Payload["sliceWorkflow"] = map[string]any{"status": "completed", "phase": "annotate"}
item.Subject.Payload["annotationWorkspace"] = map[string]any{"sliceSegments": before, "segments": []domain.AnnotationSegment{}}
f.store.UpsertHumanItem(f.ctx, item)
incoming := *label.Selection()
incoming.Tags = []string{"FORGED"}
if foreign {
incoming.ID = "foreign-label"
}
proposed := append([]domain.SliceSegment(nil), before...)
proposed[0].Dictionary = &incoming
body := sliceEditApplicationInput{ApplicationID: "classification-apply", combinedAnnotationDraftInput: combinedAnnotationDraftInput{Version: item.Version, SliceSegments: proposed}}
r := f.request("POST", "/api/plugins/annotation-qa/tasks/"+f.task.ID+"/slice-edit-applications", f.annotator, body)
if foreign {
if r.Code != 400 {
t.Fatalf("foreign classification accepted %d %s", r.Code, r.Body.String())
}
return
}
if r.Code != 200 {
t.Fatalf("%d %s", r.Code, r.Body.String())
}
saved, _ := f.store.GetHumanItem(f.ctx, item.ID)
got := decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(saved)["sliceSegments"])
if got[0].Dictionary == nil || !reflect.DeepEqual(got[0].Dictionary.Tags, label.Tags) {
t.Fatal("client classification snapshot trusted")
}
})
}
}
func TestProjectActionDictionaryReturnsVersionedItemsForAnnotator(t *testing.T) {
f := newCombinedDraftFixture(t)
label := domain.AnnotationLabelRecord{ID: "action-word", OrganizationID: f.item.OrganizationID, Code: "pick", Name: "拿取", Kind: "action", Version: 3, Tags: []string{"抓取"}, Status: domain.AnnotationLabelActive}
f.store.UpsertAnnotationLabel(f.ctx, label)
f.store.UpsertProjectAnnotationLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ID: "selection", ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID, LabelIDs: []string{label.ID}})
r := f.request("GET", "/api/projects/"+f.task.ProjectID+"/annotation-labels", f.annotator, nil)
var response struct {
ID string `json:"id"`
LabelIDs []string `json:"labelIds"`
Items []domain.AnnotationLabelRecord `json:"items"`
}
if json.Unmarshal(r.Body.Bytes(), &response) != nil || r.Code != 200 || response.ID != "selection" || len(response.LabelIDs) != 1 || len(response.Items) != 1 || response.Items[0].Version != 3 || response.Items[0].Tags[0] != "抓取" {
t.Fatalf("%d %s", r.Code, r.Body.String())
}
}
func TestUndoRestoresOnlyServerRememberedClassificationsAfterDictionaryMaintenance(t *testing.T) {
f := continuousFixture(t)
srv := &server{store: f.store}
base := domain.AnnotationLabelRecord{ID: "base-old", OrganizationID: f.item.OrganizationID, Kind: "slice", Code: "base", Name: "旧基础分类", Version: 2, Tags: []string{"原标签"}, Status: domain.AnnotationLabelActive}
action := domain.AnnotationLabelRecord{ID: "action-old", OrganizationID: f.item.OrganizationID, Kind: "action", Code: "pick", Name: "旧动作分类", Version: 3, Tags: []string{"原动作标签"}, Status: domain.AnnotationLabelActive}
f.store.UpsertAnnotationLabel(f.ctx, base)
f.store.UpsertAnnotationLabel(f.ctx, action)
selection := domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID, LabelIDs: []string{action.ID}, SliceLabelIDs: []string{base.ID}}
f.store.UpsertProjectAnnotationLabels(f.ctx, selection)
f.store.UpsertProjectSliceLabels(f.ctx, selection)
oldSlices := []domain.SliceSegment{{ID: "s", Name: base.Name, Start: 0, End: 10, Dictionary: base.Selection()}}
oldActions := []domain.AnnotationSegment{{ID: "a", Description: "原描述", Start: 0, End: 10, Status: "described", Dictionary: action.Selection()}}
item := f.item
if issues := srv.validateProjectSliceNames(f.ctx, &item, oldSlices, "draft"); len(issues) > 0 {
t.Fatal(issues)
}
if issues := srv.canonicalActionDictionaries(f.ctx, &item, oldActions); len(issues) > 0 {
t.Fatal(issues)
}
item.Subject.Payload["sliceWorkflow"] = map[string]any{"phase": "annotate", "status": "completed"}
// Later editing selected a different classification. The word is subsequently
// renamed, retagged, disabled and unlinked by an administrator.
item.Subject.Payload["annotationWorkspace"] = map[string]any{"sliceSegments": []domain.SliceSegment{{ID: "s", Name: "另一个分类", Start: 0, End: 10}}, "segments": []domain.AnnotationSegment{{ID: "a", Description: "修改后描述", Start: 0, End: 10, Status: "described"}}}
f.store.UpsertHumanItem(f.ctx, item)
base.Name = "新版"
base.Tags = []string{"新标签"}
base.Version++
base.Status = domain.AnnotationLabelInactive
f.store.UpsertAnnotationLabel(f.ctx, base)
action.Version++
action.Status = domain.AnnotationLabelInactive
f.store.UpsertAnnotationLabel(f.ctx, action)
f.store.UpsertProjectSliceLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID})
f.store.UpsertProjectAnnotationLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID})
// Simulate serialization and restart; do not use process-local snapshot state.
f.handler = NewHandler(f.store, nil)
oldSlices[0].Dictionary.Tags = []string{"FORGED"}
oldActions[0].Dictionary.Tags = []string{"FORGED"}
body := combinedAnnotationDraftInput{Version: item.Version, SliceSegments: oldSlices, Segments: oldActions}
r := f.request("POST", "/api/plugins/annotation-qa/tasks/"+f.task.ID+"/slice-edit-restorations", f.annotator, body)
if r.Code != 200 {
t.Fatalf("undo %d %s", r.Code, r.Body.String())
}
saved, _ := f.store.GetHumanItem(f.ctx, item.ID)
slices := decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(saved)["sliceSegments"])
actions := decodeMaterializationValue[domain.AnnotationSegment](annotationWorkspacePayload(saved)["segments"])
if slices[0].Dictionary.Version != 2 || slices[0].Dictionary.Tags[0] != "原标签" || actions[0].Dictionary.Version != 3 || actions[0].Dictionary.Tags[0] != "原动作标签" {
t.Fatal("undo lost or forged frozen classification")
}
}
func TestDirectDispositionCannotPersistForeignOrForgedSliceDictionary(t *testing.T) {
for _, foreign := range []bool{false, true} {
t.Run(map[bool]string{false: "forged-tags", true: "foreign-word"}[foreign], func(t *testing.T) {
f := continuousFixture(t)
body := dispositionFixturePayload(t, f)
selection, _ := f.store.GetProjectAnnotationLabels(f.ctx, f.task.ProjectID)
label, _ := f.store.GetAnnotationLabel(f.ctx, selection.SliceLabelIDs[0])
label.Tags = []string{"服务端标签"}
label.Version = 2
f.store.UpsertAnnotationLabel(f.ctx, *label)
body.SliceSegments[0].Name = label.Name
d := *label.Selection()
d.Tags = []string{"FORGED"}
if foreign {
d.ID = "foreign"
}
body.SliceSegments[0].Dictionary = &d
r := f.request("POST", "/api/plugins/annotation-qa/tasks/"+f.task.ID+"/slice-dispositions", f.annotator, body)
if foreign {
if r.Code != 400 {
t.Fatalf("foreign dictionary saved %d %s", r.Code, r.Body.String())
}
return
}
if r.Code != 200 {
t.Fatalf("%d %s", r.Code, r.Body.String())
}
item, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
saved := decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(item)["sliceSegments"])
if saved[0].Dictionary == nil || !reflect.DeepEqual(saved[0].Dictionary.Tags, label.Tags) {
t.Fatal("forged tags persisted through direct disposition")
}
})
}
}
func TestDispositionUndoRedoPreservesClassificationsAfterDictionaryRemoval(t *testing.T) {
for _, application := range []bool{false, true} {
t.Run(map[bool]string{false: "direct", true: "panel-application"}[application], func(t *testing.T) {
f := continuousFixture(t)
body := dispositionFixturePayload(t, f)
selection, _ := f.store.GetProjectAnnotationLabels(f.ctx, f.task.ProjectID)
base, _ := f.store.GetAnnotationLabel(f.ctx, selection.SliceLabelIDs[0])
base.Tags = []string{"基础标签"}
base.Version = 2
f.store.UpsertAnnotationLabel(f.ctx, *base)
reason, _ := f.store.GetAnnotationLabel(f.ctx, body.ReasonLabelID)
reason.Tags = []string{"舍弃标签"}
f.store.UpsertAnnotationLabel(f.ctx, *reason)
for i := range body.SliceSegments {
body.SliceSegments[i].Name = base.Name
body.SliceSegments[i].Dictionary = base.Selection()
}
original := append([]domain.SliceSegment(nil), body.SliceSegments...)
endpoint := "/api/plugins/annotation-qa/tasks/" + f.task.ID
r := f.request("POST", endpoint+"/slice-dispositions", f.annotator, body)
if r.Code != 200 {
t.Fatalf("discard %d %s", r.Code, r.Body.String())
}
var first struct {
Item domain.HumanItemRecord `json:"item"`
Workspace combinedAnnotationDraftInput `json:"workspace"`
}
json.Unmarshal(r.Body.Bytes(), &first)
base.Name = "新名字"
base.Version++
base.Tags = []string{"新标签"}
base.Status = domain.AnnotationLabelInactive
f.store.UpsertAnnotationLabel(f.ctx, *base)
reason.Version++
reason.Status = domain.AnnotationLabelInactive
f.store.UpsertAnnotationLabel(f.ctx, *reason)
f.store.UpsertProjectSliceLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID})
f.store.UpsertProjectDiscardReasonLabels(f.ctx, domain.ProjectAnnotationLabelsRecord{ProjectID: f.task.ProjectID, OrganizationID: f.item.OrganizationID})
item, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
item.Subject.Payload["sliceWorkflow"] = map[string]any{"phase": "annotate", "status": "completed"}
f.store.UpsertHumanItem(f.ctx, *item)
f.handler = NewHandler(f.store, nil)
if application {
input := sliceEditApplicationInput{ApplicationID: "undo-discard", RestoreClassifications: true, combinedAnnotationDraftInput: combinedAnnotationDraftInput{Version: item.Version, SliceSegments: original}, Dispositions: []sliceDispositionInput{{SliceID: body.SliceID, Disposition: "keep"}}}
r = f.request("POST", endpoint+"/slice-edit-applications", f.annotator, input)
} else {
undo := body
undo.combinedAnnotationDraftInput = first.Workspace
undo.Version = item.Version
undo.Disposition = "keep"
undo.RestoreClassifications = true
undo.RestorePreviousName = true
undo.RestoreNameFromVersion = first.Item.Version
r = f.request("POST", endpoint+"/slice-dispositions", f.annotator, undo)
}
if r.Code != 200 {
t.Fatalf("undo %d %s", r.Code, r.Body.String())
}
var restored struct {
Item domain.HumanItemRecord `json:"item"`
Workspace combinedAnnotationDraftInput `json:"workspace"`
}
json.Unmarshal(r.Body.Bytes(), &restored)
d := restored.Workspace.SliceSegments[1].Dictionary
if d == nil || d.Version != 2 || d.Tags[0] != "基础标签" {
t.Fatal("undo lost frozen base category")
}
if application {
input := sliceEditApplicationInput{ApplicationID: "redo-discard", RestoreClassifications: true, combinedAnnotationDraftInput: combinedAnnotationDraftInput{Version: restored.Item.Version, SliceSegments: first.Workspace.SliceSegments}, Dispositions: []sliceDispositionInput{{SliceID: body.SliceID, Disposition: "discard", ReasonLabelID: body.ReasonLabelID, ReasonVersion: body.ReasonVersion}}}
r = f.request("POST", endpoint+"/slice-edit-applications", f.annotator, input)
} else {
redo := body
redo.combinedAnnotationDraftInput = restored.Workspace
redo.Version = restored.Item.Version
redo.RestoreClassifications = true
r = f.request("POST", endpoint+"/slice-dispositions", f.annotator, redo)
}
if r.Code != 200 {
t.Fatalf("redo %d %s", r.Code, r.Body.String())
}
var redone struct {
Workspace combinedAnnotationDraftInput `json:"workspace"`
}
json.Unmarshal(r.Body.Bytes(), &redone)
d = redone.Workspace.SliceSegments[1].DiscardReasonDictionary
if d == nil || d.Version != 1 || d.Tags[0] != "舍弃标签" {
t.Fatal("redo lost frozen discard reason")
}
})
}
}
func TestUnconfirmedRestoredNamesCannotBecomeFinalClassification(t *testing.T) {
f := newCombinedDraftFixture(t)
srv := &server{store: f.store}
slices := []domain.SliceSegment{{ID: "s", Name: "未选择字典的临时名称", Start: 0, End: 10}}
if issues := srv.validateProjectSliceNames(f.ctx, &f.item, slices, "restore"); len(issues) > 0 {
t.Fatal(issues)
}
f.item.Subject.Payload["annotationWorkspace"] = map[string]any{"sliceSegments": slices}
for _, mode := range []string{"confirm", "submit"} {
if issues := srv.validateProjectSliceNames(f.ctx, &f.item, slices, mode); len(issues) == 0 {
t.Fatalf("unfinished name bypassed %s", mode)
}
}
}
func TestLegacyDispositionCannotSeedDictionaryRecoveryHistory(t *testing.T) {
f := newCombinedDraftFixture(t)
f.task.TemplateVersion = 5
f.store.UpsertAnnotationTask(f.ctx, f.task)
body := dispositionFixturePayload(t, f)
body.Segments[0].Handedness = "both"
body.SliceSegments[0].Name = "原有名称"
body.SliceSegments[0].TaskLabelID = "legacy-task-label"
body.SliceSegments[0].TaskLabel = "原有任务分类"
body.SliceSegments[0].Dictionary = &domain.DictionarySelection{ID: "FORGED", Kind: "action", Version: 9, Name: "伪造分类", Code: "forged", Tags: []string{"FORGED"}}
r := f.request("POST", "/api/plugins/annotation-qa/tasks/"+f.task.ID+"/slice-dispositions", f.annotator, body)
if r.Code != 200 {
t.Fatalf("legacy save %d %s", r.Code, r.Body.String())
}
saved, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
slices := decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(saved)["sliceSegments"])
if slices[0].Dictionary != nil || slices[0].Name != "原有名称" || slices[0].TaskLabelID != "legacy-task-label" || savedDictionarySelection(saved, &domain.DictionarySelection{ID: "FORGED", Version: 9}, "action") != nil {
t.Fatal("legacy task seeded a dictionary snapshot or lost legacy labels")
}
}

View File

@ -11,6 +11,19 @@ func (s *server) resolveDiscardReason(ctx context.Context, item *domain.HumanIte
issue := func(message string) (*domain.AnnotationLabelRecord, []validationIssue) {
return nil, []validationIssue{{Field: "reasonLabelId", Code: "discard_reason_dictionary_required", Message: message}}
}
if body.RestoreClassifications {
if prior := historicalDiscardReason(item, body); prior != nil {
if prior.DiscardReasonDictionary != nil {
d := prior.DiscardReasonDictionary
return &domain.AnnotationLabelRecord{ID: d.ID, Kind: d.Kind, Code: d.Code, Name: d.Name, Version: d.Version, Tags: append([]string{}, d.Tags...), OrganizationID: item.OrganizationID}, nil
}
if prior.DiscardReasonLabelID == "" {
return nil, nil
}
return &domain.AnnotationLabelRecord{ID: prior.DiscardReasonLabelID, Kind: "discard_reason", Name: prior.DiscardReasonName, Version: prior.DiscardReasonVersion, OrganizationID: item.OrganizationID}, nil
}
return issue("未找到当前任务已保存的舍弃原因,无法恢复该历史选择")
}
selection, err := s.store.GetProjectAnnotationLabels(ctx, item.ProjectID)
if err != nil {
return issue("舍弃原因字典加载失败,请重试")
@ -73,5 +86,26 @@ func describeSliceNameIssues(issues []validationIssue, slices []domain.SliceSegm
}
func discardReasonEvidence(slice domain.SliceSegment) map[string]any {
return map[string]any{"reason": slice.DiscardReason, "reasonLabelId": slice.DiscardReasonLabelID, "reasonVersion": slice.DiscardReasonVersion, "reasonName": slice.DiscardReasonName, "reasonNote": slice.DiscardReasonNote, "at": slice.DiscardedAt, "previousAnnotations": slice.DiscardedAnnotations}
return map[string]any{"dictionary": slice.DiscardReasonDictionary, "reason": slice.DiscardReason, "reasonLabelId": slice.DiscardReasonLabelID, "reasonVersion": slice.DiscardReasonVersion, "reasonName": slice.DiscardReasonName, "reasonNote": slice.DiscardReasonNote, "at": slice.DiscardedAt, "previousAnnotations": slice.DiscardedAnnotations}
}
// Restore only a reason/note combination actually recorded for this slice;
// administrator changes never let a client invent historical reason metadata.
func historicalDiscardReason(item *domain.HumanItemRecord, body sliceDispositionInput) *domain.SliceSegment {
history, _ := normalizeStructuredValue(item.Subject.Payload["sliceDispositionHistory"]).([]any)
for i := len(history) - 1; i >= 0; i-- {
entry := mapValue(history[i])
if stringValue(entry["sliceId"]) != body.SliceID {
continue
}
after := decodeMaterializationValue[domain.SliceSegment]([]any{entry["after"]})
if len(after) != 1 {
continue
}
saved := after[0]
if saved.IsDiscarded() && saved.DiscardReasonLabelID == body.ReasonLabelID && saved.DiscardReasonVersion == body.ReasonVersion && saved.DiscardReasonNote == strings.TrimSpace(body.ReasonNote) && (saved.DiscardReasonLabelID != "" || saved.DiscardReason == strings.TrimSpace(body.Reason)) {
return &saved
}
}
return nil
}

View File

@ -30,6 +30,7 @@ func TestDiscardReasonDictionaryValidationAndSnapshots(t *testing.T) {
case "other_without_note":
label.RequiresNote = true
case "success":
label.Tags = []string{"不可用"}
body.Reason = "forged"
body.ReasonNote = "遮挡持续两秒"
}
@ -54,6 +55,9 @@ func TestDiscardReasonDictionaryValidationAndSnapshots(t *testing.T) {
t.Fatal(err)
}
slice := got.Workspace.SliceSegments[1]
if slice.DiscardReasonDictionary == nil || len(slice.DiscardReasonDictionary.Tags) != 1 || slice.DiscardReasonDictionary.Tags[0] != "不可用" {
t.Fatal("discard tags missing")
}
if slice.DiscardReasonName != "相机被遮挡" || slice.DiscardReasonVersion != 1 || slice.DiscardReasonLabelID != label.ID || slice.DiscardReasonNote != body.ReasonNote || strings.Contains(slice.DiscardReason, "forged") {
t.Fatalf("snapshot %+v", slice)
}

View File

@ -948,9 +948,9 @@ func (server *server) materializeCombinedSliceSubmission(ctx context.Context, ta
}
relativeStart := segmentStart - slice.Start
relativeEnd := segmentEnd - slice.Start
actions = append(actions, map[string]any{"id": segment.ID, "status": effectiveActionStatus(segment), "isInvalid": segment.IsInvalid, "description": segment.Description, "descriptionTemplateId": segment.LabelID, "phrases": segment.Phrases, "source": segment.Source, "confidence": segment.Confidence, "clipStartMs": relativeStart * 1000, "clipEndMs": relativeEnd * 1000, "startTimestampUs": mediaSecondToTimestamp(segmentStart), "endTimestampUs": mediaSecondToTimestamp(segmentEnd)})
actions = append(actions, map[string]any{"id": segment.ID, "status": effectiveActionStatus(segment), "isInvalid": segment.IsInvalid, "description": segment.Description, "descriptionTemplateId": segment.LabelID, "dictionary": segment.Dictionary, "phrases": segment.Phrases, "source": segment.Source, "confidence": segment.Confidence, "clipStartMs": relativeStart * 1000, "clipEndMs": relativeEnd * 1000, "startTimestampUs": mediaSecondToTimestamp(segmentStart), "endTimestampUs": mediaSecondToTimestamp(segmentEnd)})
}
annotation := map[string]any{"schemaVersion": "droboticflow.timestamped-description/v3", "boundaryContract": "hardware-frame-v1", "sliceId": sliceID, "sliceName": slice.Name, "window": map[string]any{"startTimestampUs": startUS, "endTimestampUs": endUS, "semantics": "half-open"}, "descriptions": actions, "actions": actions}
annotation := map[string]any{"schemaVersion": "droboticflow.timestamped-description/v3", "boundaryContract": "hardware-frame-v1", "sliceId": sliceID, "sliceName": slice.Name, "dictionary": slice.Dictionary, "window": map[string]any{"startTimestampUs": startUS, "endTimestampUs": endUS, "semantics": "half-open"}, "descriptions": actions, "actions": actions}
annotationSHA, annotationBytes, err := writeJSONFile(filepath.Join(directory, "annotation.json"), annotation)
if err != nil {
return nil, err
@ -968,7 +968,7 @@ func (server *server) materializeCombinedSliceSubmission(ctx context.Context, ta
boundaryTimes = append(boundaryTimes, grid.Seconds[k]-slice.Start)
rightBoundaryTimes = append(rightBoundaryTimes, float64(grid.RightFrames[k]-rightStart)*rightMediaDuration/float64(len(rightPTS)))
}
synchronization := map[string]any{"schemaVersion": "droboticflow.synchronized-slice/v1", "collectionId": collectionID, "sliceIndex": index + 1, "sliceCount": len(normalizedSlices), "sliceName": slice.Name, "window": map[string]any{"startTimestampUs": startUS, "endTimestampUs": endUS, "semantics": "half-open"}, "commonRange": map[string]any{"startTimestampUs": commonStart, "endTimestampUs": commonEnd, "startSecond": commonMediaStart, "endSecond": commonMediaEnd}, "coverage": map[string]any{"mode": "full_common_range", "contiguous": true, "previousEndTimestampUs": func() any {
synchronization := map[string]any{"schemaVersion": "droboticflow.synchronized-slice/v1", "collectionId": collectionID, "sliceIndex": index + 1, "sliceCount": len(normalizedSlices), "sliceName": slice.Name, "dictionary": slice.Dictionary, "window": map[string]any{"startTimestampUs": startUS, "endTimestampUs": endUS, "semantics": "half-open"}, "commonRange": map[string]any{"startTimestampUs": commonStart, "endTimestampUs": commonEnd, "startSecond": commonMediaStart, "endSecond": commonMediaEnd}, "coverage": map[string]any{"mode": "full_common_range", "contiguous": true, "previousEndTimestampUs": func() any {
if index == 0 {
return nil
}
@ -991,7 +991,7 @@ func (server *server) materializeCombinedSliceSubmission(ctx context.Context, ta
if err != nil {
return nil, err
}
manifestItems = append(manifestItems, map[string]any{"sliceId": sliceID, "sourceSliceId": slice.ID, "sliceName": slice.Name, "startSecond": slice.Start, "endSecond": slice.End, "startTimestampUs": startUS, "endTimestampUs": endUS, "directory": filepath.ToSlash(filepath.Join("slices", sliceID)), "files": map[string]any{"leftVideo": leftVideo, "rightVideo": rightVideo, "leftPts": leftManifest, "rightPts": rightManifest, "imu": imuManifest, "annotation": map[string]any{"name": "annotation.json", "sha256": annotationSHA, "bytes": annotationBytes}, "synchronization": map[string]any{"name": "synchronized-slice.json", "sha256": syncSHA, "bytes": syncBytes}}})
manifestItems = append(manifestItems, map[string]any{"sliceId": sliceID, "sourceSliceId": slice.ID, "sliceName": slice.Name, "dictionary": slice.Dictionary, "startSecond": slice.Start, "endSecond": slice.End, "startTimestampUs": startUS, "endTimestampUs": endUS, "directory": filepath.ToSlash(filepath.Join("slices", sliceID)), "files": map[string]any{"leftVideo": leftVideo, "rightVideo": rightVideo, "leftPts": leftManifest, "rightPts": rightManifest, "imu": imuManifest, "annotation": map[string]any{"name": "annotation.json", "sha256": annotationSHA, "bytes": annotationBytes}, "synchronization": map[string]any{"name": "synchronized-slice.json", "sha256": syncSHA, "bytes": syncBytes}}})
manifestItems[len(manifestItems)-1]["deliveryDisposition"] = sliceDisposition(slice)
manifestItems[len(manifestItems)-1]["mediaKey"] = mediaKey
if slice.IsDiscarded() {
@ -1005,15 +1005,23 @@ func (server *server) materializeCombinedSliceSubmission(ctx context.Context, ta
}
coverage := map[string]any{"mode": "full_common_range", "contiguous": true, "gapMicroseconds": 0, "overlapMicroseconds": 0, "startSecond": commonMediaStart, "endSecond": commonMediaEnd, "durationSeconds": commonMediaEnd - commonMediaStart, "startTimestampUs": commonStart, "endTimestampUs": commonEnd, "ptsToMediaScale": ptsToMediaScale, "sliceCount": len(manifestItems), "reconstruction": map[string]any{"leftFrameCount": deliveredLeftFrames, "rightFrameCount": deliveredRightFrames, "imuRowCount": deliveredIMURows}}
captureEvidence := map[string]any{}
var originalSource map[string]any
if capture, captureErr := server.store.GetCaptureRecordByRunID(ctx, task.RunID); captureErr != nil {
return nil, captureErr
} else if capture != nil {
originalSource, err = annotationRevisionOriginalSource(run, capture)
if err != nil {
return nil, err
}
captureEvidence = map[string]any{
"captureRecordId": capture.ID, "captureId": capture.CaptureID, "prefix": capture.Prefix,
"fingerprint": capture.Fingerprint, "contractRevision": capture.ContractRevision,
}
}
revisionManifest := map[string]any{"schemaVersion": "droboticflow.slice-materialization/v2", "revision": revision, "humanItemId": item.ID, "taskId": task.ID, "runId": task.RunID, "collectionId": collectionID, "capture": captureEvidence, "status": "materialized", "sliceCount": len(manifestItems), "coverage": coverage, "items": manifestItems}
if originalSource != nil {
revisionManifest["originalSource"] = originalSource
}
deliverySelection := map[string]any{"keptSliceCount": keptSliceCount, "discardedSliceCount": discardedSliceCount, "keptDurationSeconds": keptDuration, "discardedDurationSeconds": discardedDuration}
if discardedSliceCount > 0 {
revisionManifest["schemaVersion"] = "droboticflow.slice-materialization/v3"

View File

@ -89,16 +89,25 @@ func TestReworkMongoHTTPRepeatedReturnAcrossRestart(t *testing.T) {
batch, e := f.store.GetHumanBatch(ctx, f.item.BatchID)
must(e)
seed("humanBatches", batch)
f.item.Subject.Payload[dictionarySelectionHistoryKey] = []domain.DictionarySelection{{ID: "historical-base", Kind: "slice", Version: 2, Code: "sort", Name: "物品操作", Tags: []string{"商品"}}}
seed("humanItems", f.item)
seed("annotationTasks", f.task)
slices, e := f.store.ListSliceRevisions(ctx, f.item.OrganizationID, f.task.ID)
must(e)
baseClass := &domain.DictionarySelection{ID: "historical-base", Kind: "slice", Version: 2, Code: "sort", Name: "物品操作", Tags: []string{"商品"}}
actionClass := &domain.DictionarySelection{ID: "historical-action", Kind: "action", Version: 3, Code: "pick", Name: "拿取", Tags: []string{"抓取"}}
for _, r := range slices {
for i := range r.Segments {
r.Segments[i].Dictionary = baseClass
}
seed("sliceRevisions", r)
}
actions, e := f.store.ListAnnotationRevisions(ctx, f.item.OrganizationID, f.task.ID)
must(e)
for _, r := range actions {
for i := range r.Segments {
r.Segments[i].Dictionary = actionClass
}
seed("annotationRevisions", r)
}
f.handler = NewHandler(ds, nil)
@ -128,6 +137,14 @@ func TestReworkMongoHTTPRepeatedReturnAcrossRestart(t *testing.T) {
if w.Code != 200 || len(out.Items) != 1 || out.Items[0].AssignedUserID != f.annotator.ID {
t.Fatalf("persisted return missing from author pool: %d %s", w.Code, w.Body.String())
}
if snap := savedDictionarySelection(&out.Items[0], &domain.DictionarySelection{ID: "historical-base", Version: 2}, "slice"); snap == nil || snap.Tags[0] != "商品" {
t.Fatal("classification recovery ledger lost across Mongo restart")
}
baseline, e := ds.GetReworkBaseline(ctx, store.ReworkBaselineID(item.ID, out.Items[0].ReviewRound))
must(e)
if baseline == nil || len(baseline.Slices) == 0 || baseline.Slices[0].Dictionary == nil || baseline.Slices[0].Dictionary.Version != 2 || baseline.Slices[0].Dictionary.Tags[0] != "商品" {
t.Fatal("base classification lost across Mongo review/restart")
}
t.Logf("%s: round %d persisted and visible after restart", operation, out.Items[0].ReviewRound)
}
item, e := ds.GetHumanItem(ctx, f.item.ID)

View File

@ -7,8 +7,8 @@ import (
"strings"
)
// Unnamed drafts are allowed, but new confirmations need project dictionary
// names. Unchanged historical names remain valid during annotation/review.
// Binding happens before persistence, and never at delivery time. Name-only
// historical records remain name-only until a new selection is made.
func (s *server) validateProjectSliceNames(ctx context.Context, item *domain.HumanItemRecord, slices []domain.SliceSegment, mode string) []validationIssue {
if item == nil {
return nil
@ -18,61 +18,93 @@ func (s *server) validateProjectSliceNames(ctx context.Context, item *domain.Hum
return []validationIssue{{Field: "sliceSegments", Message: "无法读取切片字典规则"}}
}
if task == nil || task.TemplateVersion < 6 {
// Older templates use taskLabelId/taskLabel; they cannot introduce the
// independent v6 slice dictionary field, including through undo/discard.
for i := range slices {
slices[i].Dictionary = nil
}
return nil
}
previous := decodeMaterializationValue[domain.SliceSegment](annotationWorkspacePayload(item)["sliceSegments"])
if len(previous) == 0 {
revisions, e := s.store.ListSliceRevisions(ctx, item.OrganizationID, item.LegacyTaskID)
if e != nil {
revisions, err := s.store.ListSliceRevisions(ctx, item.OrganizationID, item.LegacyTaskID)
if err != nil {
return []validationIssue{{Field: "sliceSegments", Message: "无法读取历史切片名称"}}
}
if len(revisions) > 0 {
previous = revisions[len(revisions)-1].Segments
for _, rev := range revisions {
if rev.Revision == item.CurrentRevision || len(previous) == 0 {
previous = rev.Segments
}
}
}
old := map[string]string{}
if issues := rememberDictionarySelections(item, sliceDictionarySelections(previous)...); len(issues) > 0 {
return issues
}
old := map[string]domain.SliceSegment{}
for _, p := range previous {
old[p.ID] = p.Name
old[p.ID] = p
}
needs := []int{}
for i, c := range slices {
if c.IsDiscarded() {
continue
}
if mode != "confirm" && strings.TrimSpace(c.Name) != "" {
if name, ok := old[c.ID]; ok && name == c.Name {
unconfirmed := mapValue(item.Subject.Payload["unconfirmedSliceNames"])
if unconfirmed == nil {
unconfirmed = map[string]any{}
}
var allowed map[string]domain.AnnotationLabelRecord
for i := range slices {
c := &slices[i]
p, exists := old[c.ID]
if mode == "restore" && c.Dictionary != nil {
if saved := savedDictionarySelection(item, c.Dictionary, "slice"); saved != nil && c.Name == saved.Name {
c.Dictionary = saved
delete(unconfirmed, c.ID)
continue
}
}
if mode == "draft" && strings.TrimSpace(c.Name) == "" {
if c.IsDiscarded() {
c.Dictionary = nil
delete(unconfirmed, c.ID)
continue
}
needs = append(needs, i)
}
if len(needs) == 0 {
return nil
}
selection, err := s.store.GetProjectAnnotationLabels(ctx, item.ProjectID)
if err != nil {
return []validationIssue{{Field: "sliceSegments", Message: "无法读取项目切片字典,请重试"}}
}
allowed := map[string]bool{}
if selection != nil {
for _, id := range selection.SliceLabelIDs {
label, e := s.store.GetAnnotationLabel(ctx, id)
if e != nil {
unchanged := exists && p.Name == c.Name && (c.Dictionary == nil || sameDictionaryVersion(p.Dictionary, c.Dictionary))
finalizing := mode == "confirm" || mode == "submit"
pendingName := p.Dictionary == nil && c.Name != "" && stringValue(unconfirmed[c.ID]) == c.Name
needsNameCheck := (mode == "confirm" && p.Dictionary == nil) || (mode == "submit" && strings.TrimSpace(c.Name) == "") || (finalizing && pendingName)
if unchanged && !needsNameCheck {
c.Dictionary = p.Dictionary
continue
}
if (mode == "restore" && c.Dictionary == nil) || (mode == "draft" && strings.TrimSpace(c.Name) == "") {
c.Dictionary = nil
if c.Name != "" {
unconfirmed[c.ID] = c.Name
}
continue
}
if allowed == nil {
allowed, err = s.projectDictionarySelections(ctx, item, "slice")
if err != nil {
return []validationIssue{{Field: "sliceSegments", Message: "无法读取项目切片字典,请重试"}}
}
if label != nil && label.OrganizationID == item.OrganizationID && dictionaryKind(label.Kind) == "slice" && label.Status == domain.AnnotationLabelActive {
allowed[label.Name] = true
}
var chosen *domain.DictionarySelection
for _, label := range allowed {
if label.Name == c.Name && (c.Dictionary == nil || (label.ID == c.Dictionary.ID && label.Version == c.Dictionary.Version)) {
chosen = label.Selection()
break
}
}
}
issues := []validationIssue{}
for _, i := range needs {
if !allowed[slices[i].Name] {
issues = append(issues, validationIssue{Field: fmt.Sprintf("sliceSegments.%d.name", i), Code: "slice_dictionary_name_required", Message: "请从本项目切片字典选择名称;不能自由填写或修改。未配置字典请联系管理员"})
if chosen == nil {
return []validationIssue{{Field: fmt.Sprintf("sliceSegments.%d.name", i), Code: "slice_dictionary_name_required", Message: "请从本项目切片字典选择名称;词条已更新时请重新选择"}}
}
if !(exists && p.Name == c.Name && p.Dictionary == nil && c.Dictionary == nil && stringValue(unconfirmed[c.ID]) != c.Name) {
c.Dictionary = chosen
}
delete(unconfirmed, c.ID)
}
return issues
if len(unconfirmed) > 0 {
item.Subject.Payload["unconfirmedSliceNames"] = unconfirmed
} else {
delete(item.Subject.Payload, "unconfirmedSliceNames")
}
return rememberDictionarySelections(item, sliceDictionarySelections(slices)...)
}

View File

@ -142,6 +142,7 @@ func canonicalDispositions(previous, slices []domain.SliceSegment) ([]domain.Sli
next[i].DiscardReasonLabelID = p.DiscardReasonLabelID
next[i].DiscardReasonVersion = p.DiscardReasonVersion
next[i].DiscardReasonName = p.DiscardReasonName
next[i].DiscardReasonDictionary = p.DiscardReasonDictionary
next[i].DiscardReasonNote = p.DiscardReasonNote
next[i].DiscardedByUserID = p.DiscardedByUserID
next[i].DiscardedAt = p.DiscardedAt
@ -345,6 +346,7 @@ func restoredDiscardAnnotations(slice domain.SliceSegment) []domain.AnnotationSe
type sliceDispositionInput struct {
combinedAnnotationDraftInput
RestoreClassifications bool `json:"restoreClassifications,omitempty"`
SliceID string `json:"sliceId"`
Disposition string `json:"disposition"`
Reason string `json:"reason"`
@ -357,6 +359,13 @@ type sliceDispositionInput struct {
}
func sliceNameBeforeDiscard(item *domain.HumanItemRecord, sliceID string, version int) (string, bool) {
prior := sliceBeforeDiscard(item, sliceID, version)
if prior == nil {
return "", false
}
return prior.Name, true
}
func sliceBeforeDiscard(item *domain.HumanItemRecord, sliceID string, version int) *domain.SliceSegment {
history, _ := normalizeStructuredValue(item.Subject.Payload["sliceDispositionHistory"]).([]any)
resets, _ := normalizeStructuredValue(item.Subject.Payload["sliceResetHistory"]).([]any)
resetVersion := 0
@ -375,10 +384,11 @@ func sliceNameBeforeDiscard(item *domain.HumanItemRecord, sliceID string, versio
before := decodeMaterializationValue[domain.SliceSegment]([]any{entry["before"]})
after := decodeMaterializationValue[domain.SliceSegment]([]any{entry["after"]})
if len(before) == 1 && len(after) == 1 && !before[0].IsDiscarded() && after[0].IsDiscarded() {
return before[0].Name, true
copy := before[0]
return &copy
}
}
return "", false
return nil
}
func (s *server) handleSliceDisposition(w http.ResponseWriter, r *http.Request) {
@ -468,6 +478,7 @@ func (s *server) prepareSliceDisposition(w http.ResponseWriter, r *http.Request,
}
slices, issues := s.canonicalSliceDispositionState(r.Context(), item, body.normalizedSliceSegments(), body.normalizedOperations())
issues = append(issues, s.validateProjectSliceNames(r.Context(), item, slices, "restore")...)
if task.SourceIsFullVideo {
timeline, e := s.annotationFrameTimelineForTask(r.Context(), *task, "left")
if e != nil {
@ -480,6 +491,7 @@ func (s *server) prepareSliceDisposition(w http.ResponseWriter, r *http.Request,
}
issues = append(issues, validateSliceSegments(slices, *task)...)
issues = append(issues, validateCompleteSliceCoverage(slices, *task)...)
issues = append(issues, s.canonicalActionDictionaries(r.Context(), item, body.Segments, dispositionDictionaryMode(body))...)
issues = append(issues, validateDraftCombinedAnnotationEvents(body.Segments, *task)...)
idx := -1
for i, v := range slices {
@ -536,14 +548,20 @@ func (s *server) prepareSliceDisposition(w http.ResponseWriter, r *http.Request,
if body.Disposition == "discard" {
next.DeliveryDisposition = "discard"
next.Name, next.TaskLabelID, next.TaskLabel = "", "", ""
next.Dictionary = nil
next.DiscardReason = body.Reason
next.DiscardReasonLabelID = ""
next.DiscardReasonVersion = 0
next.DiscardReasonName = ""
next.DiscardReasonDictionary = nil
if reasonLabel != nil {
next.DiscardReasonLabelID = reasonLabel.ID
next.DiscardReasonVersion = reasonLabel.Version
next.DiscardReasonName = reasonLabel.Name
next.DiscardReasonDictionary = reasonLabel.Selection()
if body.RestoreClassifications {
next.DiscardReasonDictionary = savedDictionarySelection(item, &domain.DictionarySelection{ID: reasonLabel.ID, Version: reasonLabel.Version}, "discard_reason")
}
}
next.DiscardReasonNote = body.ReasonNote
next.DiscardedByUserID = actorID
@ -622,11 +640,18 @@ func (s *server) prepareSliceDisposition(w http.ResponseWriter, r *http.Request,
// Precise undo targets a server-recorded discard, even after later
// undo/redo writes. Ordinary restore deliberately remains unnamed.
next.Name = restoredName
next.Dictionary = nil
if body.RestorePreviousName || body.RestoreNameFromVersion != 0 {
if prior := sliceBeforeDiscard(item, before.ID, body.RestoreNameFromVersion); prior != nil {
next.Dictionary = prior.Dictionary
}
}
}
next.DiscardReason = ""
next.DiscardReasonLabelID = ""
next.DiscardReasonVersion = 0
next.DiscardReasonName = ""
next.DiscardReasonDictionary = nil
next.DiscardReasonNote = ""
next.DiscardedByUserID = ""
next.DiscardedAt = ""
@ -638,13 +663,20 @@ func (s *server) prepareSliceDisposition(w http.ResponseWriter, r *http.Request,
next = slices[idx]
// Disposition is an incremental edit, not confirmation of the whole slicing
// plan. Other slices may still have draft names; confirmation validates them.
issues = validateDraftCombinedAnnotationEvents(segments, *task)
issues = s.validateProjectSliceNames(r.Context(), item, slices, "restore")
issues = append(issues, s.canonicalActionDictionaries(r.Context(), item, segments, "restore")...)
issues = append(issues, validateDraftCombinedAnnotationEvents(segments, *task)...)
if len(issues) > 0 {
renderValidationError(w, describeSliceNameIssues(issues, slices)...)
return nil, domain.SliceSegment{}, false
}
statuses := body.normalizedStatuses()
statuses[next.ID] = domain.SliceAnnotationPending
if issues := rememberDictionarySelections(item, sliceDictionarySelections(slices)...); len(issues) > 0 {
renderValidationError(w, issues...)
return nil, domain.SliceSegment{}, false
}
workspace := map[string]any{"sliceSegments": slices, "segments": segments, "statuses": statuses, "operations": body.normalizedOperations(), "userId": actorID, "savedAt": nowISO()}
if item.Subject.Payload == nil {
item.Subject.Payload = map[string]any{}
@ -665,3 +697,10 @@ func (s *server) prepareSliceDisposition(w http.ResponseWriter, r *http.Request,
}
return workspace, next, true
}
func dispositionDictionaryMode(body sliceDispositionInput) string {
if body.RestoreClassifications {
return "restore"
}
return "draft"
}

View File

@ -82,11 +82,14 @@ func TestSliceDiscardMaterialization60Seconds(t *testing.T) {
for i := 0; i < 3; i++ {
start, end := float64(i*20), float64((i+1)*20)
sl := domain.SliceSegment{ID: fmt.Sprintf("segment-%d", i), Name: fmt.Sprintf("片段 %d", i+1), Start: start, End: end}
sl.Dictionary = &domain.DictionarySelection{ID: fmt.Sprintf("base-%d", i), Kind: "slice", Version: 2, Code: fmt.Sprintf("class-%d", i), Name: sl.Name, Tags: []string{"基础分类", "商品"}}
if mode == "discard" || (mode == "mixed" && i == 1) {
sl.DeliveryDisposition = "discard"
sl.DiscardReason = "审查测试:画面被遮挡"
sl.DiscardReasonLabelID = "reason-test"
sl.DiscardReasonVersion = 3
sl.Dictionary = nil
sl.DiscardReasonDictionary = &domain.DictionarySelection{ID: "reason-test", Kind: "discard_reason", Version: 3, Code: "occluded", Name: "审查测试", Tags: []string{"不可用"}}
sl.DiscardReasonName = "审查测试"
sl.DiscardReasonNote = "画面被遮挡"
sl.DiscardedAt = "2026-09-16T00:00:00Z"
@ -94,7 +97,7 @@ func TestSliceDiscardMaterialization60Seconds(t *testing.T) {
} else if mode == "partial" && i == 0 {
segments = append(segments, domain.AnnotationSegment{ID: "partial", Start: 5, End: 10, Description: "仅描述这五秒", Source: "human"})
} else if mode != "blank" && mode != "partial" {
segments = append(segments, domain.AnnotationSegment{ID: fmt.Sprintf("action-%d", i), Start: start, End: end, Description: "测试动作", Source: "human"})
segments = append(segments, domain.AnnotationSegment{ID: fmt.Sprintf("action-%d", i), Start: start, End: end, Description: "测试动作", Dictionary: &domain.DictionarySelection{ID: "action-test", Kind: "action", Version: 4, Code: "pick", Name: "拿取", Tags: []string{"抓取"}}, Source: "human"})
}
slices = append(slices, sl)
}

View File

@ -9,8 +9,9 @@ import (
type sliceEditApplicationInput struct {
combinedAnnotationDraftInput
ApplicationID string `json:"applicationId"`
Dispositions []sliceDispositionInput `json:"dispositions"`
RestoreClassifications bool `json:"restoreClassifications,omitempty"`
ApplicationID string `json:"applicationId"`
Dispositions []sliceDispositionInput `json:"dispositions"`
}
// A panel application is one CAS, including explicit disposition commands and
@ -71,6 +72,7 @@ func (s *server) handleSliceEditApplication(w http.ResponseWriter, r *http.Reque
}
commands := map[string]sliceDispositionInput{}
for _, command := range body.Dispositions {
command.RestoreClassifications = body.RestoreClassifications
if command.SliceID == "" || commands[command.SliceID].SliceID != "" || (command.Disposition != "keep" && command.Disposition != "discard") {
renderValidationError(w, validationIssue{Field: "dispositions", Message: "切片用途操作无效或重复"})
return
@ -81,6 +83,7 @@ func (s *server) handleSliceEditApplication(w http.ResponseWriter, r *http.Reque
// Undo can first restore a discarded child before rejoining it. The restore
// remains an explicit, audited command in the same atomic application.
for _, command := range body.Dispositions {
command.RestoreClassifications = body.RestoreClassifications
if !ids[command.SliceID] {
if command.Disposition != "keep" {
renderValidationError(w, validationIssue{Field: "dispositions", Message: "舍弃操作必须保留目标切片"})
@ -143,10 +146,15 @@ func (s *server) handleSliceEditApplication(w http.ResponseWriter, r *http.Reque
}
}
baseline[i] = sliceEditPolicyBaseline(candidate, old)
if command := commands[candidate.ID]; command.Disposition == "discard" && !old.IsDiscarded() {
baseline[i].Name = old.Name
baseline[i].Dictionary = old.Dictionary
}
}
}
slices, issues := s.canonicalSliceDispositionState(r.Context(), &item, baseline, body.normalizedOperations())
issues = append(issues, s.validateProjectSliceNames(r.Context(), stored, slices, applicationDictionaryMode(body))...)
issues = append(issues, s.validateSliceEditFrameBounds(r, task, &item, slices, body.Segments)...)
issues = append(issues, validateSliceSegments(slices, *task)...)
issues = append(issues, validateCompleteSliceCoverage(slices, *task)...)
@ -194,10 +202,16 @@ func (s *server) handleSliceEditApplication(w http.ResponseWriter, r *http.Reque
segmentsForApplication = retained
}
}
if issues := s.canonicalActionDictionaries(r.Context(), stored, segmentsForApplication, applicationDictionaryMode(body)); len(issues) > 0 {
renderValidationError(w, issues...)
return
}
copyClassificationRecoveryState(&item, stored)
workspace := map[string]any{"sliceSegments": slices, "segments": segmentsForApplication, "statuses": body.normalizedStatuses(), "operations": body.normalizedOperations(), "userId": principal.User.ID, "savedAt": nowISO()}
item.Subject.Payload["annotationWorkspace"] = workspace
delete(item.Subject.Payload, "annotationDraft")
for _, command := range body.Dispositions {
command.RestoreClassifications = body.RestoreClassifications
if ids[command.SliceID] {
command.combinedAnnotationDraftInput = workspaceSliceEditInput(workspace)
var prepared bool
@ -239,10 +253,13 @@ func (s *server) handleSliceEditApplication(w http.ResponseWriter, r *http.Reque
for _, candidate := range proposed {
if candidate.ID == slices[i].ID && !slices[i].IsDiscarded() {
slices[i].Name = candidate.Name
slices[i].Dictionary = candidate.Dictionary
}
}
}
issues = s.validateProjectSliceNames(r.Context(), &item, slices, "draft")
issues = s.validateProjectSliceNames(r.Context(), stored, slices, applicationDictionaryMode(body))
issues = append(issues, s.canonicalActionDictionaries(r.Context(), stored, segments, applicationDictionaryMode(body))...)
copyClassificationRecoveryState(&item, stored)
issues = append(issues, s.validateSliceEditFrameBounds(r, task, &item, slices, segments)...)
issues = append(issues, validateDraftCombinedAnnotationEvents(segments, *task)...)
issues = append(issues, validateDraftFanyuV5CombinedSemanticSegments(segments, slices, annotationTemplateWithTaskSnapshot(template, task))...)
@ -265,6 +282,7 @@ func (s *server) handleSliceEditApplication(w http.ResponseWriter, r *http.Reque
item.Subject.Payload["annotationWorkspace"] = workspace
// Recheck dictionary versions immediately before the single commit.
for _, command := range body.Dispositions {
command.RestoreClassifications = body.RestoreClassifications
if command.Disposition == "discard" {
if _, issues := s.resolveDiscardReason(r.Context(), &item, command); len(issues) > 0 {
renderValidationError(w, issues...)
@ -302,6 +320,7 @@ func sliceEditPolicyBaseline(candidate, old domain.SliceSegment) domain.SliceSeg
candidate.DiscardReasonLabelID = old.DiscardReasonLabelID
candidate.DiscardReasonVersion = old.DiscardReasonVersion
candidate.DiscardReasonName = old.DiscardReasonName
candidate.DiscardReasonDictionary = old.DiscardReasonDictionary
candidate.DiscardReasonNote = old.DiscardReasonNote
candidate.DiscardedAt = old.DiscardedAt
candidate.DiscardedByUserID = old.DiscardedByUserID
@ -344,3 +363,10 @@ func (s *server) validateSliceEditFrameBounds(r *http.Request, task *domain.Anno
}
return issues
}
func applicationDictionaryMode(body sliceEditApplicationInput) string {
if body.RestoreClassifications {
return "restore"
}
return "draft"
}

View File

@ -122,6 +122,8 @@ func (s *server) handleSliceEditRestoration(w http.ResponseWriter, r *http.Reque
issues = append(issues, validateCompleteSliceCoverage(slices, *task)...)
issues = append(issues, validateDraftCombinedAnnotationEvents(body.Segments, *task)...)
issues = append(issues, validateDraftStatuses(body.normalizedStatuses(), slices)...)
issues = append(issues, s.validateProjectSliceNames(r.Context(), item, slices, "restore")...)
issues = append(issues, s.canonicalActionDictionaries(r.Context(), item, body.Segments, "restore")...)
template, err := s.store.GetAnnotationTemplate(r.Context(), task.TemplateID)
if err != nil {
renderError(w, 500, err)

View File

@ -16,6 +16,7 @@ import (
)
const PartSchema = "droboticflow.delivery.part-index/v3"
const ClassificationPartSchema = "droboticflow.delivery.part-index/v4"
var partName = regexp.MustCompile(`^part[0-9]{4,}$`)
@ -48,7 +49,7 @@ type Summary struct {
DiscardedCount int `json:"discardedSliceCount"`
KeptDuration float64 `json:"keptDurationSeconds"`
DiscardedDuration float64 `json:"discardedDurationSeconds"`
Acceptance map[string]string `json:"acceptance"`
Acceptance map[string]string `json:"acceptance,omitempty"`
}
type part struct {
Schema string `json:"schemaVersion"`
@ -117,6 +118,9 @@ func addSummary(a *Summary, b Summary) {
a.DiscardedCount += b.DiscardedCount
a.KeptDuration += b.KeptDuration
a.DiscardedDuration += b.DiscardedDuration
if b.Acceptance == nil {
return
}
if a.Acceptance == nil {
a.Acceptance = map[string]string{"decision": "approved"}
}
@ -128,10 +132,10 @@ func addSummary(a *Summary, b Summary) {
}
func checkPart(p part) (Summary, error) {
s := Summary{PartCount: 1, Acceptance: map[string]string{"decision": "approved", "acceptedAt": p.Acceptance.Final.AcceptedAt}}
if p.Schema != PartSchema || !safeName(p.Capture.ID) || !partName.MatchString(p.PartID) || p.Revision < 1 || !p.Acceptance.Final.Confirmed || len(p.Acceptance.Reviews) != 0 {
if (p.Schema != PartSchema && p.Schema != ClassificationPartSchema) || !safeName(p.Capture.ID) || !partName.MatchString(p.PartID) || p.Revision < 1 || !p.Acceptance.Final.Confirmed || len(p.Acceptance.Reviews) != 0 {
return s, fmt.Errorf("part must pin an accepted native v3 revision without embedded review history")
}
if _, e := time.Parse(time.RFC3339Nano, p.Acceptance.Final.AcceptedAt); e != nil {
if _, e := time.Parse(time.RFC3339Nano, p.Acceptance.Final.AcceptedAt); e != nil && p.Schema != ClassificationPartSchema {
return s, fmt.Errorf("missing acceptance timestamp")
}
if p.SourceMetadata.Path != "meta_"+p.Capture.ID+".json" || path.Base(p.Capture.Prefix) != p.Capture.ID || path.Clean(p.Capture.Prefix) != p.Capture.Prefix || strings.HasPrefix(p.Capture.Prefix, "/") || strings.Contains(p.Capture.Prefix, "../") {
@ -168,6 +172,9 @@ func checkPart(p part) (Summary, error) {
invalid = true
} else if strings.TrimSpace(a.Description) != "" {
kind = "described"
if p.Schema == ClassificationPartSchema {
kind = "annotated"
}
described = true
} else {
blank = true
@ -191,7 +198,7 @@ func checkPart(p part) (Summary, error) {
}
}
}
if s.KeptCount == 0 {
if s.KeptCount == 0 && (p.Schema != ClassificationPartSchema || s.DiscardedCount == 0) {
return s, fmt.Errorf("no deliverable slices")
}
if math.IsInf(s.KeptDuration, 0) || math.IsInf(s.DiscardedDuration, 0) {
@ -206,6 +213,9 @@ func checkPart(p part) (Summary, error) {
return s, fmt.Errorf("slice coverage gap or overlap")
}
}
if p.Schema == ClassificationPartSchema {
s.Acceptance = nil
}
return s, nil
}
@ -223,6 +233,25 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
return out, fmt.Errorf("source JSON exceeds 64 MiB")
}
}
schema := ""
for _, src := range sources {
var p part
if err := json.Unmarshal(src.Part, &p); err != nil {
return out, err
}
if schema != "" && schema != p.Schema {
return out, fmt.Errorf("cannot mix part schema versions in one batch")
}
schema = p.Schema
if p.Schema == ClassificationPartSchema {
if err := validateConsumerPart(src.Part); err != nil {
return out, err
}
if _, err := classificationAudit(src); err != nil {
return out, err
}
}
}
// Normalize JSON before hashing to ignore transport whitespace/map ordering.
for i := range sources {
var d, h any
@ -235,7 +264,17 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
sort.Strings(sources[i].ExpectedParts)
}
sort.Slice(sources, func(i, j int) bool { return string(sources[i].Part) < string(sources[j].Part) })
identity := encode([]any{"droboticflow.delivery.batch/v1", projectID, projectName, projectRoot, sources})
identitySources := append([]Source(nil), sources...)
if schema == ClassificationPartSchema {
for i := range identitySources {
identitySources[i].History = nil
}
}
identity := encode([]any{"droboticflow.delivery.batch/v1", projectID, projectName, projectRoot, identitySources})
version := 3
if schema == ClassificationPartSchema {
version = 4
}
out.ID = Hash(identity)
out.Root = projectRoot + "/index-batches/" + out.ID
type group struct {
@ -259,6 +298,15 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
if e := json.Unmarshal(src.Part, &p); e != nil {
return out, e
}
if p.Schema == ClassificationPartSchema {
// The original bytes were checked before canonicalization above.
var audit classificationHistory
if json.Unmarshal(src.History, &audit) != nil {
return out, fmt.Errorf("invalid internal audit")
}
p.Acceptance.Final.Confirmed = audit.Acceptance.Final.Confirmed
p.Acceptance.Final.AcceptedAt = audit.Acceptance.Final.AcceptedAt
}
s, e := checkPart(p)
if e != nil {
return out, e
@ -290,7 +338,7 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
Revision int `json:"revision"`
Reviews []json.RawMessage `json:"qualityReviews"`
}
if json.Unmarshal(src.History, &history) != nil || history.Schema != "droboticflow.delivery.review-history/v1" || history.CaptureID != p.Capture.ID || history.PartID != p.PartID || history.Revision != p.Revision || history.Reviews == nil {
if p.Schema != ClassificationPartSchema && (json.Unmarshal(src.History, &history) != nil || history.Schema != "droboticflow.delivery.review-history/v1" || history.CaptureID != p.Capture.ID || history.PartID != p.PartID || history.Revision != p.Revision || history.Reviews == nil) {
return out, fmt.Errorf("review history revision mismatch")
}
g := groups[p.Capture.ID]
@ -302,10 +350,17 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
}
var doc map[string]any
json.Unmarshal(src.Part, &doc)
doc["reviewHistory"] = map[string]string{"path": p.PartID + ".review-history.json", "pathBase": "capture_delivery"}
doc["delivery"] = map[string]string{"kind": "part_index", "fileName": p.PartID + ".json", "artifactName": "delivery.part-index"}
g.entries = append(g.entries, add(name+".json", encode(doc)), add(name+".review-history.json", src.History))
g.parts = append(g.parts, map[string]any{"part": p.PartID, "revision": p.Revision, "path": p.PartID + ".json", "reviewHistoryPath": p.PartID + ".review-history.json"})
g.entries = append(g.entries, add(name+".json", encode(doc)))
reference := map[string]any{"part": p.PartID, "revision": p.Revision, "path": p.PartID + ".json"}
if p.Schema != ClassificationPartSchema {
doc["reviewHistory"] = map[string]string{"path": p.PartID + ".review-history.json", "pathBase": "capture_delivery"}
totalBytes -= len(out.Files[name+".json"])
g.entries[len(g.entries)-1] = add(name+".json", encode(doc))
g.entries = append(g.entries, add(name+".review-history.json", src.History))
reference["reviewHistoryPath"] = p.PartID + ".review-history.json"
}
g.parts = append(g.parts, reference)
addSummary(&g.summary, s)
if math.IsInf(g.summary.KeptDuration, 0) || math.IsInf(g.summary.DiscardedDuration, 0) {
return out, fmt.Errorf("capture duration overflow")
@ -343,7 +398,7 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
size += local[i].SizeBytes
}
metadata := map[string]any{"path": "meta_" + n + ".json", "pathBase": "original_capture", "sha256": g.metadata.MetadataSHA256, "sizeBytes": g.metadata.MetadataBytes}
manifest := map[string]any{"schemaVersion": 3, "kind": "part-index", "capture": g.capture, "pathBase": "capture_delivery", "sourceMetadata": metadata, "summary": g.summary, "parts": g.parts, "entries": local, "fileCount": len(local), "totalBytes": size}
manifest := map[string]any{"schemaVersion": version, "kind": "part-index", "capture": g.capture, "pathBase": "capture_delivery", "sourceMetadata": metadata, "summary": g.summary, "parts": g.parts, "entries": local, "fileCount": len(local), "totalBytes": size}
allEntries = append(allEntries, g.entries...)
allEntries = append(allEntries, add(n+"/annotation_manifest.json", encode(manifest)))
captures = append(captures, map[string]any{"captureId": n, "prefix": g.capture["prefix"], "manifestPath": n + "/annotation_manifest.json", "summary": g.summary})
@ -360,7 +415,50 @@ func Build(projectID, projectName, projectRoot string, sources []Source) (Snapsh
for _, v := range allEntries {
size += v.SizeBytes
}
out.Index = encode(map[string]any{"schemaVersion": 3, "kind": "part-index", "snapshotId": out.ID, "pathBase": "batch_project", "project": map[string]string{"projectId": projectID, "deliveryName": projectName, "rootKey": out.Root}, "summary": total, "captures": captures, "entries": allEntries, "fileCount": len(allEntries), "totalBytes": size})
out.Index = encode(map[string]any{"schemaVersion": version, "kind": "part-index", "snapshotId": out.ID, "pathBase": "batch_project", "project": map[string]string{"projectId": projectID, "deliveryName": projectName, "rootKey": out.Root}, "summary": total, "captures": captures, "entries": allEntries, "fileCount": len(allEntries), "totalBytes": size})
out.Files["delivery_index.json"] = out.Index
return out, nil
}
// classificationHistory is internal acceptance proof, excluded from v4 batch
// identity and customer files. The digest binds it to the original native bytes.
type classificationHistory struct {
Schema string `json:"schemaVersion"`
CaptureID string `json:"captureId"`
PartID string `json:"partId"`
Revision int `json:"revision"`
PartSHA string `json:"partSha256"`
Acceptance struct {
Final struct {
Confirmed bool `json:"confirmed"`
AcceptedAt string `json:"acceptedAt"`
Sampled *bool `json:"sampled"`
SamplingDecision string `json:"samplingDecision"`
} `json:"finalAcceptance"`
} `json:"acceptance"`
}
func classificationAudit(src Source) (classificationHistory, error) {
var audit classificationHistory
var p part
var doc map[string]json.RawMessage
if json.Unmarshal(src.Part, &p) != nil || json.Unmarshal(src.Part, &doc) != nil || json.Unmarshal(src.History, &audit) != nil {
return audit, fmt.Errorf("invalid classification/audit JSON")
}
for _, field := range []string{"acceptance", "lineage", "reviewHistory"} {
if _, ok := doc[field]; ok {
return audit, fmt.Errorf("customer classification contains internal trace")
}
}
if audit.Schema != "droboticflow.delivery.review-history/v2" || audit.CaptureID != p.Capture.ID || audit.PartID != p.PartID || audit.Revision != p.Revision || audit.PartSHA != Hash(src.Part) || !audit.Acceptance.Final.Confirmed {
return audit, fmt.Errorf("internal audit does not pin the accepted classification")
}
if _, err := time.Parse(time.RFC3339Nano, audit.Acceptance.Final.AcceptedAt); err != nil {
final := audit.Acceptance.Final
exempt := final.AcceptedAt == "" && final.Sampled != nil && !*final.Sampled && final.SamplingDecision == "not_selected"
if !exempt {
return audit, fmt.Errorf("missing internal acceptance timestamp or explicit sampling exemption")
}
}
return audit, nil
}

View File

@ -0,0 +1,176 @@
package deliveryindex
import (
"bytes"
"encoding/json"
"strings"
"testing"
)
func classificationFixture(allDiscard bool) Source {
src := fixture("part0001", 7)
var p map[string]any
json.Unmarshal(src.Part, &p)
acceptance := p["acceptance"]
delete(p, "acceptance")
p["schemaVersion"] = ClassificationPartSchema
if allDiscard {
p["slices"] = []any{}
p["discardedSlices"] = []any{map[string]any{"sliceId": "all", "taskSplit": map[string]any{"startSecond": 0, "endSecond": 3, "semantics": "half-open"}, "classification": map[string]any{"deliveryDisposition": "discard", "discardEvidence": map[string]any{"reason": "遮挡"}}, "segments": []any{}}}
p["deliveryClassification"] = map[string]any{"keptSliceCount": 0, "discardedSliceCount": 1, "keptDurationSeconds": 0, "discardedDurationSeconds": 3}
}
enrichConsumerFixture(p)
src.Part = encode(p)
src.History = encode(map[string]any{"schemaVersion": "droboticflow.delivery.review-history/v2", "captureId": "capture-A", "partId": "part0001", "revision": 7, "partSha256": Hash(src.Part), "acceptance": acceptance, "qualityReviews": []any{map[string]string{"comment": "PRIVATE_REWORK"}}, "trace": []any{}})
return src
}
func TestClassificationBatchNeverIncludesPrivateTrace(t *testing.T) {
for _, all := range []bool{false, true} {
src := classificationFixture(all)
out := build(t, src)
if len(out.Files) != 3 {
t.Fatalf("customer files %v", out.Files)
}
for name, body := range out.Files {
if strings.Contains(name, "history") || bytes.Contains(body, []byte("PRIVATE_REWORK")) || bytes.Contains(body, []byte(`"acceptance"`)) {
t.Fatalf("private trace in %s", name)
}
}
var h map[string]any
json.Unmarshal(src.History, &h)
h["qualityReviews"] = []any{map[string]string{"comment": "changed"}}
src.History = encode(h)
again := build(t, src)
if out.ID != again.ID || !bytes.Equal(out.Index, again.Index) {
t.Fatal("audit-only edit changed customer batch")
}
src.History = bytes.ReplaceAll(src.History, []byte(Hash(src.Part)), []byte(strings.Repeat("0", 64)))
if _, err := Build("project-A", "项目A", "delivery/项目A__project-A", []Source{src}); err == nil {
t.Fatal("wrong acceptance digest accepted")
}
}
}
func TestClassificationBatchRejectsMixedVersions(t *testing.T) {
if _, err := Build("project-A", "项目A", "delivery/项目A__project-A", []Source{classificationFixture(false), fixture("part0002", 1)}); err == nil {
t.Fatal("mixed versions accepted")
}
}
func TestClassificationBatchRejectsIncompleteConsumerContract(t *testing.T) {
for _, failure := range []string{"contract", "tags", "tag-source", "part-time", "pts", "original-file", "original-source"} {
t.Run(failure, func(t *testing.T) {
src := classificationFixture(false)
var doc, audit map[string]any
json.Unmarshal(src.Part, &doc)
slice := doc["slices"].([]any)[0].(map[string]any)
action := slice["segments"].([]any)[0].(map[string]any)
switch failure {
case "contract":
delete(doc, "consumerContract")
case "tags":
delete(action, "tags")
case "tag-source":
action["tagSource"] = "action"
case "part-time":
action["partStartSecond"] = -1
case "pts":
delete(action, "startTimestampUs")
case "original-source":
delete(doc, "originalSource")
case "original-file":
delete(doc["originalSource"].(map[string]any)["files"].(map[string]any), "leftPts")
}
src.Part = encode(doc)
json.Unmarshal(src.History, &audit)
// Keep acceptance hash valid: rejection must be the consumer guard.
audit["partSha256"] = Hash(src.Part)
src.History = encode(audit)
if _, err := Build("project-A", "项目A", "delivery/项目A__project-A", []Source{src}); err == nil || !strings.Contains(err.Error(), "consumer contract") {
t.Fatalf("incomplete %s accepted: %v", failure, err)
}
})
}
}
func TestClassificationBatchAcceptsExplicitSamplingExemptionWithoutInventingHumanDate(t *testing.T) {
src := classificationFixture(false)
var audit map[string]any
json.Unmarshal(src.History, &audit)
final := audit["acceptance"].(map[string]any)["finalAcceptance"].(map[string]any)
delete(final, "acceptedAt")
final["sampled"] = false
final["samplingDecision"] = "not_selected"
src.History = encode(audit)
out := build(t, src)
if bytes.Contains(out.Index, []byte("acceptedAt")) {
t.Fatal("invented human acceptance date")
}
final["samplingDecision"] = "selected"
src.History = encode(audit)
if _, err := Build("project-A", "项目A", "delivery/项目A__project-A", []Source{src}); err == nil {
t.Fatal("unverified human acceptance accepted")
}
}
func enrichConsumerFixture(p map[string]any) {
part := p["partId"].(string)
p["consumerContract"] = "droboticflow.classification-consumer/v1"
p["timeBases"] = map[string]string{"taskSplitSeconds": "original_left_video", "segmentSeconds": "slice_left_video", "segmentPartSeconds": "original_left_video", "timestampUs": "original_hardware_pts", "interval": "half-open", "frameRanges": "zero-based-half-open"}
p["pathBases"] = map[string]string{"sources": "original_capture", "sourceMetadata": "original_capture", "calibration": "original_capture"}
files := map[string]any{}
sources := map[string]any{}
for _, role := range []string{"leftVideo", "rightVideo", "leftPts", "rightPts", "imu", "cameraCalibration", "imuCalibration", "metadata"} {
name := part + "/" + role
if role == "cameraCalibration" || role == "imuCalibration" {
name = role + ".yaml"
}
if role == "metadata" {
name = "meta_capture-A.json"
}
files[role] = map[string]any{"path": name}
if role == "leftVideo" || role == "rightVideo" || role == "leftPts" || role == "rightPts" || role == "imu" {
sources[role] = files[role]
}
}
p["sources"] = sources
p["sourceMetadata"] = files["metadata"]
p["originalSource"] = map[string]any{"provider": "BOS", "bucket": "test-bucket", "prefix": "incoming/capture-A", "partPrefix": "incoming/capture-A/" + part, "files": files}
for _, kind := range []string{"slices", "discardedSlices"} {
for _, raw := range p[kind].([]any) {
item := raw.(map[string]any)
split := item["taskSplit"].(map[string]any)
number := func(v any) float64 {
switch n := v.(type) {
case float64:
return n
case int:
return float64(n)
default:
panic("invalid fixture time")
}
}
start, end := number(split["startSecond"]), number(split["endSecond"])
split["startTimestampUs"] = int64(1e6 + start*1e6)
split["endTimestampUs"] = int64(1e6 + end*1e6)
if kind == "discardedSlices" {
continue
}
class := item["classification"].(map[string]any)
class["dictionary"] = map[string]any{"id": "base", "kind": "slice", "version": 1, "name": "操作", "tags": []string{"操作"}}
class["tags"] = []string{"操作"}
actions := item["segments"].([]any)
for i, raw := range actions {
a := raw.(map[string]any)
low, high := float64(i)*(end-start)/float64(len(actions)), float64(i+1)*(end-start)/float64(len(actions))
a["startSecond"] = low
a["endSecond"] = high
a["partStartSecond"] = start + low
a["partEndSecond"] = start + high
a["startTimestampUs"] = int64(1e6 + (start+low)*1e6)
a["endTimestampUs"] = int64(1e6 + (start+high)*1e6)
a["tags"] = []string{"操作"}
a["tagSource"] = "slice"
}
}
}
}

View File

@ -0,0 +1,148 @@
package deliveryindex
import (
"encoding/json"
"fmt"
"math"
"path"
"reflect"
"strings"
)
// v4 is unreleased: only complete consumer/v1 projections may enter new batches.
// Keep this guard at publication as well as in the package producer, so older
// local v4 artifacts cannot silently publish incomplete classification JSON.
func validateConsumerPart(body []byte) error {
type sourceFile struct {
Path string `json:"path"`
}
type selection struct {
ID string `json:"id"`
Kind string `json:"kind"`
Version int `json:"version"`
Name string `json:"name"`
Tags []string `json:"tags"`
}
type window struct {
Start *float64 `json:"startSecond"`
End *float64 `json:"endSecond"`
First *int64 `json:"startTimestampUs"`
Last *int64 `json:"endTimestampUs"`
}
type action struct {
window
PartStart *float64 `json:"partStartSecond"`
PartEnd *float64 `json:"partEndSecond"`
Tags []string `json:"tags"`
TagSource string `json:"tagSource"`
Dictionary *selection `json:"dictionary"`
}
type item struct {
Split window `json:"taskSplit"`
Class struct {
Dictionary *selection `json:"dictionary"`
Tags []string `json:"tags"`
} `json:"classification"`
Actions []action `json:"segments"`
}
var p struct {
Contract string `json:"consumerContract"`
Capture struct {
Prefix string `json:"prefix"`
} `json:"capture"`
PartID string `json:"partId"`
Original struct {
Provider string `json:"provider"`
Bucket string `json:"bucket"`
Prefix string `json:"prefix"`
PartPrefix string `json:"partPrefix"`
Files map[string]sourceFile `json:"files"`
} `json:"originalSource"`
Bases map[string]string `json:"timeBases"`
Slices []item `json:"slices"`
Discarded []struct {
Split window `json:"taskSplit"`
} `json:"discardedSlices"`
}
if err := json.Unmarshal(body, &p); err != nil {
return err
}
fail := func() error { return fmt.Errorf("incomplete v4 classification consumer contract") }
var refs struct {
Sources map[string]sourceFile `json:"sources"`
Metadata sourceFile `json:"sourceMetadata"`
Bases map[string]string `json:"pathBases"`
}
if err := json.Unmarshal(body, &refs); err != nil {
return err
}
for _, role := range []string{"leftVideo", "rightVideo", "leftPts", "rightPts", "imu"} {
if refs.Sources[role].Path != p.Original.Files[role].Path || !strings.HasPrefix(refs.Sources[role].Path, p.PartID+"/") {
return fail()
}
}
if refs.Metadata.Path != p.Original.Files["metadata"].Path || refs.Bases["sources"] != "original_capture" || refs.Bases["sourceMetadata"] != "original_capture" || refs.Bases["calibration"] != "original_capture" {
return fail()
}
if p.Contract != "droboticflow.classification-consumer/v1" || p.Original.Provider != "BOS" || p.Original.Bucket == "" || p.Original.Prefix == "" || p.Original.Prefix != p.Capture.Prefix || p.Original.PartPrefix != p.Original.Prefix+"/"+p.PartID {
return fail()
}
for key, value := range map[string]string{"taskSplitSeconds": "original_left_video", "segmentSeconds": "slice_left_video", "segmentPartSeconds": "original_left_video", "timestampUs": "original_hardware_pts", "interval": "half-open", "frameRanges": "zero-based-half-open"} {
if p.Bases[key] != value {
return fail()
}
}
for _, role := range []string{"leftVideo", "rightVideo", "leftPts", "rightPts", "imu", "cameraCalibration", "imuCalibration", "metadata"} {
file := p.Original.Files[role].Path
if file == "" || path.Clean(file) != file || strings.HasPrefix(file, "/") || strings.ContainsAny(file, "\\\x00") || file == ".." || strings.HasPrefix(file, "../") {
return fail()
}
}
validWindow := func(w window) bool {
return w.Start != nil && w.End != nil && w.First != nil && w.Last != nil && !math.IsNaN(*w.Start) && !math.IsInf(*w.Start, 0) && !math.IsNaN(*w.End) && !math.IsInf(*w.End, 0) && *w.Start >= 0 && *w.End > *w.Start && *w.First > 0 && *w.Last > *w.First
}
validTags := func(tags []string) bool {
seen := map[string]bool{}
if len(tags) == 0 {
return false
}
for _, tag := range tags {
if strings.TrimSpace(tag) == "" || tag != strings.TrimSpace(tag) || seen[tag] {
return false
}
seen[tag] = true
}
return true
}
for _, slice := range p.Slices {
d := slice.Class.Dictionary
if !validWindow(slice.Split) || d == nil || strings.TrimSpace(d.ID) == "" || strings.TrimSpace(d.Name) == "" || d.Kind != "slice" || d.Version < 1 || !validTags(d.Tags) || !reflect.DeepEqual(d.Tags, slice.Class.Tags) || len(slice.Actions) == 0 {
return fail()
}
previous := 0.0
stamp := *slice.Split.First
for _, a := range slice.Actions {
if !validWindow(a.window) || a.PartStart == nil || a.PartEnd == nil || !validTags(a.Tags) || math.Abs(*a.Start-previous) > 1e-6 || *a.First != stamp || *a.Last > *slice.Split.Last || math.Abs(*a.PartStart-*slice.Split.Start-*a.Start) > 1e-6 || math.Abs(*a.PartEnd-*slice.Split.Start-*a.End) > 1e-6 {
return fail()
}
if a.Dictionary == nil {
if a.TagSource != "slice" || !reflect.DeepEqual(a.Tags, d.Tags) {
return fail()
}
} else if a.TagSource != "action" || a.Dictionary.Kind != "action" || strings.TrimSpace(a.Dictionary.ID) == "" || strings.TrimSpace(a.Dictionary.Name) == "" || a.Dictionary.Version < 1 || !reflect.DeepEqual(a.Tags, a.Dictionary.Tags) {
return fail()
}
previous = *a.End
stamp = *a.Last
}
if math.Abs(previous-(*slice.Split.End-*slice.Split.Start)) > 1e-6 || stamp != *slice.Split.Last {
return fail()
}
}
for _, item := range p.Discarded {
if !validWindow(item.Split) {
return fail()
}
}
return nil
}

View File

@ -29,6 +29,7 @@ type AnnotationLabelRecord struct {
Description string `json:"description,omitempty" bson:"description,omitempty"`
Example string `json:"example,omitempty" bson:"example,omitempty"`
Color string `json:"color,omitempty" bson:"color,omitempty"`
Tags []string `json:"tags,omitempty" bson:"tags,omitempty"`
Aliases []string `json:"aliases,omitempty" bson:"aliases,omitempty"`
Status AnnotationLabelStatus `json:"status" bson:"status"`
Version int `json:"version" bson:"version"`
@ -48,6 +49,7 @@ type AnnotationLabelRevision struct {
Description string `json:"description,omitempty" bson:"description,omitempty"`
Example string `json:"example,omitempty" bson:"example,omitempty"`
Color string `json:"color,omitempty" bson:"color,omitempty"`
Tags []string `json:"tags,omitempty" bson:"tags,omitempty"`
Aliases []string `json:"aliases,omitempty" bson:"aliases,omitempty"`
Status AnnotationLabelStatus `json:"status" bson:"status"`
UpdatedByUserID string `json:"updatedByUserId" bson:"updatedByUserId"`
@ -64,3 +66,23 @@ type ProjectAnnotationLabelsRecord struct {
UpdatedByUserID string `json:"updatedByUserId" bson:"updatedByUserId"`
UpdatedAt string `json:"updatedAt" bson:"updatedAt"`
}
// DictionarySelection is an immutable business classification, never a lookup
// against the current dictionary during review or delivery.
type DictionarySelection struct {
ID string `json:"id" bson:"id"`
Kind string `json:"kind" bson:"kind"`
Version int `json:"version" bson:"version"`
Code string `json:"code" bson:"code"`
Name string `json:"name" bson:"name"`
Tags []string `json:"tags" bson:"tags"`
}
func (label AnnotationLabelRecord) Selection() *DictionarySelection {
kind := label.Kind
if kind == "" {
kind = "action"
}
return &DictionarySelection{ID: label.ID, Kind: kind, Version: label.Version,
Code: label.Code, Name: label.Name, Tags: append([]string{}, label.Tags...)}
}

View File

@ -66,11 +66,12 @@ type ProjectAnnotationMembershipRecord struct {
}
type SemanticLabelDefinition struct {
ID string `json:"id" bson:"id"`
Name string `json:"name" bson:"name"`
Description string `json:"description,omitempty" bson:"description,omitempty"`
Aliases []string `json:"aliases,omitempty" bson:"aliases,omitempty"`
Color string `json:"color,omitempty" bson:"color,omitempty"`
Dictionary *DictionarySelection `json:"dictionary,omitempty" bson:"dictionary,omitempty"`
ID string `json:"id" bson:"id"`
Name string `json:"name" bson:"name"`
Description string `json:"description,omitempty" bson:"description,omitempty"`
Aliases []string `json:"aliases,omitempty" bson:"aliases,omitempty"`
Color string `json:"color,omitempty" bson:"color,omitempty"`
}
type AnnotationPrelabelSettings struct {
@ -99,20 +100,22 @@ type AnnotationTemplateRecord struct {
}
type AnnotationSegment struct {
Status string `json:"status,omitempty" bson:"status,omitempty"`
ID string `json:"id" bson:"id"`
LabelID string `json:"labelId" bson:"labelId"`
Label string `json:"label" bson:"label"`
Description string `json:"description,omitempty" bson:"description,omitempty"`
Handedness string `json:"handedness,omitempty" bson:"handedness,omitempty"`
Start float64 `json:"start" bson:"start"`
End float64 `json:"end" bson:"end"`
StartTimestampUS int64 `json:"startTimestampUs,omitempty" bson:"startTimestampUs,omitempty"`
EndTimestampUS int64 `json:"endTimestampUs,omitempty" bson:"endTimestampUs,omitempty"`
Confidence float64 `json:"confidence,omitempty" bson:"confidence,omitempty"`
Phrases []string `json:"phrases,omitempty" bson:"phrases,omitempty"`
Source string `json:"source,omitempty" bson:"source,omitempty"`
IsInvalid bool `json:"isInvalid,omitempty" bson:"isInvalid,omitempty"`
ClearDictionary bool `json:"clearDictionary,omitempty" bson:"-"`
Dictionary *DictionarySelection `json:"dictionary,omitempty" bson:"dictionary,omitempty"`
Status string `json:"status,omitempty" bson:"status,omitempty"`
ID string `json:"id" bson:"id"`
LabelID string `json:"labelId" bson:"labelId"`
Label string `json:"label" bson:"label"`
Description string `json:"description,omitempty" bson:"description,omitempty"`
Handedness string `json:"handedness,omitempty" bson:"handedness,omitempty"`
Start float64 `json:"start" bson:"start"`
End float64 `json:"end" bson:"end"`
StartTimestampUS int64 `json:"startTimestampUs,omitempty" bson:"startTimestampUs,omitempty"`
EndTimestampUS int64 `json:"endTimestampUs,omitempty" bson:"endTimestampUs,omitempty"`
Confidence float64 `json:"confidence,omitempty" bson:"confidence,omitempty"`
Phrases []string `json:"phrases,omitempty" bson:"phrases,omitempty"`
Source string `json:"source,omitempty" bson:"source,omitempty"`
IsInvalid bool `json:"isInvalid,omitempty" bson:"isInvalid,omitempty"`
}
type SliceAnnotationStatus string
@ -124,10 +127,12 @@ const (
)
type SliceSegment struct {
DiscardReasonLabelID string `json:"discardReasonLabelId,omitempty" bson:"discardReasonLabelId,omitempty"`
DiscardReasonVersion int `json:"discardReasonVersion,omitempty" bson:"discardReasonVersion,omitempty"`
DiscardReasonName string `json:"discardReasonName,omitempty" bson:"discardReasonName,omitempty"`
DiscardReasonNote string `json:"discardReasonNote,omitempty" bson:"discardReasonNote,omitempty"`
Dictionary *DictionarySelection `json:"dictionary,omitempty" bson:"dictionary,omitempty"`
DiscardReasonDictionary *DictionarySelection `json:"discardReasonDictionary,omitempty" bson:"discardReasonDictionary,omitempty"`
DiscardReasonLabelID string `json:"discardReasonLabelId,omitempty" bson:"discardReasonLabelId,omitempty"`
DiscardReasonVersion int `json:"discardReasonVersion,omitempty" bson:"discardReasonVersion,omitempty"`
DiscardReasonName string `json:"discardReasonName,omitempty" bson:"discardReasonName,omitempty"`
DiscardReasonNote string `json:"discardReasonNote,omitempty" bson:"discardReasonNote,omitempty"`
DeliveryDisposition string `json:"deliveryDisposition,omitempty" bson:"deliveryDisposition,omitempty"`
DiscardReason string `json:"discardReason,omitempty" bson:"discardReason,omitempty"`

View File

@ -65,3 +65,9 @@ type DeliveryPackageQuery struct {
Status DeliveryPackageStatus
Mode RunMode
}
// IsInternalDeliveryAudit also recognizes the schema so historical workflow
// output bindings cannot accidentally expose this internal artifact.
func IsInternalDeliveryAudit(artifact ExecutionArtifact) bool {
return artifact.PortID == "deliveryAudit" || artifact.PortName == "deliveryAudit" || artifact.Data["schemaVersion"] == "droboticflow.delivery.review-history/v2"
}

View File

@ -1292,6 +1292,7 @@ func (runtime *Runtime) releasePartitionArtifacts(
}
for _, review := range reviews {
qualityReviews = append(qualityReviews, map[string]any{
"id": review.ID, "reviewRound": review.ReviewRound, "stage": review.Stage, "confirmations": review.Confirmations, "nextRevision": review.NextRevision, "rejectTargets": review.RejectTargets,
"taskId": review.TaskID, "revision": review.Revision, "decision": review.Decision,
"reasonCodes": review.ReasonCodes, "problemSliceIds": review.ProblemSliceIDs,
"comment": review.Comment, "reviewerUserId": review.ReviewerUserID, "createdAt": review.CreatedAt,
@ -1308,12 +1309,44 @@ func (runtime *Runtime) releasePartitionArtifacts(
}
}
summary := map[string]any{"schemaVersion": "droboticflow.acceptance-summary/v2", "decision": "approved", "partitionId": partition.ID, "itemCount": len(selected), "capture": captureEvidence, "qualityReviews": qualityReviews, "finalAcceptance": acceptanceEvidence}
classificationMode, err := runtime.classificationDeliveryEnabled(ctx, run.WorkflowID)
if err != nil {
return nil, err
}
if classificationMode {
auditItems := []domain.HumanItemRecord{}
for _, itemID := range partition.ItemIDs {
auditItems = append(auditItems, selected[itemID])
}
if acceptanceID, ok := acceptanceEvidence["humanItemId"].(string); ok && acceptanceID != "" {
acceptedItem, err := runtime.store.GetHumanItem(ctx, acceptanceID)
if err != nil {
return nil, err
}
if acceptedItem == nil {
return nil, fmt.Errorf("acceptance audit item missing")
}
auditItems = append(auditItems, *acceptedItem)
}
trace, err := runtime.deliveryAuditTrace(ctx, batch.OrganizationID, auditItems)
if err != nil {
return nil, err
}
summary["trace"] = trace
}
summary["acceptedMaterializations"] = acceptedMaterializations
if batch.ProcessingProfile == domain.ProcessingProfileRawHuman {
summary["processingProfile"] = domain.ProcessingProfileRawHuman
}
encoded, _ := json.Marshal(summary)
artifacts = append(artifacts, domain.ExecutionArtifact{ID: fmt.Sprintf("%s-%s-acceptance-summary", run.ID, node.ID), RunID: run.ID, NodeID: node.ID, PortID: summaryPortID, PortName: summaryPortName, Name: "acceptance-summary.json", Type: domain.PortTypeJSON, Size: annotationArtifactSize(int64(len(encoded))), SizeBytes: int64(len(encoded)), Data: summary, Payload: &domain.ArtifactPayload{Mode: domain.ArtifactTransferModeInline, Type: domain.PortTypeJSON, Inline: summary}})
summaryArtifact := domain.ExecutionArtifact{ID: fmt.Sprintf("%s-%s-acceptance-summary", run.ID, node.ID), RunID: run.ID, NodeID: node.ID, PortID: summaryPortID, PortName: summaryPortName, Name: "acceptance-summary.json", Type: domain.PortTypeJSON, Size: annotationArtifactSize(int64(len(encoded))), SizeBytes: int64(len(encoded)), Data: summary, Payload: &domain.ArtifactPayload{Mode: domain.ArtifactTransferModeInline, Type: domain.PortTypeJSON, Inline: summary}}
if classificationMode {
summaryArtifact, err = runtime.persistClassificationSummary(ctx, run, node, partition.ID, summaryArtifact, encoded)
if err != nil {
return nil, err
}
}
artifacts = append(artifacts, summaryArtifact)
return artifacts, nil
}
clips := annotationClipItems(execution.InputArtifacts)

View File

@ -324,7 +324,19 @@ func (runtime *Runtime) normalizeOutputArtifacts(ctx context.Context, workflow *
// the exact already-uploaded object refs instead of copying every byte a
// second time.
sharedDirectoryFileRefs := map[string]domain.ArtifactReference{}
order := make([]int, 0, len(artifacts))
for index, artifact := range artifacts {
if domain.IsInternalDeliveryAudit(artifact) {
order = append(order, index)
}
}
for index, artifact := range artifacts {
if !domain.IsInternalDeliveryAudit(artifact) {
order = append(order, index)
}
}
for _, index := range order {
artifact := artifacts[index]
result[index] = artifact
if artifact.Payload != nil && artifact.Payload.Mode == domain.ArtifactTransferModeRef {
if err := domain.ValidateArtifactPayload(*artifact.Payload, artifact.Type); err != nil {

View File

@ -0,0 +1,124 @@
package worker
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"os"
"path/filepath"
"droboticflow/internal/domain"
"droboticflow/internal/store"
)
// Internal evidence is pinned with the accepted package, independently from
// customer classification JSON. Never resolve it from a later live workspace.
func (runtime *Runtime) deliveryAuditTrace(ctx context.Context, org string, items []domain.HumanItemRecord) ([]map[string]any, error) {
trace := []map[string]any{}
seen := map[string]bool{}
for _, item := range items {
if seen[item.ID] {
continue
}
seen[item.ID] = true
if item.OrganizationID != org || item.ReviewRound > 10000 {
return nil, fmt.Errorf("invalid audit item scope/round")
}
attempts, err := runtime.store.ListHumanStageAttempts(ctx, org, item.ID)
if err != nil {
return nil, err
}
events, err := runtime.store.ListHumanEvents(ctx, org, item.ID)
if err != nil {
return nil, err
}
rejects, err := runtime.store.ListHumanRejects(ctx, org, domain.HumanRejectQuery{ItemID: item.ID})
if err != nil {
return nil, err
}
baselines := []domain.ReworkBaselineRecord{}
handlingRecords := []domain.ReworkHandlingRecord{}
for round := 1; round <= item.ReviewRound; round++ {
baseline, err := runtime.store.GetReworkBaseline(ctx, store.ReworkBaselineID(item.ID, round))
if err != nil {
return nil, err
}
handling, err := runtime.store.GetReworkHandling(ctx, store.ReworkHandlingID(item.ID, round))
if err != nil {
return nil, err
}
if handling != nil {
handlingRecords = append(handlingRecords, *handling)
}
if baseline != nil {
baselines = append(baselines, *baseline)
}
}
trace = append(trace, map[string]any{"humanItemId": item.ID, "taskId": item.LegacyTaskID, "revision": item.CurrentRevision, "reviewRound": item.ReviewRound, "stageAttempts": attempts, "events": events, "rejects": rejects, "reworkBaselines": baselines, "reworkHandling": handlingRecords})
}
body, err := json.Marshal(trace)
if err != nil {
return nil, err
}
if len(body) > 6<<20 {
return nil, fmt.Errorf("internal delivery trace exceeds 6 MiB; refusing incomplete export")
}
return trace, nil
}
func (runtime *Runtime) classificationDeliveryEnabled(ctx context.Context, workflowID string) (bool, error) {
workflow, err := runtime.store.GetWorkflow(ctx, workflowID)
if err != nil {
return false, err
}
if workflow == nil {
return false, nil
}
for _, node := range workflow.Nodes {
if node.Data.OperatorID != "operator-builtin-package" {
continue
}
for _, param := range node.Data.Hyperparams {
if param.Key == "DELIVERY_JSON_V4" && param.Value == "true" {
return true, nil
}
}
}
return false, nil
}
// Reuse normal internal-artifact persistence rather than exceeding the 64 KiB
// inline transport limit with a multi-round audit. Customer publication is a
// different path and never sees this source artifact.
func (runtime *Runtime) persistClassificationSummary(ctx context.Context, run *domain.RunRecord, node domain.WorkflowNode, partitionID string, artifact domain.ExecutionArtifact, body []byte) (domain.ExecutionArtifact, error) {
if len(body) > 8<<20 {
return artifact, fmt.Errorf("acceptance audit summary exceeds 8 MiB")
}
if len(body) <= domain.MaxInlinePayloadBytes && !structuredJSONRequiresReference(artifact, artifact.Data) {
return artifact, nil
}
if runtime.workspaceRoot == "" {
return artifact, fmt.Errorf("classification audit requires a persistent workspace")
}
sum := sha256.Sum256(body)
directory := filepath.Join(runtime.workspaceRoot, "system", "runs", run.ID, "release-audits", artifactSlug(partitionID), hex.EncodeToString(sum[:]))
if err := os.MkdirAll(directory, 0700); err != nil {
return artifact, err
}
artifact.Path = filepath.Join(directory, "acceptance-summary.json")
if err := os.WriteFile(artifact.Path, body, 0600); err != nil {
return artifact, err
}
artifact.Payload = nil
workflow, err := runtime.store.GetWorkflow(ctx, run.WorkflowID)
if err != nil {
return artifact, err
}
result, err := runtime.normalizeOutputArtifacts(ctx, workflow, run, node.ID, []domain.ExecutionArtifact{artifact})
if err != nil {
return artifact, err
}
return result[0], nil
}

View File

@ -0,0 +1,98 @@
package worker
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"droboticflow/internal/domain"
"droboticflow/internal/store"
)
func TestClassificationAuditUsesInternalStorageBeyondInlineLimit(t *testing.T) {
ctx := context.Background()
ds := store.NewMemoryStore()
publisher := &memoryArtifactPublisher{}
runtime := &Runtime{store: ds, workspaceRoot: t.TempDir(), artifactPublisher: publisher}
c := domain.StorageConnectionRecord{ID: "audit-storage", OrganizationID: "audit-org", Provider: domain.StorageProviderBOS, Bucket: "internal-audit", Endpoint: "https://bos.example.test"}
if err := ds.UpsertStorageConnection(ctx, c); err != nil {
t.Fatal(err)
}
workflow := domain.WorkflowRecord{ID: "wf", OrganizationID: c.OrganizationID, ArtifactStorage: &domain.ArtifactStoragePolicy{ConnectionID: c.ID, Bucket: c.Bucket, PrefixTemplate: "artifacts/{runId}/{nodeId}"}}
if err := ds.UpsertWorkflow(ctx, workflow); err != nil {
t.Fatal(err)
}
run := &domain.RunRecord{ID: "run-audit", OrganizationID: c.OrganizationID, WorkflowID: workflow.ID}
data := map[string]any{"schemaVersion": "droboticflow.acceptance-summary/v2", "trace": []any{map[string]any{"comment": strings.Repeat("详细返工记录", 20000)}}}
body, _ := json.Marshal(data)
artifact := domain.ExecutionArtifact{ID: "summary", RunID: run.ID, NodeID: "human", Name: "acceptance-summary.json", PortID: "acceptanceSummary", Type: domain.PortTypeJSON, Data: data, SizeBytes: int64(len(body))}
saved, err := runtime.persistClassificationSummary(ctx, run, domain.WorkflowNode{ID: "human"}, "partition", artifact, body)
if err != nil {
t.Fatal(err)
}
if saved.Payload == nil || saved.Payload.Mode != domain.ArtifactTransferModeRef || saved.Payload.Ref == nil {
t.Fatal("oversized trace remained inline")
}
ref := saved.Payload.Ref
if !strings.HasPrefix(ref.Key, "artifacts/run-audit/human/") || strings.Contains(ref.Key, "deliveries/") {
t.Fatal("audit in customer location")
}
raw := publisher.objects[ref.Bucket+"/"+ref.Key]
if string(raw) != string(body) || ref.SizeBytes != int64(len(body)) || len(ref.SHA256) != 64 {
t.Fatal("internal trace truncated or unpinned")
}
}
func TestClassificationAuditTraceRetainsFormalReworkAndScopedEvidence(t *testing.T) {
ctx := context.Background()
ds := store.NewMemoryStore()
runtime := &Runtime{store: ds}
item := domain.HumanItemRecord{ID: "item", OrganizationID: "audit-org", LegacyTaskID: "task", CurrentRevision: 3, ReviewRound: 2}
for round := 1; round <= 2; round++ {
if err := ds.UpsertReworkBaseline(ctx, domain.ReworkBaselineRecord{ID: store.ReworkBaselineID(item.ID, round), OrganizationID: item.OrganizationID, ItemID: item.ID, Round: round, Problems: []domain.ReviewProblem{{ID: "problem", Text: "正式退回意见"}}}); err != nil {
t.Fatal(err)
}
}
ds.AppendHumanEvent(ctx, domain.HumanEventRecord{ID: "event", OrganizationID: item.OrganizationID, ItemID: item.ID, EventType: "review_rejected"})
trace, err := runtime.deliveryAuditTrace(ctx, item.OrganizationID, []domain.HumanItemRecord{item})
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(trace)
if !strings.Contains(string(raw), "正式退回意见") || !strings.Contains(string(raw), "review_rejected") {
t.Fatal("formal trace lost")
}
if _, err := runtime.deliveryAuditTrace(ctx, "foreign", []domain.HumanItemRecord{item}); err == nil {
t.Fatal("foreign trace accepted")
}
audit := domain.ExecutionArtifact{Data: compactArtifactData(map[string]any{"schemaVersion": "droboticflow.delivery.review-history/v2"})}
if !domain.IsInternalDeliveryAudit(audit) {
t.Fatal("compacted audit no longer excluded from final export")
}
}
func TestInternalAuditFailurePreventsCustomerDirectoryPublication(t *testing.T) {
ctx := context.Background()
ds := store.NewMemoryStore()
p := &deliveryPublisher{}
runtime := &Runtime{store: ds, artifactPublisher: p}
connection := domain.StorageConnectionRecord{ID: "audit-store", OrganizationID: "org", Provider: domain.StorageProviderBOS, Bucket: "b"}
ds.UpsertStorageConnection(ctx, connection)
workflow := &domain.WorkflowRecord{ID: "wf", OrganizationID: "org", DefinitionState: domain.WorkflowDefinitionStateReleased, ArtifactStorage: &domain.ArtifactStoragePolicy{ConnectionID: connection.ID, Bucket: connection.Bucket, PrefixTemplate: "artifacts/{runId}/{nodeId}"}}
run := &domain.RunRecord{ID: "run-audit-first", OrganizationID: "org", WorkflowID: "wf"}
ds.UpsertCaptureRecord(ctx, domain.CaptureRecord{ID: "capture", OrganizationID: "org", RunID: run.ID, Delivery: &domain.CaptureDeliveryTarget{ConnectionID: connection.ID, Bucket: connection.Bucket, Prefix: "deliveries/项目__project/capture/part0001"}})
directory := t.TempDir()
if err := os.WriteFile(filepath.Join(directory, "part.json"), []byte(`{"classification":"synthetic"}`), 0600); err != nil {
t.Fatal(err)
}
artifacts := []domain.ExecutionArtifact{{ID: "customer", RunID: run.ID, Name: "delivery.directory.index", Type: domain.PortTypeFile, Path: directory}, {ID: "audit", RunID: run.ID, PortID: "deliveryAudit", Type: domain.PortTypeJSON, Path: filepath.Join(directory, "missing-internal-audit.json"), Data: map[string]any{"schemaVersion": "droboticflow.delivery.review-history/v2"}}}
if _, err := runtime.normalizeOutputArtifacts(ctx, workflow, run, "package", artifacts); err == nil {
t.Fatal("missing audit accepted")
}
if len(p.published) != 0 || len(p.puts) != 0 {
t.Fatal("customer directory became visible before audit persisted")
}
}

View File

@ -256,7 +256,7 @@ func (runtime *Runtime) syncDeliveryPackage(ctx context.Context, batch domain.Hu
handposeIDs := make([]string, 0)
for _, artifact := range child.Artifacts {
artifactIDs = append(artifactIDs, artifact.ID)
if artifact.RunID == child.ID {
if artifact.RunID == child.ID && !domain.IsInternalDeliveryAudit(artifact) {
finalArtifactIDs = append(finalArtifactIDs, artifact.ID)
}
identity := strings.ToLower(artifact.Name + " " + artifact.Path + " " + artifact.NodeID)

View File

@ -5,6 +5,7 @@ import hashlib
import bisect
import csv
import json
import math
import pathlib
import re
import shutil
@ -752,9 +753,11 @@ def _part_index_discarded_slice(
"reason": str(evidence.get("reason") or ""),
"reasonLabelId": str(evidence.get("reasonLabelId") or ""),
"reasonName": str(evidence.get("reasonName") or ""),
"note": str(evidence.get("note") or ""),
"note": str(evidence.get("note") or evidence.get("reasonNote") or ""),
"reasonVersion": evidence.get("reasonVersion"),
"dictionary": evidence.get("dictionary"),
"discardedByUserId": str(evidence.get("discardedByUserId") or ""),
"discardedAt": str(evidence.get("discardedAt") or ""),
"discardedAt": str(evidence.get("discardedAt") or evidence.get("at") or ""),
},
},
"segments": [],
@ -843,13 +846,184 @@ def _description_status(actions: list[dict[str, Any]]) -> str:
return "empty_confirmed"
def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
"""Use the established immutable directory publisher for the two JSON files.
def _v4_tags(selection: Any, kind: str) -> list[str]:
if not isinstance(selection, dict) or selection.get("kind") != kind or not str(selection.get("id") or "").strip() or type(selection.get("version")) is not int or selection["version"] < 1 or not str(selection.get("name") or "").strip():
raise RuntimeError(f"v4 missing frozen {kind} dictionary; confirm historical classification before delivery")
tags = selection.get("tags")
if not isinstance(tags, list) or not tags or any(not isinstance(tag, str) or not tag.strip() or tag != tag.strip() for tag in tags) or len(set(tags)) != len(tags):
raise RuntimeError(f"v4 missing frozen {kind} tags; confirm classification before delivery")
return list(tags)
def _v4_original_sources(manifest, capture_id, part_id, capture_prefix):
source = manifest.get("originalSource")
if not isinstance(source, dict) or source.get("provider") != "BOS" or not str(source.get("bucket") or "").strip():
raise RuntimeError("v4 requires frozen original BOS source selection; workspace paths cannot identify original data")
prefix = str(source.get("prefix") or "")
part_prefix = str(source.get("partPrefix") or "")
if prefix != capture_prefix or part_prefix != prefix + "/" + part_id or source.get("captureUnit") != capture_id + "/" + part_id:
raise RuntimeError("v4 original source identity disagrees with accepted capture/part")
if not prefix or str(pathlib.PurePosixPath(prefix)) != prefix or pathlib.PurePosixPath(prefix).is_absolute() or ".." in pathlib.PurePosixPath(prefix).parts or any(c in prefix for c in ("\\", "\x00")):
raise RuntimeError("unsafe v4 original source prefix")
keys = source.get("objectKeys")
if not isinstance(keys, list) or not keys or any(not isinstance(key, str) for key in keys) or len(set(keys)) != len(keys):
raise RuntimeError("v4 requires an unambiguous original object selection")
files = {}
cap, part = re.escape(capture_id), re.escape(part_id)
patterns = {
"leftVideo": rf"{part}/{cap}_camera_left_{part}(?:_[a-z0-9_]+)?\.(?:mp4|mov|mkv)",
"rightVideo": rf"{part}/{cap}_camera_right_{part}(?:_[a-z0-9_]+)?\.(?:mp4|mov|mkv)",
"leftPts": rf"{part}/{cap}_camera_left_{part}(?:_[a-z0-9_]+)?_pts\.csv",
"rightPts": rf"{part}/{cap}_camera_right_{part}(?:_[a-z0-9_]+)?_pts\.csv",
"imu": rf"{part}/{cap}_imu_{part}\.csv",
"cameraCalibration": rf"{cap}_calibration_camera\.yaml",
"imuCalibration": rf"{cap}_calibration_imu\.yaml",
"metadata": rf"meta_{cap}\.json",
}
for key in keys:
if not key.startswith(prefix + "/") or str(pathlib.PurePosixPath(key)) != key or ".." in pathlib.PurePosixPath(key).parts or any(c in key for c in ("\\", "\x00")):
raise RuntimeError("unsafe v4 original source object")
relative = key[len(prefix) + 1:]
if "/" in relative and not relative.startswith(part_id + "/"):
raise RuntimeError("v4 original selection contains another part")
for role, pattern in patterns.items():
if re.fullmatch(pattern, relative, re.IGNORECASE):
if role in files: raise RuntimeError(f"ambiguous v4 original {role} selection")
files[role] = {"path": relative}
missing = sorted(set(patterns) - files.keys())
if missing: raise RuntimeError("v4 missing original files: " + ", ".join(missing))
return {"provider": "BOS", "bucket": source["bucket"], "prefix": prefix, "partPrefix": part_prefix,
"snapshotFingerprint": str(source.get("snapshotFingerprint") or ""), "files": files}
def _v4_number(value, label):
if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value):
raise RuntimeError(f"v4 missing or invalid {label}")
return float(value)
def _v4_timestamp(value, label):
if type(value) is not int or value <= 0: raise RuntimeError(f"v4 missing or invalid {label}")
return value
def _v4_consumer_payload(payload, manifest, source_root):
"""Project immutable accepted facts into a self-contained consumer contract.
No dictionary lookup, source media reads, transcoding or BOS writes here.
Hardware PTS are authoritative; media seconds are explicitly named and never
used to manufacture missing hardware timestamps.
"""
if not isinstance(manifest, dict): raise RuntimeError("v4 requires accepted source manifest")
original = _v4_original_sources(manifest, payload["capture"]["captureId"], payload["partId"], payload["capture"]["prefix"])
payload["originalSource"] = original
payload["consumerContract"] = "droboticflow.classification-consumer/v1"
payload["timeBases"] = {"taskSplitSeconds": "original_left_video", "segmentSeconds": "slice_left_video",
"segmentPartSeconds": "original_left_video", "timestampUs": "original_hardware_pts",
"interval": "half-open", "frameRanges": "zero-based-half-open"}
payload["pathBases"] = {"sources": "original_capture", "sourceMetadata": "original_capture", "calibration": "original_capture"}
# Workspace sources are private transport references, not the original files.
payload["sources"] = {kind: original["files"][kind] for kind in ("leftVideo", "rightVideo", "leftPts", "rightPts", "imu")}
payload["sourceMetadata"]["path"] = original["files"]["metadata"]["path"]
for item in payload["calibration"]:
# Bind by actual calibration kind, not input ordering.
kind = "imuCalibration" if "imu" in item["name"].lower() else "cameraCalibration" if "camera" in item["name"].lower() else ""
if not kind: raise RuntimeError("v4 unknown calibration kind")
item["path"] = original["files"][kind]["path"]
rows = {str(row["sliceId"]): row for row in manifest["items"]}
if len(rows) != len(manifest["items"]): raise RuntimeError("v4 duplicate slice identities")
for item in payload["slices"] + payload["discardedSlices"]:
split = item["taskSplit"]
start, end = (_v4_number(split[k], k) for k in ("startSecond", "endSecond"))
first, last = (_v4_timestamp(split[k], k) for k in ("startTimestampUs", "endTimestampUs"))
if start < 0 or end <= start or last <= first: raise RuntimeError("v4 invalid accepted slice range")
if item["classification"]["deliveryDisposition"] == "discard":
item["sliceName"] = ""
evidence = item["classification"]["discardEvidence"]
selected = evidence.get("dictionary")
if selected is not None:
if (not isinstance(selected, dict) or selected.get("kind") != "discard_reason" or
not selected.get("id") or not selected.get("name") or
type(selected.get("version")) is not int or selected["version"] < 1):
raise RuntimeError("v4 invalid frozen discard dictionary")
for alias, field in (("reasonLabelId", "id"), ("reasonName", "name"), ("reasonVersion", "version")):
if evidence.get(alias) not in (None, "", 0, selected[field]):
raise RuntimeError("v4 discard reason disagrees with frozen dictionary")
evidence[alias] = selected[field]
continue
parent_tags = _v4_tags(item["classification"].get("dictionary"), "slice")
item["classification"]["tags"] = parent_tags
row = rows[item["sliceId"]]
sync = json.loads((source_root / row["directory"] / "synchronized-slice.json").read_text())
seconds, stamps = sync.get("boundarySeconds", []), sync.get("boundaryTimestampUs", [])
if seconds or stamps:
if not isinstance(seconds, list) or not isinstance(stamps, list) or len(seconds) < 2 or len(seconds) != len(stamps):
raise RuntimeError("v4 invalid hardware boundary mapping")
seconds = [_v4_number(value, "boundary") for value in seconds]
stamps = [_v4_timestamp(value, "boundary timestamp") for value in stamps]
if (abs(seconds[0]) > 1e-7 or abs(seconds[-1] - (end-start)) > 1e-7 or
stamps[0] != first or stamps[-1] != last or
any(a >= b for a, b in zip(seconds, seconds[1:])) or
any(a >= b for a, b in zip(stamps, stamps[1:]))):
raise RuntimeError("v4 invalid hardware boundary mapping")
def timestamp(second):
if abs(second) < 1e-7: return first
if abs(second - (end-start)) < 1e-7: return last
hits = [stamps[i] for i, value in enumerate(seconds) if abs(_v4_number(value,"boundary") - second) < 1e-7]
if len(hits) != 1: raise RuntimeError("v4 action boundary has no original PTS mapping")
return _v4_timestamp(hits[0], "boundary timestamp")
actions = []
for action in item["segments"]:
a = _v4_number(action.get("clipStartMs"), "action clipStartMs") / 1000
b = _v4_number(action.get("clipEndMs"), "action clipEndMs") / 1000
if a < -1e-7 or b <= a or b > end-start+1e-7: raise RuntimeError("v4 action escapes its parent slice")
ta, tb = (_v4_timestamp(action.get(k), k) for k in ("startTimestampUs", "endTimestampUs"))
if not first <= ta < tb <= last: raise RuntimeError("v4 action timestamps escape its parent slice")
if seconds and (timestamp(a) != ta or timestamp(b) != tb): raise RuntimeError("v4 action seconds disagree with original PTS")
action.update(startSecond=a, endSecond=b, partStartSecond=start+a, partEndSecond=start+b)
own = action.get("dictionary")
action["tags"] = _v4_tags(own, "action") if own is not None else list(parent_tags)
action["tagSource"] = "action" if own is not None else "slice"
actions.append(action)
actions.sort(key=lambda a: a["startSecond"])
complete, previous = [], 0.0
def gap(a,b):
ta,tb=timestamp(a),timestamp(b)
return {"id":f"{item['sliceId']}:gap:{ta}:{tb}", "description":"", "isInvalid":False,
"contentKind":"gap_placeholder", "startSecond":a,"endSecond":b,
"clipStartMs":a*1000,"clipEndMs":b*1000,"partStartSecond":start+a,"partEndSecond":start+b,
"startTimestampUs":ta,"endTimestampUs":tb,"tags":list(parent_tags),"tagSource":"slice"}
for action in actions:
if action["startSecond"] < previous-1e-7: raise RuntimeError("v4 overlapping actions")
if action["startSecond"] > previous+1e-7: complete.append(gap(previous,action["startSecond"]))
complete.append(action);previous=action["endSecond"]
if previous < end-start-1e-7: complete.append(gap(previous,end-start))
if not complete: raise RuntimeError("v4 kept slice has no classified action coverage")
previous_stamp = first
for action in complete:
if action["startTimestampUs"] != previous_stamp or action["endTimestampUs"] <= previous_stamp:
raise RuntimeError("v4 action PTS coverage is discontinuous")
previous_stamp = action["endTimestampUs"]
if previous_stamp != last: raise RuntimeError("v4 action PTS coverage is incomplete")
item["segments"] = complete
counts = {kind:sum(a["contentKind"]==kind for a in complete) for kind in ("annotated","gap_placeholder","invalid_marked")}
item["classification"]["contentCounts"] = counts
item["classification"]["descriptionStatus"] = _description_status(complete)
if not counts["annotated"] and counts["invalid_marked"] and counts["gap_placeholder"]: item["classification"]["descriptionStatus"]="invalid_and_blank"
item["synchronization"]["rightBoundarySeconds"] = sync.get("rightBoundarySeconds", [])
def _emit_part_index_directory(run_id, node_id, output, node_data, helpers, payload, internal_audit=None, revision_manifest=None, source_root=None):
"""Use the established immutable publisher for versioned customer JSON.
V3 retains its historical sibling review file; v4 stores that audit in a
separate internal artifact before publishing the customer directory.
Legacy single-file publishers deliberately accept only v1/v2. A directory
artifact preserves their ownership/conditional-publication protocol without
teaching an old worker to misinterpret a new single-file schema.
"""
v4 = internal_audit is not None
capture = dict(payload["capture"])
prefix = pathlib.PurePosixPath(str(capture.get("prefix") or ""))
part_id = prefix.name
@ -860,7 +1034,7 @@ def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
if prefix.is_absolute() or ".." in prefix.parts or len(prefix.parts) < 3:
raise RuntimeError("unsafe v3 capture identity prefix")
capture.update(captureId=capture_id, prefix=str(prefix.parent))
payload.update(schemaVersion="droboticflow.delivery.part-index/v3", capture=capture, partId=part_id)
payload.update(schemaVersion="droboticflow.delivery.part-index/v4" if v4 else "droboticflow.delivery.part-index/v3", capture=capture, partId=part_id)
payload["pathBases"] = {"sourceMetadata": "original_capture", "calibration": "original_capture", "sources": "workspace"}
payload["sourceMetadata"] = {"path": f"meta_{capture_id}.json"}
history = {
@ -869,7 +1043,8 @@ def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
"qualityReviews": payload["acceptance"].pop("qualityReviews", []),
}
payload["reviewHistory"] = {"path": "review-history.json", "pathBase": "part_package"}
payload["acceptance"]["finalAcceptance"].pop("confirmedByUserId", None)
if not v4:
payload["acceptance"]["finalAcceptance"].pop("confirmedByUserId", None)
for item in payload["calibration"]:
if pathlib.PurePosixPath(item["name"]).name != item["name"] or "\\" in item["name"]:
raise RuntimeError("unsafe calibration filename")
@ -889,6 +1064,54 @@ def _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload):
classification["contentCounts"] = counts
if not counts["described"] and counts["invalid_marked"] and counts["gap_placeholder"]:
classification["descriptionStatus"] = "invalid_and_blank"
if v4:
if not isinstance(internal_audit.get("trace"), list):
raise RuntimeError("DELIVERY_JSON_V4 requires complete internal trace input")
if _output(node_data, helpers, "deliveryAudit") is None:
raise RuntimeError("DELIVERY_JSON_V4 requires an internal deliveryAudit output")
audit = {
"schemaVersion": "droboticflow.delivery.review-history/v2",
"captureId": capture_id, "partId": part_id, "revision": payload["revision"],
"acceptance": payload.pop("acceptance"), "lineage": payload.pop("lineage"),
"qualityReviews": internal_audit.get("qualityReviews", []),
"trace": internal_audit.get("trace", []),
"acceptedMaterializations": internal_audit.get("acceptedMaterializations", []),
}
audit["acceptance"]["finalAcceptance"] = internal_audit.get("finalAcceptance", {})
payload.pop("reviewHistory", None)
payload["capture"] = {"captureId": capture_id, "prefix": str(prefix.parent)}
for item in payload["slices"] + payload["discardedSlices"]:
classification = item["classification"]
# Blank descriptions retain the independently selected base class.
if classification.get("dictionary") is None:
classification.pop("dictionary", None)
evidence = classification.get("discardEvidence", {})
evidence.pop("discardedAt", None)
if evidence.get("dictionary") is None:
evidence.pop("dictionary", None)
counts = classification.get("contentCounts")
if counts is not None:
counts["annotated"] = counts.pop("described")
for action in item["segments"]:
if action["contentKind"] == "described": action["contentKind"] = "annotated"
for key in ("source", "confidence", "phrases", "labelId"):
action.pop(key, None)
if action.get("dictionary") is None: action.pop("dictionary", None)
audit["materializedSources"] = payload.get("sources", {})
_v4_consumer_payload(payload, revision_manifest, source_root)
body = (json.dumps(payload, ensure_ascii=False, sort_keys=True, indent=2) + "\n").encode("utf8")
audit["partSha256"] = hashlib.sha256(body).hexdigest()
if len(json.dumps(audit, ensure_ascii=False, indent=2).encode("utf8")) > 8 * 1024 * 1024:
raise RuntimeError("internal delivery audit exceeds 8 MiB; refusing incomplete export")
staging = pathlib.Path(tempfile.mkdtemp(prefix="part-index-v4-", dir=output))
(staging / "part.json").write_bytes(body)
# A sibling artifact stays in the internal run, never in the customer tree.
audit_artifact = helpers["write_json_artifact"](run_id, node_id, output, "review-history.json", audit, output=_output(node_data, helpers, "deliveryAudit"))
directory = helpers["build_virtual_artifact"](
run_id, node_id, "delivery.directory.index", str(staging), "file", 0,
data={"schemaVersion": "droboticflow.directory-index/v1", "presentation": "collection", "datasetType": "directory", "dataSource": "FILESYSTEM", "rootPath": str(staging), "sliceCount": len(payload["slices"]), "classificationOnly": True},
output=_output(node_data, helpers, "deliveryDirectory"))
return [directory, audit_artifact]
staging = pathlib.Path(tempfile.mkdtemp(prefix="part-index-v3-", dir=output))
for name, document in (("part.json", payload), ("review-history.json", history)):
(staging / name).write_text(json.dumps(document, ensure_ascii=False, sort_keys=True, indent=2) + "\n", encoding="utf8")
@ -944,6 +1167,7 @@ def _part_index_slice(
"clipStartMs": _optional_float(action.get("clipStartMs")),
"clipEndMs": _optional_float(action.get("clipEndMs")),
"labelId": str(action.get("descriptionTemplateId") or ""),
"dictionary": action.get("dictionary"),
"phrases": [str(item) for item in action.get("phrases", [])] if isinstance(action.get("phrases"), list) else [],
"source": str(action.get("source") or ""),
"confidence": _optional_float(action.get("confidence")),
@ -976,6 +1200,7 @@ def _part_index_slice(
"deliveryDisposition": declared,
"taskLabel": str(synchronization.get("taskLabel") or ""),
"taskLabelId": str(synchronization.get("taskLabelId") or ""),
"dictionary": synchronization.get("dictionary"),
"descriptionStatus": _description_status(actions),
},
"segments": actions,
@ -999,9 +1224,11 @@ def _part_index_slice(
"reason": str(evidence.get("reason") or ""),
"reasonLabelId": str(evidence.get("reasonLabelId") or ""),
"reasonName": str(evidence.get("reasonName") or ""),
"note": str(evidence.get("note") or ""),
"note": str(evidence.get("note") or evidence.get("reasonNote") or ""),
"reasonVersion": evidence.get("reasonVersion"),
"dictionary": evidence.get("dictionary"),
"discardedByUserId": str(evidence.get("discardedByUserId") or ""),
"discardedAt": str(evidence.get("discardedAt") or ""),
"discardedAt": str(evidence.get("discardedAt") or evidence.get("at") or ""),
}
# Pose frames deliberately stay out of the index: the customer contract for
# this mode is the split information, and embedded per-frame pose would
@ -1651,6 +1878,12 @@ def _run_accepted_slice_directory(
raise RuntimeError("acceptedSliceDirectory does not prove gap-free full-video coverage")
summary_artifact = next((item for item in inputs if _port(item) == "acceptanceSummary"), None)
summary_payload = summary_artifact.get("data") if isinstance((summary_artifact or {}).get("data"), dict) else {}
if summary_artifact is not None and not isinstance(summary_payload.get("finalAcceptance"), dict):
summary_path = _materialize_input_file(summary_artifact, output / "_acceptance", helpers)
if summary_path.stat().st_size > 8 * 1024 * 1024:
raise RuntimeError("acceptance summary exceeds 8 MiB")
summary_payload = json.loads(summary_path.read_text(encoding="utf8"))
if not isinstance(summary_payload, dict): raise RuntimeError("invalid acceptance summary reference")
final_acceptance = summary_payload.get("finalAcceptance") if isinstance(summary_payload.get("finalAcceptance"), dict) else {}
capture_record = _part_index_capture_record(summary_payload, revision_manifest)
all_slices = revision_manifest.get("items", [])
@ -1669,7 +1902,11 @@ def _run_accepted_slice_directory(
discarded_slices.append(row)
else:
kept_slices.append(row)
if disposition_contract:
params = helpers["hyperparams"](node_data) if callable(helpers.get("hyperparams")) else {}
classification_v4 = _bool(params, "DELIVERY_JSON_V4", False)
if classification_v4 and not coverage_required:
raise RuntimeError("DELIVERY_JSON_V4 requires verified full-video coverage")
if disposition_contract or classification_v4:
pins = summary_payload.get("acceptedMaterializations", [])
expected_sha = _sha256_file(revision_manifest_path)
if summary_payload.get("decision") != "approved" or final_acceptance.get("confirmed") is not True or not any(
@ -1679,7 +1916,7 @@ def _run_accepted_slice_directory(
raise RuntimeError("slice dispositions are not pinned to the accepted manifest revision")
if int(coverage.get("sliceCount", -1)) != len(all_slices):
raise RuntimeError("review coverage slice count mismatch")
if disposition_contract and not kept_slices:
if disposition_contract and not kept_slices and not _bool(params, "DELIVERY_JSON_V4", False):
if any(p.get("key") == "PROCESSING_PROFILE" and p.get("value") == "raw_human" for p in node_data.get("hyperparams", [])) and (summary_payload.get("processingProfile") != "raw_human" or final_acceptance.get("processingProfile") != "raw_human"):
raise RuntimeError("raw-human package requires matching accepted task profile")
# Internal evidence only: deliberately no deliveryDirectory output and
@ -1697,7 +1934,6 @@ def _run_accepted_slice_directory(
payloads = _payload_tree(_payloads_with_full_pose_manifests(inputs, output, helpers))
headpose = _first_payload(payloads, "droboticflow.headpose/v1")
handpose = _first_payload(payloads, "droboticflow.wilor.hand_pose_result/v1")
params = helpers["hyperparams"](node_data) if callable(helpers.get("hyperparams")) else {}
if _bool(params, "PART_INDEX_DELIVERY", False):
if source_index and not _bool(params, "SOURCE_INDEX_DELIVERY", False):
raise RuntimeError("source-index JSON delivery requires explicit SOURCE_INDEX_DELIVERY approval; use directory delivery to preserve physical discard exclusion")
@ -1715,8 +1951,10 @@ def _run_accepted_slice_directory(
for item in payload["slices"] + payload["discardedSlices"]:
item["sourceFrames"] = frames[item["sliceId"]]
body = (json.dumps(payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + "\n").encode("utf8")
if _bool(params, "DELIVERY_JSON_V4", False):
return _emit_part_index_directory(run_id, node_id, output, node_data, helpers, payload, summary_payload, revision_manifest, source_root)
if _bool(params, "DELIVERY_JSON_V3", False):
return _emit_part_index_v3(run_id, node_id, output, node_data, helpers, payload)
return _emit_part_index_directory(run_id, node_id, output, node_data, helpers, payload)
return _emit_part_index(run_id, node_id, output, node_data, helpers, body)
if source_index:
source_root = _source_index_physical_export(source_root, revision_manifest, kept_slices, output)
@ -1953,6 +2191,8 @@ def _run_multiport(
def run(run_id: str, node_id: str, output_dir: Any, node_data: dict[str, Any], inputs: list[dict[str, Any]], helpers: HelperMap) -> list[dict[str, Any]]:
params = helpers["hyperparams"](node_data) if callable(helpers.get("hyperparams")) else {}
if _bool(params, "DELIVERY_JSON_V4", False) and (not _bool(params, "PART_INDEX_DELIVERY", False) or _bool(params, "DELIVERY_JSON_V3", False)):
raise RuntimeError("DELIVERY_JSON_V4 requires PART_INDEX_DELIVERY and cannot combine with DELIVERY_JSON_V3")
if _bool(params, "DELIVERY_JSON_V3", False) and not _bool(params, "PART_INDEX_DELIVERY", False):
raise RuntimeError("DELIVERY_JSON_V3 requires PART_INDEX_DELIVERY")
# A part-index delivery intentionally has no directory port; it must still

View File

@ -39,6 +39,7 @@ export function AnnotationLabelsPanel({
const [error, setError] = useState("");
const [editingLabelId, setEditingLabelId] = useState("");
const [inputText, setInputText] = useState("");
const [tagText, setTagText] = useState("");
const [checkedIds, setCheckedIds] = useState<string[]>([]);
const [importOpen, setImportOpen] = useState(false);
const [importText, setImportText] = useState("");
@ -83,6 +84,7 @@ export function AnnotationLabelsPanel({
setEnabled([]);
setEditingLabelId("");
setInputText("");
setTagText("");
setCheckedIds([]);
setImportOpen(false);
setImportText("");
@ -99,6 +101,7 @@ export function AnnotationLabelsPanel({
const resetEditing = () => {
setEditingLabelId("");
setInputText("");
setTagText("");
};
const startEdit = (label: AnnotationLabelRecord) => {
@ -106,6 +109,7 @@ export function AnnotationLabelsPanel({
setError("");
setEditingLabelId(label.id);
setInputText(label.name || label.description || "");
setTagText((label.tags || []).join(","));
};
const saveLabel = async (e?: React.FormEvent) => {
@ -133,7 +137,16 @@ export function AnnotationLabelsPanel({
try {
const existing = labels.find((item) => item.id === editingLabelId);
const code = `desc_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 6)}`;
const tags = [
...new Set(
tagText
.split(/[,,\n]/)
.map((tag) => tag.trim())
.filter(Boolean),
),
];
const payload = {
tags,
kind,
code,
name: text,
@ -143,7 +156,7 @@ export function AnnotationLabelsPanel({
status: existing?.status || ("active" as const),
};
const saved = await reuseOrCreateDictionaryLabel({
let saved = await reuseOrCreateDictionaryLabel({
kind,
name: text,
preferredId: editingLabelId,
@ -154,6 +167,19 @@ export function AnnotationLabelsPanel({
});
if (scope.current !== generation) return;
// Explicit tag maintenance versions the shared dictionary word; saved
// selections retain their original tags.
if (
(editingLabelId === saved.id || tags.length > 0) &&
JSON.stringify(saved.tags || []) !== JSON.stringify(tags)
) {
saved = await apiRequest<AnnotationLabelRecord>(
`/api/annotation-labels/${saved.id}`,
{ method: "PATCH" },
{ ...saved, tags },
);
if (scope.current !== generation) return;
}
// Relink only this project; never rename a term shared by other projects.
const selection = await apiRequest<ProjectAnnotationLabelsRecord>(
selectionPath,
@ -321,6 +347,14 @@ export function AnnotationLabelsPanel({
}
className="flex-1 min-w-[280px] rounded-lg border border-[color:var(--app-border)] bg-[color:var(--app-panel)] px-4 py-2.5 text-sm text-[color:var(--app-text)] shadow-sm focus:border-blue-500 focus:outline-none"
/>
<input
aria-label="分类标签"
placeholder="分类标签,用逗号分隔"
value={tagText}
onChange={(e) => setTagText(e.target.value)}
disabled={loading || saving}
className="min-w-48 flex-1 rounded-lg border border-[color:var(--app-border)] bg-[color:var(--app-panel)] px-3 py-2 text-sm"
/>
<button
type="submit"
disabled={loading || saving || !inputText.trim()}
@ -508,6 +542,14 @@ export function AnnotationLabelsPanel({
className="flex-1 text-sm text-slate-800 dark:text-gray-200 cursor-pointer select-none"
>
{text}
{(item.tags || []).map((tag) => (
<span
key={tag}
className="ml-2 rounded bg-blue-50 px-2 py-0.5 text-xs text-blue-700 dark:bg-blue-900/30 dark:text-blue-200"
>
{tag}
</span>
))}
</span>
{canManage && (
<div className="flex items-center gap-1 shrink-0">
@ -537,9 +579,7 @@ export function AnnotationLabelsPanel({
</div>
)}
</div>
<p className="text-sm text-slate-500">
支持导出 CSV、批量导入与勾选批量删除;编辑或删除只影响本项目的可选词条,不影响其他项目及已保存的标注、审核记录。
</p>
<p className="text-sm text-slate-500">名称增删只影响本项目;标签随字典词条共享,已保存的分类保持原版本。</p>
</section>
);
}

View File

@ -4,6 +4,8 @@ import {
sliceEditWorkspaceChanged,
} from "../lib/slice-edit-session.js";
import { checkSliceEditResolution } from "../lib/slice-edit-drag-quick-path.js";
import type { DictionaryItem } from "../lib/dictionary-selection";
import type { DictionarySelection } from "../types";
import { formatAnnotationTime } from "../lib/annotation-time.js";
import { userErrorMessage } from "../lib/user-error-message.js";
import { reviewCommandRequiresWrite, reviewShortcut, reviewShortcutHelp } from "../lib/review-shortcuts.js";
@ -156,6 +158,9 @@ type Props = ReviewViewingProps & {
slices: SliceSegment[];
segments: AnnotationSegment[];
labels?: SemanticLabelDefinition[];
actionDictionaryLoading?: boolean;
actionDictionaryError?: string;
onRefreshActionDictionary?: () => void;
discardReasons?: AnnotationLabelRecord[];
discardReasonsLoading?: boolean;
discardReasonsAllowManual?: boolean;
@ -163,7 +168,7 @@ type Props = ReviewViewingProps & {
operationNotice?: string;
discardReasonsError?: string;
onRefreshDiscardReasons?: () => void;
sliceDictionary?: { id: string; name: string }[];
sliceDictionary?: DictionaryItem[];
sliceDictionaryLoading?: boolean;
sliceDictionaryError?: string;
onRefreshSliceDictionary?: () => void;
@ -220,7 +225,7 @@ type Props = ReviewViewingProps & {
onDraftsChange?: (drafts: AnnotationDraft[]) => void;
/** 换片通知(A01 草稿落盘触发):选中切片变化时上报;初始挂载不触发。 */
onSelectedSliceChange?: (sliceId: string) => void;
onSliceNameChange?: (sliceId: string, name: string) => void;
onSliceNameChange?: (sliceId: string, name: string, dictionary?: DictionarySelection) => void;
onSliceDispositionChange?: (
sliceId: string,
disposition: "keep" | "discard",
@ -348,6 +353,9 @@ export function CaptureAnnotationWorkbench({
slices,
segments: providedSegments,
labels = [],
actionDictionaryLoading = false,
actionDictionaryError = "",
onRefreshActionDictionary,
discardReasons = [],
discardReasonsLoading = false,
discardReasonsAllowManual = false,
@ -2028,8 +2036,27 @@ export function CaptureAnnotationWorkbench({
setEditingAnnotation((current) => (current ? { ...current, ...patch } : current));
};
const actionDictionaryStatus = actionDictionaryLoading ? (
<p role="status">正在加载动作字典…</p>
) : actionDictionaryError ? (
<div role="alert">
{userErrorMessage(actionDictionaryError)}
<button type="button" onClick={onRefreshActionDictionary}>
重试加载
</button>
</div>
) : !labels.length ? (
<p>暂无字典词条,可直接填写描述。</p>
) : undefined;
const chooseTemplate = (definition: SemanticLabelDefinition) => {
updateEditingAnnotation({ labelId: "", label: "", description: definition.description?.trim() || definition.name });
updateEditingAnnotation({
labelId: "",
label: "",
dictionary: definition.dictionary,
clearDictionary: false,
description: definition.description?.trim() || definition.name,
});
setShowDictionary(false);
focusDescriptionEditor();
};
@ -2255,10 +2282,12 @@ export function CaptureAnnotationWorkbench({
if (isInvalid) {
candidate.isInvalid = true;
candidate.labelId = "";
delete candidate.dictionary;
} else {
delete candidate.isInvalid;
if (!description && continuousActions) {
candidate.labelId = "";
delete candidate.dictionary;
delete candidate.phrases;
delete candidate.handedness;
}
@ -5005,7 +5034,7 @@ export function CaptureAnnotationWorkbench({
loading={sliceDictionaryLoading}
error={userErrorMessage(sliceDictionaryError)}
onRefresh={onRefreshSliceDictionary}
onSelect={(name) => onSliceNameChange?.(selectedSlice.id, name)}
onSelect={(name, dictionary) => onSliceNameChange?.(selectedSlice.id, name, dictionary)}
onCloseFocus={focusWorkbench}
/>
)}
@ -5752,7 +5781,7 @@ export function CaptureAnnotationWorkbench({
text: item.description || item.name,
searchText: `${item.name} ${item.description || ""}`,
}))}
status={!labels.length ? <p>暂无字典词条,可直接填写描述。</p> : undefined}
status={actionDictionaryStatus}
onSelect={(id) => {
const item = labels.find((item) => item.id === id);
if (item) chooseTemplate(item);
@ -6029,7 +6058,10 @@ export function CaptureAnnotationWorkbench({
disabled={isInvalid || saving}
title="从字典选择(F / D,填写文本时不触发)"
aria-keyshortcuts="F D"
onClick={() => setShowDictionary((value) => !value)}
onClick={() => {
if (!showDictionary) onRefreshActionDictionary?.();
setShowDictionary((value) => !value);
}}
className="inline-flex items-center gap-1.5 rounded border border-slate-300 px-2 py-1 text-xs text-slate-700 dark:border-slate-600 dark:text-gray-300 disabled:opacity-40 disabled:cursor-not-allowed"
>
<BookOpen className="h-3.5 w-3.5" />
@ -6054,11 +6086,7 @@ export function CaptureAnnotationWorkbench({
<p className="px-2 py-1 text-[10px] text-slate-500">
↑ / ↓ 选择 · Enter 填入 · Esc 关闭
</p>
{!labels.length && (
<p className="p-2 text-xs text-slate-500">
暂无字典词条,可直接填写描述。
</p>
)}
{actionDictionaryStatus}
{labels.map((item) => (
<button
data-dictionary-option
@ -6076,6 +6104,24 @@ export function CaptureAnnotationWorkbench({
</div>
</div>
{!isInvalid && editingAnnotation.dictionary && (
<div className="flex flex-wrap items-center gap-2 text-xs text-slate-600">
<span>分类:{editingAnnotation.dictionary.name}</span>
{editingAnnotation.dictionary.tags.map((tag) => (
<span key={tag} className="rounded bg-blue-50 px-2 py-0.5 text-blue-700">
{tag}
</span>
))}
<button
type="button"
onClick={() =>
updateEditingAnnotation({ dictionary: undefined, clearDictionary: true })
}
>
取消分类
</button>
</div>
)}
{isInvalid ? (
<div className="flex min-h-24 flex-col items-center justify-center rounded border border-dashed border-amber-300 bg-amber-50/60 p-4 text-center dark:border-amber-800/60 dark:bg-amber-950/30">
<div className="flex items-center gap-1.5 text-sm font-semibold text-amber-800 dark:text-amber-300">
@ -6091,7 +6137,11 @@ export function CaptureAnnotationWorkbench({
ref={descriptionInputRef}
aria-label="动作描述"
value={editingAnnotation.description || ""}
onChange={(event) => updateEditingAnnotation({ description: event.target.value })}
onChange={(event) =>
updateEditingAnnotation({
description: event.target.value,
})
}
placeholder="直接填写详细动作描述..."
rows={3}
className="min-h-24 w-full resize-y rounded border border-slate-300 p-2.5 text-sm text-slate-800 outline-none focus:border-orange-400 dark:border-[#2a2b2f] dark:bg-[#1a1b1e] dark:text-gray-200"
@ -6627,9 +6677,9 @@ export function CaptureAnnotationWorkbench({
loading: sliceDictionaryLoading,
error: userErrorMessage(sliceDictionaryError),
refresh: onRefreshSliceDictionary,
onSelect: (name) =>
onSelect: (name, dictionary) =>
applySliceEditPanelOp(
setPendingSliceName(sliceEditPanel.pendingSlices, sliceEditPanel.sliceId, name),
setPendingSliceName(sliceEditPanel.pendingSlices, sliceEditPanel.sliceId, name, dictionary),
),
}}
discard={{

View File

@ -1,3 +1,5 @@
import type { DictionaryItem } from "../lib/dictionary-selection";
import type { DictionarySelection } from "../types";
import { useEffect, useLayoutEffect, useRef, useState } from "react";
import type { AnnotationLabelRecord, SliceSegment } from "../types";
import { formatAnnotationTime } from "../lib/annotation-time.js";
@ -57,11 +59,11 @@ export function SliceEditPanel({
onCancel: () => void;
error?: string;
dictionary: {
items: { id: string; name: string }[];
items: DictionaryItem[];
loading?: boolean;
error?: string;
refresh?: () => void;
onSelect: (name: string) => void;
onSelect: (name: string, dictionary?: DictionarySelection) => void;
};
discard: {
draft: SliceEditPanelDiscardDraft | null;

View File

@ -1,3 +1,5 @@
import { dictionarySelection, type DictionaryItem } from "../lib/dictionary-selection";
import type { DictionarySelection } from "../types";
import { userErrorMessage } from "../lib/user-error-message.js";
import { useEffect, useLayoutEffect, useRef, useState } from "react";
import { createPortal } from "react-dom";
@ -20,13 +22,13 @@ export function SliceNameDictionaryPicker({
inline = false,
}: {
key?: string;
items: { id: string; name: string }[];
items: DictionaryItem[];
value: string;
disabled?: boolean;
loading?: boolean;
error?: string;
onRefresh?: () => void;
onSelect: (name: string) => void;
onSelect: (name: string, dictionary?: DictionarySelection) => void;
triggerRef?: RefObject<HTMLButtonElement | null>;
onCloseFocus?: () => void;
portal?: boolean;
@ -128,7 +130,7 @@ export function SliceNameDictionaryPicker({
}
onSelect={(id) => {
const item = items.find((item) => item.id === id);
if (item) onSelect(item.name);
if (item) onSelect(item.name, dictionarySelection(item));
close();
}}
onClose={close}
@ -178,7 +180,7 @@ export function SliceNameDictionaryPicker({
type="button"
key={item.id}
onClick={() => {
onSelect(item.name);
onSelect(item.name, dictionarySelection(item));
close();
}}
className={`flex w-full items-center justify-between gap-2 rounded px-2 py-2 text-left text-sm hover:bg-orange-50 dark:hover:bg-orange-950/40 ${value === item.name ? "bg-orange-50 text-orange-700 dark:bg-orange-950/40" : ""}`}

View File

@ -2,7 +2,7 @@ import { useEffect, useState } from "react";
import { apiRequest } from "../services/api/client";
import type { AnnotationLabelRecord } from "../types";
export function useProjectSliceDictionary(projectId?: string, taskId?: string) {
export function useProjectSliceDictionary(projectId?: string, taskId?: string, kind: "slice" | "action" = "slice") {
const [refreshKey, setRefreshKey] = useState(0);
const [state, setState] = useState<{
owner: string;
@ -10,23 +10,26 @@ export function useProjectSliceDictionary(projectId?: string, taskId?: string) {
loading: boolean;
error: string;
}>({ owner: "", items: [], loading: false, error: "" });
const owner = `${projectId || ""}:${taskId || ""}`;
const owner = `${projectId || ""}:${taskId || ""}:${kind}`;
useEffect(() => {
const controller = new AbortController();
if (!projectId || !taskId) return () => controller.abort();
setState({ owner, items: [], loading: true, error: "" });
void apiRequest<{ items: AnnotationLabelRecord[] }>(`/api/projects/${projectId}/slice-labels`, {
signal: controller.signal,
})
void apiRequest<{ items: AnnotationLabelRecord[] }>(
`/api/projects/${projectId}/${kind === "slice" ? "slice-labels" : "annotation-labels"}`,
{
signal: controller.signal,
},
)
.then((result) => {
if (!controller.signal.aborted) setState({ owner, items: result.items || [], loading: false, error: "" });
})
.catch(() => {
if (!controller.signal.aborted)
setState({ owner, items: [], loading: false, error: "切片字典加载失败,仍可手动输入名称。" });
setState({ owner, items: [], loading: false, error: "字典加载失败,请刷新重试。" });
});
return () => controller.abort();
}, [projectId, taskId, owner, refreshKey]);
}, [projectId, taskId, owner, refreshKey, kind]);
return {
...(state.owner === owner ? state : { items: [], loading: Boolean(projectId && taskId), error: "" }),
refresh: () => setRefreshKey((n) => n + 1),

View File

@ -27,6 +27,7 @@ export function normalizeAnnotationSegments(segments, clipStart, clipEnd) {
return (Array.isArray(segments) ? segments : [])
.map((segment, index) => ({
id: String(segment?.id || `segment-${index + 1}`),
dictionary: segment?.dictionary,
labelId: String(segment?.labelId || "").trim(),
label: String(segment?.label || "").trim(),
description: String(segment?.description || "").trim(),

View File

@ -0,0 +1,15 @@
import type { AnnotationLabelRecord, DictionarySelection } from "../types";
export type DictionaryItem = Pick<AnnotationLabelRecord, "id" | "name"> & Partial<AnnotationLabelRecord>;
export function dictionarySelection(item: DictionaryItem): DictionarySelection | undefined {
if (!item.code || !item.version) return undefined;
return {
id: item.id,
kind: item.kind || "action",
version: item.version,
code: item.code,
name: item.name,
tags: [...(item.tags || [])],
};
}

View File

@ -1,3 +1,5 @@
const dictionaryKey = (value) => value ? JSON.stringify([value.id,value.kind,value.version,value.code,value.name,value.tags || []]) : "";
// Candidate discovery only. The server revalidates explicit selections and
// creates a new revision; viewing or marking an action invalid never merges.
export function invalidSliceConsolidationCandidates(slices, actions) {
@ -12,6 +14,8 @@ export function invalidSliceConsolidationCandidates(slices, actions) {
!current.name?.trim() ||
left.name !== current.name ||
right.name !== current.name ||
dictionaryKey(left.dictionary) !== dictionaryKey(current.dictionary) ||
dictionaryKey(right.dictionary) !== dictionaryKey(current.dictionary) ||
left.end !== current.start ||
current.end !== right.start
)

View File

@ -137,10 +137,10 @@ export function mergePendingSliceBoundary(slices, sliceId, side) {
}
/** ④ 切片字典描述(名称)编辑:只改目标片名称。 */
export function setPendingSliceName(slices, sliceId, name) {
export function setPendingSliceName(slices, sliceId, name, dictionary = undefined) {
const index = indexOfSliceId(slices, sliceId);
if (index < 0) return { slices, issue: "未找到目标切片。" };
return { slices: slices.map((slice, at) => (at === index ? { ...slice, name } : slice)) };
return { slices: slices.map((slice, at) => (at === index ? { ...slice, name, dictionary } : slice)) };
}
/**

View File

@ -1,6 +1,8 @@
import { createReviewDecisionCompletion } from "../lib/review-decision-completion.js";
import { createSliceEditSession, freezeSliceEditSession } from "../lib/slice-edit-session.js";
import { createAnnotationPersistence } from "../lib/annotation-persistence.js";
import { dictionarySelection } from "../lib/dictionary-selection";
import type { DictionarySelection } from "../types";
import { AnnotationWorkModeToggle } from "../components/AnnotationWorkModeToggle";
import { loadOwnedAnnotationProject, loadAnnotationPoolPage } from "../lib/annotation-project-loading.js";
import { FailedSliceRecoveryNotice } from "../components/FailedSliceRecoveryNotice";
@ -767,6 +769,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?:
const [detail, setDetailState] = useState<AnnotationTaskDetail | null>(null);
const discardReasons = useProjectDiscardReasons(detail?.task.projectId, detail?.task.id);
const sliceDictionary = useProjectSliceDictionary(detail?.task.projectId, detail?.task.id);
const actionDictionary = useProjectSliceDictionary(detail?.task.projectId, detail?.task.id, "action");
const detailRef = useRef<AnnotationTaskDetail | null>(null);
const annotationMutationScopeRef = useRef(createAuthoritativeMutationScope());
useEffect(() => {
@ -1857,7 +1860,17 @@ export default function AnnotationOperations({ workPage = false }: { workPage?:
null,
[detail?.task.templateId, templates],
);
const selectedLabels = detail?.task.labelSnapshot || [];
const selectedLabels =
(detail?.task.templateVersion || 0) >= 6
? actionDictionary.items.map((item) => ({
id: item.code,
name: item.name,
description: item.description,
aliases: item.aliases,
color: item.color,
dictionary: dictionarySelection(item),
}))
: detail?.task.labelSnapshot || [];
const modelPrelabels = useMemo(
() =>
detail?.revisions
@ -3750,12 +3763,17 @@ export default function AnnotationOperations({ workPage = false }: { workPage?:
setSliceEditQuickConflict(prompt);
};
const updateSliceName = (id: string, name: string) => {
const updateSliceName = (id: string, name: string, dictionary?: DictionarySelection) => {
if (!annotationAuthoringRef.current) return;
if (!exactFrameEditing || saving || combinedSubmissionLocked || annotationMutationScopeRef.current.busy()) return;
const slice = sliceSegments.find((item) => item.id === id);
if (!slice || slice.deliveryDisposition === "discard" || slice.name === name) return;
const next = sliceSegments.map((item) => (item.id === id ? { ...item, name } : item));
if (
!slice ||
slice.deliveryDisposition === "discard" ||
(slice.name === name && JSON.stringify(slice.dictionary) === JSON.stringify(dictionary))
)
return;
const next = sliceSegments.map((item) => (item.id === id ? { ...item, name, dictionary } : item));
const before = sliceEditSnapshot(id);
editHistoryRef.current.record({ before, after: { ...before, slices: next } });
setSliceSegments(next);
@ -3802,6 +3820,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?:
...(recordHistory && disposition === "keep" ? { resetToPending: true } : {}),
restorePreviousName,
...(restorePreviousName && restoreNameFromVersion ? { restoreNameFromVersion } : {}),
restoreClassifications: !recordHistory,
...reason,
sliceSegments,
segments,
@ -3874,6 +3893,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?:
const payload = {
version: token.version,
applicationId,
restoreClassifications: !recordHistory,
sliceSegments: nextSlices,
segments: nextSegments,
operations: restoredOperations,
@ -7624,6 +7644,9 @@ export default function AnnotationOperations({ workPage = false }: { workPage?:
slices={sliceSegments}
segments={segments}
labels={selectedLabels}
actionDictionaryLoading={actionDictionary.loading}
actionDictionaryError={actionDictionary.error}
onRefreshActionDictionary={actionDictionary.refresh}
sliceDictionary={sliceDictionary.items}
sliceDictionaryLoading={sliceDictionary.loading}
sliceDictionaryError={userErrorMessage(sliceDictionary.error)}

View File

@ -1904,7 +1904,17 @@ export interface ObjectStorageUploadSessionSummary {
pendingCount: number;
}
export interface DictionarySelection {
id: string;
kind: "action" | "slice" | "discard_reason";
version: number;
code: string;
name: string;
tags: string[];
}
export interface SemanticLabelDefinition {
dictionary?: DictionarySelection;
id: string;
name: string;
description?: string;
@ -1913,6 +1923,7 @@ export interface SemanticLabelDefinition {
}
export interface AnnotationLabelRecord {
tags?: string[];
kind?: "action" | "slice" | "discard_reason";
requiresNote?: boolean;
id: string;
@ -1928,6 +1939,7 @@ export interface AnnotationLabelRecord {
version: number;
updatedAt: string;
history?: Array<{
tags?: string[];
version: number;
code: string;
name: string;
@ -1985,6 +1997,8 @@ export interface AnnotationTemplate {
}
export interface AnnotationSegment {
clearDictionary?: boolean;
dictionary?: DictionarySelection;
status?: "pending" | "described" | "invalid";
id: string;
labelId: string;
@ -2000,6 +2014,8 @@ export interface AnnotationSegment {
}
export interface SliceSegment {
dictionary?: DictionarySelection;
discardReasonDictionary?: DictionarySelection;
deliveryDisposition?: "keep" | "discard";
discardReason?: string;
discardReasonLabelId?: string;

View File

@ -0,0 +1,68 @@
"""Read-only independent fixture consumer: stdlib, JSON and original files only.
Produces a processing plan; never cuts, copies, uploads or modifies source data.
"""
import argparse
import bisect
import csv
import hashlib
import json
from pathlib import Path
def inspect(part, original):
original=original.resolve()
def file(relative):
p=(original/relative).resolve()
if not p.is_relative_to(original) or not p.is_file():raise ValueError("original file unavailable or outside source root")
return p
assert part["consumerContract"]=="droboticflow.classification-consumer/v1"
assert part["timeBases"]["timestampUs"]=="original_hardware_pts"
assert part["timeBases"]["interval"]=="half-open"
source=part["originalSource"]
assert source["provider"]=="BOS" and source["bucket"]
assert source["prefix"]==part["capture"]["prefix"]
assert source["partPrefix"]==source["prefix"]+"/"+part["partId"]
assert part["pathBases"]["sources"]=="original_capture"
assert part["sources"]=={k:source["files"][k] for k in ("leftVideo","rightVideo","leftPts","rightPts","imu")}
clocks={}
for side in ("left","right"):
file(part["sources"][side+"Video"]["path"])
with file(part["sources"][side+"Pts"]["path"]).open() as f:clocks[side]=[int(row["timestamp_us"]) for row in csv.DictReader(f)]
assert clocks[side] and all(a<b for a,b in zip(clocks[side],clocks[side][1:]))
for c in part["calibration"]:assert hashlib.sha256(file(c["path"]).read_bytes()).hexdigest()==c["sha256"]
file(part["sourceMetadata"]["path"])
with file(part["sources"]["imu"]["path"]).open() as f:imu=[int(r["timestamp_us"]) for r in csv.DictReader(f)]
covered={side:[] for side in clocks};actions=[];discard=[]
def range_for(a,b):return {side:[bisect.bisect_left(pts,a),bisect.bisect_left(pts,b)] for side,pts in clocks.items()}
for item in sorted(part["slices"]+part["discardedSlices"],key=lambda s:s["taskSplit"]["startTimestampUs"]):
parent=item["taskSplit"];a,b=parent["startTimestampUs"],parent["endTimestampUs"]
if item["classification"]["deliveryDisposition"]=="discard":
assert item["segments"]==[]
reason=item["classification"]["discardEvidence"];assert reason["reason"]
frames=range_for(a,b);discard.append({"sliceId":item["sliceId"],"reason":reason,"frames":frames})
for side,(start,end) in frames.items():covered[side]+=list(range(start,end))
continue
previous=a
for action in item["segments"]:
start,end=action["startTimestampUs"],action["endTimestampUs"]
assert start==previous and start<end<=b
previous=end
assert abs(action["partStartSecond"]-parent["startSecond"]-action["startSecond"])<1e-6
assert abs(action["partEndSecond"]-parent["startSecond"]-action["endSecond"])<1e-6
dictionary=action.get("dictionary") if action["tagSource"]=="action" else item["classification"]["dictionary"]
assert dictionary["tags"]==action["tags"] and action["tags"]
frames=range_for(start,end)
for side,(first,last) in frames.items():
assert last>first
covered[side]+=list(range(first,last))
actions.append({"slice":item["sliceId"],"tags":action["tags"],"tagSource":action["tagSource"],"dictionaryId":dictionary["id"],"dictionaryVersion":dictionary["version"],"invalid":action["isInvalid"],"range":[action["partStartSecond"],action["partEndSecond"]],"timestampsUs":[start,end],"frames":frames,"imuRows":bisect.bisect_left(imu,end)-bisect.bisect_left(imu,start)})
assert previous==b
low=part["coverage"]["startTimestampUs"];high=part["coverage"]["endTimestampUs"]
for side,pts in clocks.items():assert covered[side]==[i for i,t in enumerate(pts) if low<=t<high]
return {"passed":True,"actions":actions,"discarded":discard,"coveredFrames":{side:len(frames) for side,frames in covered.items()},"sourceModified":False}
if __name__=="__main__":
parser=argparse.ArgumentParser(description=__doc__);parser.add_argument("part",type=Path);parser.add_argument("original",type=Path)
args=parser.parse_args();print(json.dumps(inspect(json.loads(args.part.read_text()),args.original),ensure_ascii=False))

View File

@ -0,0 +1,39 @@
"""Synthetic v4 examples from the actual package entry point, with private audit kept separately."""
import argparse
import hashlib
import json
import shutil
from pathlib import Path
from backend.operators.builtin.package import operator as package
from tests.backend.test_delivery_v4_package import NODE, ACTION, prepare, original_source
from tests.backend.test_part_index_package import helpers
def export(destination):
destination.mkdir(parents=True, exist_ok=True)
for part in ("part0001", "part0002"):
work=destination/("work-"+part)
work.mkdir()
inputs=prepare(work,[{"id":"blank","description":"","clipStartMs":0,"clipEndMs":500,"startTimestampUs":1000000,"endTimestampUs":1500000},{"id":"pick","description":"拿起杯子","dictionary":ACTION,"clipStartMs":500,"clipEndMs":1000,"startTimestampUs":1500000,"endTimestampUs":2000000}])
inputs[1]["data"]["capture"].update(captureId="capture-A/"+part,prefix="incoming/capture-A/"+part)
manifest_path=Path(inputs[0]["path"])/"revision-manifest.json"
m=json.loads(manifest_path.read_text());m["runId"]="source-run-"+part;m["originalSource"]=original_source(part=part)
if part=="part0002":
m["items"][0]["deliveryDisposition"]="discard"
m["items"][0]["discardEvidence"]=m["items"][1]["discardEvidence"]
manifest_path.write_text(json.dumps(m))
inputs[1]["data"]["acceptedMaterializations"][0]["manifestSha256"]=hashlib.sha256(manifest_path.read_bytes()).hexdigest()
artifacts=package.run("package-"+part,"package",work/"out",NODE,inputs,helpers())
shutil.copytree(artifacts[0]["path"],destination/part)
audit=destination/"internal"/part
audit.mkdir(parents=True)
shutil.copy2(artifacts[1]["path"],audit/"review-history.json")
shutil.rmtree(work)
(destination/"meta_capture-A.json").write_text('{"synthetic":true,"parts":["part0001","part0002"]}\n')
if __name__=="__main__":
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument("output",type=Path)
export(parser.parse_args().output)

View File

@ -0,0 +1,124 @@
"""Consumers must classify and locate every interval without platform workspaces."""
import csv
import hashlib
import json
import os
import shutil
import subprocess
import sys
from pathlib import Path
import pytest
from backend.operators.builtin.package import operator as package
from tests.backend.test_delivery_v4_package import ACTION, BASE, NODE, prepare, read_outputs
from tests.backend.test_part_index_package import helpers
from tests.backend.test_slice_discard_package import accepted_inputs
def test_tags_inherit_by_dictionary_selection_not_description_text(tmp_path):
inputs=prepare(tmp_path,[{"description":""},{"description":"人工补充文字"},{"description":"","dictionary":ACTION},{"description":"无效","isInvalid":True}])
doc,_,_=read_outputs(tmp_path,inputs)
actions=doc["slices"][0]["segments"]
assert [a["tags"] for a in actions]==[BASE["tags"],BASE["tags"],ACTION["tags"],BASE["tags"]]
assert [a["tagSource"] for a in actions]==["slice","slice","action","slice"]
assert actions[-1]["isInvalid"] is True
assert "dictionary" not in actions[0] # Inheritance must not invent an action selection.
assert actions[2]["dictionary"]==ACTION
assert doc["consumerContract"]=="droboticflow.classification-consumer/v1"
assert doc["pathBases"]["sources"]=="original_capture"
assert "workspaceKey" not in json.dumps(doc)
@pytest.mark.parametrize("failure,match",[("parent-tags","slice tags"),("action-tags","action tags"),("source","original BOS"),("timestamps","startTimestampUs"),("overlap","overlapping"),("clock","disagree"),("missing-file","missing original files"),("unsafe-key","unsafe"),("boundary","boundary mapping"),("pts-gap","discontinuous"),("discard-dictionary","discard reason disagrees")])
def test_v4_refuses_incomplete_consumer_instructions(tmp_path,failure,match):
inputs=prepare(tmp_path,[{"description":"one"},{"description":"two","dictionary":dict(ACTION)}])
root=Path(inputs[0]["path"])
manifest_path=root/"revision-manifest.json"
manifest=json.loads(manifest_path.read_text())
sync_path=root/"slices/slice-000001/synchronized-slice.json"
sync=json.loads(sync_path.read_text())
ann_path=root/"slices/slice-000001/annotation.json"
ann=json.loads(ann_path.read_text())
if failure=="parent-tags":sync["dictionary"]["tags"]=[]
elif failure=="action-tags":ann["actions"][1]["dictionary"]["tags"]=[]
elif failure=="source":manifest.pop("originalSource")
elif failure=="missing-file":manifest["originalSource"]["objectKeys"]=[k for k in manifest["originalSource"]["objectKeys"] if "camera_left" not in k]
elif failure=="timestamps":ann["actions"][0].pop("startTimestampUs")
elif failure=="overlap":ann["actions"][1].update(clipStartMs=0,startTimestampUs=1000000)
elif failure=="clock":ann["actions"][0]["endTimestampUs"]-=1
elif failure=="unsafe-key":manifest["originalSource"]["objectKeys"][0]=manifest["originalSource"]["objectKeys"][0].replace("/part0001/", "//part0001/")
elif failure=="boundary":sync["boundaryTimestampUs"][-1]-=1
elif failure=="discard-dictionary":manifest["items"][1]["discardEvidence"]["reasonVersion"]=999
elif failure=="pts-gap":
sync.pop("boundarySeconds");sync.pop("boundaryTimestampUs")
ann["actions"][1]["startTimestampUs"]+=1
sync_path.write_text(json.dumps(sync));ann_path.write_text(json.dumps(ann));manifest_path.write_text(json.dumps(manifest))
inputs[1]["data"]["acceptedMaterializations"][0]["manifestSha256"]=hashlib.sha256(manifest_path.read_bytes()).hexdigest()
with pytest.raises(RuntimeError,match=match):read_outputs(tmp_path,inputs)
def test_legacy_partial_annotations_keep_inherited_gap_coverage(tmp_path):
inputs=prepare(tmp_path,[{"description":"拿起","dictionary":ACTION,"clipStartMs":250,"clipEndMs":750}])
doc,_,_=read_outputs(tmp_path,inputs)
actions=doc["slices"][0]["segments"]
assert [(a["startSecond"],a["endSecond"],a["tagSource"]) for a in actions]==[(0,.25,"slice"),(.25,.75,"action"),(.75,1,"slice")]
assert [(a["startTimestampUs"],a["endTimestampUs"]) for a in actions]==[(1000000,1250000),(1250000,1750000),(1750000,2000000)]
def test_discard_aliases_use_frozen_dictionary_without_zero_version(tmp_path):
inputs=prepare(tmp_path,[])
manifest_path=Path(inputs[0]["path"])/"revision-manifest.json"
manifest=json.loads(manifest_path.read_text())
reason=manifest["items"][1]["discardEvidence"]
reason.update(reasonLabelId="",reasonName="",reasonVersion=0)
manifest_path.write_text(json.dumps(manifest))
inputs[1]["data"]["acceptedMaterializations"][0]["manifestSha256"]=hashlib.sha256(manifest_path.read_bytes()).hexdigest()
doc,_,_=read_outputs(tmp_path,inputs)
reason=doc["discardedSlices"][0]["classification"]["discardEvidence"]
assert reason["reasonLabelId"]==reason["dictionary"]["id"]
assert reason["reasonVersion"]==reason["dictionary"]["version"]==4
assert reason["reasonName"]==reason["dictionary"]["name"]
@pytest.mark.parametrize("mode",["physical","source-index"])
def test_real_v4_roundtrip_uses_only_customer_json_and_original_data(tmp_path,monkeypatch,mode):
root=os.environ.get("FANYU_DELIVERY_CONSUMER_FIXTURE_DIR")
if not root:pytest.skip("generate retained local Go source fixture first")
context=json.loads((Path(root)/mode/"context.json").read_text())
artifact=context["artifact"]
inputs=accepted_inputs(artifact)[:2]
inputs[1]["data"].update(trace=[],qualityReviews=[])
inputs[1]["data"]["finalAcceptance"]["acceptedAt"]="2026-10-21T00:00:00Z"
original=Path(context["original"])
for kind in ("camera","imu"):
p=original/f"capture-A_calibration_{kind}.yaml"
inputs.append({"portId":"calibrations","name":p.name,"path":str(p)})
node={**NODE,"hyperparams":NODE["hyperparams"]+[{"key":"SOURCE_INDEX_DELIVERY","value":"true"}]}
with monkeypatch.context() as guard:
def no_subprocess(*args, **kwargs):
raise AssertionError("classification export must not transcode or recut source data")
original_sha = package._sha256_file
def metadata_only(path):
assert Path(path).suffix not in {".mp4", ".mov", ".mkv"}, "classification export rehashed video bodies"
return original_sha(path)
guard.setattr(package.subprocess, "run", no_subprocess)
guard.setattr(package.subprocess, "Popen", no_subprocess)
guard.setattr(package, "_sha256_file", metadata_only)
result=package.run("consumer","package",tmp_path/"out",node,inputs,helpers())
doc=json.loads((Path(result[0]["path"])/"part.json").read_text())
assert not any(word in json.dumps(doc) for word in ("workspaceKey","system/runs/","PRIVATE_CONNECTION"))
# A fresh process can consume copies of only the customer file and original source.
consumer=tmp_path/"downstream";shutil.copytree(original,consumer/"original")
(consumer/"part.json").write_text(json.dumps(doc))
before={p.relative_to(consumer/"original"):hashlib.sha256(p.read_bytes()).hexdigest() for p in (consumer/"original").rglob("*") if p.is_file()}
result=subprocess.run([sys.executable,str(Path(__file__).with_name("consume_delivery_v4.py").resolve()),str(consumer/"part.json"),str(consumer/"original")],cwd=consumer,capture_output=True,text=True,check=True)
assert before=={p.relative_to(consumer/"original"):hashlib.sha256(p.read_bytes()).hexdigest() for p in (consumer/"original").rglob("*") if p.is_file()}
report=json.loads(result.stdout)
assert report["passed"] and report["sourceModified"] is False
assert report["coveredFrames"]=={"left":32,"right":32}
groups=report["actions"]
assert [g["tags"] for g in groups]==[["商品","手部"],["抓取"],["商品","手部"],["商品","手部"]]
assert [g["invalid"] for g in groups]==[False,False,True,False]
assert [g["range"] for g in groups]==[[0,.5],[.5,1],[3,3.5],[3.5,4]]
assert doc["slices"][1]["segments"][0]["startSecond"]==0
assert doc["slices"][1]["segments"][0]["partStartSecond"]==3

View File

@ -0,0 +1,120 @@
"""Final classifications and private review history are different outputs."""
import hashlib
import json
from pathlib import Path
import pytest
from backend.operators.builtin.package import operator as package
from tests.backend.test_delivery_v3_package import inputs_for
from tests.backend.test_part_index_package import helpers
from tests.backend.test_slice_discard_package import accepted_inputs, fixture
NODE = {
"outputs": [{"id": n, "name": n} for n in ("deliveryDirectory", "deliverySummary", "deliveryAudit")],
"hyperparams": [{"key": k, "value": v} for k, v in {
"PART_INDEX_DELIVERY": "true", "DELIVERY_JSON_V4": "true", "PROCESSING_PROFILE": "raw_human"
}.items()],
}
BASE = {"id": "slice-word", "kind": "slice", "version": 2, "code": "base", "name": "物品操作", "tags": ["商品", "手部"]}
ACTION = {"id": "action-word", "kind": "action", "version": 3, "code": "pick", "name": "拿取", "tags": ["抓取"]}
def original_source(capture="capture-A", part="part0001"):
root=f"incoming/{capture}"
paths=[f"{part}/{capture}_camera_{side}_{part}{suffix}" for side in ("left","right") for suffix in (".mp4","_pts.csv")]
paths += [f"{part}/{capture}_imu_{part}.csv",f"{capture}_calibration_camera.yaml",f"{capture}_calibration_imu.yaml",f"meta_{capture}.json"]
return {"provider":"BOS","bucket":"test-bucket","prefix":root,"partPrefix":root+"/"+part,"captureUnit":capture+"/"+part,"objectKeys":[root+"/"+p for p in paths],"snapshotFingerprint":"synthetic-source-version"}
def prepare(tmp_path, actions):
inputs = inputs_for(tmp_path, actions)
root = Path(inputs[0]["path"])
sync = root / "slices/slice-000001/synchronized-slice.json"
doc = json.loads(sync.read_text())
doc["dictionary"] = BASE
n=max(1,len(actions))
for i,action in enumerate(actions):
action.setdefault("id",f"action-{i}")
action.setdefault("clipStartMs",i*1000/n)
action.setdefault("clipEndMs",(i+1)*1000/n)
action.setdefault("startTimestampUs",1000000+round(action["clipStartMs"]*1000))
action.setdefault("endTimestampUs",1000000+round(action["clipEndMs"]*1000))
(root/"slices/slice-000001/annotation.json").write_text(json.dumps({"actions":actions}))
points=sorted({0.0,1.0,*[a[k]/1000 for a in actions for k in ("clipStartMs","clipEndMs")]})
doc.update(boundarySeconds=points,boundaryTimestampUs=[1000000+round(p*1000000) for p in points],rightBoundarySeconds=points)
sync.write_text(json.dumps(doc))
manifest_path = root / "revision-manifest.json"
m = json.loads(manifest_path.read_text())
m["runId"] = "source-run"
m["originalSource"] = original_source()
m["items"][0]["sliceName"] = BASE["name"]
reason = m["items"][1]["discardEvidence"]
reason.update(reasonLabelId="discard-word", reasonVersion=4, reasonName="遮挡", reasonNote="持续遮挡", at="2026-10-20T00:00:00Z", dictionary={"id":"discard-word","kind":"discard_reason","version":4,"code":"occluded","name":"遮挡","tags":["不可用"]})
manifest_path.write_text(json.dumps(m))
inputs[1]["data"]["acceptedMaterializations"][0]["manifestSha256"] = hashlib.sha256(manifest_path.read_bytes()).hexdigest()
inputs[1]["data"]["qualityReviews"] = [{"reviewRound":0,"decision":"rejected","comment":"PRIVATE_REWORK"},{"reviewRound":1,"decision":"approved","confirmations":[{"opinionId":"old","resolved":True}]}]
inputs[1]["data"]["trace"] = [{"humanItemId":"item","reworkBaselines":[{"round":1,"comment":"PRIVATE_BASELINE"}],"reworkHandling":[{"round":1}],"events":[{"type":"rework_submitted"}]}]
return inputs
def read_outputs(tmp_path, inputs, out="out"):
result = package.run("r", "p", tmp_path / out, NODE, inputs, helpers())
assert len(result) == 2
directory = Path(result[0]["path"])
assert {p.name for p in directory.iterdir()} == {"part.json"}
raw = (directory / "part.json").read_bytes()
audit = json.loads(Path(result[1]["path"]).read_text())
assert result[1]["portId"] == "deliveryAudit"
assert Path(result[1]["path"]).parent != directory
assert audit["partSha256"] == hashlib.sha256(raw).hexdigest()
assert audit["revision"] == json.loads(raw)["revision"]
return json.loads(raw), audit, raw
@pytest.mark.parametrize("actions,kinds,status", [
([{"description":"", "status":"pending"}], ["gap_placeholder"], "empty_confirmed"),
([{"description":"拿起杯子", "dictionary":ACTION},{"description":""},{"description":"无效", "isInvalid":True}], ["annotated","gap_placeholder","invalid_marked"], "described"),
([{"description":"无效", "isInvalid":True}], ["invalid_marked"], "invalid_only"),
])
def test_v4_freezes_classification_without_exporting_review_process(tmp_path, actions, kinds, status):
inputs = prepare(tmp_path, actions)
doc, audit, raw = read_outputs(tmp_path, inputs)
first = doc["slices"][0]
assert doc["schemaVersion"] == "droboticflow.delivery.part-index/v4"
assert first["classification"]["dictionary"] == BASE
assert first["classification"]["descriptionStatus"] == status
assert [a["contentKind"] for a in first["segments"]] == kinds
if kinds[0] == "annotated": assert first["segments"][0]["dictionary"] == ACTION
discard = doc["discardedSlices"][0]["classification"]["discardEvidence"]
assert discard["note"] == "持续遮挡" and discard["dictionary"]["tags"] == ["不可用"]
assert discard["reasonVersion"] == 4
for key in ("acceptance","reviewHistory","lineage"): assert key not in doc
for text in (b"PRIVATE_REWORK",b"PRIVATE_BASELINE",b"reviewerUserId",b"sampling",b"discardedAt",b"confirmedByUserId"):
assert text not in raw
assert audit["qualityReviews"][0]["comment"] == "PRIVATE_REWORK"
assert audit["trace"][0]["reworkBaselines"]
assert audit["lineage"]["sourceRunId"] == "source-run"
assert read_outputs(tmp_path, inputs, "retry")[2] == raw
# Intermediate audit-only changes cannot change final business bytes.
inputs[1]["data"]["qualityReviews"][0]["comment"] = "OTHER_REVIEW_NOTE"
assert read_outputs(tmp_path, inputs, "audit-change")[2] == raw
def test_v4_all_discard_outputs_classification_and_legacy_stays_unchanged(tmp_path):
inputs = fixture(tmp_path,["discard","discard"])
inputs[1]["data"].update(capture={"captureId":"capture-A/part0001","prefix":"incoming/capture-A/part0001"},trace=[])
inputs[1]["data"]["finalAcceptance"]["acceptedAt"] = "2026-10-20T01:00:00Z"
root=Path(inputs[0]["path"]);p=root/"revision-manifest.json";m=json.loads(p.read_text());m["originalSource"]=original_source();p.write_text(json.dumps(m));inputs[1]["data"]["acceptedMaterializations"][0]["manifestSha256"]=hashlib.sha256(p.read_bytes()).hexdigest()
doc, _, _ = read_outputs(tmp_path, inputs)
assert doc["slices"] == [] and len(doc["discardedSlices"]) == 2
assert doc["deliveryClassification"]["discardedSliceCount"] == 2
legacy={**NODE,"hyperparams":[p for p in NODE["hyperparams"] if p["key"]!="DELIVERY_JSON_V4"]}
result=package.run("r","p",tmp_path/"legacy",legacy,inputs,helpers())
assert len(result)==1 and result[0]["data"]["outcome"]=="no_deliverables"
def test_v4_rejects_unaccepted_or_wrong_revision(tmp_path):
inputs = prepare(tmp_path,[])
inputs[1]["data"]["acceptedMaterializations"][0]["revision"] = 999
with pytest.raises(RuntimeError,match="not pinned"):
read_outputs(tmp_path, inputs)

View File

@ -0,0 +1,57 @@
"""Consume real Go materialization revisions, including metadata-only reuse."""
import hashlib
import json
import shutil
from pathlib import Path
import pytest
from backend.operators.builtin.package import operator as package
from tests.backend.test_part_index_real_materialization import load_context
from tests.backend.test_part_index_package import helpers
from tests.backend.test_delivery_v4_package import NODE, original_source
@pytest.mark.parametrize("mode", ["keep", "mixed", "discard", "blank"])
def test_v4_real_materialization_retains_final_dictionary_snapshots(tmp_path, mode):
context=load_context()
artifact=context[mode]["artifact"]
root=tmp_path/"accepted"
shutil.copytree(artifact["path"],root)
artifact={**artifact,"path":str(root)}
mpath=root/"revision-manifest.json"
manifest=json.loads(mpath.read_text())
# Explicit synthetic original reference for this historical, local-only fixture.
manifest["originalSource"]=original_source("real-capture")
mpath.write_text(json.dumps(manifest))
summary={"decision":"approved","processingProfile":"raw_human","capture":{"captureId":"real-capture/part0001","prefix":"incoming/real-capture/part0001"},"trace":[],"qualityReviews":[],"acceptedMaterializations":[{"artifactId":artifact["id"],"revision":manifest["revision"],"manifestSha256":hashlib.sha256(mpath.read_bytes()).hexdigest()}],"finalAcceptance":{"confirmed":True,"acceptedAt":"2026-10-21T00:00:00Z","processingProfile":"raw_human","confirmedByUserId":"internal-acceptor"}}
inputs=[{**artifact,"portId":"acceptedSliceDirectory"},{"portId":"acceptanceSummary","data":summary}]
for kind in ("camera","imu"):
calibration=tmp_path/f"calibration_{kind}.yaml"
calibration.write_text(f"{kind}: synthetic-calibration\n")
inputs.append({"portId":"calibrations","name":calibration.name,"path":str(calibration)})
# Also exercise a durable referenced acceptance summary (large trace transport).
reference=tmp_path/"accepted.json"
reference.write_text(json.dumps(summary))
inputs[1]={"portId":"acceptanceSummary","path":str(reference),"data":{"schemaVersion":"droboticflow.acceptance-summary/v2"}}
artifacts=package.run("real-package","package",tmp_path/"out",NODE,inputs,helpers())
customer=Path(artifacts[0]["path"])
assert {p.name for p in customer.iterdir()}=={"part.json"}
part=json.loads((customer/"part.json").read_text())
assert len(part["slices"])+len(part["discardedSlices"])==3
for item in part["slices"]:
snap=item["classification"]["dictionary"]
assert snap["version"]==2 and snap["tags"]==["基础分类","商品"]
if mode=="blank": assert item["classification"]["descriptionStatus"]=="empty_confirmed"
for action in item["segments"]:
if "dictionary" in action:
assert action["dictionary"]["version"]==4 and action["dictionary"]["tags"]==["抓取"]
assert action["tags"]==["抓取"] and action["tagSource"]=="action"
else:
assert mode=="blank"
assert action["tags"]==snap["tags"] and action["tagSource"]=="slice"
for item in part["discardedSlices"]:
reason=item["classification"]["discardEvidence"]
assert reason["note"]=="画面被遮挡" and reason["dictionary"]["tags"]==["不可用"]
audit=json.loads(Path(artifacts[1]["path"]).read_text())
assert audit["acceptance"]["finalAcceptance"]["confirmedByUserId"]=="internal-acceptor"
assert "internal-acceptor" not in (customer/"part.json").read_text()

View File

@ -0,0 +1,71 @@
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import { chromium } from "playwright";
const base = process.env.DICTIONARY_TEST_URL || "http://127.0.0.1:5198";
const out = process.env.DICTIONARY_TEST_OUTPUT || "local-data/verification/delivery-classification-20261021/browser";
await fs.mkdir(out, { recursive: true });
const browser = await chromium.launch({
executablePath: process.env.BROWSER_EXECUTABLE || "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",
headless: true,
});
const page = await browser.newPage({ viewport: { width: 1365, height: 900 } });
const errors = [];
const failedRequests = [];
page.on("requestfailed", (request) => failedRequests.push({ url: request.url(), failure: request.failure() }));
page.on("pageerror", (error) => errors.push(error.message));
try {
page.setDefaultTimeout(60000);
let label = {
id: "base",
name: "物品操作",
kind: "slice",
code: "base",
version: 2,
tags: ["商品"],
status: "active",
};
let patch;
await page.route("**/api/**", async (route) => {
const request = route.request(),
path = new URL(request.url()).pathname;
if (path === "/api/annotation-labels/base" && request.method() === "PATCH") {
patch = request.postDataJSON();
label = { ...label, ...patch, version: label.version + 1 };
return route.fulfill({ json: label });
}
if (path === "/api/annotation-labels") return route.fulfill({ json: { items: [label] } });
if (path === "/api/projects/project/slice-labels")
return route.fulfill({ json: { labelIds: [label.id], items: [label] } });
return route.fulfill({ status: 404, json: { error: "unexpected route" } });
});
await page.goto(base + "/tests/browser/fixtures/dictionary-classification-tags.html", { waitUntil: "commit" });
await page.getByRole("button", { name: "编辑切片名称", exact: true }).click();
await page.getByRole("textbox", { name: "分类标签" }).fill("商品,手部, 商品");
await page.getByRole("button", { name: "保存修改", exact: true }).click();
await page.getByText("手部", { exact: true }).waitFor();
assert.deepEqual(patch.tags, ["商品", "手部"]);
assert.equal(label.version, 3);
await page.getByRole("button", { name: "从切片字典选择", exact: true }).click();
await page.getByRole("button", { name: "物品操作", exact: true }).click();
const selected = await page.evaluate(() => window.selected);
assert.equal(selected.dictionary.id, "base");
assert.equal(selected.dictionary.version, 2); // Picker represents the version actually shown, not the later edit.
assert.deepEqual(selected.dictionary.tags, ["商品"]);
assert.deepEqual(errors, []);
await page.screenshot({ path: `${out}/dictionary-tags.png`, fullPage: true });
await fs.writeFile(
`${out}/result.json`,
JSON.stringify({ passed: true, edited: label, selected, pageErrors: errors }, null, 2),
);
} catch (error) {
await fs.writeFile(
`${out}/failure.json`,
JSON.stringify({ errors, failedRequests, url: page.url(), html: await page.content() }, null, 2),
);
await page.screenshot({ path: `${out}/failure.png`, fullPage: true });
throw error;
} finally {
await browser.close();
}

View File

@ -0,0 +1,13 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
import tailwindcss from "@tailwindcss/vite";
export default defineConfig({
plugins: [react(), tailwindcss()],
build: {
outDir: "local-data/verification/delivery-classification-20261021/browser-ui-dist",
emptyOutDir: true,
rollupOptions: { input: "tests/browser/fixtures/dictionary-classification-tags.html" },
},
preview: { host: "127.0.0.1", port: 5198, strictPort: true },
});

View File

@ -0,0 +1,11 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<title>字典分类验证</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="./dictionary-classification-tags.tsx"></script>
</body>
</html>

View File

@ -0,0 +1,34 @@
import React from "react";
import { createRoot } from "react-dom/client";
import "../../../src/index.css";
import { AnnotationLabelsPanel } from "../../../src/components/AnnotationLabelsPanel";
import { SliceNameDictionaryPicker } from "../../../src/components/SliceNameDictionaryPicker";
import type { AnnotationLabelRecord, DictionarySelection } from "../../../src/types";
const label: AnnotationLabelRecord = {
id: "base",
name: "物品操作",
kind: "slice",
code: "base",
version: 2,
tags: ["商品"],
status: "active",
organizationId: "fixture",
updatedAt: "2026-10-21T00:00:00Z",
};
createRoot(document.getElementById("root")!).render(
<main className="space-y-8 p-8">
<AnnotationLabelsPanel projectId="project" canManage kind="slice" />
<SliceNameDictionaryPicker
items={[label]}
value=""
inline
onSelect={(name, dictionary) => {
(window as Window & { selected?: { name: string; dictionary?: DictionarySelection } }).selected = {
name,
dictionary,
};
}}
/>
</main>,
);

View File

@ -4,7 +4,10 @@ import fs from "node:fs/promises";
const base = process.env.DICTIONARY_TEST_URL || "http://127.0.0.1:5178",
out = process.env.DICTIONARY_TEST_OUTPUT || "verification/dictionary-reuse";
await fs.mkdir(out, { recursive: true });
const browser = await chromium.launch({ channel: "chrome", headless: true });
const browser = await chromium.launch({
...(process.env.BROWSER_EXECUTABLE ? { executablePath: process.env.BROWSER_EXECUTABLE } : { channel: "chrome" }),
headless: true,
});
try {
for (const kind of ["slice", "action", "discard_reason"]) {
const page = await browser.newPage({ viewport: { width: 1280, height: 900 } });
@ -54,7 +57,7 @@ try {
);
await page.goto(base + "/__dictionary_reuse");
assert.equal(await page.title(), "项目字典复用验证");
const input = page.getByRole("textbox"),
const input = page.getByRole("textbox").first(),
add = page.getByRole("button", {
name: kind === "slice" ? "添加名称" : kind === "discard_reason" ? "添加原因" : "添加描述",
exact: true,

View File

@ -38,3 +38,18 @@ test("never propose discarded, different-name, gapped or partially invalid parti
test("invalid inside a single business slice stays inside it", () => {
assert.deepEqual(candidates([{ id: "a", name: "盘点", start: 0, end: 3 }], actions), []);
});
test("never merge same-name slices with distinct frozen dictionary classifications", () => {
const dictionary = { id: "base", kind: "slice", version: 2, code: "sort", name: "盘点", tags: ["商品"] };
const classified = slices.map((slice) => ({ ...slice, dictionary }));
assert.equal(candidates(classified, actions).length, 1);
for (const replacement of [{ ...dictionary, version: 3 }, { ...dictionary, tags: ["其他"] }, undefined]) {
assert.deepEqual(
candidates(
classified.map((slice) => (slice.id === "c" ? { ...slice, dictionary: replacement } : slice)),
actions,
),
[],
);
}
});

View File

@ -28,10 +28,12 @@ test("theme utilities follow the application class rather than OS preference", a
assert.match(css, /:where\(\.dark, \.dark \*\)/);
assert.doesNotMatch(css, /prefers-color-scheme/);
});
test("project dictionary only shows explicitly selected descriptions and never edits shared labels", () => {
test("project dictionary only shows selected descriptions and relinks names while versioning explicit tags", () => {
const s = read("components/AnnotationLabelsPanel.tsx");
assert.match(s, /enabled\.includes\(item\.id\)/);
assert.doesNotMatch(s, /method: "PATCH"/);
assert.match(s, /reuseOrCreateDictionaryLabel/);
assert.match(s, /editingLabelId === saved.id \|\| tags.length > 0/);
assert.match(s, /\{ \.\.\.saved, tags \}/);
assert.match(s, /await apiRequest<ProjectAnnotationLabelsRecord>/);
assert.match(s, /controller\.abort\(\)/);
});

View File

@ -2,15 +2,15 @@ import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
const read = (p) => fs.readFileSync(new URL("../../" + p, import.meta.url), "utf8");
test("slice dictionary selection only fills the existing name callback", () => {
test("slice dictionary selection carries the displayed classification snapshot without changing boundaries", () => {
const picker = read("src/components/SliceNameDictionaryPicker.tsx");
assert.match(picker, /onSelect\(item.name\)/);
assert.match(picker, /onSelect\(item.name, dictionarySelection\(item\)\)/);
assert.doesNotMatch(picker, /apiRequest|\.start\s*=|\.end\s*=/);
assert.match(picker, /搜索本项目切片字典/);
assert.match(picker, /请联系管理员配置后再确认切片/);
assert.doesNotMatch(picker, /可直接输入名称/);
const workbench = read("src/components/CaptureAnnotationWorkbench.tsx");
assert.match(workbench, /onSliceNameChange\?\.\(selectedSlice\.id,\s*name\)/);
assert.match(workbench, /onSliceNameChange\?\.\(selectedSlice\.id,\s*name,\s*dictionary\)/);
});
test("dictionary management is separated and stale task results are hidden", () => {
assert.match(read("src/pages/ProjectDetail.tsx"), /动作描述字典/);