diff --git a/.scratch/project-production-dashboard/design.md b/.scratch/project-production-dashboard/design.md new file mode 100644 index 00000000..cc8c1700 --- /dev/null +++ b/.scratch/project-production-dashboard/design.md @@ -0,0 +1,85 @@ +# 项目与供应商生产统计:整合设计 + +Status: approved + +2026-09-28 用户确认定稿并授权本地实现、核验、备份部署配置后上线、线上只读验证成功后提交相关 PR。不得改动客户业务数据。线上不初始化统计范围,由管理员自行配置。审查基线为 5f2d796d;测试边界为统计 API 与权限、来源/阶段/事件归并及页面联动和只读下钻。 + +## 页面层次 + +在平台管理员的现有仪表盘顶部增加生产统计区;原有服务状态、工作量记录、运行记录等内容保留在下方。沿用平台配色、字体、表格和控件风格。 + +1. 顶部筛选:项目、供应商、项目状态(全部含归档 / 仅进行中),显示当前统计范围,提供返回全部。 +2. 两种统计视图:默认“当前进度”,可切“期间产出”。项目和供应商筛选作用于整个新统计区,不出现顶部全局、表格局部却没有标识的混用。 +3. 当前进度摘要:项目源数据量、当前已交付、异常/待核对。阶段分布在下一行,避免把所有细状态做成常驻卡片。已交付与异常均是总量中的子集,不是额外数据量。 +4. 阶段分布:接入与自动处理、标注、质检、终审、交付处理、已交付、已结束未交付。待核对有数据时单列。异常以所属阶段内的醒目数字提示,不再作为可相加的独立主阶段。 +5. 一张汇总表:切换按项目 / 按供应商。点击名称进入对应的另一维度;点击数值打开右侧明细。 +6. 明细保留范围与原表位置,显示源采集、项目、生产供应商、主阶段/细状态及关联任务,支持搜索、分页和只读查看。关闭不重置筛选,不自动领取、启动或改动业务。 + +桌面保留右侧明细;较窄内容区允许表格横向滚动、控件换行,避免把数字挤成不可读的小字。布局示意中的数据全部为模拟值。 + +## 数据范围与单位 + +- 仅当前管理员有权管理、且明确纳入生产统计的项目。纳入范围由管理员在管理设置维护;既有项目先核对清单,新项目确认后才计入。 +- 内部测试不出现在统计数、明细、项目筛选或供应商产出中。不得按名称包含 test、供应商默认状态或客户字段为空猜测。 +- 已归档生产项目默认计入,可选仅进行中项目。未配置统计范围时显示待配置,不用 0 冒充客户没有数据。 +- 单位仅为源数据份数,不展示或统计视频小时数。本需求不删除业务视频时长、不改动其他页面的既有数据字段。 +- 同项目、同一来源目录的 Capture 只计一份;多个工作流、重试、重跑、切片、动作及左右镜头不增加源数据份数。存储命名空间不同不能只因目录尾名相同而合并。 +- 同源用于两个项目,各项目各计一份,总览计两份;不做跨项目内容去重,也不读取视频做内容比对。 +- 已发现登记即计入,包括文件未齐、待处理和失败。未被平台发现的存储文件不计入。 + +## 当前进度 + +- 以最新一轮正式处理的权威记录归并,不以列表中“最近更新的一条 HumanItem”猜测;调试运行不影响生产进度。 +- 同轮多任务按最早尚未完成环节归并,所有应完成任务完成后才视作该源完成;按现有流程的必要阶段处理,不强制给未要求终审的任务补终审步骤。 +- 已交付后正式重跑回标注,当前只在标注计一份;既有交付证据留在历史/期间记录中。 +- 切片生成和送检尚未实际完成交接时,不提前算进入质检;领取、处理中、返工等是主阶段内的细状态。 +- 失败、待核对、已结束未交付与仍需处理的在途任务明确区分;全舍弃、取消等终结原因在明细中解释,不混同“已交付”。 +- 所有已知主阶段加待核对等于范围内项目源数据量。阶段条和项目行共享同一统计口径,不从有分页上限的列表求和。 + +## 供应商维度 + +- 当前进度按生产任务供应商归属;未领取公共源任务单列未分配,不复制给所有获授权供应商。 +- 匿名终审项即使 supplierId 为空,也追溯原生产任务;不会因为审核人属于另一供应商而改变生产归属。 +- 撤销授权、停用人员不删除历史贡献或既有任务统计。 +- 缺失/冲突导致无法可靠确定归属时列待核对,禁止猜测分配或重复加总。 + +## 期间产出 + +- 默认近7个北京时间自然日,含今天;提供今日、近7日、近30日、自定义。自定义结束日包含整天,查询边界使用次日零点,避免重复包含相邻窗口边界。 +- 日期只影响期间产出,不声称可还原某历史时刻的完整在途快照。 +- 新增源数据、成功送检、质检通过、终审通过、成功交付分别按期间内发生的可靠事件统计,每个指标内按项目源数据去重;同源在本周送检三次仍计一份。 +- 2026-09-28 补充确认 A:一源包含多个 part 时,送检、质检通过、终审通过和交付均须全部所需 part 有可靠完成证据,完成时间为最后一个 part 达成的时刻;之前完成的 part 可跨日期参与汇总。正式重跑及退回使对应 part 的旧完成证据失效,旧轮次已经成立的历史整源完成仍保留。供应商计参与完成整源的贡献,明细分别显示整源完成时间及原 part 证据时间。 +- 退回单列次数;不同指标不能相加当作新增数据量。一个源跨天重复完成时,每天可各有记录,但整个周范围仍应去重,不能简单累加日去重数。 +- 供应商按事件当时生产归属计期间参与产出。历史同源由两家供应商先后完成同一环节,各家各计一份,总览仍去重一份;只有存在交叉时提示供应商小计不可直接相加。 +- 来源不明的历史事件不倒推给当前人员或供应商;新增源数据尚无生产供应商时不能补算给之后领取者。 +- 缺少可靠历史事件则标明记录不完整,已知明细可查看;不估算为零、不复用现有语义混杂的 workforce 时长/产出字段冒充完整事实。 + +## 已确认的常规交互默认值 + +- 默认表格按源数据总量降序,支持名称搜索和数字列排序;不新增绩效排名或评分。 +- 范围内暂无数据的项目可显示零;未分配与待核对为明确特殊行。有历史数据的供应商即使授权撤销仍可查看,不列入仅与测试项目有关的供应商。 +- 前台可见时约30秒刷新,支持手动刷新并显示统计截至时间;具体频率以聚合成本验证为准。 +- 查询失败保留最后成功结果,标记更新失败/数据时间;初次加载失败显示不可用,不把请求失败绘制成全零。 +- 一个页面各分项尽量来自同一统计时点;切换条件后不让旧条件的慢响应覆盖新视图。 +- 控制说明密度:口径集中在简短帮助或明确的异常提示,不在每个卡片上重复长段文字。 + +## 兼容与验证边界 + +- 本功能主要是只读查询与统计范围配置,不重写任务、供应商归属、工作流、切片、审核和交付数据。 +- 服务端执行管理员权限及组织隔离,不能依赖前端隐藏;供应商管理员现有仪表盘权限保持原状。 +- 新聚合不复用 Capture automation 最近200条统计,不改变既有接口含义以免破坏调用方;具体索引和接口方案在实施阶段审查。 +- 验证覆盖:大于200源数据全量聚合、双目/切片/重跑去重、同名不同源、跨项目累计、公共池未分配、匿名终审溯源、撤销授权保留、同源多任务归并、重跑后的当前状态、时区边界及期间重复事件、跨供应商重叠、缺失证据、测试排除、归档筛选、空范围/零数据、请求失败与筛选竞态。 +- 本地可使用隔离合成数据验证;公司集群只读默认仍有效。设计确认不等于线上范围配置或其他写入操作授权。 + +## 关联材料 + +- [确认记录](./map.md) +- [现有代码事实与限制](./facts.md) +- 领域词汇以根目录 `CONTEXT.md` 为准。 + +## 布局示意核验 + +- 使用模拟数据演示项目/供应商交叉筛选、期间产出、数字下钻、明细搜索分页;未请求生产 API。 +- 本地浏览器检查覆盖约1024、736、320像素内容宽度及深色显示;主筛选联动、项目切换供应商维度、明细搜索与任务摘要、期间跨供应商重叠提示无脚本错误。 +- 这仅证明设计示意可交互,不代表真实统计接口、历史数据归并、查询性能或产品验收已完成。 +- 核验材料保存在未跟踪目录 `local-data/verification/production-dashboard-design/`。 diff --git a/.scratch/project-production-dashboard/facts.md b/.scratch/project-production-dashboard/facts.md new file mode 100644 index 00000000..c636f49d --- /dev/null +++ b/.scratch/project-production-dashboard/facts.md @@ -0,0 +1,35 @@ +# 生产统计代码事实 + +研究日期:2026-09-28。代码基点:`93695e22`。只读研究,不代表新增需求已批准。 + +## 源对象与数量 + +- v5.8 整个 Capture 创建一条主标注任务,任务内可产出多个切片;左右镜头属于同一 Capture。`docs/annotation-operations.md:65`。 +- CaptureRecord 没有直接的 duration 或 supplierId;源时长存在 AnnotationTaskRecord,尚未进入人工阶段的源数据可能缺失时长。`internal/domain/capture_automation.go:61`、`internal/domain/annotation_operations.go:181`。 +- 源记录查询以 workflowId + prefix 关联。跨工作流、跨项目的相同来源如何统计仍需业务确认。`internal/store/capture_automation.go:203`。 +- 旧/通用路径可按视频或 clip 建立多条人工任务,不能假设所有历史记录都严格一源一任务。`internal/worker/annotation_gate.go:783`、`:855`。 + +## 当前阶段与供应商 + +- 主标注/质检项和抽检后的最终验收项可同时存在;最终验收项以源 batch 为 ItemKey,supplierId 刻意留空以支持匿名验收。不能直接按 HumanItem 行数和 supplierId 聚合。`internal/api/human_loop.go:2428`。 +- 首次领取绑定供应商;质检返工、终审退回不清空供应商,原作者仍有权限时可直接返原作者。`internal/store/human_claim.go:83`、`internal/api/review_decision_plan.go:31`、`internal/api/rework_assignment.go:11`。 +- 默认供应商是真实供应商,不是公共池未分配项。`docs/project-member-scope.md:3`。 +- 撤销授权后存量仍闭环,不能从统计中删除供应商历史及在途记录。`CONTEXT.md` 的撤销授权定义。 + +## 现有统计不能直接复用 + +- Capture automation 每工作流仅取最近更新的 200 条,并据此算总数。`internal/api/capture_automation.go:210`、`:147`;`internal/store/capture_automation.go:180`。 +- 现有 Dashboard 按 HumanItem ItemKey 去重数量,但逐项累加时长,验收项可能产生额外 key。`internal/api/server.go:1986`。 +- Workforce 基于工作量事件的用户 × 供应商聚合;事件有 projectId,但当前汇总没有项目维度。`internal/api/dashboard_workforce.go:12`、`:81`。 +- outputCount 的含义是首次标注提交产出,不等于交付数;durationSeconds 某些路径会退化为切片视频时长,不能当作源时长或纯操作工时。`internal/api/annotation_operations.go:2408`、`:2815`。 +- 新最终验收路径有正式决定/事件,但不能假设均写入 acceptance worklog。旧版本清理也使任意过去日期的完整存量快照无法保证。`internal/api/review_decision_plan.go:73`;`docs/annotation-operations.md:47`。 + +## 生产与内部测试范围 + +- 项目目前只有 active/archived 等属性,没有正式的生产/测试用途字段;RunMode production/debug 不能替代数据用途分类。`internal/domain/types.go:115`、`:346`。 +- ADR-0004 是独立测试环境的已批准方向,不能仅据 ADR 假定已全部迁移。 +- 生产库历史测试项目的归类需明确依据,例如经确认的项目 ID 或新增显式用途;不得按名字包含 test、customer 为空或默认供应商猜测。 + +## 仍需确认的业务决策 + +源身份与跨项目计数、有效处理链路选择、同源跨供应商历史任务、阶段归并、未知时长、历史事件完整性、测试用途识别和期间产出指标。均不应因现有字段方便而默认为某种用户要求。 diff --git a/.scratch/project-production-dashboard/implementation.md b/.scratch/project-production-dashboard/implementation.md new file mode 100644 index 00000000..ff898234 --- /dev/null +++ b/.scratch/project-production-dashboard/implementation.md @@ -0,0 +1,45 @@ +# Production dashboard implementation + +Status: implemented; deployed; readonly acceptance passed + +Approved design: [design.md](./design.md). Review baseline: `5f2d796d`. + +## Boundaries + +- `internal/productionstats`: deterministic source identity, latest formal attempt, human/package stages, historical event ownership, period deduplication and scoped drill-down. No storage writes. +- `internal/store/production_stats.go`: projected, project-scoped full reads (including archived runs); isolated `productionStatsScopes` settings keyed by organization `_id`, revision CAS. No bootstrap migration, customer backfill or default scope initialization. +- `/api/dashboard/production` and `/scope`: server-side platform-admin and organization checks; explicit production allowlist. Existing dashboard contract unchanged. +- Dashboard top section and system-settings scope panel; UI refresh/filters/race fencing/error retention; drawer only displays references or opens existing readonly execution view. + +## Verification plan and evidence + +Local synthetic Mongo container `codex-production-dashboard-20260928`, port 27028, database `dashboard_synthetic_20260928`. No company database connection. Existing 27018 is confirmed Colima-forwarded local snapshot, not used by these tests. + +Tests cover source dedup, latest formal attempt/debug exclusion, >200 records, multiple tasks, anonymous acceptance, ended tasks, completed outbox, packaging child/legacy post-package acceptance, missing links, historical ownership, timezone/period dedup, cross-org/role isolation, scope CAS and restart persistence. + +Browser fixture: 251 production examples across two projects plus one explicitly excluded example. Results under untracked `local-data/verification/production-dashboard/`. + +Historical reporting deliberately identifies incomplete evidence rather than substituting current owner or treating missing events as proof of zero output. No arbitrary historical in-progress snapshots are claimed. + +## Release constraints + +User approved release and PR after successful live verification. Live scope remains unconfigured until an administrator explicitly selects projects. Preserve `DROBOTICFLOW_DATABASE_BOOTSTRAP=verify-existing`; no migrations, business writes, task claims, workflow edits, or synthetic customer records. Back up deployment state and preserve previous immutable image for conditional rollback; no database restore. + +## Review corrections and local results (2026-09-28) + +- Resolve latest formal attempts per physical part before source-level aggregation. Ignore superseded unadmitted parent placeholders for progress while preserving the earliest discovery timestamp. Admitted, discarded and launching parents remain evidence. +- Normalize legacy review worklog names. Use persisted post-package acceptance reviews as canonical decisions; exclude packaging-time copies of pre-package decisions. Legacy streams lacking shared decision IDs cannot be reconciled exactly: prefer HumanEvent evidence per package/metric rather than adding potentially duplicate worklogs, retain earlier evidence, and explicitly mark history incomplete. +- Period drilldown separates current state from event-time supplier, timestamp and run references. Current multipart details list individual run references. +- Index task/package/ownership facts once and debounce drawer search. A local synthetic 10K aggregation benchmark was approximately 46ms; this does not establish Mongo/concurrent production capacity. +- Targeted aggregation, HTTP permission/scope tests, local Mongo persistence/projection tests, TypeScript checking and frontend build passed. Browser checks passed on 251 included synthetic sources plus one excluded project: configured scope, filters, project/supplier drilldown, pagination/search, historical warning, archived projects and refresh-error retention. No business mutation was sent by the drawer. +- Full-suite comparison against 5f2d796d found the same 79 Go top-level failing cases and the same 14 frontend failing cases (608/622 passing). These remain baseline debt, not a clean full-suite pass. Raw evidence is in the untracked local verification directory. + +## Multipart period decision + +User selected A: all required parts must complete before counting a source milestone. Temporal aggregation now uses the last part completion time, keeps original contributing evidence, and invalidates corresponding part evidence on formal retry or rejection. Regression tests cover cross-day completion, missing parts, retries with late old events, historical supplier ownership and rejection/resubmission. + +Integrated candidate 36eb8540 is deployed to droboticflow-fanyu. Readonly live acceptance passed: five roles across seven existing isolated accounts, admin-only statistics API, unchanged unconfigured scope, six timeline/video range checks and actual left/right playback. No customer business writes occurred. Full annotation/delivery/review mutation cycles and live slice restart remain explicitly unverified. PR preparation excludes changes already submitted in #20 and #21; production scope remains administrator-configured. + +The delayed-response browser check exposed filter options disappearing during reload. The options now retain their last authorized response while numeric results reload. Re-tested by holding project A's response, selecting project B, then releasing A: B's 51-source result and selection remained intact. Typecheck/build and browser race verification passed. + +All-parts follow-up also verifies multiple tasks within one part, acceptance returns preserving submitted work, dependency-scoped sibling invalidation, and replacement packages completing the same production units. Current progress excludes explicitly superseded packages and their linked acceptance items, while retaining historical decisions. Final full-suite comparison still has the baseline's 79 Go top-level failures and 14 frontend failures; differing diagnostics were checked for cause (membership 403, deleted-account 410, concurrent route 404, video-fixture dependency failure), with no new cause found. diff --git a/.scratch/project-production-dashboard/map.md b/.scratch/project-production-dashboard/map.md new file mode 100644 index 00000000..421ef859 --- /dev/null +++ b/.scratch/project-production-dashboard/map.md @@ -0,0 +1,104 @@ +# 项目与供应商生产统计面板:需求讨论 + +Type: grilling +Status: discussing + +## Notes + +- 用户要求在平台管理员仪表盘增加生产统计:总体、分项目、分供应商、项目内各供应商,直观、清爽、便于客户管理。 +- 本轮为需求讨论和文档记录,尚未批准实施;不创建 PR、不推送、不部署、不修改线上数据。 +- 基于 2026-09-28 本地版本 `93695e22` 检查现有代码。当前管理员仪表盘有总体任务状态和成员原始工作记录,不能直接替代项目 × 供应商全量统计。 +- 现有 Capture、人工任务、切片、动作区间属于不同层级;供应商授权多对多不代表公共池任务复制给每家供应商。 +- 既有内部测试隔离约定参见 `docs/adr/0004-isolated-internal-test-lane.md`;不能用命名猜测生产/测试属性。 +- 当前工作流 Capture 接口仍以 200 条列表计算统计(`internal/api/capture_automation.go:210`、`:147`),不可复用为项目全量汇总。 +- 当前仪表盘成员记录基于工作量事件,不能将领取/提交/返工次数作为去重后的数据量。 +- 发现历史规则差异:ADR-0003 记载退回公共可领取池,当前 combined 质检实现会优先退给原作者(`internal/api/annotation_operations.go:3176` 起),供应商归属不因此清空。本统计需求不应顺带改动返工工作流,后续需要确认其呈现口径。 + +## Decisions-so-far + +- 确认目标入口:平台管理员的仪表盘。用户称“超级管理员”,实现需沿用当前管理员权限模型。 +- 确认至少覆盖总体、项目、供应商及项目内供应商四种查看需要。 +- 确认通过多轮讨论确定口径与交互;不把候选建议当作已确认需求。 +- Q1 已确认并由 Q6 简化:以去重后的源 Capture 份数为主要数据量口径;双目只计一次,切片和动作放展开明细,重跑不重复增加源数据。原先的时长统计已取消;各环节任务计数仍需单独确认。 +- Q2 已确认:默认展示当前进度;期间新增、送检、完成等独立展示,不将累计量、当前存量和操作次数混算。 +- Q3 已确认:顶部新增生产总览及阶段分布,下方一张表切换按项目/按供应商;支持项目内供应商、供应商内项目下钻及交叉筛选。原仪表盘内容保留在下方,沿用现有风格。 +- Q4 已确认并由 Q8、Q10 明确:默认统计明确纳入的全部生产项目,累计包含已归档;可筛选仅进行中项目。内部测试完全不出现在本面板,取消原“测试单列查看”方案。 +- Q5 已确认:按项目累计。同源用于两个项目时各计 1 份、总览计 2 份;同项目内重跑或重复处理仍只计 1 份。面板主口径称“项目源数据量”,总览等于项目之和;此规则明确 Q1 的去重范围。 +- Q6 已确认:新统计面板取消时长统计与展示,仅展示数量;这不是删除现有视频时长或改动业务数据的要求,也不新增未知时长提示。 +- Q7 已确认:供应商维度按生产任务归属统计;公共未领取独立未分配,质检和终审追溯生产供应商,授权撤销仍保留存量/历史。 +- Q8 已确认:内部测试不显示在本面板,不提供测试总览、行或筛选项;识别并排除方式见 Q10。 +- Q9 已确认:平台发现并登记 Capture 即计入,含文件不齐、自动处理和失败;尚未发现的存储文件不计入。 +- Q10 已确认:管理员明确维护纳入生产统计的项目范围;旧项目清单先核对,新项目确认纳入后才统计;范围设置在管理设置中。 +- Q11 已确认:主阶段简洁、细状态展开。主阶段为接入与自动处理、标注、质检、终审、交付处理、已交付、已结束未交付;每源只归一个主阶段。异常为阶段内提示,无法判定历史数据单列待核对,不重复加总。 +- Q12 已确认:当前进度跟随最新正式处理;重跑后回到标注则当前只计标注,旧交付保留历史与期间记录。调试运行不改变生产统计。 +- Q13 已确认:期间新增、成功送检、质检通过、终审通过、成功交付分别按项目源数据去重,退回次数另列。同一份数据在选定期间重复送检只计 1 份,统计对象不同的指标不相加。 +- Q14 已确认:期间产出默认近 7 个北京时间自然日;可选今日、近7日、近30日、自定义。日期只影响期间产出,当前进度不是历史快照。 +- Q15 已确认:点击数值打开右侧明细,继承项目/供应商/阶段/日期,支持搜索分页及任务查看页跳转;关闭保留筛选和表格位置,不自动领取或修改。 +- Q16 已确认:同源最新正式轮次有多任务时,按最早尚未完成环节归类;需完成的任务全部完成才算整份完成,各任务状态可展开。 +- Q17 已确认:展示可靠已知数据,阶段/供应商无法判定的列待核对;期间缺历史证据明确提示记录不完整,不当作零、不假装全量。 +- Q18 已确认:期间按事件当时生产供应商归属,各家分别计参与产出,总览仍去重;出现交叉处理时提示供应商小计不可直接相加。 +- Q19 已确认:项目/供应商筛选联动顶部总览、阶段分布及明细,明确显示当前范围,可一键返回全部。 +- Q20 已确认:项目内按同一来源目录的 Capture 去重,不因跨工作流重复处理增加;同名不同来源目录分别计数,不做视频内容比对去重。 + +## Design tree + +1. 管理用途、统计对象与单位、纳入范围 + - 当前存量与期间产出分别如何计算 + - Capture / 人工任务 / 切片 / 视频时长的去重与关系 + - 内部测试、历史、归档、未接入、失败数据 +2. 归属与状态 + - 公共未领取、已绑定供应商、跨阶段审核归属 + - 退回、重新切片、重跑与撤销授权 + - 阶段互斥与总数核对、异常与未知状态 +3. 信息结构 + - 总览卡片、按项目 / 按供应商明细、交叉筛选 + - 仪表盘现有内容如何安置 + - 明细跳转、搜索排序、日期筛选、刷新与错误反馈 +4. 验收与权限 + - 服务端聚合覆盖全量;不以列表分页代替统计 + - 平台/供应商权限范围、只读业务行为 + - 性能、历史缺失、统计时点一致性与可解释性 + +## Fog + +- Q1–Q20 已确认(Q6 取消时长,Q8 取消测试显示)。核心业务问题已收敛。 +- 只读代码研究已完成,见 [facts.md](./facts.md)。整合设计见 [design.md](./design.md)。 +- 2026-09-28 整体设计已批准,按 design.md 的实施、验证与发布边界执行。线上统计范围由管理员自行配置。时长相关规则不再属于本需求。 +- 设计稿中刷新频率、默认排序、宽度适配等常规交互是最终确认时一并审阅的建议,不冒充新的用户回答。 + +## 第一轮(Q1–Q4 已确认) + +- Q1 原回答:Capture 份数+源视频时长为主;后续 Q6 取消时长,仅保留份数,双目不翻倍;切片、动作放明细,任务数量另定。 +- Q2 已确认:默认当前阶段分布与已交付量,期间新增/送检/完成独立。 +- Q3 已确认:顶部总览+阶段分布,下方按项目/供应商切换一张表,支持交叉筛选和下钻;原内容保留在下方。 +- Q4 原回答:默认全部生产项目含归档,可选仅进行中;后续 Q8 取消测试查看入口,内部测试全部排除。 + +## 第二轮(Q5–Q9 已确认) + +- Q5 已确认:同源数据跨项目分别计算,总览为项目之和;同项目重跑仍去重,主口径称项目源数据量。 +- Q6 用户自定回答:直接不记录时长。按本统计功能取消时长处理,不删除既有业务时长。 +- Q7 选择 A:按生产任务所属供应商归属,公共未领取单列未分配;质检和匿名终审追溯原生产供应商,撤销授权不抹掉历史。 +- Q8 用户自定回答:不要内部测试,内部测试数据不显示在上面。排除机制随后由 Q10 确认。 +- Q9 选择 A:已发现并登记的 Capture 即计入,包含文件不齐、自动处理及失败;未发现存储对象不计入。 + +## 第三轮(Q10–Q13 已确认,均选 A) + +- Q10 已确认:管理员明确维护纳入生产统计的项目范围,既有项目清单先确认,新项目确认后纳入,范围管理在设置中;内部测试不进入统计面板或筛选项。 +- Q11 已确认:主阶段列为接入与自动处理、标注、质检、终审、交付处理、已交付、已结束未交付;细状态下钻,异常作为阶段内提示,无法判定历史数据待核对,不重复加总。 +- Q12 已确认:当前状态跟随最新正式处理,旧交付保留历史/期间记录,调试运行不改变生产统计。 +- Q13 已确认:期间新增、送检、质检通过、终审通过、交付按项目源数据在所选期间去重,退回次数独立。 + +## 第四轮(Q14–Q17 已确认,均选 A) + +- Q14 已确认:期间默认近 7 个北京时间自然日,提供今日/近7日/近30日/自定义;不影响当前进度。 +- Q15 已确认:点击数值打开右侧明细,携带项目/供应商/阶段/日期,搜索分页,必要时打开既有任务查看页;关闭保留原视图,不自动领取或修改。 +- Q16 已确认:同一源最新一轮多任务按最早未完成环节归并,全部应完成任务结束后才算整源完成。 +- Q17 已确认:已知总量照常展示,无法判定阶段/供应商列待核对,期间缺证据明确记录不完整;不估算、不假装为零或全量。 + +## 第五轮(Q18–Q20 已确认,均选 A) + +- Q18 已确认:同源同期间先后由两家供应商完成同一环节时,按事件当时生产归属分别计各家涉及完成份数,总览仍去重为1;出现重叠时明确供应商小计不可直接相加。当前进度始终按当前正式轮次归属,不与期间产出混用。 +- Q19 已确认:项目/供应商筛选同时影响顶部总览、阶段分布和明细,范围标签明确,可一键返回全部。 +- Q20 已确认:项目内同一来源目录的 Capture 去重,不同工作流重复处理不增加;名字相同但来源目录不同则分别计数,不做视频内容去重。 + +除明确标注已确认的回答外,其余均为建议,未收到回答前不作为已确认规则。 diff --git a/AGENTS.md b/AGENTS.md index 2f7ea682..c901611c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,6 +5,44 @@ - This repository directory is the single working root; the sibling `~/Desktop/fanyu` folder is legacy and will be retired. - Cluster kubeconfig (`fanyu.conf`), BOS credential CSV, downloaded captures, and the local workspace mount are being migrated under this repo's untracked local data path (see `docs/HANDOFF.md` § 0 for the live layout). +## Engineering Principles + +These principles guide implementation choices. Specific business contracts and this document's release, authorization, and data safety rules take precedence. + +1. Check contracts and implementation first. Follow the Domain docs reading rules to review relevant business documentation, + ADRs, code, and tests. Distinguish current behavior, approved designs that are not yet implemented, and superseded rules. + Identify specific conflicts between the current request and existing agreements; ask the user when a conflict affects business outcomes and cannot be resolved independently. + +2. Define the compatibility scope before making changes. For APIs, Mongo records, workflow definitions, operator inputs and outputs, + delivery formats, or task states, check callers, existing data, and in-flight tasks. + Before removing an old path, establish that compatibility is no longer needed or migration is complete, and explain the rollback impact. + Preserve migration and recovery paths that still serve a purpose. Internal implementations may be refactored directly when contracts remain unchanged. + +3. Choose the simplest solution that meets current requirements and known constraints. Abstractions, configuration, and indirection must serve a concrete purpose. + Authentication, permission isolation, data consistency, and necessary retries and recovery are part of the requirements. + For temporary solutions, state their scope, limitations, and replacement conditions; avoid shortcuts known to fall short of confirmed requirements. + +4. Implement incrementally through verifiable business flows. For changes spanning multiple layers, first connect the smallest complete flow required by the current request, + then extend its capabilities. Validate replacement paths before removing existing implementations. + A minimal version determines implementation order; the final delivery must still meet all acceptance criteria for the current request. + +5. Follow existing responsibility boundaries. Locate the relevant components, API wrappers, business logic, storage, and execution entry points, + and place changes in the module responsible for that behavior. Enforce permissions and business constraints on the server; + use frontend validation for interaction feedback. Keep shared rules consistent and update generated files from their authoritative sources. + +6. Reuse before adding dependencies. Before writing a new implementation or adding a dependency, inspect existing project code, standard libraries, + and the documentation, types, and available capabilities for installed dependency versions. When existing capabilities are insufficient, choose well-maintained + libraries that reduce overall complexity, and explain their purpose and adoption costs. + +7. Consult established approaches where useful. For unfamiliar, consequential, or hard-to-reverse design choices, + consult official sources or relevant approaches in mature products, and explain the conditions under which they apply. + Follow existing project patterns for routine changes; adapt external approaches to this project's business contracts. + +8. Complete delivery with verification evidence. Run the checks required for the scope of the change and verify relevant behavior in the local replica environment. + For changes involving asynchronous processing, persistence, or compatibility, cover the relevant failure, restart recovery, and legacy data scenarios. + Compare existing failures against the baseline by specific test case and cause, rather than failure counts alone. + Update the corresponding contracts when behavior changes, and clearly report what passed, failed, or remains unverified. + ## Deployment Environment - The `dev` branch uses Fanyu. Leave `DEPLOY_ENV` unset or set it to `fanyu`; the loader rejects other targets, including `poc` and `wjzh`. diff --git a/CONTEXT.md b/CONTEXT.md index fd14dd15..ad770914 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -6,6 +6,16 @@ **Capture(源采集)**:一次源采集所关联的双目视频、时间戳与传感器数据,是本流程组织人工处理和审核的对象。 +**项目源数据量**:每个项目内已发现并登记、按同一来源目录的 Capture 去重后的份数;同源用于两个项目时各计一份,总览为项目之和。同项目内双目、跨工作流重复处理、切片、动作划分或重跑不增加源数据量;同名但不同来源目录分别计数,不按视频内容去重。 +_Avoid_:全局唯一源文件数、累计提交次数、切片数量(均不等于项目源数据量) + +**当前生产进度**:项目源数据在最新一轮正式处理中的当前主阶段,每份数据只占一个主阶段;同轮存在多条任务时按最早尚未完成的环节归类。正式重跑可使当前阶段回退,旧交付结果保留为历史;调试运行不改变当前生产进度。 + +**期间阶段完成量**:在所选期间内发生过指定阶段成功完成事件的项目源数据份数,同一项目的同一源数据在该指标中只计一次。新增、送检、质检通过、终审通过与交付分别计算,退回另计发生次数。一源包含多个 part 时,阶段完成须所有所需 part 达成,按最后一个 part 完成时间计入期间;正式重跑或退回后的对应 part 不能复用已失效的完成证据。 +_Avoid_:操作次数(同一份数据可以重复操作,但期间完成份数不因此重复累计) + +**供应商期间参与产出量**:供应商在所选期间内参与完成某环节的项目源数据份数,按事件当时的生产归属统计。同一份数据在期间内可先后由多家供应商处理,各家可分别计入,总览仍按项目源数据去重;存在交叉时供应商小计不能直接相加。 + **切片**:标注员在源采集的统一有效帧范围内划定的连续区间。全部切片完整覆盖该范围,互不重叠;一个切片可以包含多个动作区间。 _Avoid_:动作描述区间(两者不是同一层级) @@ -93,6 +103,8 @@ _Avoid_:公共可领取任务池(其中的任务尚未归属当前标注员 **项目公共任务池**:供项目已授权供应商共同领取的待处理任务集合;同一任务只允许一家供应商领取,领取后归属该供应商,其他供应商不能重复领取或查看其标注内容。 _Avoid_:供应商任务副本(公共池不为每家供应商复制同一任务) +**生产供应商归属**:生产任务所属的供应商,不随质检或最终验收的处理人员变化而改变;公共池尚未归属供应商的任务为未分配。撤销项目授权不抹去该供应商已归属任务的存量与历史统计。 + ## 内部测试 **内部测试任务池**:仅供内部人员复现问题和验收版本的独立任务集合;测试中的领取、标注、审查和交付不改变客户生产任务。 diff --git a/internal/api/annotation_operations.go b/internal/api/annotation_operations.go index f3332531..fc9abede 100644 --- a/internal/api/annotation_operations.go +++ b/internal/api/annotation_operations.go @@ -618,6 +618,17 @@ func (server *server) handleAnnotationTasksList(writer http.ResponseWriter, requ return } } + reviewedIDs := map[string]bool{} + if principal != nil && (principal.User.Role == domain.UserRoleQualityReviewer || principal.User.Role == domain.UserRoleAcceptanceReviewer) { + ids, historyErr := server.store.ListReviewedHumanItemIDs(request.Context(), server.requestOrganizationID(request), principal.User.ID) + if historyErr != nil { + renderError(writer, http.StatusInternalServerError, historyErr) + return + } + for _, id := range ids { + reviewedIDs[id] = true + } + } type filteredTask struct { task domain.AnnotationTaskRecord humanStageID string @@ -658,7 +669,7 @@ func (server *server) handleAnnotationTasksList(writer http.ResponseWriter, requ } if hasHumanItem { isAuthor := principal != nil && linkedHumanItem.LatestAuthorUserID == principal.User.ID - if !server.canAccessCurrentHumanStage(request.Context(), request, linkedHumanItem) && !isAuthor { + if !server.canAccessCurrentHumanStage(request.Context(), request, linkedHumanItem) && !isAuthor && !reviewedIDs[linkedHumanItem.ID] { continue } } @@ -762,7 +773,7 @@ func (server *server) handleAnnotationTaskGet(writer http.ResponseWriter, reques renderJSON(writer, http.StatusForbidden, map[string]string{"error": "请先选择项目随机领取任务", "code": "random_claim_required"}) return } - if linkedHumanItem != nil && !annotationManager(principal) && !server.canAccessCurrentHumanStage(request.Context(), request, *linkedHumanItem) && !isAuthor { + if linkedHumanItem != nil && !annotationManager(principal) && !server.canAccessCurrentHumanStage(request.Context(), request, *linkedHumanItem) && !isAuthor && !server.canReadReviewedHumanItem(request.Context(), request, *linkedHumanItem) { renderJSON(writer, http.StatusForbidden, map[string]string{"error": "current human stage role is required"}) return } @@ -812,6 +823,8 @@ func (server *server) handleAnnotationTaskGet(writer http.ResponseWriter, reques renderError(writer, http.StatusInternalServerError, err) return } + } + if linkedHumanItem != nil && request.URL.Query().Get("view") != "history" { if err := server.recoverInterruptedSlice(request.Context(), linkedHumanItem, time.Now()); err != nil { renderError(writer, http.StatusInternalServerError, err) return @@ -1068,7 +1081,7 @@ func (server *server) handleAnnotationTaskFrameTimelineGet(writer http.ResponseW return } principal, _ := currentPrincipal(request) - if linkedHumanItem != nil && !annotationManager(principal) && !server.canAccessCurrentHumanStage(request.Context(), request, *linkedHumanItem) { + if linkedHumanItem != nil && !annotationManager(principal) && !server.canAccessCurrentHumanStage(request.Context(), request, *linkedHumanItem) && !server.canReadReviewedHumanItem(request.Context(), request, *linkedHumanItem) { renderJSON(writer, http.StatusForbidden, map[string]string{"error": "current human stage role is required"}) return } diff --git a/internal/api/human_loop.go b/internal/api/human_loop.go index c7b9faea..6b92497d 100644 --- a/internal/api/human_loop.go +++ b/internal/api/human_loop.go @@ -226,8 +226,8 @@ func (server *server) handleHumanItemsList(writer http.ResponseWriter, request * pool = "available" } } - if pool != "available" && pool != "mine" && pool != "all" { - renderValidationError(writer, validationIssue{Field: "pool", Message: "pool must be available, mine, or all"}) + if pool != "available" && pool != "mine" && pool != "all" && pool != "reviewed" { + renderValidationError(writer, validationIssue{Field: "pool", Message: "pool must be available, mine, reviewed, or all"}) return } if pool == "all" && !annotationManager(principal) { @@ -235,6 +235,19 @@ func (server *server) handleHumanItemsList(writer http.ResponseWriter, request * return } query := parseHumanItemQuery(request) + if pool == "reviewed" { + if principal == nil || (principal.User.Role != domain.UserRoleQualityReviewer && principal.User.Role != domain.UserRoleAcceptanceReviewer && !annotationManager(principal)) { + renderJSON(writer, http.StatusForbidden, map[string]string{"error": "review history requires reviewer access"}) + return + } + ids, err := server.store.ListReviewedHumanItemIDs(request.Context(), server.requestOrganizationID(request), principal.User.ID) + if err != nil { + renderError(writer, http.StatusInternalServerError, err) + return + } + query.IDs = ids + } + if pool == "mine" && principal != nil { query.ActorUserID = principal.User.ID } @@ -269,7 +282,7 @@ func (server *server) handleHumanItemsList(writer http.ResponseWriter, request * continue } isAuthor := principal != nil && item.LatestAuthorUserID == principal.User.ID - if !server.canAccessHumanItem(request.Context(), request, item, annotationRoleForHumanStage(stage)) && !isAuthor { + if pool != "reviewed" && !server.canAccessHumanItem(request.Context(), request, item, annotationRoleForHumanStage(stage)) && !isAuthor { continue } if pool == "mine" { @@ -287,6 +300,34 @@ func (server *server) handleHumanItemsList(writer http.ResponseWriter, request * } filtered = append(filtered, item) } + if pool == "reviewed" { + acceptanceIDs := []string{} + for _, item := range filtered { + if item.Status == domain.HumanItemAccepted && isFanyuV58ThreeRoleBatch(batchByID[item.BatchID]) { + acceptanceIDs = append(acceptanceIDs, fanyuV5PrePackageAcceptanceItemID(item.BatchID)) + } + } + accepted := map[string]bool{} + if len(acceptanceIDs) > 0 { + downstream, err := server.store.ListHumanItems(request.Context(), server.requestOrganizationID(request), domain.HumanItemQuery{IDs: acceptanceIDs}) + if err != nil { + renderError(writer, http.StatusInternalServerError, err) + return + } + for _, item := range downstream { + accepted[item.ID] = item.Status == domain.HumanItemAccepted + } + } + for index := range filtered { + item := &filtered[index] + if item.Status == domain.HumanItemAccepted && isFanyuV58ThreeRoleBatch(batchByID[item.BatchID]) { + item.ReviewHistoryStatus = "pending_acceptance" + if accepted[fanyuV5PrePackageAcceptanceItemID(item.BatchID)] { + item.ReviewHistoryStatus = "accepted" + } + } + } + } resolver := newHumanDisplayResolver(server) for index := range filtered { if err := resolver.enrichItem(request.Context(), &filtered[index]); err != nil { @@ -2524,3 +2565,24 @@ func (server *server) flushHumanReleasePartitions(ctx context.Context, batch *do } return server.store.UpsertHumanBatch(ctx, *batch) } + +// History grants read access only; mutation handlers retain current-stage checks. +func (server *server) canReadReviewedHumanItem(ctx context.Context, request *http.Request, item domain.HumanItemRecord) bool { + principal, ok := currentPrincipal(request) + if !ok || principal == nil || request.Method != http.MethodGet || !server.canAccessHumanItem(ctx, request, item, "") { + return false + } + if principal.User.Role != domain.UserRoleQualityReviewer && principal.User.Role != domain.UserRoleAcceptanceReviewer { + return false + } + ids, err := server.store.ListReviewedHumanItemIDs(ctx, item.OrganizationID, principal.User.ID) + if err != nil { + return false + } + for _, id := range ids { + if id == item.ID { + return true + } + } + return false +} diff --git a/internal/api/production_dashboard.go b/internal/api/production_dashboard.go new file mode 100644 index 00000000..a97efd16 --- /dev/null +++ b/internal/api/production_dashboard.go @@ -0,0 +1,207 @@ +package api + +import ( + "context" + "droboticflow/internal/domain" + "droboticflow/internal/productionstats" + "encoding/json" + "net/http" + "sort" + "strconv" + "strings" + "time" +) + +func (s *server) productionAdmin(w http.ResponseWriter, r *http.Request) (*authenticatedPrincipal, bool) { + p, ok := currentPrincipal(r) + if !ok || !isOrganizationAdmin(p) { + renderJSON(w, http.StatusForbidden, map[string]string{"error": "platform administrator required"}) + return nil, false + } + return p, true +} +func (s *server) handleProductionScope(w http.ResponseWriter, r *http.Request) { + p, ok := s.productionAdmin(w, r) + if !ok { + return + } + org := s.requestOrganizationID(r) + projects, err := s.store.ListProjectsByOrganization(r.Context(), org) + if err != nil { + renderError(w, 500, err) + return + } + scope, err := s.store.GetProductionStatsScope(r.Context(), org) + if err != nil { + renderError(w, 500, err) + return + } + if r.Method == http.MethodPut { + var input productionstats.Scope + if err = json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20)).Decode(&input); err != nil { + renderError(w, 400, err) + return + } + allowed := map[string]bool{} + for _, project := range projects { + allowed[project.ID] = true + } + ids := []string{} + seen := map[string]bool{} + for _, id := range input.ProjectIDs { + if !allowed[id] { + renderJSON(w, 400, map[string]string{"error": "统计范围包含无权限或已删除的项目"}) + return + } + if !seen[id] { + seen[id] = true + ids = append(ids, id) + } + } + sort.Strings(ids) + next := productionstats.Scope{ProjectIDs: ids, Revision: input.Revision + 1, UpdatedAt: time.Now().UTC().Format(time.RFC3339Nano), UpdatedBy: p.User.ID} + saved, saveErr := s.store.SaveProductionStatsScope(r.Context(), org, next, input.Revision) + if saveErr != nil { + renderError(w, 500, saveErr) + return + } + if !saved { + renderJSON(w, 409, map[string]string{"error": "统计范围已被其他管理员修改,请重新加载"}) + return + } + scope = next + } + options := []map[string]string{} + for _, project := range projects { + options = append(options, map[string]string{"id": project.ID, "name": project.Name, "status": string(project.Status)}) + } + renderJSON(w, 200, map[string]any{"scope": scope, "projects": options}) +} +func (s *server) handleProductionDashboard(w http.ResponseWriter, r *http.Request) { + if _, ok := s.productionAdmin(w, r); !ok { + return + } + started := time.Now().UTC() + ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second) + defer cancel() + org := s.requestOrganizationID(r) + scope, err := s.store.GetProductionStatsScope(ctx, org) + if err != nil { + renderError(w, 500, err) + return + } + params := r.URL.Query() + q, err := productionstats.ParseQuery(params.Get("view"), params.Get("from"), params.Get("to"), started) + if err != nil { + renderError(w, 400, err) + return + } + q.ProjectID = params.Get("projectId") + q.SupplierID = params.Get("supplierId") + q.Group = params.Get("group") + if q.Group == "" { + q.Group = "project" + } + if q.Group != "project" && q.Group != "supplier" { + renderJSON(w, 400, map[string]string{"error": "invalid group"}) + return + } + projects, err := s.store.ListProjectsByOrganization(ctx, org) + if err != nil { + renderError(w, 500, err) + return + } + allowed := map[string]bool{} + for _, id := range scope.ProjectIDs { + allowed[id] = true + } + ids := []string{} + projectOptions := []map[string]string{} + for _, p := range projects { + if !allowed[p.ID] || (params.Get("activeOnly") == "true" && p.Status == domain.ProjectStatusArchived) { + continue + } + ids = append(ids, p.ID) + projectOptions = append(projectOptions, map[string]string{"id": p.ID, "name": p.Name}) + } + if q.ProjectID != "" { + found := false + for _, id := range ids { + if id == q.ProjectID { + found = true + } + } + if !found { + renderJSON(w, 400, map[string]string{"error": "项目不在当前生产统计范围"}) + return + } + } + snapshot, err := s.store.LoadProductionStats(ctx, org, ids) + if err != nil { + renderError(w, 500, err) + return + } + report := productionstats.Build(snapshot) + summary := productionstats.Summarize(report, q) + suppliers, err := s.store.ListSuppliers(ctx, org) + if err != nil { + renderError(w, 500, err) + return + } + names := map[string]string{productionstats.Unassigned: "未分配", productionstats.Unknown: "待核对"} + for _, supplier := range suppliers { + names[supplier.ID] = supplier.Name + } + present := map[string]bool{} + for _, source := range report.Sources { + present[source.SupplierID] = true + } + for _, event := range report.Events { + present[event.SupplierID] = true + } + supplierOptions := []map[string]string{} + for id := range present { + name := names[id] + if name == "" { + name = id + } + supplierOptions = append(supplierOptions, map[string]string{"id": id, "name": name}) + } + sort.Slice(supplierOptions, func(i, j int) bool { return supplierOptions[i]["name"] < supplierOptions[j]["name"] }) + // Include configured projects with no data without leaking excluded projects. + if q.Group == "project" && q.SupplierID == "" { + seen := map[string]bool{} + for _, row := range summary.Rows { + seen[row.ID] = true + } + for _, p := range projectOptions { + if !seen[p["id"]] && (q.ProjectID == "" || q.ProjectID == p["id"]) { + summary.Rows = append(summary.Rows, productionstats.Row{ID: p["id"], Counts: map[string]int{}}) + } + } + } + response := map[string]any{"configured": scope.Revision > 0, "asOf": started.Format(time.RFC3339Nano), "summary": summary, "projects": projectOptions, "suppliers": supplierOptions, "from": q.From.Format(time.RFC3339), "to": q.To.Format(time.RFC3339)} + if params.Get("details") == "true" { + items := productionstats.Details(report, q, params.Get("metric"), strings.TrimSpace(params.Get("search"))) + page, pageErr := strconv.Atoi(params.Get("page")) + if params.Get("page") != "" && (pageErr != nil || page < 1 || page > 10000000) { + renderJSON(w, 400, map[string]string{"error": "invalid page"}) + return + } + if page < 1 { + page = 1 + } + start := (page - 1) * 25 + if start > len(items) { + start = len(items) + } + end := start + 25 + if end > len(items) { + end = len(items) + } + response["items"] = items[start:end] + response["total"] = len(items) + response["page"] = page + } + renderJSON(w, 200, response) +} diff --git a/internal/api/production_dashboard_test.go b/internal/api/production_dashboard_test.go new file mode 100644 index 00000000..f95c2567 --- /dev/null +++ b/internal/api/production_dashboard_test.go @@ -0,0 +1,85 @@ +package api + +import ( + "droboticflow/internal/domain" + "droboticflow/internal/store" + "encoding/json" + "fmt" + "testing" +) + +func TestProductionDashboardScopeIsolationAndFullCorpus(t *testing.T) { + t.Setenv("DROBOTICFLOW_REQUIRE_SESSION_AUTH", "true") + ds := store.NewMemoryStore() + h := NewHandler(ds, nil) + root := bootstrapSuperAdminSession(t, h) + org := adminCreateOrganization(t, h, root, "Statistics", "stats") + admin := adminCreateUser(t, h, root, org.ID, "Admin", "stats-admin@example.com", domain.UserRoleAdmin, "test-password") + cookie := loginSession(t, h, admin.Phone, "test-password") + p, err := ds.CreateProject(t.Context(), domain.CreateProjectInput{OrganizationID: org.ID, Name: "Production"}) + if err != nil { + t.Fatal(err) + } + test, err := ds.CreateProject(t.Context(), domain.CreateProjectInput{OrganizationID: org.ID, Name: "Excluded"}) + if err != nil { + t.Fatal(err) + } + for n := 0; n < 251; n++ { + if err = ds.UpsertCaptureRecord(t.Context(), domain.CaptureRecord{ID: fmt.Sprint(n), OrganizationID: org.ID, ProjectID: p.ID, Prefix: fmt.Sprint(n), CaptureID: fmt.Sprint(n), Status: domain.CaptureRecordIncomplete}); err != nil { + t.Fatal(err) + } + } + if err = ds.UpsertCaptureRecord(t.Context(), domain.CaptureRecord{ID: "excluded", OrganizationID: org.ID, ProjectID: test.ID, Prefix: "excluded", CaptureID: "excluded"}); err != nil { + t.Fatal(err) + } + path := "/api/dashboard/production" + r := serveJSONWithCookie(h, "GET", path, nil, cookie) + if r.Code != 200 || !jsonContains(r.Body.String(), `"configured":false`) { + t.Fatalf("unconfigured: %d %s", r.Code, r.Body.String()) + } + r = serveJSONWithCookie(h, "PUT", path+"/scope", []byte(fmt.Sprintf(`{"projectIds":[%q],"revision":0}`, p.ID)), cookie) + if r.Code != 200 { + t.Fatalf("scope: %d %s", r.Code, r.Body.String()) + } + r = serveJSONWithCookie(h, "PUT", path+"/scope", []byte(`{"projectIds":[],"revision":0}`), cookie) + if r.Code != 409 { + t.Fatalf("concurrent scope overwrite: %d", r.Code) + } + r = serveJSONWithCookie(h, "GET", path+"?details=true&page=11", nil, cookie) + var got struct { + Total int `json:"total"` + Items []any `json:"items"` + Summary struct { + Totals map[string]int `json:"totals"` + } `json:"summary"` + } + if err = json.Unmarshal(r.Body.Bytes(), &got); err != nil { + t.Fatal(err) + } + if r.Code != 200 || got.Total != 251 || len(got.Items) != 1 || got.Summary.Totals["total"] != 251 || jsonContains(r.Body.String(), "Excluded") { + t.Fatalf("full scope and page: %d %s", r.Code, r.Body.String()) + } + r = serveJSONWithCookie(h, "GET", path+"?projectId="+test.ID, nil, cookie) + if r.Code != 400 { + t.Fatalf("excluded project allowed: %d", r.Code) + } + other := adminCreateOrganization(t, h, root, "Other", "stats-other") + otherAdmin := adminCreateUser(t, h, root, other.ID, "Other", "stats-other@example.com", domain.UserRoleAdmin, "test-password") + otherCookie := loginSession(t, h, otherAdmin.Phone, "test-password") + r = serveJSONWithCookie(h, "PUT", path+"/scope", []byte(fmt.Sprintf(`{"projectIds":[%q],"revision":0}`, p.ID)), otherCookie) + if r.Code != 400 { + t.Fatalf("cross tenant scope allowed: %d", r.Code) + } + supplier := domain.SupplierRecord{ID: "stats-vendor", OrganizationID: org.ID, Name: "Vendor", Status: "active"} + if err = ds.UpsertSupplier(t.Context(), supplier); err != nil { + t.Fatal(err) + } + manager := adminCreateUser(t, h, root, org.ID, "Vendor manager", "stats-vendor@example.com", domain.UserRoleSupplierManager, "test-password", supplier.ID) + mc := loginSession(t, h, manager.Phone, "test-password") + for _, url := range []string{path, path + "/scope"} { + r = serveJSONWithCookie(h, "GET", url, nil, mc) + if r.Code != 403 { + t.Fatalf("vendor accessed %s: %d", url, r.Code) + } + } +} diff --git a/internal/api/review_decision_test.go b/internal/api/review_decision_test.go index 61f4d970..ae3d5615 100644 --- a/internal/api/review_decision_test.go +++ b/internal/api/review_decision_test.go @@ -36,6 +36,14 @@ func TestReviewDecisionReceiptReplaysAfterQualityTaskLeavesReviewer(t *testing.T if first.Code != 201 { t.Fatalf("first decision %d %s", first.Code, first.Body.String()) } + history := f.request(http.MethodGet, "/api/human-items?pool=reviewed&projectId="+f.task.ProjectID, f.reviewer, nil) + var historyBody struct { + Items []domain.HumanItemRecord `json:"items"` + } + _ = json.Unmarshal(history.Body.Bytes(), &historyBody) + if history.Code != 200 || len(historyBody.Items) != 1 || historyBody.Items[0].ID != f.item.ID { + t.Fatalf("completed decision missing from own history: %d %s", history.Code, history.Body.String()) + } again := reviewSessionRequest(f, http.MethodPost, endpoint, f.reviewer, input, headers) if again.Code != 201 { t.Fatalf("lost response replay %d %s", again.Code, again.Body.String()) diff --git a/internal/api/review_history_test.go b/internal/api/review_history_test.go new file mode 100644 index 00000000..cadba84e --- /dev/null +++ b/internal/api/review_history_test.go @@ -0,0 +1,129 @@ +package api + +import ( + "droboticflow/internal/domain" + "encoding/json" + "net/http" + "strings" + "testing" +) + +func TestReviewedPoolIncludesOwnDecisionsAcrossStages(t *testing.T) { + for _, stage := range []string{"annotation_combined", "acceptance", "quality_review"} { + t.Run(stage, func(t *testing.T) { + f := newCombinedDraftFixture(t) + batch, _ := f.store.GetHumanBatch(f.ctx, f.item.BatchID) + batch.Stages = append(batch.Stages, domain.HumanStageDefinition{ID: "acceptance", Kind: domain.HumanStageAcceptance, RequiredRole: "acceptance_reviewer"}) + if err := f.store.UpsertHumanBatch(f.ctx, *batch); err != nil { + t.Fatal(err) + } + item := f.item + item.CurrentStageID = stage + item.Status = domain.HumanItemReady + item.AssignedUserID = "" + if err := f.store.UpsertHumanItem(f.ctx, item); err != nil { + t.Fatal(err) + } + list := func() []domain.HumanItemRecord { + res := f.request(http.MethodGet, "/api/human-items?pool=reviewed&projectId="+f.task.ProjectID, f.reviewer, nil) + if res.Code != 200 { + t.Fatalf("%d: %s", res.Code, res.Body.String()) + } + var body struct { + Items []domain.HumanItemRecord `json:"items"` + } + if err := json.Unmarshal(res.Body.Bytes(), &body); err != nil { + t.Fatal(err) + } + return body.Items + } + if len(list()) != 0 { + t.Fatal("unreviewed task leaked") + } + for _, event := range []domain.HumanEventRecord{ + {ID: "other", OrganizationID: item.OrganizationID, ItemID: item.ID, ActorUserID: "other-reviewer", EventType: "approved"}, + {ID: "draft", OrganizationID: item.OrganizationID, ItemID: item.ID, ActorUserID: f.reviewer.ID, EventType: "claimed"}, + } { + if err := f.store.AppendHumanEvent(f.ctx, event); err != nil { + t.Fatal(err) + } + } + if len(list()) != 0 { + t.Fatal("other reviewer or non-decision leaked") + } + for _, decision := range []string{"approved", "rejected"} { + if err := f.store.AppendHumanEvent(f.ctx, domain.HumanEventRecord{ID: decision, OrganizationID: item.OrganizationID, ItemID: item.ID, ActorUserID: f.reviewer.ID, EventType: decision}); err != nil { + t.Fatal(err) + } + } + // The companion task list and read-only detail must also permit history. + res := f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks?projectId="+f.task.ProjectID, f.reviewer, nil) + if res.Code != 200 || !strings.Contains(res.Body.String(), f.task.ID) { + t.Fatalf("task history missing: %d %s", res.Code, res.Body.String()) + } + res = f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks/"+f.task.ID+"?view=summary", f.reviewer, nil) + if res.Code != 200 { + t.Fatalf("history detail denied: %d %s", res.Code, res.Body.String()) + } + item.PendingCombinedSubmissionID = "history-must-not-recover" + if err := f.store.UpsertHumanItem(f.ctx, item); err != nil { + t.Fatal(err) + } + res = f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks/"+f.task.ID+"?view=history", f.reviewer, nil) + if res.Code != 200 { + t.Fatalf("full history denied: %d %s", res.Code, res.Body.String()) + } + unchanged, _ := f.store.GetHumanItem(f.ctx, item.ID) + if unchanged.PendingCombinedSubmissionID != item.PendingCombinedSubmissionID { + t.Fatal("history read mutated task") + } + items := list() + if len(items) != 1 || items[0].ID != item.ID { + t.Fatalf("own history missing or duplicated: %v", items) + } + f.reviewer.Status = domain.UserStatusDisabled + if err := f.store.UpsertUser(f.ctx, f.reviewer); err != nil { + t.Fatal(err) + } + if len(list()) != 0 { + t.Fatal("revoked access leaked") + } + }) + } +} + +func TestReviewedPoolProjectsAggregateAcceptanceWithoutChangingSource(t *testing.T) { + f := newCombinedDraftFixture(t) + item := f.item + item.CurrentStageID = "quality_review" + item.Status = domain.HumanItemAccepted + item.AssignedUserID = "" + if err := f.store.UpsertHumanItem(f.ctx, item); err != nil { + t.Fatal(err) + } + if err := f.store.AppendHumanEvent(f.ctx, domain.HumanEventRecord{ID: "approved", OrganizationID: item.OrganizationID, ItemID: item.ID, ActorUserID: f.reviewer.ID, EventType: "approved"}); err != nil { + t.Fatal(err) + } + check := func(want string) { + res := f.request(http.MethodGet, "/api/human-items?pool=reviewed&projectId="+f.task.ProjectID, f.reviewer, nil) + var body struct { + Items []domain.HumanItemRecord `json:"items"` + } + _ = json.Unmarshal(res.Body.Bytes(), &body) + if res.Code != 200 || len(body.Items) != 1 || body.Items[0].ReviewHistoryStatus != want { + t.Fatalf("want %s: %d %s", want, res.Code, res.Body.String()) + } + source, _ := f.store.GetHumanItem(f.ctx, item.ID) + if source.ReviewHistoryStatus != "" || source.Status != domain.HumanItemAccepted { + t.Fatal("read projection modified source") + } + } + check("pending_acceptance") + downstream := item + downstream.ID = fanyuV5PrePackageAcceptanceItemID(item.BatchID) + downstream.CurrentStageID = "prepackage_acceptance" + if err := f.store.UpsertHumanItem(f.ctx, downstream); err != nil { + t.Fatal(err) + } + check("accepted") +} diff --git a/internal/api/server.go b/internal/api/server.go index 3981e90d..07cf7b0f 100644 --- a/internal/api/server.go +++ b/internal/api/server.go @@ -1334,6 +1334,9 @@ func NewHandlerWithConfig(dataStore store.Store, aiService ai.Service, appConfig }) router.With(instance.requireAdminMonitoringAccess).Get("/api/acceptance/poc-readiness", instance.handlePocAcceptanceReadiness) router.Get("/api/dashboard", instance.handleDashboard) + router.Get("/api/dashboard/production", instance.handleProductionDashboard) + router.Get("/api/dashboard/production/scope", instance.handleProductionScope) + router.Put("/api/dashboard/production/scope", instance.handleProductionScope) router.Get("/api/plugins", instance.handlePluginsList) router.Get("/api/organizations/{id}/plugins/{pluginId}", instance.handleOrganizationPluginGet) router.Group(func(router chi.Router) { diff --git a/internal/domain/human_loop.go b/internal/domain/human_loop.go index 5e4f0cca..37806f4f 100644 --- a/internal/domain/human_loop.go +++ b/internal/domain/human_loop.go @@ -133,7 +133,8 @@ type HumanBatchRecord struct { } type HumanItemRecord struct { - ReviewOperationID string `json:"reviewOperationId,omitempty" bson:"reviewOperationId,omitempty"` + ReviewHistoryStatus string `json:"reviewHistoryStatus,omitempty" bson:"-"` + ReviewOperationID string `json:"reviewOperationId,omitempty" bson:"reviewOperationId,omitempty"` ProcessingJob *AnnotationProcessingJob `json:"-" bson:"processingJob,omitempty"` ProcessingScope string `json:"processingScope,omitempty" bson:"processingScope,omitempty"` ProcessingProfile string `json:"processingProfile,omitempty" bson:"processingProfile,omitempty"` @@ -288,6 +289,8 @@ type HumanBatchQuery struct { } type HumanItemQuery struct { + // Server-populated review history scope; nil means unrestricted. + IDs []string LegacyTaskID string // Server-populated only; limits mine queries before loading heavy payloads. ActorUserID string diff --git a/internal/productionstats/current.go b/internal/productionstats/current.go new file mode 100644 index 00000000..78f1df14 --- /dev/null +++ b/internal/productionstats/current.go @@ -0,0 +1,448 @@ +package productionstats + +import ( + "crypto/sha256" + "droboticflow/internal/domain" + "fmt" + "path" + "sort" + "strings" + "time" +) + +func sourceKey(c domain.CaptureRecord) string { + prefix := c.SourcePrefix + if prefix == "" { + prefix = c.Prefix + } + // Keep storage namespace and full directory; never deduplicate by display name. + if prefix == "" { + prefix = "missing:" + c.ID + } + return fmt.Sprintf("%x", sha256.Sum256([]byte(strings.Join([]string{c.ProjectID, c.ConnectionID, c.Bucket, strings.TrimRight(prefix, "/")}, "\x00")))) +} +func partKey(c domain.CaptureRecord) string { c.SourcePrefix = c.Prefix; return sourceKey(c) } +func mergeStage(a, b string) string { + if a == "unknown" || b == "unknown" { + return "unknown" + } + if rank(b) < rank(a) { + return b + } + return a +} +func stageForItem(i domain.HumanItemRecord, batches map[string]domain.HumanBatchRecord) string { + if i.Status == domain.HumanItemCancelled { + return "ended" + } + if i.Status == domain.HumanItemAccepted { + return "delivery" + } + if i.PendingCombinedSubmissionID != "" || (i.ProcessingJob != nil && (i.ProcessingJob.Status == "queued" || i.ProcessingJob.Status == "running" || i.ProcessingJob.Status == "failed")) { + return "annotation" + } + for _, s := range batches[i.BatchID].Stages { + if s.ID == i.CurrentStageID { + switch s.Kind { + case domain.HumanStageAnnotation: + return "annotation" + case domain.HumanStageReview: + return "quality" + case domain.HumanStageAcceptance: + return "acceptance" + } + } + } + switch i.CurrentStageID { + case "annotation_combined", "split_annotation", "cut_adjustment", "semantic_annotation": + return "annotation" + case "quality_review", "cut_review", "semantic_review": + return "quality" + case "acceptance", "final_acceptance", "prepackage_acceptance": + return "acceptance" + } + return "unknown" +} +func rank(s string) int { + for n, x := range Stages { + if x == s { + return n + } + } + return 7 +} + +func Build(d Snapshot) Report { + result := Report{Sources: []Source{}, Events: []Event{}} + captures := map[string][]domain.CaptureRecord{} + parents := map[string]string{} + parentNames := map[string]string{} + captureSource := map[string]string{} + runSource := map[string]string{} + runs := map[string]domain.RunRecord{} + batches := map[string]domain.HumanBatchRecord{} + for _, r := range d.Runs { + runs[r.ID] = r + } + for _, b := range d.Batches { + batches[b.ID] = b + } + for _, c := range d.Captures { + key := partKey(c) + parents[key] = sourceKey(c) + name := c.CaptureID + if c.SourcePrefix != "" && strings.TrimRight(c.SourcePrefix, "/") != strings.TrimRight(c.Prefix, "/") { + name = path.Base(strings.TrimRight(c.SourcePrefix, "/")) + } + parentNames[sourceKey(c)] = name + captures[key] = append(captures[key], c) + captureSource[c.ID] = key + if c.RunID != "" { + runSource[c.RunID] = key + } + } + for _, r := range d.Runs { + if key := captureSource[r.TaskID]; key != "" { + runSource[r.ID] = key + } + } + // Human-release packaging children belong to their parent's attempt. Retries don't. + for changed := true; changed; { + changed = false + for _, r := range d.Runs { + if runSource[r.ID] == "" && r.Trigger == domain.RunTriggerHumanRelease && runSource[r.ParentRunID] != "" { + runSource[r.ID] = runSource[r.ParentRunID] + changed = true + } + } + } + root := func(id string) string { + seen := map[string]bool{} + for !seen[id] { + seen[id] = true + r, ok := runs[id] + if !ok || r.Trigger != domain.RunTriggerHumanRelease || r.ParentRunID == "" { + break + } + id = r.ParentRunID + } + return id + } + latest := map[string]domain.RunRecord{} + for _, r := range d.Runs { + key := runSource[r.ID] + if key == "" || r.Mode == domain.RunModeDebug || r.Trigger == domain.RunTriggerHumanRelease { + continue + } + old := latest[key] + if old.ID == "" || later(r.CreatedAt, old.CreatedAt) || (r.CreatedAt == old.CreatedAt && r.ID > old.ID) { + latest[key] = r + } + } + itemSource := map[string]string{} + taskSource := map[string]string{} + for _, i := range d.Items { + if runs[i.RunID].Mode != domain.RunModeDebug { + itemSource[i.ID] = runSource[i.RunID] + if i.LegacyTaskID != "" { + taskSource[i.LegacyTaskID] = runSource[i.RunID] + } + } + } + for _, t := range d.Tasks { + if runs[t.RunID].Mode != domain.RunModeDebug { + taskSource[t.ID] = runSource[t.RunID] + } + } + attemptItems := map[string][]domain.HumanItemRecord{} + attemptTasks := map[string][]domain.AnnotationTaskRecord{} + represented := map[string]bool{} + for _, item := range d.Items { + attemptItems[root(item.RunID)] = append(attemptItems[root(item.RunID)], item) + if item.LegacyTaskID != "" { + represented[item.LegacyTaskID] = true + } + } + for _, task := range d.Tasks { + attemptTasks[root(task.RunID)] = append(attemptTasks[root(task.RunID)], task) + } + itemPartitions := map[string]string{} + for _, item := range d.Items { + itemPartitions[item.ID] = item.ReleasePartitionID + } + supersededAcceptance := map[string]bool{} + for _, p := range d.Packages { + if supersededPackage(p, itemPartitions) { + supersededAcceptance[p.AcceptanceHumanItemID] = true + } + } + attemptPackages := map[string][]domain.DeliveryPackageRecord{} + for _, p := range d.Packages { + rid := root(p.ParentRunID) + if rid == "" { + rid = root(p.ChildRunID) + } + attemptPackages[rid] = append(attemptPackages[rid], p) + } + for key, cs := range captures { + c := cs[0] + r := latest[key] + s := Source{ID: key, ProjectID: c.ProjectID, Name: c.CaptureID, RunID: r.ID, SupplierID: Unassigned, Stage: "intake", Detail: string(c.Status), Tasks: []Task{}} + for _, candidate := range cs { + if candidate.DiscoveredAt != "" && (s.DiscoveredAt == "" || later(s.DiscoveredAt, candidate.DiscoveredAt)) { + s.DiscoveredAt = candidate.DiscoveredAt + } + if root(candidate.RunID) == r.ID { + c = candidate + } + } + if s.Name == "" { + s.Name = c.Prefix + } + s.Detail = string(c.Status) + if r.ID == "" && c.RunID != "" && runs[c.RunID].Mode != domain.RunModeDebug { + s.Stage = "unknown" + s.Detail = "缺少正式处理记录" + } + s.WorkflowID = r.WorkflowID + s.Runs = []RunReference{} + if r.ID != "" { + s.Runs = append(s.Runs, RunReference{r.ID, r.WorkflowID, c.CaptureID}) + } + suppliers := map[string]bool{} + hasTasks := false + minStage := "ended" + for _, i := range attemptItems[r.ID] { + if supersededAcceptance[i.ID] { + continue + } + if itemSource[i.ID] != key || root(i.RunID) != r.ID || r.ID == "" { + continue + } + hasTasks = true + stage := stageForItem(i, batches) + s.Tasks = append(s.Tasks, Task{i.ID, stage, string(i.Status)}) + minStage = mergeStage(minStage, stage) + if batches[i.BatchID].SourceBatchID == "" { + supplier := i.SupplierID + if supplier == "" { + supplier = Unassigned + } + suppliers[supplier] = true + } + if i.Status == domain.HumanItemEscalated || i.Status == domain.HumanItemRejected || (i.ProcessingJob != nil && i.ProcessingJob.Status == "failed") { + s.Exception = true + } + } + + for _, task := range attemptTasks[r.ID] { + if represented[task.ID] || taskSource[task.ID] != key || root(task.RunID) != r.ID || r.ID == "" { + continue + } + stage := "unknown" + switch task.Status { + case domain.AnnotationTaskStatusPendingAnnotation, domain.AnnotationTaskStatusRework: + stage = "annotation" + case domain.AnnotationTaskStatusPendingQuality: + stage = "quality" + case domain.AnnotationTaskStatusPendingAcceptance: + stage = "acceptance" + case domain.AnnotationTaskStatusAccepted: + stage = "delivery" + case domain.AnnotationTaskStatusRejected: + s.Exception = true + } + hasTasks = true + s.Tasks = append(s.Tasks, Task{task.ID, stage, string(task.Status)}) + minStage = mergeStage(minStage, stage) + supplier := task.SupplierID + if supplier == "" { + supplier = Unassigned + } + suppliers[supplier] = true + } + if len(suppliers) == 1 { + for supplier := range suppliers { + s.SupplierID = supplier + } + } else if len(suppliers) > 1 || (hasTasks && len(suppliers) == 0) { + s.SupplierID = Unknown + s.Exception = true + } + if hasTasks { + s.Stage = minStage + s.Detail = minStage + } else if r.ID != "" { + s.Detail = string(r.Status) + switch r.Status { + case domain.RunStatusCompleted: + s.Stage = "unknown" + case domain.RunStatusCancelled: + s.Stage = "ended" + } + } + if root(c.RunID) == r.ID || r.ID == "" { + switch c.BusinessStage { + case domain.CaptureStageDelivered: + if !hasTasks || minStage == "delivery" { + s.Stage = "delivered" + } + case domain.CaptureStageNoDeliverables, domain.CaptureStageDiscarded, domain.CaptureStageCancelled: + s.Stage = "ended" + } + } + + packageStage := "ended" + packageCount := 0 + for _, p := range attemptPackages[r.ID] { + if p.Mode == domain.RunModeDebug || p.Temporary || r.ID == "" || (root(p.ParentRunID) != r.ID && root(p.ChildRunID) != r.ID) { + continue + } + if supersededPackage(p, itemPartitions) { + continue + } + packageCount++ + stage := "unknown" + switch p.Status { + case domain.DeliveryPackagePendingAcceptance, domain.DeliveryPackageRejected: + stage = "acceptance" + case domain.DeliveryPackageBuilding, domain.DeliveryPackageFailed: + stage = "delivery" + case domain.DeliveryPackageAccepted: + stage = "delivered" + case domain.DeliveryPackageNoDeliverables: + stage = "ended" + } + if p.Status == domain.DeliveryPackageFailed || p.Status == domain.DeliveryPackageRejected { + s.Exception = true + } + packageStage = mergeStage(packageStage, stage) + } + if packageCount > 0 && (!hasTasks || minStage == "delivery") { + s.Stage = packageStage + s.Detail = packageStage + } + if (root(c.RunID) == r.ID || c.RunID == "") && (c.Status == domain.CaptureRecordDiscarded || c.Status == domain.CaptureRecordCancelled) { + s.Stage = "ended" + s.Detail = string(c.Status) + } + if r.DeliveryOutcome == "no_deliverables" { + s.Stage = "ended" + s.Detail = "no_deliverables" + } + if r.Status == domain.RunStatusFailed || (root(c.RunID) == r.ID && c.Status == domain.CaptureRecordFailed) { + s.Exception = true + } + if s.SupplierID == Unknown || s.Stage == "unknown" { + result.Incomplete = true + } + sort.Slice(s.Tasks, func(i, j int) bool { return s.Tasks[i].ID < s.Tasks[j].ID }) + result.Sources = append(result.Sources, s) + } + sort.Slice(result.Sources, func(i, j int) bool { return result.Sources[i].ID < result.Sources[j].ID }) + buildEvents(d, &result, runSource, itemSource, taskSource, runs) + for n := range result.Events { + result.Events[n].AttemptID = root(result.Events[n].RunID) + } + for _, run := range d.Runs { + if run.Mode != domain.RunModeDebug && run.Trigger != domain.RunTriggerHumanRelease && runSource[run.ID] != "" { + result.Events = append(result.Events, Event{ID: run.ID, SourceID: runSource[run.ID], Metric: "attempt_started", At: run.CreatedAt, AttemptID: run.ID}) + } + } + // An unadmitted parent may be replaced by discovered part directories. + // Preserve its registration date, but it is no longer required production work. + hasParts := map[string]bool{} + for part, parent := range parents { + if part != parent { + hasParts[parent] = true + } + } + placeholders := map[string]bool{} + for key, cs := range captures { + if !hasParts[key] || latest[key].ID != "" { + continue + } + unadmitted := true + for _, c := range cs { + if c.RunID != "" || c.Status == domain.CaptureRecordDiscarded || c.Status == domain.CaptureRecordLaunching { + unadmitted = false + } + } + placeholders[key] = unadmitted + } + return rollupParts(result, parents, parentNames, placeholders, requiredMilestoneUnits(d, result.Events, root)) +} + +func later(a, b string) bool { + left, e1 := time.Parse(time.RFC3339Nano, a) + right, e2 := time.Parse(time.RFC3339Nano, b) + return e1 == nil && (e2 != nil || left.After(right)) +} + +// A physical Capture may register several part directories. Retries are resolved +// per part first; completion of one part must not hide unfinished sibling work. +func rollupParts(parts Report, parents, names map[string]string, placeholders map[string]bool, units milestoneUnits) Report { + out := Report{Sources: []Source{}, Events: parts.Events, Incomplete: parts.Incomplete} + combined := map[string]*Source{} + discovery := map[string]string{} + for _, part := range parts.Sources { + id := parents[part.ID] + if part.DiscoveredAt != "" && (discovery[id] == "" || later(discovery[id], part.DiscoveredAt)) { + discovery[id] = part.DiscoveredAt + } + if placeholders[part.ID] { + continue + } + s := combined[id] + if s == nil { + copy := part + copy.ID = id + if names[id] != "" { + copy.Name = names[id] + } + combined[id] = © + continue + } + s.Stage = mergeStage(s.Stage, part.Stage) + s.Detail = s.Stage + s.Exception = s.Exception || part.Exception + if s.SupplierID != part.SupplierID { + s.SupplierID = Unknown + s.Exception = true + out.Incomplete = true + } + if s.DiscoveredAt == "" || later(s.DiscoveredAt, part.DiscoveredAt) { + s.DiscoveredAt = part.DiscoveredAt + } + s.Tasks = append(s.Tasks, part.Tasks...) + s.Runs = append(s.Runs, part.Runs...) + s.RunID = "" + s.WorkflowID = "" + } + for _, s := range combined { + s.DiscoveredAt = discovery[s.ID] + out.Sources = append(out.Sources, *s) + } + sort.Slice(out.Sources, func(i, j int) bool { return out.Sources[i].ID < out.Sources[j].ID }) + out.Events = sourceMilestones(parts.Events, parents, placeholders, units) + for _, s := range out.Sources { + if s.DiscoveredAt != "" { + out.Events = append(out.Events, Event{ID: "new:" + s.ID, SourceID: s.ID, SupplierID: Unassigned, Metric: "new", At: s.DiscoveredAt}) + } + } + return out +} + +// Rework preserves the old package for audit but moves its production items to +// a new partition. Only explicit current pointers justify excluding it. +func supersededPackage(p domain.DeliveryPackageRecord, itemPartitions map[string]string) bool { + if p.ReleasePartitionID == "" || len(p.ItemIDs) == 0 { + return false + } + for _, id := range p.ItemIDs { + if itemPartitions[id] == "" || itemPartitions[id] == p.ReleasePartitionID { + return false + } + } + return true +} diff --git a/internal/productionstats/current_test.go b/internal/productionstats/current_test.go new file mode 100644 index 00000000..79b03651 --- /dev/null +++ b/internal/productionstats/current_test.go @@ -0,0 +1,51 @@ +package productionstats + +import ( + "droboticflow/internal/domain" + "fmt" + "testing" +) + +func TestSourcesDeduplicateWithinProjectAndSelectFormalAttempt(t *testing.T) { + data := Snapshot{Captures: []domain.CaptureRecord{ + {ID: "a", ProjectID: "p", ConnectionID: "bos", Bucket: "b", Prefix: "one/c", CaptureID: "c", RunID: "old", DiscoveredAt: "2026-09-01T00:00:00Z"}, + {ID: "b", ProjectID: "p", ConnectionID: "bos", Bucket: "b", Prefix: "one/c/", CaptureID: "c", RunID: "new"}, + {ID: "c", ProjectID: "p", ConnectionID: "bos", Bucket: "b", Prefix: "two/c", CaptureID: "c"}, + {ID: "d", ProjectID: "q", ConnectionID: "bos", Bucket: "b", Prefix: "one/c", CaptureID: "c"}, + }, Runs: []domain.RunRecord{ + {ID: "old", TaskID: "a", ProjectID: "p", CreatedAt: "2026-09-01T00:00:00Z", Status: domain.RunStatusCompleted}, + {ID: "new", TaskID: "b", ProjectID: "p", CreatedAt: "2026-09-02T00:00:00Z"}, + {ID: "debug", TaskID: "b", ProjectID: "p", CreatedAt: "2026-09-03T00:00:00Z", Mode: domain.RunModeDebug}, + }, Items: []domain.HumanItemRecord{ + {ID: "i", ProjectID: "p", RunID: "new", SupplierID: "s", CurrentStageID: domain.FanyuV58StageAnnotationCombined, Status: domain.HumanItemReady}, + }} + result := Build(data) + if len(result.Sources) != 3 { + t.Fatalf("want 3 sources, got %+v", result.Sources) + } + for _, s := range result.Sources { + if s.ProjectID == "p" && s.Name == "c" && s.RunID == "new" { + if s.Stage != "annotation" || s.SupplierID != "s" { + t.Fatalf("latest formal attempt: %+v", s) + } + return + } + } + t.Fatal("latest formal source absent") +} +func BenchmarkBuildSources(b *testing.B) { + data := Snapshot{} + for n := 0; n < 10000; n++ { + id := fmt.Sprint(n) + data.Captures = append(data.Captures, domain.CaptureRecord{ID: id, ProjectID: "p", Prefix: id, RunID: id}) + data.Runs = append(data.Runs, domain.RunRecord{ID: id, TaskID: id, CreatedAt: "2026-09-01T00:00:00Z"}) + data.Items = append(data.Items, domain.HumanItemRecord{ID: id, RunID: id, LegacyTaskID: id, CurrentStageID: "quality_review", SupplierID: "s"}) + data.Packages = append(data.Packages, domain.DeliveryPackageRecord{ID: id, ParentRunID: id, Status: domain.DeliveryPackageAccepted}) + data.Worklogs = append(data.Worklogs, domain.AnnotationWorklogEventRecord{ID: id, TaskID: id, SupplierID: "s", EventType: "quality_approved", CreatedAt: "2026-09-01T01:00:00Z"}) + data.Events = append(data.Events, domain.HumanEventRecord{ID: id, ItemID: id, StageID: "prepackage_acceptance", EventType: "approved", Payload: map[string]any{"sourceItemIds": []string{id}}, CreatedAt: "2026-09-01T02:00:00Z"}) + } + b.ResetTimer() + for n := 0; n < b.N; n++ { + Build(data) + } +} diff --git a/internal/productionstats/events.go b/internal/productionstats/events.go new file mode 100644 index 00000000..ca6bb27c --- /dev/null +++ b/internal/productionstats/events.go @@ -0,0 +1,271 @@ +package productionstats + +import ( + "droboticflow/internal/domain" + "reflect" + "sort" + "strings" + "time" +) + +type ownershipFact struct { + at time.Time + supplier string +} +type eventIndex struct { + itemTasks map[string]string + owners map[string][]ownershipFact +} + +func indexEvents(d Snapshot) eventIndex { + index := eventIndex{map[string]string{}, map[string][]ownershipFact{}} + for _, i := range d.Items { + index.itemTasks[i.ID] = i.LegacyTaskID + } + for _, e := range d.Worklogs { + at, err := time.Parse(time.RFC3339Nano, e.CreatedAt) + if err == nil && e.Mode != domain.RunModeDebug && !e.Temporary { + index.owners[e.TaskID] = append(index.owners[e.TaskID], ownershipFact{at, e.SupplierID}) + } + } + for task, facts := range index.owners { + sort.Slice(facts, func(i, j int) bool { return facts[i].at.Before(facts[j].at) }) + index.owners[task] = facts + } + return index +} +func worklogMetric(kind string) string { + switch kind { + case "annotation_submitted", "combined_submitted", "rework_submitted": + return "submitted" + case "quality_approved", "quality_review_approved", "split_review_approved", "cut_review_approved", "semantic_review_approved": + return "quality" + case "acceptance_approved": + return "acceptance" + case "quality_rejected", "quality_review_rejected", "split_review_rejected", "cut_review_rejected", "semantic_review_rejected", "acceptance_rejected": + return "rejected" + } + return "" +} +func buildEvents(d Snapshot, out *Report, runSource, itemSource, taskSource map[string]string, runs map[string]domain.RunRecord) { + index := indexEvents(d) + taskRuns := map[string]string{} + itemRuns := map[string]string{} + for _, i := range d.Items { + itemRuns[i.ID] = i.RunID + if i.LegacyTaskID != "" { + taskRuns[i.LegacyTaskID] = i.RunID + } + } + for _, t := range d.Tasks { + taskRuns[t.ID] = t.RunID + } + // Package acceptance worklogs use package IDs rather than annotation-task IDs. + packageReviews := map[string][]domain.DeliveryPackageAcceptanceReview{} + packageOwners := map[string]string{} + packageUnits := map[string][]string{} + firstReview := map[string]string{} + for _, p := range d.Packages { + if p.Mode == domain.RunModeDebug || p.Temporary { + continue + } + packageOwners[p.ID] = p.SupplierID + packageUnits[p.ID] = index.productionTasks(p.ItemIDs) + if _, copied := p.QualitySummary["prePackageAcceptance"]; copied { + continue + } + for _, review := range p.AcceptanceReviews { + if worklogMetric("acceptance_"+string(review.Decision)) == "" || review.ID == "" { + continue + } + if _, err := time.Parse(time.RFC3339Nano, review.CreatedAt); err != nil { + continue + } + packageReviews[p.ID] = append(packageReviews[p.ID], review) + if firstReview[p.ID] == "" || later(firstReview[p.ID], review.CreatedAt) { + firstReview[p.ID] = review.CreatedAt + } + } + } + coveredByReview := func(pkg, at string) bool { return firstReview[pkg] != "" && !later(firstReview[pkg], at) } + // Legacy streams have no common decision identifier. Prefer HumanEvents for + // a package/metric when canonical reviews are absent, rather than adding + // potentially duplicate worklogs. Incomplete historical evidence stays marked. + legacyHuman := map[string]bool{} + for _, e := range d.Events { + pkg, _ := e.Payload["deliveryPackageId"].(string) + if pkg != "" && itemSource[e.ItemID] != "" && !coveredByReview(pkg, e.CreatedAt) && acceptanceStage(e.StageID) { + legacyHuman[pkg+":"+worklogMetric("acceptance_"+e.EventType)] = true + } + } + for _, p := range d.Packages { + if p.Mode == domain.RunModeDebug || p.Temporary { + continue + } + rid := p.ChildRunID + if runSource[rid] == "" { + rid = p.ParentRunID + } + taskSource[p.ID] = runSource[rid] + taskRuns[p.ID] = rid + } + for _, e := range d.Worklogs { + key := taskSource[e.TaskID] + metric := worklogMetric(e.EventType) + if key == "" || metric == "" || e.Mode == domain.RunModeDebug || e.Temporary { + continue + } + if _, pkg := packageOwners[e.TaskID]; pkg && (e.EventType == "acceptance_approved" || e.EventType == "acceptance_rejected") { + if coveredByReview(e.TaskID, e.CreatedAt) || legacyHuman[e.TaskID+":"+metric] { + continue + } + } + supplier := e.SupplierID + if supplier == "" { + supplier = Unknown + out.Incomplete = true + } + rid := taskRuns[e.TaskID] + productionUnits := []string{e.TaskID} + if _, pkg := packageOwners[e.TaskID]; pkg { + productionUnits = packageUnits[e.TaskID] + } + out.Events = append(out.Events, Event{ID: e.ID, SourceID: key, SupplierID: supplier, Metric: metric, At: e.CreatedAt, RunID: rid, WorkflowID: runs[rid].WorkflowID, TaskID: e.TaskID, InvalidateFrom: rejectionReturn(e.EventType), ProductionUnits: productionUnits}) + } + for _, e := range d.Events { + key := itemSource[e.ItemID] + if key == "" { + continue + } + if !acceptanceStage(e.StageID) { + continue + } + metric := "" + if e.EventType == "approved" { + metric = "acceptance" + } + if e.EventType == "rejected" { + metric = "rejected" + } + if metric == "" { + continue + } + if pkg, _ := e.Payload["deliveryPackageId"].(string); pkg != "" && coveredByReview(pkg, e.CreatedAt) { + continue + } + supplier := index.acceptanceSupplier(e) + if pkg, _ := e.Payload["deliveryPackageId"].(string); pkg != "" && packageOwners[pkg] != "" { + supplier = packageOwners[pkg] + } + rid := itemRuns[e.ItemID] + unit := e.ItemID + productionUnits := index.productionTasks(e.Payload["sourceItemIds"]) + if pkg, _ := e.Payload["deliveryPackageId"].(string); pkg != "" { + unit = pkg + productionUnits = packageUnits[pkg] + } + out.Events = append(out.Events, Event{ID: e.ID, SourceID: key, SupplierID: supplier, Metric: metric, At: e.CreatedAt, RunID: rid, WorkflowID: runs[rid].WorkflowID, TaskID: unit, InvalidateFrom: "quality", ProductionUnits: productionUnits}) + if supplier == Unknown { + out.Incomplete = true + } + } + for pkg, reviews := range packageReviews { + key := taskSource[pkg] + if key == "" { + continue + } + supplier := packageOwners[pkg] + if supplier == "" { + supplier = Unknown + } + rid := taskRuns[pkg] + for _, review := range reviews { + out.Events = append(out.Events, Event{ID: review.ID, SourceID: key, SupplierID: supplier, Metric: worklogMetric("acceptance_" + string(review.Decision)), At: review.CreatedAt, RunID: rid, WorkflowID: runs[rid].WorkflowID, TaskID: pkg, InvalidateFrom: "quality", ProductionUnits: packageUnits[pkg]}) + } + } + for _, p := range d.Packages { + key := taskSource[p.ID] + if key == "" || p.Mode == domain.RunModeDebug || p.Temporary || p.FinalExportedAt == "" { + continue + } + supplier := p.SupplierID + if supplier == "" { + supplier = Unknown + } + rid := taskRuns[p.ID] + out.Events = append(out.Events, Event{ID: "delivery:" + p.ID, SourceID: key, SupplierID: supplier, Metric: "delivered", At: p.FinalExportedAt, RunID: rid, WorkflowID: runs[rid].WorkflowID, TaskID: p.ID, ProductionUnits: packageUnits[p.ID]}) + } + // Older versions did not persist all transitions; reliable facts are a lower + // bound, never evidence that an absent event means zero actual production. + if len(out.Sources) > 0 { + out.Incomplete = true + } +} + +// Decisions reference source items; their earlier worklog snapshots carry the +// production owner. A reviewer account or today's task owner is not evidence. +func (index eventIndex) acceptanceSupplier(event domain.HumanEventRecord) string { + ids := reflect.ValueOf(event.Payload["sourceItemIds"]) + if !ids.IsValid() || (ids.Kind() != reflect.Slice && ids.Kind() != reflect.Array) { + return Unknown + } + at, err := time.Parse(time.RFC3339Nano, event.CreatedAt) + if err != nil { + return Unknown + } + owners := map[string]bool{} + for n := 0; n < ids.Len(); n++ { + id, ok := ids.Index(n).Interface().(string) + if !ok { + return Unknown + } + task := index.itemTasks[id] + facts := index.owners[task] + last := sort.Search(len(facts), func(i int) bool { return facts[i].at.After(at) }) - 1 + if task == "" || last < 0 || facts[last].supplier == "" { + return Unknown + } + owner := facts[last].supplier + for j := last - 1; j >= 0 && facts[j].at.Equal(facts[last].at); j-- { + if facts[j].supplier != owner { + return Unknown + } + } + owners[owner] = true + } + if len(owners) != 1 { + return Unknown + } + for owner := range owners { + return owner + } + return Unknown +} + +func acceptanceStage(stage string) bool { + return stage == "acceptance" || stage == "final_acceptance" || stage == "prepackage_acceptance" +} + +// Quality rejection reopens annotation; acceptance rejection reopens quality. +func rejectionReturn(kind string) string { + if strings.HasPrefix(kind, "acceptance_") { + return "quality" + } + return "submitted" +} + +func (index eventIndex) productionTasks(value any) []string { + ids := reflect.ValueOf(value) + if !ids.IsValid() || (ids.Kind() != reflect.Slice && ids.Kind() != reflect.Array) { + return nil + } + tasks := []string{} + for n := 0; n < ids.Len(); n++ { + id, ok := ids.Index(n).Interface().(string) + if !ok || index.itemTasks[id] == "" { + return nil + } + tasks = append(tasks, index.itemTasks[id]) + } + return tasks +} diff --git a/internal/productionstats/events_test.go b/internal/productionstats/events_test.go new file mode 100644 index 00000000..16e10229 --- /dev/null +++ b/internal/productionstats/events_test.go @@ -0,0 +1,65 @@ +package productionstats + +import ( + "droboticflow/internal/domain" + "testing" +) + +func TestPrepackageAcceptanceUsesHistoricalProductionOwner(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c"}}, Items: []domain.HumanItemRecord{{ID: "production", RunID: "r", LegacyTaskID: "t", SupplierID: "new-owner"}, {ID: "review", RunID: "r", CurrentStageID: "prepackage_acceptance"}}, Worklogs: []domain.AnnotationWorklogEventRecord{{ID: "qc", TaskID: "t", SupplierID: "original", EventType: "quality_approved", CreatedAt: "2026-09-25T00:00:00Z"}}, Events: []domain.HumanEventRecord{{ID: "accept", ItemID: "review", StageID: "prepackage_acceptance", EventType: "approved", Payload: map[string]any{"sourceItemIds": []string{"production"}}, CreatedAt: "2026-09-26T00:00:00Z"}}} + r := Build(d) + for _, e := range r.Events { + if e.Metric == "acceptance" { + if e.SupplierID != "original" { + t.Fatalf("event-time ownership: %+v", e) + } + return + } + } + t.Fatal("missing final acceptance event") +} +func TestLegacyReviewAndPackageAcceptanceFacts(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c", WorkflowID: "w"}}, Tasks: []domain.AnnotationTaskRecord{{ID: "task", RunID: "r"}}, Packages: []domain.DeliveryPackageRecord{{ID: "pkg", ParentRunID: "r"}}, Worklogs: []domain.AnnotationWorklogEventRecord{{ID: "q", TaskID: "task", SupplierID: "supplier", EventType: "quality_review_approved", CreatedAt: "2026-09-26T00:00:00Z"}, {ID: "a", TaskID: "pkg", SupplierID: "supplier", EventType: "acceptance_approved", CreatedAt: "2026-09-26T01:00:00Z"}}} + r := Build(d) + seen := map[string]bool{} + for _, e := range r.Events { + if e.Metric == "quality" || e.Metric == "acceptance" { + if e.SupplierID != "supplier" || e.RunID != "r" { + t.Fatalf("legacy provenance lost: %+v", e) + } + seen[e.Metric] = true + } + } + if !seen["quality"] || !seen["acceptance"] { + t.Fatalf("legacy facts missing: %+v", r.Events) + } +} + +func TestPackageDecisionStreamsPreserveEarlierRejection(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c"}}, Items: []domain.HumanItemRecord{{ID: "review", RunID: "r"}}, Packages: []domain.DeliveryPackageRecord{{ID: "pkg", ParentRunID: "r", SupplierID: "v"}}, Worklogs: []domain.AnnotationWorklogEventRecord{{ID: "later", TaskID: "pkg", SupplierID: "v", EventType: "acceptance_approved", CreatedAt: "2026-09-26T00:00:00Z"}}, Events: []domain.HumanEventRecord{{ID: "earlier", ItemID: "review", StageID: "acceptance", EventType: "rejected", CreatedAt: "2026-09-25T00:00:00Z", Payload: map[string]any{"deliveryPackageId": "pkg"}}}} + r := Build(d) + counts := map[string]int{} + for _, e := range r.Events { + counts[e.Metric]++ + } + if counts["rejected"] != 1 || counts["acceptance"] != 1 { + t.Fatalf("distinct decisions lost: %+v", r.Events) + } + d.Packages[0].AcceptanceReviews = []domain.DeliveryPackageAcceptanceReview{{ID: "reject", Decision: domain.HumanDecisionRejected, CreatedAt: "2026-09-24T23:59:59Z"}, {ID: "approve", Decision: domain.HumanDecisionApproved, CreatedAt: "2026-09-25T23:59:59Z"}} + r = Build(d) + counts = map[string]int{} + for _, e := range r.Events { + counts[e.Metric]++ + } + if counts["rejected"] != 1 || counts["acceptance"] != 1 { + t.Fatalf("canonical decisions double counted: %+v", r.Events) + } + // Packaging-time copies of pre-package acceptance are not a second decision. + d.Packages[0].QualitySummary = map[string]any{"prePackageAcceptance": true} + r = Build(d) + for _, e := range r.Events { + if e.ID == "approve" || e.ID == "reject" { + t.Fatalf("synthetic review counted: %+v", e) + } + } +} diff --git a/internal/productionstats/milestones.go b/internal/productionstats/milestones.go new file mode 100644 index 00000000..ed2d4e17 --- /dev/null +++ b/internal/productionstats/milestones.go @@ -0,0 +1,234 @@ +package productionstats + +import ( + "droboticflow/internal/domain" + "sort" + "time" +) + +// A source milestone occurs at the last required part's completion. Contributors +// retain their actual evidence timestamps, while period filters use that common +// completion time. Missing part evidence is never inferred from today's state. +func sourceMilestones(events []Event, parents map[string]string, placeholders map[string]bool, units milestoneUnits) []Event { + required := map[string]map[string]bool{} + for part, parent := range parents { + if placeholders[part] { + continue + } + if required[parent] == nil { + required[parent] = map[string]bool{} + } + required[parent][part] = true + } + ordered := append([]Event(nil), events...) + sort.SliceStable(ordered, func(i, j int) bool { + if ordered[i].At == ordered[j].At { + if (ordered[i].Metric == "attempt_started") != (ordered[j].Metric == "attempt_started") { + return ordered[i].Metric == "attempt_started" + } + return ordered[i].ID < ordered[j].ID + } + return later(ordered[j].At, ordered[i].At) + }) + ready := map[string]map[string]map[string]Event{} + attempts := map[string]string{} + out := []Event{} + for _, e := range ordered { + part := e.SourceID + parent := parents[part] + if parent == "" || placeholders[part] { + continue + } + if _, err := time.Parse(time.RFC3339Nano, e.At); err != nil { + continue + } + if e.Metric == "attempt_started" { + attempts[part] = e.AttemptID + for _, metric := range Metrics { + delete(ready[parent+":"+metric], part) + } + continue + } + if e.Metric == "rejected" { + if attempts[part] == "" || e.AttemptID == attempts[part] { + for _, metric := range []string{"submitted", "quality", "acceptance", "delivered"} { + if metric == "submitted" && e.InvalidateFrom == "quality" { + continue + } + for unit, fact := range ready[parent+":"+metric][part] { + if affectedProductionUnits(e.ProductionUnits, fact.ProductionUnits) { + delete(ready[parent+":"+metric][part], unit) + } + } + } + } + e.SourceID = parent + out = append(out, e) + continue + } + if attempts[part] != "" && e.AttemptID != attempts[part] { + continue + } + key := parent + ":" + e.Metric + if ready[key] == nil { + ready[key] = map[string]map[string]Event{} + } + if ready[key][part] == nil { + ready[key][part] = map[string]Event{} + } + for _, unit := range completionUnits(e) { + ready[key][part][unit] = e + } + complete := true + for requiredPart := range required[parent] { + facts := ready[key][requiredPart] + if len(facts) == 0 { + complete = false + break + } + for _, fact := range facts { + for unit := range units[fact.AttemptID+":"+e.Metric] { + if _, ok := facts[unit]; !ok { + complete = false + break + } + } + break + } + if !complete { + break + } + } + if !complete { + continue + } + emitted := map[string]bool{} + for _, facts := range ready[key] { + for _, fact := range facts { + if emitted[fact.ID] { + continue + } + emitted[fact.ID] = true + if fact.ID != e.ID { + fact.ID = e.ID + ":" + fact.ID + } + fact.SourceID = parent + fact.EvidenceAt = fact.At + fact.At = e.At + out = append(out, fact) + } + } + } + return out +} + +// Required units come from persisted tasks, not just successful events. Event +// references also cover legacy package/item identities absent from task records. +type milestoneUnits map[string]map[string]bool + +func requiredMilestoneUnits(d Snapshot, events []Event, root func(string) string) milestoneUnits { + units := milestoneUnits{} + add := func(attempt, metric, id string) { + if attempt == "" || id == "" { + return + } + key := attempt + ":" + metric + if units[key] == nil { + units[key] = map[string]bool{} + } + units[key][id] = true + } + for _, e := range events { + if e.Metric != "rejected" && e.Metric != "attempt_started" { + for _, unit := range completionUnits(e) { + add(e.AttemptID, e.Metric, unit) + } + } + } + for _, task := range d.Tasks { + for _, metric := range []string{"submitted", "quality"} { + add(root(task.RunID), metric, task.ID) + } + } + eventProduction := map[string][]string{} + for _, e := range events { + if len(e.ProductionUnits) > 0 { + eventProduction[e.TaskID] = e.ProductionUnits + } + } + index := indexEvents(d) + packageForItem := map[string]string{} + packageProduction := map[string][]string{} + for _, event := range d.Events { + if pkg, _ := event.Payload["deliveryPackageId"].(string); pkg != "" { + packageForItem[event.ItemID] = pkg + } + } + for _, p := range d.Packages { + if p.Mode == domain.RunModeDebug || p.Temporary { + continue + } + attempt := root(p.ParentRunID) + if attempt == "" { + attempt = root(p.ChildRunID) + } + production := index.productionTasks(p.ItemIDs) + packageProduction[p.ID] = production + for _, unit := range completionUnits(Event{TaskID: p.ID, Metric: "delivered", ProductionUnits: production}) { + add(attempt, "delivered", unit) + } + if p.AcceptanceHumanItemID != "" { + packageForItem[p.AcceptanceHumanItemID] = p.ID + } + if _, copied := p.QualitySummary["prePackageAcceptance"]; !copied && (p.AcceptanceHumanItemID != "" || len(p.AcceptanceReviews) > 0 || p.Status == domain.DeliveryPackagePendingAcceptance || p.Status == domain.DeliveryPackageRejected) { + for _, unit := range completionUnits(Event{TaskID: p.ID, Metric: "acceptance", ProductionUnits: production}) { + add(attempt, "acceptance", unit) + } + } + } + for _, item := range d.Items { + for _, metric := range []string{"submitted", "quality"} { + add(root(item.RunID), metric, item.LegacyTaskID) + } + if acceptanceStage(item.CurrentStageID) { + unit := item.ID + if packageForItem[unit] != "" { + unit = packageForItem[unit] + } + production := packageProduction[unit] + // Aggregate pre-package decisions name their production dependencies. + if len(production) == 0 { + production = eventProduction[unit] + } + for _, key := range completionUnits(Event{TaskID: unit, Metric: "acceptance", ProductionUnits: production}) { + add(root(item.RunID), "acceptance", key) + } + } + } + return units +} + +// Missing historical dependency links are conservatively invalidated, never +// treated as proof that downstream completion survived a return. +func affectedProductionUnits(rejected, completed []string) bool { + if len(rejected) == 0 || len(completed) == 0 { + return true + } + for _, a := range rejected { + for _, b := range completed { + if a == b { + return true + } + } + } + return false +} + +// Replacement packages complete the same production units; rejected historical +// package IDs are not additional work that must somehow succeed later. +func completionUnits(e Event) []string { + if (e.Metric == "acceptance" || e.Metric == "delivered") && len(e.ProductionUnits) > 0 { + return e.ProductionUnits + } + return []string{e.TaskID} +} diff --git a/internal/productionstats/milestones_test.go b/internal/productionstats/milestones_test.go new file mode 100644 index 00000000..b9e4891e --- /dev/null +++ b/internal/productionstats/milestones_test.go @@ -0,0 +1,151 @@ +package productionstats + +import ( + "droboticflow/internal/domain" + "testing" + "time" +) + +func multipartHistory() Snapshot { + return Snapshot{Captures: []domain.CaptureRecord{{ID: "a", ProjectID: "p", Prefix: "source/part1", SourcePrefix: "source", RunID: "ra"}, {ID: "b", ProjectID: "p", Prefix: "source/part2", SourcePrefix: "source", RunID: "rb"}}, Runs: []domain.RunRecord{{ID: "ra", TaskID: "a", CreatedAt: "2026-09-20T00:00:00Z"}, {ID: "rb", TaskID: "b", CreatedAt: "2026-09-20T00:00:00Z"}}, Tasks: []domain.AnnotationTaskRecord{{ID: "ta", RunID: "ra"}, {ID: "tb", RunID: "rb"}}, Worklogs: []domain.AnnotationWorklogEventRecord{{ID: "ea", TaskID: "ta", SupplierID: "va", EventType: "combined_submitted", CreatedAt: "2026-09-21T01:00:00Z"}, {ID: "eb", TaskID: "tb", SupplierID: "vb", EventType: "combined_submitted", CreatedAt: "2026-09-22T01:00:00Z"}}} +} +func period(t *testing.T, date string) Query { + t.Helper() + q, err := ParseQuery("period", date, date, time.Now()) + if err != nil { + t.Fatal(err) + } + return q +} +func TestMultipartPeriodWaitsForAllParts(t *testing.T) { + d := multipartHistory() + r := Build(d) + if got := Summarize(r, period(t, "2026-09-21")).Totals["submitted"]; got != 0 { + t.Fatalf("partial source counted: %d", got) + } + q := period(t, "2026-09-22") + got := Summarize(r, q) + if got.Totals["submitted"] != 1 || !got.SupplierOverlap { + t.Fatalf("whole-source completion missing: %+v", got) + } + q.SupplierID = "va" + if got := Summarize(r, q).Totals["submitted"]; got != 1 { + t.Fatal("first part contributor lost") + } + d.Worklogs = d.Worklogs[:1] + if got := Summarize(Build(d), period(t, "2026-09-21")).Totals["submitted"]; got != 0 { + t.Fatal("missing part counted as complete") + } +} + +func TestMultipartMilestoneDoesNotReuseSupersededAttempt(t *testing.T) { + d := multipartHistory() + d.Runs = append(d.Runs, domain.RunRecord{ID: "ra2", TaskID: "a", CreatedAt: "2026-09-21T12:00:00Z"}) + d.Tasks = append(d.Tasks, domain.AnnotationTaskRecord{ID: "ta2", RunID: "ra2"}) + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "late-old", TaskID: "ta", SupplierID: "va", EventType: "combined_submitted", CreatedAt: "2026-09-22T02:00:00Z"}) + if got := Summarize(Build(d), period(t, "2026-09-22")).Totals["submitted"]; got != 0 { + t.Fatal("old attempt completed new source round") + } + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "new", TaskID: "ta2", SupplierID: "vc", EventType: "combined_submitted", CreatedAt: "2026-09-23T01:00:00Z"}) + r := Build(d) + q := period(t, "2026-09-23") + if got := Summarize(r, q).Totals["submitted"]; got != 1 { + t.Fatal("new attempt not counted") + } + q.SupplierID = "va" + if got := Summarize(r, q).Totals["submitted"]; got != 0 { + t.Fatal("superseded owner counted") + } +} + +func TestMultipartRejectionInvalidatesCompletionEvidence(t *testing.T) { + d := multipartHistory() + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "reject-b", TaskID: "tb", SupplierID: "vb", EventType: "quality_rejected", CreatedAt: "2026-09-23T01:00:00Z"}, domain.AnnotationWorklogEventRecord{ID: "repeat-a", TaskID: "ta", SupplierID: "va", EventType: "combined_submitted", CreatedAt: "2026-09-24T01:00:00Z"}) + r := Build(d) + if got := Summarize(r, period(t, "2026-09-24")).Totals["submitted"]; got != 0 { + t.Fatal("rejected part reused as complete") + } + if got := Summarize(r, period(t, "2026-09-23")).Totals["rejected"]; got != 1 { + t.Fatal("rejection should remain an event count") + } + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "repeat-b", TaskID: "tb", SupplierID: "vb", EventType: "combined_submitted", CreatedAt: "2026-09-25T01:00:00Z"}) + if got := Summarize(Build(d), period(t, "2026-09-25")).Totals["submitted"]; got != 1 { + t.Fatal("resubmission missing") + } +} + +func TestAcceptanceReturnPreservesSubmittedMilestone(t *testing.T) { + d := multipartHistory() + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "acceptance-return", TaskID: "ta", SupplierID: "va", EventType: "acceptance_rejected", CreatedAt: "2026-09-21T12:00:00Z"}) + if got := Summarize(Build(d), period(t, "2026-09-22")).Totals["submitted"]; got != 1 { + t.Fatal("acceptance return to quality must not require re-annotation") + } +} + +func TestMultipartWaitsForEveryRequiredTaskWithinPart(t *testing.T) { + d := multipartHistory() + d.Tasks = append(d.Tasks, domain.AnnotationTaskRecord{ID: "ta-extra", RunID: "ra"}) + if got := Summarize(Build(d), period(t, "2026-09-22")).Totals["submitted"]; got != 0 { + t.Fatal("unfinished task inside part counted as complete") + } + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "extra", TaskID: "ta-extra", SupplierID: "va", EventType: "combined_submitted", CreatedAt: "2026-09-23T01:00:00Z"}) + if got := Summarize(Build(d), period(t, "2026-09-23")).Totals["submitted"]; got != 1 { + t.Fatal("all required tasks completed but source not counted") + } +} + +func TestRejectionPreservesUnaffectedSiblingTask(t *testing.T) { + d := multipartHistory() + d.Tasks = append(d.Tasks, domain.AnnotationTaskRecord{ID: "ta2", RunID: "ra"}) + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "extra", TaskID: "ta2", SupplierID: "va", EventType: "combined_submitted", CreatedAt: "2026-09-21T02:00:00Z"}, domain.AnnotationWorklogEventRecord{ID: "reject-one", TaskID: "ta", SupplierID: "va", EventType: "quality_rejected", CreatedAt: "2026-09-21T03:00:00Z"}, domain.AnnotationWorklogEventRecord{ID: "resubmit-one", TaskID: "ta", SupplierID: "va", EventType: "combined_submitted", CreatedAt: "2026-09-23T00:00:00Z"}) + if got := Summarize(Build(d), period(t, "2026-09-23")).Totals["submitted"]; got != 1 { + t.Fatal("unaffected sibling completion erased") + } +} + +func TestAggregateReturnOnlyInvalidatesReferencedProductionTasks(t *testing.T) { + d := multipartHistory() + d.Tasks = append(d.Tasks, domain.AnnotationTaskRecord{ID: "ta2", RunID: "ra"}) + d.Items = []domain.HumanItemRecord{{ID: "ia", RunID: "ra", LegacyTaskID: "ta"}, {ID: "ia2", RunID: "ra", LegacyTaskID: "ta2"}, {ID: "accept", RunID: "ra", CurrentStageID: "prepackage_acceptance"}} + d.Worklogs = append(d.Worklogs, domain.AnnotationWorklogEventRecord{ID: "qa1", TaskID: "ta", SupplierID: "va", EventType: "quality_approved", CreatedAt: "2026-09-21T03:00:00Z"}, domain.AnnotationWorklogEventRecord{ID: "qa2", TaskID: "ta2", SupplierID: "va", EventType: "quality_approved", CreatedAt: "2026-09-21T04:00:00Z"}, domain.AnnotationWorklogEventRecord{ID: "reapprove", TaskID: "ta", SupplierID: "va", EventType: "quality_approved", CreatedAt: "2026-09-23T00:00:00Z"}, domain.AnnotationWorklogEventRecord{ID: "qb", TaskID: "tb", SupplierID: "vb", EventType: "quality_approved", CreatedAt: "2026-09-23T01:00:00Z"}) + d.Events = []domain.HumanEventRecord{{ID: "return", ItemID: "accept", StageID: "prepackage_acceptance", EventType: "rejected", CreatedAt: "2026-09-22T00:00:00Z", Payload: map[string]any{"sourceItemIds": []string{"ia"}}}} + r := Build(d) + q := period(t, "2026-09-23") + if got := Summarize(r, q).Totals["quality"]; got != 1 { + t.Fatal("aggregate return erased unrelated task") + } + rows := Details(r, q, "quality", "") + if len(rows) != 1 { + t.Fatal("missing detail") + } + found := false + for _, e := range rows[0].Evidence { + if e.TaskID == "ta2" { + found = true + if e.At != "2026-09-23T01:00:00Z" || e.EvidenceAt != "2026-09-21T04:00:00Z" { + t.Fatalf("original evidence time lost: %+v", e) + } + } + } + if !found { + t.Fatal("sibling evidence absent") + } +} + +func TestReplacementPackageDoesNotRequireRejectedPackageSuccess(t *testing.T) { + d := multipartHistory() + d.Items = []domain.HumanItemRecord{{ID: "ia", RunID: "ra", LegacyTaskID: "ta"}, {ID: "ib", RunID: "rb", LegacyTaskID: "tb"}} + d.Packages = []domain.DeliveryPackageRecord{ + {ID: "old", ParentRunID: "ra", ItemIDs: []string{"ia"}, SupplierID: "va", Status: domain.DeliveryPackageRejected, AcceptanceReviews: []domain.DeliveryPackageAcceptanceReview{{ID: "old-reject", Decision: domain.HumanDecisionRejected, CreatedAt: "2026-09-23T00:00:00Z"}}}, + {ID: "replacement", ParentRunID: "ra", ItemIDs: []string{"ia"}, SupplierID: "va", Status: domain.DeliveryPackageAccepted, FinalExportedAt: "2026-09-25T02:00:00Z", AcceptanceReviews: []domain.DeliveryPackageAcceptanceReview{{ID: "new-approve", Decision: domain.HumanDecisionApproved, CreatedAt: "2026-09-25T01:00:00Z"}}}, + {ID: "other", ParentRunID: "rb", ItemIDs: []string{"ib"}, SupplierID: "vb", Status: domain.DeliveryPackageAccepted, FinalExportedAt: "2026-09-24T02:00:00Z", AcceptanceReviews: []domain.DeliveryPackageAcceptanceReview{{ID: "other-approve", Decision: domain.HumanDecisionApproved, CreatedAt: "2026-09-24T01:00:00Z"}}}, + } + r := Build(d) + totals := Summarize(r, period(t, "2026-09-25")).Totals + if totals["acceptance"] != 1 || totals["delivered"] != 1 { + t.Fatalf("superseded package blocked replacement: %+v", totals) + } + if Summarize(r, period(t, "2026-09-23")).Totals["rejected"] != 1 { + t.Fatal("rejection history lost") + } +} diff --git a/internal/productionstats/model.go b/internal/productionstats/model.go new file mode 100644 index 00000000..a359ee34 --- /dev/null +++ b/internal/productionstats/model.go @@ -0,0 +1,76 @@ +// Package productionstats derives read-only source-level reporting from production facts. +// It never changes task state, claims work, or materializes customer records. +package productionstats + +import "droboticflow/internal/domain" + +type Scope struct { + ProjectIDs []string `json:"projectIds" bson:"projectIds"` + Revision int `json:"revision" bson:"revision"` + UpdatedAt string `json:"updatedAt,omitempty" bson:"updatedAt,omitempty"` + UpdatedBy string `json:"updatedBy,omitempty" bson:"updatedBy,omitempty"` +} +type Snapshot struct { + Captures []domain.CaptureRecord + Runs []domain.RunRecord + Items []domain.HumanItemRecord + Batches []domain.HumanBatchRecord + Tasks []domain.AnnotationTaskRecord + Worklogs []domain.AnnotationWorklogEventRecord + Events []domain.HumanEventRecord + Packages []domain.DeliveryPackageRecord +} +type Task struct { + ID string `json:"id"` + Stage string `json:"stage"` + Status string `json:"status"` +} +type Source struct { + ID string `json:"id"` + ProjectID string `json:"projectId"` + Name string `json:"name"` + WorkflowID string `json:"workflowId,omitempty"` + RunID string `json:"runId,omitempty"` + SupplierID string `json:"supplierId"` + Stage string `json:"stage"` + Detail string `json:"detail"` + Exception bool `json:"exception"` + DiscoveredAt string `json:"discoveredAt"` + Tasks []Task `json:"tasks"` + Runs []RunReference `json:"runs"` + Evidence []Evidence `json:"evidence,omitempty"` +} +type Event struct { + ID, SourceID, SupplierID, Metric, At string + RunID, WorkflowID, TaskID string + EvidenceAt string + AttemptID string + InvalidateFrom string + ProductionUnits []string +} +type RunReference struct { + ID string `json:"id"` + WorkflowID string `json:"workflowId"` + Part string `json:"part"` +} +type Evidence struct { + EvidenceAt string `json:"evidenceAt,omitempty"` + ID string `json:"id"` + Metric string `json:"metric"` + SupplierID string `json:"supplierId"` + At string `json:"at"` + RunID string `json:"runId,omitempty"` + WorkflowID string `json:"workflowId,omitempty"` + TaskID string `json:"taskId,omitempty"` +} +type Report struct { + Sources []Source + Events []Event + Incomplete bool +} + +var Stages = []string{"intake", "annotation", "quality", "acceptance", "delivery", "delivered", "ended", "unknown"} +var Metrics = []string{"new", "submitted", "quality", "acceptance", "delivered", "rejected"} + +const Unassigned = "__unassigned__" +const Unknown = "__unknown__" diff --git a/internal/productionstats/query.go b/internal/productionstats/query.go new file mode 100644 index 00000000..6858ec7d --- /dev/null +++ b/internal/productionstats/query.go @@ -0,0 +1,184 @@ +package productionstats + +import ( + "fmt" + "sort" + "strings" + "time" +) + +type Query struct { + View, ProjectID, SupplierID, Group string + From, To time.Time +} +type Row struct { + ID string `json:"id"` + Counts map[string]int `json:"counts"` +} +type Summary struct { + Totals map[string]int `json:"totals"` + StageErrors map[string]int `json:"stageErrors"` + Rows []Row `json:"rows"` + SupplierOverlap bool `json:"supplierOverlap"` + Incomplete bool `json:"incomplete"` +} + +func ParseQuery(view, from, to string, now time.Time) (Query, error) { + q := Query{View: view, Group: "project"} + if q.View == "" { + q.View = "current" + } + if q.View != "current" && q.View != "period" { + return q, fmt.Errorf("unknown reporting view") + } + zone := time.FixedZone("Asia/Shanghai", 8*3600) + today := now.In(zone).Format("2006-01-02") + if to == "" { + to = today + } + if from == "" { + from = now.In(zone).AddDate(0, 0, -6).Format("2006-01-02") + } + var err error + q.From, err = time.ParseInLocation("2006-01-02", from, zone) + if err != nil { + return q, fmt.Errorf("invalid start date") + } + q.To, err = time.ParseInLocation("2006-01-02", to, zone) + if err != nil { + return q, fmt.Errorf("invalid end date") + } + q.To = q.To.AddDate(0, 0, 1) + if !q.From.Before(q.To) || q.To.Sub(q.From) > 366*24*time.Hour { + return q, fmt.Errorf("date range must be 1–366 days") + } + return q, nil +} +func (q Query) eventInRange(e Event) bool { + at, err := time.Parse(time.RFC3339Nano, e.At) + return err == nil && !at.Before(q.From) && at.Before(q.To) +} +func Summarize(r Report, q Query) Summary { + out := Summary{Totals: map[string]int{}, StageErrors: map[string]int{}, Rows: []Row{}, Incomplete: r.Incomplete} + rows := map[string]map[string]int{} + sources := map[string]Source{} + for _, s := range r.Sources { + if q.ProjectID != "" && s.ProjectID != q.ProjectID { + continue + } + sources[s.ID] = s + } + add := func(row, metric string) { + if rows[row] == nil { + rows[row] = map[string]int{} + } + rows[row][metric]++ + } + if q.View == "current" { + for _, s := range sources { + if q.SupplierID != "" && s.SupplierID != q.SupplierID { + continue + } + row := s.ProjectID + if q.Group == "supplier" { + row = s.SupplierID + } + out.Totals["total"]++ + out.Totals[s.Stage]++ + add(row, "total") + add(row, s.Stage) + if s.Exception || s.Stage == "unknown" { + out.Totals["attention"]++ + out.StageErrors[s.Stage]++ + add(row, "attention") + } + } + } else { + seen := map[string]bool{} + rowSeen := map[string]bool{} + owners := map[string]map[string]bool{} + for _, e := range r.Events { + s, ok := sources[e.SourceID] + if !ok || !q.eventInRange(e) || (q.SupplierID != "" && e.SupplierID != q.SupplierID) { + continue + } + row := s.ProjectID + if q.Group == "supplier" { + row = e.SupplierID + } + key := e.Metric + ":" + e.SourceID + if e.Metric == "rejected" { + key = e.Metric + ":" + e.ID + } + if !seen[key] { + out.Totals[e.Metric]++ + seen[key] = true + } + rk := row + ":" + key + if !rowSeen[rk] { + add(row, e.Metric) + rowSeen[rk] = true + } + if owners[key] == nil { + owners[key] = map[string]bool{} + } + owners[key][e.SupplierID] = true + if e.Metric != "rejected" && len(owners[key]) > 1 { + out.SupplierOverlap = true + } + } + } + for id, counts := range rows { + out.Rows = append(out.Rows, Row{id, counts}) + } + sort.Slice(out.Rows, func(i, j int) bool { + a, b := out.Rows[i], out.Rows[j] + metric := "total" + if q.View == "period" { + metric = "new" + } + if a.Counts[metric] != b.Counts[metric] { + return a.Counts[metric] > b.Counts[metric] + } + return a.ID < b.ID + }) + return out +} +func Details(r Report, q Query, metric, search string) []Source { + selected := map[string][]Evidence{} + if q.View == "period" { + for _, e := range r.Events { + if (metric == "" || metric == e.Metric) && q.eventInRange(e) && (q.SupplierID == "" || q.SupplierID == e.SupplierID) { + selected[e.SourceID] = append(selected[e.SourceID], Evidence{ID: e.ID, Metric: e.Metric, SupplierID: e.SupplierID, At: e.At, RunID: e.RunID, WorkflowID: e.WorkflowID, TaskID: e.TaskID, EvidenceAt: e.EvidenceAt}) + } + } + } + items := []Source{} + for _, s := range r.Sources { + if q.ProjectID != "" && s.ProjectID != q.ProjectID { + continue + } + if q.View == "period" { + if len(selected[s.ID]) == 0 { + continue + } + s.Evidence = selected[s.ID] + } else { + if q.SupplierID != "" && q.SupplierID != s.SupplierID { + continue + } + if metric == "attention" { + if !s.Exception && s.Stage != "unknown" { + continue + } + } else if metric != "" && metric != "total" && metric != s.Stage { + continue + } + } + if !strings.Contains(strings.ToLower(s.Name), strings.ToLower(search)) { + continue + } + items = append(items, s) + } + return items +} diff --git a/internal/productionstats/query_test.go b/internal/productionstats/query_test.go new file mode 100644 index 00000000..7c4710ef --- /dev/null +++ b/internal/productionstats/query_test.go @@ -0,0 +1,36 @@ +package productionstats + +import ( + "testing" + "time" +) + +func TestPeriodCountsDeduplicateAndKeepHistoricalSupplier(t *testing.T) { + r := Report{Sources: []Source{{ID: "one", ProjectID: "p", SupplierID: "b", Stage: "annotation"}}, Events: []Event{ + {ID: "e1", SourceID: "one", SupplierID: "a", Metric: "submitted", At: "2026-09-27T16:00:00Z"}, {ID: "e2", SourceID: "one", SupplierID: "b", Metric: "submitted", At: "2026-09-28T00:00:00Z"}, {ID: "e3", SourceID: "one", SupplierID: "b", Metric: "submitted", At: "2026-09-28T01:00:00Z"}, {ID: "e4", SourceID: "one", SupplierID: "a", Metric: "submitted", At: "2026-09-28T16:00:00Z"}}} + q, err := ParseQuery("period", "2026-09-28", "2026-09-28", time.Now()) + if err != nil { + t.Fatal(err) + } + got := Summarize(r, q) + if got.Totals["submitted"] != 1 || !got.SupplierOverlap { + t.Fatalf("overall dedup: %+v", got) + } + q.SupplierID = "a" + got = Summarize(r, q) + if got.Totals["submitted"] != 1 { + t.Fatalf("event ownership: %+v", got) + } +} +func TestHistoricalDetailsKeepEventReferences(t *testing.T) { + r := Report{Sources: []Source{{ID: "s", ProjectID: "p", SupplierID: "new", RunID: "new-run", Stage: "annotation"}}, Events: []Event{{ID: "e", SourceID: "s", SupplierID: "old", Metric: "quality", At: "2026-09-28T01:00:00Z", RunID: "old-run", WorkflowID: "w", TaskID: "old-task"}}} + q, err := ParseQuery("period", "2026-09-28", "2026-09-28", time.Now()) + if err != nil { + t.Fatal(err) + } + q.SupplierID = "old" + rows := Details(r, q, "quality", "") + if len(rows) != 1 || len(rows[0].Evidence) != 1 || rows[0].Evidence[0].RunID != "old-run" || rows[0].Evidence[0].SupplierID != "old" { + t.Fatalf("historical evidence replaced by current state: %+v", rows) + } +} diff --git a/internal/productionstats/stages_test.go b/internal/productionstats/stages_test.go new file mode 100644 index 00000000..c3b3c0fd --- /dev/null +++ b/internal/productionstats/stages_test.go @@ -0,0 +1,120 @@ +package productionstats + +import ( + "droboticflow/internal/domain" + "fmt" + "testing" +) + +func TestStageRollupAndAnonymousAcceptance(t *testing.T) { + base := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "one", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c", ProjectID: "p", CreatedAt: "2026-09-01T00:00:00Z"}}, Batches: []domain.HumanBatchRecord{{ID: "main"}, {ID: "accept", SourceBatchID: "main"}}, Items: []domain.HumanItemRecord{{ID: "a", BatchID: "main", RunID: "r", SupplierID: "vendor", CurrentStageID: "quality_review", Status: domain.HumanItemSubmitted}, {ID: "b", BatchID: "accept", RunID: "r", CurrentStageID: "acceptance", Status: domain.HumanItemReady}}} + r := Build(base) + if r.Sources[0].Stage != "quality" || r.Sources[0].SupplierID != "vendor" { + t.Fatalf("earliest stage and owner: %+v", r) + } + base.Items[0].Status = domain.HumanItemAccepted + r = Build(base) + if r.Sources[0].Stage != "acceptance" { + t.Fatalf("anonymous acceptance: %+v", r) + } + base.Items = base.Items[:1] + base.Items[0].Status = domain.HumanItemCancelled + r = Build(base) + if r.Sources[0].Stage != "ended" { + t.Fatalf("all cancelled: %+v", r) + } +} +func TestAllDiscoveredSourcesAndConflictingOwnership(t *testing.T) { + d := Snapshot{} + for i := 0; i < 251; i++ { + d.Captures = append(d.Captures, domain.CaptureRecord{ID: fmt.Sprint(i), ProjectID: "p", Prefix: fmt.Sprint(i), Status: domain.CaptureRecordIncomplete}) + } + if got := Summarize(Build(d), Query{View: "current"}); got.Totals["total"] != 251 || got.Totals["intake"] != 251 { + t.Fatalf("full discovered corpus: %+v", got) + } + d = Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c"}}, Items: []domain.HumanItemRecord{{ID: "a", RunID: "r", SupplierID: "a", CurrentStageID: "annotation_combined"}, {ID: "b", RunID: "r", SupplierID: "b", CurrentStageID: "annotation_combined"}}} + r := Build(d) + if r.Sources[0].SupplierID != Unknown || !r.Sources[0].Exception { + t.Fatalf("conflict hidden: %+v", r) + } +} +func TestCompletedProcessingJobDoesNotUndoHandoff(t *testing.T) { + i := domain.HumanItemRecord{ID: "i", RunID: "r", CurrentStageID: "quality_review", Status: domain.HumanItemSubmitted, ProcessingJob: &domain.AnnotationProcessingJob{Status: "completed"}} + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c"}}, Items: []domain.HumanItemRecord{i}} + if r := Build(d); r.Sources[0].Stage != "quality" { + t.Fatalf("completed job: %+v", r) + } +} +func TestPackagingChildKeepsProductionOwnerAndDeliveredState(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "child", BusinessStage: domain.CaptureStageDelivered}}, Runs: []domain.RunRecord{{ID: "parent", TaskID: "c", CreatedAt: "2026-09-01T00:00:00Z"}, {ID: "child", TaskID: "c", ParentRunID: "parent", Trigger: domain.RunTriggerHumanRelease, Status: domain.RunStatusCompleted, CreatedAt: "2026-09-02T00:00:00Z"}}, Items: []domain.HumanItemRecord{{ID: "i", RunID: "parent", SupplierID: "v", Status: domain.HumanItemAccepted}}} + r := Build(d) + if r.Sources[0].Stage != "delivered" || r.Sources[0].SupplierID != "v" { + t.Fatalf("packaging is not a new attempt: %+v", r) + } +} +func TestMissingRunDoesNotPretendSourceIsStillIntake(t *testing.T) { + r := Build(Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "missing"}}}) + if r.Sources[0].Stage != "unknown" { + t.Fatalf("missing historical linkage: %+v", r) + } +} +func TestLegacyAnnotationTaskRetainsItsStage(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c"}}, Tasks: []domain.AnnotationTaskRecord{{ID: "task", RunID: "r", SupplierID: "v", Status: domain.AnnotationTaskStatusPendingQuality}}} + r := Build(d) + if r.Sources[0].Stage != "quality" || r.Sources[0].SupplierID != "v" { + t.Fatalf("legacy task hidden: %+v", r) + } +} +func TestLegacyPostPackageAcceptanceNotPrematurelyDelivered(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r", BusinessStage: domain.CaptureStageDelivered}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c", Status: domain.RunStatusCompleted}}, Items: []domain.HumanItemRecord{{ID: "i", RunID: "r", SupplierID: "v", Status: domain.HumanItemAccepted}}, Packages: []domain.DeliveryPackageRecord{{ID: "pkg", ParentRunID: "r", Status: domain.DeliveryPackagePendingAcceptance}}} + r := Build(d) + if r.Sources[0].Stage != "acceptance" { + t.Fatalf("legacy acceptance skipped: %+v", r) + } +} +func TestMultipartSourceRollsUpAllPartsAndRetriesPerPart(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "part1", ProjectID: "p", Prefix: "capture/part1", SourcePrefix: "capture", CaptureID: "capture/part1", RunID: "r1"}, {ID: "part2", ProjectID: "p", Prefix: "capture/part2", SourcePrefix: "capture", CaptureID: "capture/part2", RunID: "r2", BusinessStage: domain.CaptureStageDelivered}}, Runs: []domain.RunRecord{{ID: "r1", TaskID: "part1", CreatedAt: "2026-09-01T00:00:00Z"}, {ID: "r2", TaskID: "part2", CreatedAt: "2026-09-02T00:00:00Z", Status: domain.RunStatusCompleted}}, Items: []domain.HumanItemRecord{{ID: "i1", RunID: "r1", SupplierID: "v", CurrentStageID: "annotation_combined"}, {ID: "i2", RunID: "r2", SupplierID: "v", Status: domain.HumanItemAccepted}}} + r := Build(d) + if len(r.Sources) != 1 || r.Sources[0].Stage != "annotation" || len(r.Sources[0].Tasks) != 2 { + t.Fatalf("unfinished part hidden: %+v", r) + } + d.Runs = append(d.Runs, domain.RunRecord{ID: "retry1", TaskID: "part1", CreatedAt: "2026-09-03T00:00:00Z"}) + d.Captures[0].RunID = "retry1" + d.Items = append(d.Items, domain.HumanItemRecord{ID: "i3", RunID: "retry1", SupplierID: "v", CurrentStageID: "quality_review"}) + r = Build(d) + if r.Sources[0].Stage != "quality" || len(r.Sources[0].Tasks) != 2 { + t.Fatalf("retry retained obsolete part state: %+v", r) + } +} + +func TestSupersededParentKeepsDiscoveryButNotCurrentStage(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{ + {ID: "parent", ProjectID: "p", Prefix: "source", Status: domain.CaptureRecordIncomplete, DiscoveredAt: "2026-09-20T00:00:00Z"}, + {ID: "part", ProjectID: "p", Prefix: "source/part0001", SourcePrefix: "source", RunID: "r", DiscoveredAt: "2026-09-21T00:00:00Z"}, + }, Runs: []domain.RunRecord{{ID: "r", TaskID: "part"}}, Items: []domain.HumanItemRecord{{ID: "i", RunID: "r", CurrentStageID: "quality_review", SupplierID: "v"}}} + r := Build(d) + if len(r.Sources) != 1 || r.Sources[0].Stage != "quality" || r.Sources[0].SupplierID != "v" || r.Sources[0].DiscoveredAt != "2026-09-20T00:00:00Z" { + t.Fatalf("placeholder polluted progress: %+v", r.Sources) + } + // A parent actually admitted to production must not be silently suppressed. + d.Captures[0].RunID = "parent-run" + d.Runs = append(d.Runs, domain.RunRecord{ID: "parent-run", TaskID: "parent"}) + r = Build(d) + if r.Sources[0].Stage != "intake" { + t.Fatalf("admitted parent lost: %+v", r.Sources) + } +} + +func TestCurrentProgressIgnoresReplacedRejectedPackage(t *testing.T) { + d := Snapshot{Captures: []domain.CaptureRecord{{ID: "c", ProjectID: "p", Prefix: "c", RunID: "r"}}, Runs: []domain.RunRecord{{ID: "r", TaskID: "c"}}, Items: []domain.HumanItemRecord{{ID: "i", RunID: "r", Status: domain.HumanItemAccepted, ReleasePartitionID: "new"}}, Packages: []domain.DeliveryPackageRecord{{ID: "old", ParentRunID: "r", ReleasePartitionID: "old", ItemIDs: []string{"i"}, Status: domain.DeliveryPackageRejected}, {ID: "new", ParentRunID: "r", ReleasePartitionID: "new", ItemIDs: []string{"i"}, Status: domain.DeliveryPackageAccepted}}} + d.Runs = append(d.Runs, domain.RunRecord{ID: "old-child", ParentRunID: "r", Trigger: domain.RunTriggerHumanRelease}, domain.RunRecord{ID: "new-child", ParentRunID: "r", Trigger: domain.RunTriggerHumanRelease}) + d.Packages[0].ChildRunID = "old-child" + d.Packages[0].AcceptanceHumanItemID = "old-review" + d.Packages[1].ChildRunID = "new-child" + d.Packages[1].AcceptanceHumanItemID = "new-review" + d.Items = append(d.Items, domain.HumanItemRecord{ID: "old-review", RunID: "old-child", CurrentStageID: "acceptance", Status: domain.HumanItemRejected}, domain.HumanItemRecord{ID: "new-review", RunID: "new-child", CurrentStageID: "acceptance", Status: domain.HumanItemAccepted}) + r := Build(d) + if r.Sources[0].Stage != "delivered" || r.Sources[0].Exception { + t.Fatalf("obsolete package still affects current progress: %+v", r.Sources) + } +} diff --git a/internal/store/human_loop.go b/internal/store/human_loop.go index c9799c0d..075d4ce4 100644 --- a/internal/store/human_loop.go +++ b/internal/store/human_loop.go @@ -26,7 +26,7 @@ func humanBatchMatches(item domain.HumanBatchRecord, query domain.HumanBatchQuer } func humanItemMatches(item domain.HumanItemRecord, query domain.HumanItemQuery) bool { - return (query.LegacyTaskID == "" || item.LegacyTaskID == query.LegacyTaskID) && + return (query.IDs == nil || containsHumanItemID(query.IDs, item.ID)) && (query.LegacyTaskID == "" || item.LegacyTaskID == query.LegacyTaskID) && (query.ActorUserID == "" || item.AssignedUserID == query.ActorUserID || item.LatestAuthorUserID == query.ActorUserID) && (query.PluginID == "" || item.PluginID == query.PluginID) && (query.BatchID == "" || item.BatchID == query.BatchID) && @@ -436,6 +436,9 @@ func humanBatchMongoFilter(organizationID string, query domain.HumanBatchQuery) func humanItemMongoFilter(organizationID string, query domain.HumanItemQuery) bson.M { filter := bson.M{"organizationId": normalizedOrganizationID(organizationID)} + if query.IDs != nil { + filter["id"] = bson.M{"$in": query.IDs} + } if query.LegacyTaskID != "" { filter["legacyTaskId"] = query.LegacyTaskID } @@ -657,3 +660,40 @@ func (store *MemoryStore) ListHumanBatchAccessRecords(ctx context.Context, organ } return items, nil } + +func containsHumanItemID(ids []string, id string) bool { + for _, candidate := range ids { + if candidate == id { + return true + } + } + return false +} + +// Query decision events, not current assignments: completion releases ownership. +func (store *MemoryStore) ListReviewedHumanItemIDs(_ context.Context, organizationID, actorID string) ([]string, error) { + store.mu.Lock() + defer store.mu.Unlock() + ids := []string{} + seen := map[string]bool{} + for _, event := range store.humanEvents { + if humanOrganizationMatches(event.OrganizationID, organizationID) && event.ActorUserID == actorID && (event.EventType == "approved" || event.EventType == "rejected" || event.EventType == "combined_quality_approved" || event.EventType == "combined_quality_rejected") && !seen[event.ItemID] { + seen[event.ItemID] = true + ids = append(ids, event.ItemID) + } + } + return ids, nil +} +func (store *MongoStore) ListReviewedHumanItemIDs(ctx context.Context, organizationID, actorID string) ([]string, error) { + values, err := store.humanEvents.Distinct(ctx, "itemId", bson.M{"organizationId": normalizedOrganizationID(organizationID), "actorUserId": actorID, "eventType": bson.M{"$in": []string{"approved", "rejected", "combined_quality_approved", "combined_quality_rejected"}}}) + if err != nil { + return nil, err + } + ids := []string{} + for _, value := range values { + if id, ok := value.(string); ok { + ids = append(ids, id) + } + } + return ids, nil +} diff --git a/internal/store/memory.go b/internal/store/memory.go index 2e5c2cfb..cb2c270e 100644 --- a/internal/store/memory.go +++ b/internal/store/memory.go @@ -9,9 +9,11 @@ import ( "time" "droboticflow/internal/domain" + "droboticflow/internal/productionstats" ) type MemoryStore struct { + productionStatsScopes map[string]productionstats.Scope reviewContinuations map[string]domain.ReviewContinuation reviewOperations map[string]domain.ReviewOperation reviewDrafts map[string]domain.ReviewDraft diff --git a/internal/store/production_stats.go b/internal/store/production_stats.go new file mode 100644 index 00000000..7e56dba5 --- /dev/null +++ b/internal/store/production_stats.go @@ -0,0 +1,172 @@ +package store + +import ( + "context" + "droboticflow/internal/productionstats" + "go.mongodb.org/mongo-driver/bson" + "go.mongodb.org/mongo-driver/mongo" + "go.mongodb.org/mongo-driver/mongo/options" +) + +func (s *MemoryStore) GetProductionStatsScope(_ context.Context, org string) (productionstats.Scope, error) { + s.mu.Lock() + defer s.mu.Unlock() + scope := s.productionStatsScopes[org] + if scope.ProjectIDs == nil { + scope.ProjectIDs = []string{} + } + return clone(scope), nil +} +func (s *MemoryStore) SaveProductionStatsScope(_ context.Context, org string, next productionstats.Scope, revision int) (bool, error) { + s.mu.Lock() + defer s.mu.Unlock() + if s.productionStatsScopes[org].Revision != revision { + return false, nil + } + if s.productionStatsScopes == nil { + s.productionStatsScopes = map[string]productionstats.Scope{} + } + s.productionStatsScopes[org] = clone(next) + return true, nil +} +func (s *MongoStore) GetProductionStatsScope(ctx context.Context, org string) (productionstats.Scope, error) { + result := productionstats.Scope{ProjectIDs: []string{}} + err := s.db.Collection("productionStatsScopes").FindOne(ctx, bson.M{"_id": org}).Decode(&result) + if err == mongo.ErrNoDocuments { + err = nil + } + return result, err +} +func (s *MongoStore) SaveProductionStatsScope(ctx context.Context, org string, next productionstats.Scope, revision int) (bool, error) { + c := s.db.Collection("productionStatsScopes") + if revision == 0 { + _, err := c.InsertOne(ctx, bson.M{"_id": org, "projectIds": next.ProjectIDs, "revision": next.Revision, "updatedAt": next.UpdatedAt, "updatedBy": next.UpdatedBy}) + if mongo.IsDuplicateKeyError(err) { + return false, nil + } + return err == nil, err + } + result, err := c.UpdateOne(ctx, bson.M{"_id": org, "revision": revision}, bson.M{"$set": next}) + if err != nil { + return false, err + } + return result.MatchedCount == 1, nil +} +func (s *MemoryStore) LoadProductionStats(ctx context.Context, org string, projects []string) (productionstats.Snapshot, error) { + s.mu.Lock() + defer s.mu.Unlock() + d := productionstats.Snapshot{} + allowed := map[string]bool{} + for _, id := range projects { + allowed[id] = true + } + belongs := func(o, p string) bool { return organizationMatches(o, org) && allowed[p] } + for _, v := range s.captureRecords { + if belongs(v.OrganizationID, v.ProjectID) { + d.Captures = append(d.Captures, clone(v)) + } + } + for _, v := range s.runs { + if belongs(v.OrganizationID, v.ProjectID) { + d.Runs = append(d.Runs, clone(v)) + } + } + items := map[string]bool{} + for _, v := range s.humanItems { + if belongs(v.OrganizationID, v.ProjectID) { + d.Items = append(d.Items, clone(v)) + items[v.ID] = true + } + } + for _, v := range s.humanBatches { + if belongs(v.OrganizationID, v.ProjectID) { + d.Batches = append(d.Batches, clone(v)) + } + } + for _, v := range s.annotationTasks { + if belongs(v.OrganizationID, v.ProjectID) { + d.Tasks = append(d.Tasks, clone(v)) + } + } + for _, v := range s.annotationWorklogEvents { + if belongs(v.OrganizationID, v.ProjectID) { + d.Worklogs = append(d.Worklogs, clone(v)) + } + } + for _, v := range s.humanEvents { + if organizationMatches(v.OrganizationID, org) && items[v.ItemID] { + d.Events = append(d.Events, clone(v)) + } + } + for _, v := range s.deliveryPackages { + if belongs(v.OrganizationID, v.ProjectID) { + d.Packages = append(d.Packages, clone(v)) + } + } + return d, nil +} + +// Project-scoped, unpaginated and projected reads: do not load video/artifact +// payloads or inherit the automation list's 200-record UI limit. No migrations. +func (s *MongoStore) LoadProductionStats(ctx context.Context, org string, projects []string) (productionstats.Snapshot, error) { + d := productionstats.Snapshot{} + if len(projects) == 0 { + return d, nil + } + filter := organizationScopedFilter(org, bson.M{"projectId": bson.M{"$in": projects}}) + read := func(c *mongo.Collection, f bson.M, projection bson.M, target any) error { + cursor, err := c.Find(ctx, f, options.Find().SetProjection(projection)) + if err != nil { + return err + } + defer cursor.Close(ctx) + return cursor.All(ctx, target) + } + fields := func(names ...string) bson.M { + m := bson.M{} + for _, name := range names { + m[name] = 1 + } + return m + } + if err := read(s.captureRecords, filter, fields("id", "projectId", "connectionId", "bucket", "prefix", "sourcePrefix", "captureId", "runId", "status", "businessStage", "discoveredAt", "completedAt"), &d.Captures); err != nil { + return d, err + } + captureIDs := make([]string, 0, len(d.Captures)) + runIDs := make([]string, 0, len(d.Captures)) + for _, c := range d.Captures { + captureIDs = append(captureIDs, c.ID) + if c.RunID != "" { + runIDs = append(runIDs, c.RunID) + } + } + runFilter := organizationScopedFilter(org, bson.M{"$or": bson.A{bson.M{"projectId": bson.M{"$in": projects}}, bson.M{"taskId": bson.M{"$in": captureIDs}}, bson.M{"id": bson.M{"$in": runIDs}}}}) + if err := read(s.runs, runFilter, fields("id", "projectId", "workflowId", "taskId", "trigger", "mode", "parentRunId", "createdAt", "endedAt", "status", "deliveryOutcome"), &d.Runs); err != nil { + return d, err + } + if err := read(s.humanItems, filter, fields("id", "projectId", "supplierId", "runId", "batchId", "legacyTaskId", "currentStageId", "status", "pendingCombinedSubmissionId", "releasePartitionId", "processingJob.status"), &d.Items); err != nil { + return d, err + } + if err := read(s.humanBatches, filter, fields("id", "sourceBatchId", "stages"), &d.Batches); err != nil { + return d, err + } + if err := read(s.annotationTasks, filter, fields("id", "runId", "supplierId", "status"), &d.Tasks); err != nil { + return d, err + } + if err := read(s.annotationWorklogEvents, filter, fields("id", "taskId", "supplierId", "eventType", "mode", "temporary", "createdAt"), &d.Worklogs); err != nil { + return d, err + } + ids := make([]string, 0, len(d.Items)) + for _, item := range d.Items { + ids = append(ids, item.ID) + } + if len(ids) > 0 { + if err := read(s.humanEvents, organizationScopedFilter(org, bson.M{"itemId": bson.M{"$in": ids}}), fields("id", "itemId", "stageId", "eventType", "createdAt", "payload.sourceItemIds", "payload.deliveryPackageId"), &d.Events); err != nil { + return d, err + } + } + if err := read(s.deliveryPackages, filter, fields("id", "parentRunId", "childRunId", "supplierId", "mode", "temporary", "finalExportedAt", "status", "itemIds", "releasePartitionId", "acceptanceHumanItemId", "acceptanceReviews.id", "acceptanceReviews.decision", "acceptanceReviews.createdAt", "qualitySummary.prePackageAcceptance"), &d.Packages); err != nil { + return d, err + } + return d, nil +} diff --git a/internal/store/production_stats_test.go b/internal/store/production_stats_test.go new file mode 100644 index 00000000..e3cbce37 --- /dev/null +++ b/internal/store/production_stats_test.go @@ -0,0 +1,104 @@ +package store + +import ( + "context" + "droboticflow/internal/domain" + "droboticflow/internal/productionstats" + "fmt" + "go.mongodb.org/mongo-driver/mongo" + "go.mongodb.org/mongo-driver/mongo/options" + "os" + "testing" + "time" +) + +func TestProductionStatsMongoPersistenceAndReadonlyProjection(t *testing.T) { + uri := os.Getenv("PRODUCTION_STATS_TEST_MONGO_URI") + if uri == "" { + t.Skip("isolated Mongo required") + } + if uri != "mongodb://127.0.0.1:27028" { + t.Fatal("only isolated loopback test server allowed") + } + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + client, err := mongo.Connect(ctx, options.Client().ApplyURI(uri)) + if err != nil { + t.Fatal(err) + } + defer client.Disconnect(context.Background()) + db := client.Database(fmt.Sprintf("production_stats_test_%d", time.Now().UnixNano())) + defer db.Drop(context.Background()) + s := NewMongoStore(db) + c := domain.CaptureRecord{ID: "c", OrganizationID: "o", ProjectID: "p", Prefix: "c", RunID: "r"} + if err = s.UpsertCaptureRecord(ctx, c); err != nil { + t.Fatal(err) + } + if err = s.UpsertRun(ctx, domain.RunRecord{ID: "r", OrganizationID: "o", TaskID: "c", ProjectID: "p", Status: domain.RunStatusWaitingForHuman}); err != nil { + t.Fatal(err) + } + if err = s.UpsertHumanItem(ctx, domain.HumanItemRecord{ID: "i", OrganizationID: "o", ProjectID: "p", RunID: "r", LegacyTaskID: "task", CurrentStageID: "quality_review", SupplierID: "v"}); err != nil { + t.Fatal(err) + } + ok, err := s.SaveProductionStatsScope(ctx, "o", productionstats.Scope{ProjectIDs: []string{"p"}, Revision: 1}, 0) + if !ok || err != nil { + t.Fatalf("save %v %v", ok, err) + } + restarted := NewMongoStore(db) + scope, err := restarted.GetProductionStatsScope(ctx, "o") + if err != nil || scope.Revision != 1 || len(scope.ProjectIDs) != 1 { + t.Fatalf("restart %+v %v", scope, err) + } + if ok, err = restarted.SaveProductionStatsScope(ctx, "o", productionstats.Scope{Revision: 1}, 0); ok || err != nil { + t.Fatalf("stale creation %v %v", ok, err) + } + + if err = s.UpsertAnnotationTask(ctx, domain.AnnotationTaskRecord{ID: "task", OrganizationID: "o", ProjectID: "p", RunID: "r"}); err != nil { + t.Fatal(err) + } + if err = s.UpsertAnnotationWorklogEvent(ctx, domain.AnnotationWorklogEventRecord{ID: "legacy-q", OrganizationID: "o", ProjectID: "p", TaskID: "task", SupplierID: "historic", EventType: "quality_review_approved", CreatedAt: "2026-09-26T00:00:00Z"}); err != nil { + t.Fatal(err) + } + if err = s.UpsertDeliveryPackage(ctx, domain.DeliveryPackageRecord{ID: "pkg", OrganizationID: "o", ProjectID: "p", ParentRunID: "r", SupplierID: "historic", FinalExportedAt: "2026-09-26T01:00:00Z", Status: domain.DeliveryPackageAccepted, AcceptanceReviews: []domain.DeliveryPackageAcceptanceReview{{ID: "canonical-approval", Decision: domain.HumanDecisionApproved, CreatedAt: "2026-09-26T00:45:00Z"}}}); err != nil { + t.Fatal(err) + } + if err = s.AppendHumanEvent(ctx, domain.HumanEventRecord{ID: "pre-accept", OrganizationID: "o", ItemID: "i", StageID: "prepackage_acceptance", EventType: "approved", CreatedAt: "2026-09-26T00:30:00Z", Payload: map[string]any{"sourceItemIds": []string{"i"}}}); err != nil { + t.Fatal(err) + } + if err = s.AppendHumanEvent(ctx, domain.HumanEventRecord{ID: "pkg-reject", OrganizationID: "o", ItemID: "i", StageID: "acceptance", EventType: "rejected", CreatedAt: "2026-09-25T00:00:00Z", Payload: map[string]any{"deliveryPackageId": "pkg"}}); err != nil { + t.Fatal(err) + } + d, err := restarted.LoadProductionStats(ctx, "o", scope.ProjectIDs) + if err != nil { + t.Fatal(err) + } + r := productionstats.Build(d) + if len(r.Sources) != 1 || r.Sources[0].Stage != "quality" || r.Sources[0].SupplierID != "v" { + t.Fatalf("projected facts %+v", r) + } + + facts := map[string]bool{} + canonical := false + for _, event := range r.Events { + if event.ID == "canonical-approval" { + canonical = true + } + if event.Metric == "quality" || event.Metric == "delivered" || event.Metric == "acceptance" || event.Metric == "rejected" { + if event.SupplierID != "historic" { + t.Fatalf("projection lost supplier: %+v", event) + } + facts[event.Metric] = true + } + } + if !canonical || !facts["quality"] || !facts["delivered"] || !facts["acceptance"] || !facts["rejected"] { + t.Fatalf("projection lost event/package: %+v", r.Events) + } + other, err := restarted.LoadProductionStats(ctx, "other", scope.ProjectIDs) + if err != nil || len(other.Captures) != 0 { + t.Fatalf("tenant leak %+v %v", other, err) + } + still, err := s.GetCaptureRecord(ctx, "c") + if err != nil || still.Prefix != c.Prefix || still.RunID != c.RunID { + t.Fatalf("report mutated source %+v %v", still, err) + } +} diff --git a/internal/store/store.go b/internal/store/store.go index 28787fba..3931a94b 100644 --- a/internal/store/store.go +++ b/internal/store/store.go @@ -6,6 +6,7 @@ import ( "time" "droboticflow/internal/domain" + "droboticflow/internal/productionstats" ) var ( @@ -22,6 +23,9 @@ type WorkerRunCounts struct { } type Store interface { + GetProductionStatsScope(context.Context, string) (productionstats.Scope, error) + SaveProductionStatsScope(context.Context, string, productionstats.Scope, int) (bool, error) + LoadProductionStats(context.Context, string, []string) (productionstats.Snapshot, error) GetReviewOperation(context.Context, string) (*domain.ReviewOperation, error) FindReviewOperation(context.Context, string, string, string, string) (*domain.ReviewOperation, error) CompareAndSwapReviewOperation(context.Context, domain.ReviewOperation, int) (bool, error) @@ -255,6 +259,7 @@ type Store interface { ClaimHumanItemVersioned(context.Context, domain.HumanItemRecord, int, domain.UserRole) (bool, error) ListHumanStageAttempts(context.Context, string, string) ([]domain.HumanStageAttemptRecord, error) UpsertHumanStageAttempt(context.Context, domain.HumanStageAttemptRecord) error + ListReviewedHumanItemIDs(context.Context, string, string) ([]string, error) ListHumanEvents(context.Context, string, string) ([]domain.HumanEventRecord, error) AppendHumanEvent(context.Context, domain.HumanEventRecord) error ListHumanRejects(context.Context, string, domain.HumanRejectQuery) ([]domain.HumanRejectRecord, error) diff --git a/src/components/ProductionStatistics.tsx b/src/components/ProductionStatistics.tsx new file mode 100644 index 00000000..a9aa2e4d --- /dev/null +++ b/src/components/ProductionStatistics.tsx @@ -0,0 +1,512 @@ +import React, { useEffect, useMemo, useRef, useState } from "react"; +import { Link } from "react-router-dom"; +import { productionDashboardApi, type ProductionResult } from "../services/api/production-dashboard"; +import { getActionErrorMessage } from "../lib/action-errors.js"; +const stages: Record = { + intake: "接入与自动处理", + annotation: "标注", + quality: "质检", + acceptance: "终审", + delivery: "交付处理", + delivered: "已交付", + ended: "已结束未交付", + unknown: "待核对", +}; +const metrics: Record = { + new: "新增数据", + submitted: "成功送检", + quality: "质检通过", + acceptance: "终审通过", + delivered: "成功交付", + rejected: "退回次数", +}; +const control = "rounded-lg border border-[color:var(--app-border)] bg-[color:var(--app-panel)] px-3 py-2 text-sm"; +const beijingDate = (offset = 0) => new Date(Date.now() + 8 * 3600000 + offset * 86400000).toISOString().slice(0, 10); +export function ProductionStatistics() { + const [view, setView] = useState("current"), + [projectId, setProject] = useState(""), + [supplierId, setSupplier] = useState(""), + [group, setGroup] = useState("project"), + [activeOnly, setActive] = useState(false), + [from, setFrom] = useState(() => beijingDate(-6)), + [to, setTo] = useState(() => beijingDate()), + [search, setSearch] = useState(""), + [sort, setSort] = useState(""), + [tick, setTick] = useState(0); + const [options, setOptions] = useState>({ + projects: [], + suppliers: [], + }); + const [result, setResult] = useState(null), + [error, setError] = useState(""), + [loading, setLoading] = useState(false); + const [drawer, setDrawer] = useState | null>(null), + [detail, setDetail] = useState(null), + [detailError, setDetailError] = useState(""), + [detailSearch, setDetailSearch] = useState(""), + [page, setPage] = useState(1); + const sequence = useRef(0), + detailSequence = useRef(0), + lastKey = useRef(""); + const params = useMemo( + () => ({ + view, + projectId, + supplierId, + group, + activeOnly: String(activeOnly), + ...(view === "period" ? { from, to } : {}), + }), + [view, projectId, supplierId, group, activeOnly, from, to], + ); + const key = JSON.stringify(params); + useEffect(() => { + const seq = ++sequence.current; + setLoading(true); + if (lastKey.current !== key) { + setResult(null); + lastKey.current = key; + } + productionDashboardApi + .get(params) + .then((r) => { + if (seq === sequence.current) { + setResult(r); + setOptions({ projects: r.projects, suppliers: r.suppliers }); + setError(""); + } + }) + .catch((e) => { + if (seq === sequence.current) setError(getActionErrorMessage(e, "统计加载失败")); + }) + .finally(() => { + if (seq === sequence.current) setLoading(false); + }); + return () => { + sequence.current++; + }; + }, [key, tick]); + useEffect(() => { + const timer = setInterval(() => { + if (!document.hidden) setTick((x) => x + 1); + }, 30000); + return () => clearInterval(timer); + }, []); + useEffect(() => { + if (!drawer) return; + const seq = ++detailSequence.current; + setDetail(null); + setDetailError(""); + const timer = setTimeout(() => { + productionDashboardApi + .get({ ...drawer, details: "true", search: detailSearch, page: String(page) }) + .then((r) => { + if (seq === detailSequence.current) setDetail(r); + }) + .catch((e) => { + if (seq === detailSequence.current) setDetailError(getActionErrorMessage(e, "明细加载失败")); + }); + }, 200); + return () => { + clearTimeout(timer); + detailSequence.current++; + }; + }, [drawer, detailSearch, page]); + const open = (metric: string, extra: Record = {}) => { + setPage(1); + setDetailSearch(""); + setDrawer({ ...params, metric, ...extra }); + }; + const reset = () => { + setProject(""); + setSupplier(""); + setSearch(""); + setActive(false); + setDrawer(null); + }; + const names = new Map([...(result?.projects || []), ...(result?.suppliers || [])].map((x) => [x.id, x.name])); + const columns = view === "current" ? { total: "源数据量", ...stages } : metrics; + const rows = (result?.summary.rows || []) + .filter((r) => (names.get(r.id) || r.id).toLowerCase().includes(search.toLowerCase())) + .slice() + .sort((a, b) => (sort ? (b.counts[sort] || 0) - (a.counts[sort] || 0) : 0)); + const cards = + view === "current" + ? { total: "项目源数据量", delivered: "当前已交付", attention: "异常/待核对" } + : { new: "新增数据", submitted: "成功送检", delivered: "成功交付" }; + return ( +
+
+
+

生产总览

+

+ 按项目来源去重 · 单位:份 + {result + ? ` · 截至 ${new Date(result.asOf).toLocaleString("zh-CN", { timeZone: "Asia/Shanghai" })} 北京时间` + : ""} +

+
+
+ + 统计范围 + + +
+
+
+ + + + + + +
+

+ 当前范围:{names.get(projectId) || projectId || "全部生产项目"} /{" "} + {names.get(supplierId) || supplierId || "全部供应商"} + {activeOnly ? " · 仅进行中" : " · 含已归档"} +

+ {view === "period" && ( +
+ {[ + ["今日", 0], + ["近7日", -6], + ["近30日", -29], + ].map(([label, days]) => ( + + ))} + setFrom(e.target.value)} + /> + 至 + setTo(e.target.value)} + /> + 北京时间,含结束日 +
+ )} + {error && ( +

+ {error} + {result ? ",保留上次成功结果" : ",统计暂不可用"} +

+ )} + {!result && loading &&

正在加载统计…

} + {result && !result.configured ? ( +

尚未配置生产统计范围,请先到系统配置选择生产项目。

+ ) : ( + result && ( + <> +
+ {Object.entries(cards).map(([metric, label]) => ( + + ))} +
+
+ {Object.entries(view === "current" ? stages : metrics) + .filter(([id]) => id !== "unknown" || result.summary.totals[id]) + .map(([id, label]) => ( + + ))} +
+ {view === "period" && result.summary.incomplete && ( +

+ 历史记录不完整,展示可核实的完成记录;缺失记录不代表实际产出为零。 +

+ )} + {view === "period" && result.summary.supplierOverlap && ( +

同源由多家供应商参与,供应商小计不能直接相加。

+ )} +
+ + setSearch(e.target.value)} + /> +
+
+ + + + + {Object.entries(columns).map(([id, label]) => ( + + ))} + + + + {rows.map((row) => ( + + + {Object.keys(columns).map((id) => ( + + ))} + + ))} + +
{group === "project" ? "项目" : "供应商"} + +
+ + + +
+ {!rows.length &&

当前范围暂无数据

} +
+ + ) + )} + {drawer && ( +
setDrawer(null)}> + +
+ )} +
+ ); +} diff --git a/src/components/ProductionStatisticsScope.tsx b/src/components/ProductionStatisticsScope.tsx new file mode 100644 index 00000000..97d4b1af --- /dev/null +++ b/src/components/ProductionStatisticsScope.tsx @@ -0,0 +1,103 @@ +import React, { useEffect, useState } from "react"; +import { + productionDashboardApi, + type ProductionScope, + type ProductionOption, +} from "../services/api/production-dashboard"; +import { getActionErrorMessage } from "../lib/action-errors.js"; +export function ProductionStatisticsScope() { + const [scope, setScope] = useState(null), + [projects, setProjects] = useState([]), + [error, setError] = useState(""), + [saving, setSaving] = useState(false), + [saved, setSaved] = useState(false); + const load = () => + productionDashboardApi + .scope() + .then((r) => { + setScope(r.scope); + setProjects(r.projects); + setError(""); + }) + .catch((e) => setError(getActionErrorMessage(e, "统计范围加载失败"))); + useEffect(() => { + void load(); + }, []); + const save = async () => { + if (!scope) return; + setSaving(true); + setSaved(false); + try { + const r = await productionDashboardApi.saveScope(scope); + setScope(r.scope); + setError(""); + setSaved(true); + } catch (e) { + setError(getActionErrorMessage(e, "保存失败,请重新加载后重试")); + } finally { + setSaving(false); + } + }; + return ( +
+

生产统计范围

+

+ 仅勾选客户生产项目,内部测试请勿纳入。此设置只影响统计展示,不修改项目权限、任务或工作流。 +

+ {error && ( +

+ {error}{" "} + +

+ )} + {!scope && !error ?

加载中…

: null} + {scope && ( + <> +
+ {projects.map((p) => ( + + ))} +
+
+ + + {saved + ? "已保存" + : scope.revision === 0 + ? "尚未配置,仪表盘不会把未配置显示成零数据" + : `已选择 ${scope.projectIds.length} 个项目`} + +
+ + )} +
+ ); +} diff --git a/src/pages/AnnotationOperations.tsx b/src/pages/AnnotationOperations.tsx index 35f5fa21..4a3f4222 100644 --- a/src/pages/AnnotationOperations.tsx +++ b/src/pages/AnnotationOperations.tsx @@ -146,7 +146,7 @@ import { BusinessExecutionPath } from "../components/BusinessExecutionPath"; type Tab = "queue" | "rejects" | "templates" | "workload"; type WorkspaceMode = "clip" | "clip-quality" | "annotate" | "quality" | "acceptance"; -type TaskPoolView = "available" | "mine" | "all"; +type TaskPoolView = "available" | "mine" | "all" | "reviewed"; type HumanPools = Record; type CombinedEditorPhase = "slice" | "materializing" | "annotate"; type AnnotationEditSnapshot = { @@ -176,7 +176,7 @@ type SliceEditSnapshot = { location: { sliceId: string; time: number }; }; -const emptyHumanPools: HumanPools = { available: [], mine: [], all: [] }; +const emptyHumanPools: HumanPools = { available: [], mine: [], all: [], reviewed: [] }; const projectLoadRecoveryAttempts = 3; const projectLoadRecoveryDelayMs = 2_000; @@ -205,6 +205,7 @@ export const deriveDetailedTaskStatus = ( humanItem?: HumanItem, livePhase?: CombinedEditorPhase, ): DetailedTaskStatus => { + if (humanItem?.reviewHistoryStatus) return humanItem.reviewHistoryStatus; if (humanItem?.status === "accepted" || task.status === "accepted") return "accepted"; if ( task.status === "pending_quality" || @@ -477,6 +478,7 @@ function taskVisibleInMode(task: AnnotationTaskRecord, mode: WorkspaceMode, pool } function humanItemVisibleInMode(item: HumanItem, mode: WorkspaceMode, poolView?: TaskPoolView) { + if (poolView === "reviewed") return true; if (mode === "clip") return item.currentStageId === "split_annotation" || item.currentStageId === "cut_adjustment"; if (mode === "clip-quality") return item.currentStageId === "split_quality_review" || item.currentStageId === "cut_review"; @@ -965,7 +967,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: setTemplates(nextTemplates); setWorkflows(nextWorkflows); setTasks(details.map((d) => d.task)); - setHumanPools({ available: [], mine, all: [] }); + setHumanPools({ available: [], mine, all: [], reviewed: [] }); setClaimableCount({ projectId, count: count?.count ?? null }); projectLoadRecoveryAttemptRef.current = 0; if (projectLoadRecoveryTimerRef.current !== null) { @@ -985,6 +987,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: availableHumanItems, myHumanItems, allHumanItems, + reviewedHumanItems, nextWorkload, nextRejects, nextPackages, @@ -1031,6 +1034,15 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: pool: "all", }) : Promise.resolve([] as HumanItem[]), + reviewOnly && workspaceMode !== "acceptance" + ? humanLoopApi.listItems({ + pluginId: "annotation-qa", + projectId: projectId || undefined, + workflowId: requestedWorkflowId || undefined, + runId: requestedRunId || undefined, + pool: "reviewed", + }) + : Promise.resolve([] as HumanItem[]), reviewOnly ? Promise.resolve(null) : annotationOperationsApi.workloadSummary(projectId || undefined), reviewOnly ? Promise.resolve(null) @@ -1066,6 +1078,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: submittedItemIdsRef.current.has(i.id) ? { ...i, status: "ready", currentStageId: "quality_review" } : i, ), all: allHumanItems, + reviewed: reviewedHumanItems, }); if (nextWorkload) setWorkload(nextWorkload); if (nextRejects) setRejects(nextRejects); @@ -1165,6 +1178,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: setLoading(false); setRefreshingProject(false); setHumanPools((current) => ({ + ...current, available: current.available.filter((i) => i.id !== item.id), mine: [item, ...current.mine.filter((i) => i.id !== item.id)], all: current.all.map((i) => (i.id === item.id ? item : i)), @@ -1173,6 +1187,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: const retireReviewedItem = (itemId: string, taskId?: string) => { ++projectLoadRequestRef.current; setHumanPools((current) => ({ + ...current, available: current.available.filter((i) => i.id !== itemId), mine: current.mine.filter((i) => i.id !== itemId), all: current.all, @@ -1388,7 +1403,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: setCombinedEditorPhase("slice"); } annotationOperationsApi - .getTask(selectedTaskId, controller.signal) + .getTask(selectedTaskId, controller.signal, taskPoolView === "reviewed") .then((next) => { if (requestID !== taskDetailRequestRef.current || annotationMutationScopeRef.current.busy()) return; if ( @@ -1474,9 +1489,22 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: setCombinedEditorPhase(nextPhase); if (next.humanItem) { setHumanPools((current) => ({ - available: current.available.map((item) => (item.id === next.humanItem?.id ? next.humanItem! : item)), - mine: current.mine.map((item) => (item.id === next.humanItem?.id ? next.humanItem! : item)), - all: current.all.map((item) => (item.id === next.humanItem?.id ? next.humanItem! : item)), + ...current, + available: current.available.map((item) => + item.id === next.humanItem?.id + ? { ...next.humanItem!, reviewHistoryStatus: item.reviewHistoryStatus } + : item, + ), + mine: current.mine.map((item) => + item.id === next.humanItem?.id + ? { ...next.humanItem!, reviewHistoryStatus: item.reviewHistoryStatus } + : item, + ), + all: current.all.map((item) => + item.id === next.humanItem?.id + ? { ...next.humanItem!, reviewHistoryStatus: item.reviewHistoryStatus } + : item, + ), })); } annotationBaselineRef.current = annotationWorkspaceFingerprint( @@ -1517,7 +1545,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: } }); return () => controller.abort(); - }, [currentUser?.id, selectedTaskId, taskDetailRefreshVersion, loadFrameTimeline]); + }, [currentUser?.id, selectedTaskId, taskPoolView, taskDetailRefreshVersion, loadFrameTimeline]); useEffect(() => { if (combinedEditorPhase !== "materializing" || !selectedTaskId) return; @@ -1682,6 +1710,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: prepackageAcceptanceItems.find((item) => item.id === expandedDeliveryPackageId) || detail?.humanItem; const reviewSessionEnabled = Boolean( (workspaceMode === "quality" || workspaceMode === "clip-quality" || workspaceMode === "acceptance") && + taskPoolView !== "reviewed" && reviewSessionItem?.assignedUserId === currentUser?.id && ["claimed", "in_progress"].includes(reviewSessionItem?.status || ""), ); @@ -1769,27 +1798,29 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: // any pause latches the scope so recovery stays manual. const reviewAutoEntryReady = Boolean( reviewOnly && - reviewSessionEnabled && - // Legacy-protocol tasks keep manual-only review: an explicit entry must - // never send a continuation start the task cannot support, and the - // resulting session error would invalidate a usable legacy review. - reviewReceiptSupported && - reviewSession.writable && - !viewportPaused && - !reviewWindowUndersized && - !reviewDecision.pending && - !reviewDraft.dirty && - !reviewContinuation.busy && - (workspaceMode === "acceptance" - ? expandedDeliveryPackageId && reviewSessionItem && expandedDeliveryPackageId === reviewSessionItem.id - : detail?.humanItem && detail.humanItem.id === reviewSessionItem?.id), + reviewSessionEnabled && + // Legacy-protocol tasks keep manual-only review: an explicit entry must + // never send a continuation start the task cannot support, and the + // resulting session error would invalidate a usable legacy review. + reviewReceiptSupported && + reviewSession.writable && + !viewportPaused && + !reviewWindowUndersized && + !reviewDecision.pending && + !reviewDraft.dirty && + !reviewContinuation.busy && + (workspaceMode === "acceptance" + ? expandedDeliveryPackageId && reviewSessionItem && expandedDeliveryPackageId === reviewSessionItem.id + : detail?.humanItem && detail.humanItem.id === reviewSessionItem?.id), ); const reviewEntryItemId = reviewSessionEnabled ? reviewSessionItem?.id || "" : ""; // Prune uses the item the user is actually navigating to, never the async // detail/session state of the previous task: while entering B, a stale A // context must neither consume nor drop B's fresh entry intent. const reviewSelectedItemId = - workspaceMode === "acceptance" ? expandedDeliveryPackageId || requestedAcceptanceId : selectedTaskId || requestedTaskId; + workspaceMode === "acceptance" + ? expandedDeliveryPackageId || requestedAcceptanceId + : selectedTaskId || requestedTaskId; const reviewSelectedHumanItemId = workspaceMode === "acceptance" ? reviewSelectedItemId @@ -1924,7 +1955,10 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: new Map( visibleTasks.map((task) => { const poolItem = visiblePoolItems.find((item) => item.legacyTaskId === task.id); - const humanItem = task.id === selectedTaskId && detail?.humanItem ? detail.humanItem : poolItem; + const humanItem = + task.id === selectedTaskId && detail?.humanItem + ? { ...detail.humanItem, reviewHistoryStatus: poolItem?.reviewHistoryStatus } + : poolItem; const livePhase = task.id === selectedTaskId ? combinedEditorPhase : undefined; return [ task.id, @@ -1952,7 +1986,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: const poolCounts = useMemo( () => Object.fromEntries( - (["available", "mine", "all"] as const).map((pool) => [ + (["available", "mine", "all", "reviewed"] as const).map((pool) => [ pool, humanPools[pool].filter((item) => matchesPoolFilters(item, pool)).length, ]), @@ -2163,6 +2197,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: } }; const selectedClaimedByMe = Boolean( + taskPoolView !== "reviewed" && selectedHumanItem && selectedHumanItem.assignedUserId === currentUser?.id && ["claimed", "in_progress"].includes(selectedHumanItem.status), @@ -2647,6 +2682,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: if (!workspace) throw new Error("保存切片用途失败:服务器返回的标注工作区无效"); setDetail((current) => (current ? { ...current, humanItem: savedItem } : current)); setHumanPools((current) => ({ + ...current, ...current, mine: current.mine.map((i) => (i.id === response.item.id ? savedItem : i)), })); @@ -2962,6 +2998,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: setDetail((current) => (current ? { ...current, humanItem: response.item } : current)); if (response.item) { setHumanPools((current) => ({ + ...current, available: current.available.map((item) => (item.id === response.item.id ? response.item : item)), mine: current.mine.map((item) => (item.id === response.item.id ? response.item : item)), all: current.all.map((item) => (item.id === response.item.id ? response.item : item)), @@ -3063,6 +3100,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: setDetail((current) => (current ? { ...current, humanItem: confirmedItem } : current)); if (confirmedItem) { setHumanPools((current) => ({ + ...current, available: current.available.map((item) => (item.id === confirmedItem.id ? confirmedItem : item)), mine: current.mine.map((item) => (item.id === confirmedItem.id ? confirmedItem : item)), all: current.all.map((item) => (item.id === confirmedItem.id ? confirmedItem : item)), @@ -4127,6 +4165,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: if (!sameScope()) return null; setWorkBuffer(buffer); setHumanPools((pools) => ({ + ...pools, ...pools, mine: [...buffer.items, ...pools.mine.filter((old) => !buffer.items.some((item) => item.id === old.id))], })); @@ -4454,8 +4493,12 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: data-workspace-compact={focusedStageLayout || undefined} > @@ -5073,6 +5116,9 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: [ ["available", blindAnnotator ? "随机领取" : "可领取", poolCounts.available], ["mine", "我的任务", poolCounts.mine], + ...(reviewOnly && workspaceMode !== "acceptance" + ? ([["reviewed", "我已审核", poolCounts.reviewed]] as const) + : []), ...(isManager ? ([["all", "全部任务", poolCounts.all]] as const) : []), ] as Array<[TaskPoolView, string, number]> ).map(([value, label, count]) => ( @@ -5203,18 +5249,23 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: ))} {filteredVisibleTasks.map((task) => { const poolItem = visibleHumanItemByTaskId.get(task.id); - const humanItem = task.id === selectedTaskId && detail?.humanItem ? detail.humanItem : poolItem; + const humanItem = + task.id === selectedTaskId && detail?.humanItem + ? { ...detail.humanItem, reviewHistoryStatus: poolItem?.reviewHistoryStatus } + : poolItem; const livePhase = task.id === selectedTaskId ? combinedEditorPhase : undefined; const detailedStatus = deriveDetailedTaskStatus(task, humanItem, livePhase); const assignee = users.find((user) => user.id === humanItem?.assignedUserId); const captureLabel = captureTaskDisplayName(humanItem, task); const claimable = Boolean( + taskPoolView !== "reviewed" && humanItem && !humanItem.assignedUserId && ["ready", "rework", "submitted"].includes(humanItem.status) && canCurrentUserClaimItem(humanItem), ); const claimedByMe = Boolean( + taskPoolView !== "reviewed" && humanItem?.assignedUserId === currentUser?.id && ["claimed", "in_progress"].includes(humanItem?.status || ""), ); @@ -5226,7 +5277,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: workspaceMode === "annotate" && (isTaskPendingQuality || isTaskPendingAcceptance || isTaskAccepted); const taskAction = annotationTaskAction({ - awaitingDownstream, + awaitingDownstream: taskPoolView !== "reviewed" && awaitingDownstream, status: detailedStatus, claimable, claimedByMe, @@ -5738,6 +5789,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: unifiedBoundaries={frameTimeline?.boundaryContract === "hardware-frame-v1"} readOnlyStatusLabel={statusLabels[detail.task.status] || "只读查看"} readOnly={ + taskPoolView === "reviewed" || (workspaceMode === "annotate" && (detail.task.status === "pending_quality" || detail.task.status === "pending_acceptance" || @@ -5750,6 +5802,7 @@ export default function AnnotationOperations({ workPage = false }: { workPage?: combinedSubmissionLocked } isQualityReview={ + taskPoolView !== "reviewed" && (workspaceMode === "quality" || workspaceMode === "acceptance" || workspaceMode === "clip-quality") && diff --git a/src/pages/Dashboard.tsx b/src/pages/Dashboard.tsx index a3d53b49..1426fe4b 100644 --- a/src/pages/Dashboard.tsx +++ b/src/pages/Dashboard.tsx @@ -1,3 +1,4 @@ +import { ProductionStatistics } from "../components/ProductionStatistics"; import React, { useCallback, useEffect, useState } from "react"; import { AlertCircle, @@ -343,6 +344,7 @@ export default function Dashboard() {

Drobodata 仪表盘

+ {errorMessage && (
diff --git a/src/pages/Settings.tsx b/src/pages/Settings.tsx index 2c645166..df78d25c 100644 --- a/src/pages/Settings.tsx +++ b/src/pages/Settings.tsx @@ -1,3 +1,4 @@ +import { ProductionStatisticsScope } from "../components/ProductionStatisticsScope"; import React, { useEffect, useState } from "react"; import { AlertCircle, Bot, CheckCircle2, Clock3, Cpu, Database, GitBranch, Loader2, Save, Server } from "lucide-react"; import { useTimeZone } from "../context/TimeZoneContext"; @@ -195,6 +196,7 @@ export default function Settings() {

系统配置

+ {errorMessage && (
diff --git a/src/services/api/annotation-operations.ts b/src/services/api/annotation-operations.ts index 8f6f20a3..871591ea 100644 --- a/src/services/api/annotation-operations.ts +++ b/src/services/api/annotation-operations.ts @@ -114,10 +114,10 @@ export const annotationOperationsApi = { artifactId?: string; } = {}, ) => (await apiRequest<{ items: AnnotationTaskRecord[] }>(`/api/plugins/annotation-qa/tasks${query(filters)}`)).items, - getTask: (id: string, signal?: AbortSignal) => - apiRequest(`/api/plugins/annotation-qa/tasks/${id}`, { signal }).then( - normalizeAnnotationTaskDetail, - ), + getTask: (id: string, signal?: AbortSignal, history = false) => + apiRequest(`/api/plugins/annotation-qa/tasks/${id}${history ? "?view=history" : ""}`, { + signal, + }).then(normalizeAnnotationTaskDetail), getTaskSummary: (id: string) => apiRequest<{ task: AnnotationTaskRecord }>(`/api/plugins/annotation-qa/tasks/${id}?view=summary`), getTaskStatus: (id: string, signal?: AbortSignal) => diff --git a/src/services/api/human-loop.ts b/src/services/api/human-loop.ts index 2bbc5fe4..07a34661 100644 --- a/src/services/api/human-loop.ts +++ b/src/services/api/human-loop.ts @@ -10,7 +10,7 @@ type HumanItemFilters = { runId?: string; stageId?: string; status?: HumanItemStatus; - pool?: "available" | "mine" | "all"; + pool?: "available" | "mine" | "all" | "reviewed"; }; const queryString = (filters: Record) => { diff --git a/src/services/api/production-dashboard.ts b/src/services/api/production-dashboard.ts new file mode 100644 index 00000000..11ec2d5f --- /dev/null +++ b/src/services/api/production-dashboard.ts @@ -0,0 +1,53 @@ +import { apiRequest } from "./client"; +export type ProductionOption = { id: string; name: string; status?: string }; +export type ProductionScope = { projectIds: string[]; revision: number; updatedAt?: string }; +export type ProductionSource = { + id: string; + projectId: string; + name: string; + runId?: string; + workflowId?: string; + supplierId: string; + stage: string; + detail: string; + exception: boolean; + discoveredAt: string; + tasks: { id: string; stage: string; status: string }[]; + runs: { id: string; workflowId: string; part: string }[]; + evidence?: { + id: string; + metric: string; + supplierId: string; + at: string; + evidenceAt?: string; + runId?: string; + workflowId?: string; + taskId?: string; + }[]; +}; +export type ProductionResult = { + configured: boolean; + asOf: string; + projects: ProductionOption[]; + suppliers: ProductionOption[]; + summary: { + totals: Record; + stageErrors: Record; + rows: { id: string; counts: Record }[]; + supplierOverlap: boolean; + incomplete: boolean; + }; + items?: ProductionSource[]; + total?: number; + page?: number; +}; +export const productionDashboardApi = { + get: (params: Record) => + apiRequest(`/api/dashboard/production?${new URLSearchParams(params)}`), + scope: () => apiRequest<{ scope: ProductionScope; projects: ProductionOption[] }>("/api/dashboard/production/scope"), + saveScope: (scope: ProductionScope) => + apiRequest<{ scope: ProductionScope; projects: ProductionOption[] }>("/api/dashboard/production/scope", { + method: "PUT", + body: JSON.stringify(scope), + }), +}; diff --git a/src/types.ts b/src/types.ts index 3bb67290..8f1d1fad 100644 --- a/src/types.ts +++ b/src/types.ts @@ -2131,6 +2131,7 @@ export interface HumanTaskDisplayContext { } export interface HumanItem { + reviewHistoryStatus?: "pending_acceptance" | "accepted"; claimedAt?: string; processingScope?: "public" | "internal" | "supplier"; processingProfile?: string;