forked from longtao.wu/droboticflow
♻️ Make annotator deletion handoffs recoverable
This commit is contained in:
parent
4810602703
commit
6b42476962
@ -137,7 +137,9 @@ _Avoid_:回退意见(易与撤销操作混淆)
|
||||
|
||||
**连续审查**:审查者在当前审核环节接续处理任务的工作方式;最终验收跟随项目筛选,“全部项目”跨有权限的项目接续,指定项目仅在该项目接续,质检仍在当前任务所属项目接续。单纯领取保留可领取列表,“继续处理”进入任务并启动连续审查;当前决定实际成功后才进入下一条,暂停不撤回已提交决定。
|
||||
|
||||
**返工任务**:质检退回后定向归还给原标注员、按意见修改并重新送检的任务;原标注员仍有资格时直接归还本人,不再有资格时按既有任务归属规则处理,不回到公共池随机领取。
|
||||
**返工任务**:质检退回后按意见修改并重新送检的任务。原标注员仍有资格时定向归还本人;原标注员已删除时,保留正式标注和质检意见,在原供应商有项目权限且容量允许的标注员中按负载优先、同负载随机分配,不混入普通随机领取。
|
||||
|
||||
**接手返工**:接手已删除原标注员留下的返工任务。每名标注员跨项目最多同时接手一条,不触发本人返工阻止领取新任务的规则;历史意见与责任不因接手而改写。
|
||||
_Avoid_:公共池任务(返工任务定向归属,不参与随机领取)
|
||||
|
||||
**意见处理状态**:标注员对某条退回意见的处理在界面上的呈现——"未修改(红)"或"已修改待复审(灰)"。已修改由"意见目标相对退回基线已改动"自动判定(切片内容含片内动作,不落库、不等于质检确认解决);另有免打字的一键"不改了"(跳过留痕,落库且复审可见),用于误报/不适用。它不是审核意见,也不改写原意见。
|
||||
|
||||
33
docs/operations/deleted-annotator-handoff.md
Normal file
33
docs/operations/deleted-annotator-handoff.md
Normal file
@ -0,0 +1,33 @@
|
||||
# 删除标注员后的任务交接
|
||||
|
||||
状态:规则已确认,本地实现;未执行生产重置、回填或索引建设。
|
||||
|
||||
## 业务规则
|
||||
|
||||
- 从未正式进入质检:删除原标注员后清除当前切片、动作、个人草稿、编辑备注及分类选择,回原供应商标注池重新开始。源视频、任务来源和删除审计保留。
|
||||
- 已正式进入质检:删除原作者不撤销提交,不改变质检、验收和交付结果。已经退回标注的任务也属于这一类,不能按首次标注清空。
|
||||
- 仅原作者已删除的返工,进入供应商内接手分配。候选必须账号有效、同供应商、有该项目标注权限且个人池有容量;负载优先,同负载随机。跨项目每人最多同时接手一条,接手返工不阻止领取普通新任务。
|
||||
- 原作者仍有效时保留定向返工;新规则启用后不再应用旧的公共池返工回退开关。仅停用、撤权等情形不扩大适用删除规则。
|
||||
- 无合适接手人时保留待分配;缺少可靠的正式版本或历史阶段证据时保留内容并报告交接未完成,不能按空白猜测恢复。
|
||||
|
||||
## 并发与重启
|
||||
|
||||
删除意图先持久化并停用账号。任务重置以 HumanItem 版本 CAS 为分界:正式送检先提交则保留质检任务;重置先提交则旧保存和旧送检不能再生效。
|
||||
|
||||
重置提升编辑代次,记录历史修订高水位并撤销后台作业身份。修订写入和重置共用任务粒度的写入锁,锁内重读代次并更新修订高水位,避免旧作业在检查后插入更高修订号。旧 Task 发布、旧切片/标注修订、续租和完成回包均不得复活被重置内容。已失效修订只在工作流读取中隔离,不物理删除审计证据,后续代次不复用旧修订号。
|
||||
|
||||
HumanItem 保留交接待完成记录;重建 Task 投影及审计完成后才释放领取关系。API 重启按持久意图继续,重复请求不重复重置,也不能覆盖新接手人的工作。删除进行中禁止恢复账号为有效状态。
|
||||
|
||||
质检退回决定保留一次待核对标记,落库后重新检查原作者是否已经删除,关闭“审核提交时作者刚好被删除”的窗口。接手从本轮冻结基线开始,旧数据缺少独立基线时只允许使用准确的正式提交修订,不能载入原账号私有草稿。原质检记录和退回意见不改写。
|
||||
|
||||
## 启用及回退
|
||||
|
||||
新入口由 `DELETED_ANNOTATOR_HANDOFF=true` 启用,默认关闭。先升级 API、修订处理器及其他可能写入这些记录的组件,再批准开启,避免混合旧写入者丢弃新代次或私有恢复字段。测试通过不表示已上线。
|
||||
|
||||
启用前须单独批准并建设 [两个查询索引](deleted-annotator-indexes.json)。后台路由在索引缺失或键不符时拒绝扫描,任务保留待交接,不在 API 启动时自动建索引。索引会增加存储、构建 IO 和后续写维护成本,需上线前只读对照及资源评估。
|
||||
|
||||
恢复器每 30 秒检查持久意图;单次有时间预算,失败保留待处理状态。候选负载在单次恢复中复用,领取提交仍重新核对账号、权限、源状态和全局接手上限,不跨请求缓存权限。
|
||||
|
||||
关闭开关暂停自动恢复,并停止创建新的删除重置/返工路由意图;已有交接记录保留,重新开启后继续。已有代次隔离和待交接记录不能用不认识这些字段的旧二进制处理,回退须保留兼容读写者,不能直接回退旧镜像。
|
||||
|
||||
不自动改写历史已删除账号及已领取旧返工。生产补处理须先只读盘点、列明任务范围并单独授权。历史独立切分/语义标注工作流不在此自动重置路径中,遇到这类仍有归属的任务会保留并阻止删除完成,须先按原流程交接。
|
||||
4
docs/operations/deleted-annotator-indexes.json
Normal file
4
docs/operations/deleted-annotator-indexes.json
Normal file
@ -0,0 +1,4 @@
|
||||
[
|
||||
{"collection":"humanItems","name":"deleted_author_routing_v1","keys":{"organizationId":1,"currentStageId":1,"reworkRoutingPending":1,"status":1}},
|
||||
{"collection":"humanItems","name":"deleted_author_unassigned_v1","keys":{"organizationId":1,"currentStageId":1,"inheritedRework":1,"assignedUserId":1,"status":1}}
|
||||
]
|
||||
@ -1,6 +1,6 @@
|
||||
# 返工与复审操作手册(v5.8 三角色流程)
|
||||
|
||||
适用版本:返工与复审工作流上线版(B 块)。术语与 [CONTEXT.md](../CONTEXT.md) 一致:**返工任务**(定向归还原标注员,不回公共池)、**质检审核**、**复审**(质检对返工重提结果的核对模式,不是新阶段)、**复验**(终审对整批退回重检后的核对,必含原失败样本)、**确认已解决**(正式判定,提交后不可撤销)。
|
||||
适用版本:返工与复审工作流上线版(B 块)。术语与 [CONTEXT.md](../CONTEXT.md) 一致:**返工任务**(原作者有效时定向归还,原作者已删除时按供应商内接手规则分配)、**质检审核**、**复审**(质检对返工重提结果的核对模式,不是新阶段)、**复验**(终审对整批退回重检后的核对,必含原失败样本)、**确认已解决**(正式判定,提交后不可撤销)。
|
||||
|
||||
---
|
||||
|
||||
@ -11,8 +11,8 @@
|
||||
### 1. 怎么知道有返工?
|
||||
|
||||
- 返工任务**定向归还本人**,出现在「我的任务」列表**顶部**,带红色「打回」徽标。
|
||||
- 原标注员不可用(停用/换供应商等)时由管理员改派,改派后的归属人同样在「我的任务」看到。
|
||||
- 连续工作模式下**返工优先**:先清完自己的返工任务,系统才派下一条新任务。
|
||||
- 原标注员已删除时,保留正式标注和退回意见,在同供应商有项目标注权限的人员中按负载优先、同负载随机分配;每人跨项目最多同时接手一条。没有合适人员时保持待分配。仅停用或失去授权等其他情况不扩大适用此次删除规则。
|
||||
- 连续工作模式下**返工优先**:先清完自己的返工任务,系统才派下一条新任务。接手已删除人员的返工不执行此限制,仍占正常个人池容量。
|
||||
|
||||
### 2. 打开后看到什么?
|
||||
|
||||
|
||||
@ -2059,7 +2059,7 @@ func (server *server) finishSliceConfirmation(ctx context.Context, taskID string
|
||||
defer cancel()
|
||||
task, taskErr := server.getAnnotationTaskForAccess(ctx, taskID)
|
||||
item, itemErr := server.store.GetHumanItem(ctx, itemID)
|
||||
if taskErr != nil || itemErr != nil || task == nil || item == nil {
|
||||
if taskErr != nil || itemErr != nil || task == nil || item == nil || task.AnnotationEpoch != item.AnnotationEpoch || (item.AccountHandoff != nil && item.AccountHandoff.Pending) {
|
||||
return
|
||||
}
|
||||
w := mapValue(item.Subject.Payload["sliceWorkflow"])
|
||||
@ -2075,7 +2075,7 @@ func (server *server) finishSliceConfirmation(ctx context.Context, taskID string
|
||||
}
|
||||
slices := descriptionOnlySlices(payload.SliceSegments)
|
||||
sliceRevision := domain.SliceRevisionRecord{
|
||||
ID: "slice-confirmation-" + requestID, OrganizationID: task.OrganizationID, TaskID: task.ID,
|
||||
ID: "slice-confirmation-" + requestID, OrganizationID: task.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch,
|
||||
Revision: revision, Segments: slices, Operations: payload.SliceOperations, AuthorUserID: actorUserID,
|
||||
ChangeReason: strings.TrimSpace(payload.ChangeReason), CreatedAt: nowISO(),
|
||||
}
|
||||
@ -2348,7 +2348,7 @@ func (server *server) handleSliceConfirmationCreate(writer http.ResponseWriter,
|
||||
renderError(writer, 500, e)
|
||||
return
|
||||
}
|
||||
revision := 1
|
||||
revision := max(1, task.AnnotationRevisionFloor+1)
|
||||
previous := mapValue(item.Subject.Payload["sliceWorkflow"])
|
||||
retryOf := sliceRetryFailureEvidence(item)
|
||||
if issues := server.validateProjectSliceNames(request.Context(), item, payload.SliceSegments, "confirm"); len(issues) > 0 {
|
||||
@ -2751,6 +2751,10 @@ func (server *server) handleCombinedAnnotationSubmission(writer http.ResponseWri
|
||||
}
|
||||
|
||||
func (server *server) completeCombinedAnnotationSubmission(writer http.ResponseWriter, request *http.Request, task *domain.AnnotationTaskRecord, humanItem *domain.HumanItemRecord, payload combinedAnnotationSubmissionInput, principal *authenticatedPrincipal, submissionRef string, resumed bool) {
|
||||
if task.AnnotationEpoch != humanItem.AnnotationEpoch || (humanItem.AccountHandoff != nil && humanItem.AccountHandoff.Pending) {
|
||||
renderAnnotationSubmissionError(writer, http.StatusConflict, store.ErrHumanClaimConflict)
|
||||
return
|
||||
}
|
||||
if err := server.assertAnnotationLease(request.Context()); err != nil {
|
||||
renderAnnotationSubmissionError(writer, http.StatusConflict, err)
|
||||
return
|
||||
@ -2766,7 +2770,7 @@ func (server *server) completeCombinedAnnotationSubmission(writer http.ResponseW
|
||||
annotationID := "annotation-" + submissionRef
|
||||
// Older releases could delete revision documents while retaining reviews.
|
||||
// Their audit references remain a lower bound even when history is sparse.
|
||||
revisionNumber := max(1, task.CurrentRevision+1, humanItem.CurrentRevision+1)
|
||||
revisionNumber := max(1, task.CurrentRevision+1, humanItem.CurrentRevision+1, task.AnnotationRevisionFloor+1)
|
||||
for _, review := range reviews {
|
||||
revisionNumber = max(revisionNumber, review.Revision+1, review.NextRevision)
|
||||
}
|
||||
@ -2792,8 +2796,8 @@ func (server *server) completeCombinedAnnotationSubmission(writer http.ResponseW
|
||||
}
|
||||
isRework := task.CurrentRevision > 0 || fanyuV5CombinedSubmissionIsRework(annotationRevisions, task.Status)
|
||||
now := nowISO()
|
||||
sliceRevision := domain.SliceRevisionRecord{ID: sliceID, OrganizationID: task.OrganizationID, TaskID: task.ID, Revision: revisionNumber, Segments: payload.SliceSegments, Operations: payload.SliceOperations, AuthorUserID: principal.User.ID, ChangeReason: submissionRef + ":" + strings.TrimSpace(payload.ChangeReason), DurationSeconds: payload.DurationSeconds, CreatedAt: now}
|
||||
annotationRevision := domain.AnnotationRevisionRecord{ID: annotationID, OrganizationID: task.OrganizationID, TaskID: task.ID, Revision: revisionNumber, TemplateVersion: task.TemplateVersion, Segments: payload.Segments, ObjectHints: payload.ObjectHints, AuthorUserID: principal.User.ID, ChangeReason: submissionRef + ":" + strings.TrimSpace(payload.ChangeReason), IsRework: isRework, CountsAsOutput: !isRework, DurationSeconds: payload.DurationSeconds, CreatedAt: now}
|
||||
sliceRevision := domain.SliceRevisionRecord{ID: sliceID, OrganizationID: task.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch, Revision: revisionNumber, Segments: payload.SliceSegments, Operations: payload.SliceOperations, AuthorUserID: principal.User.ID, ChangeReason: submissionRef + ":" + strings.TrimSpace(payload.ChangeReason), DurationSeconds: payload.DurationSeconds, CreatedAt: now}
|
||||
annotationRevision := domain.AnnotationRevisionRecord{ID: annotationID, OrganizationID: task.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch, Revision: revisionNumber, TemplateVersion: task.TemplateVersion, Segments: payload.Segments, ObjectHints: payload.ObjectHints, AuthorUserID: principal.User.ID, ChangeReason: submissionRef + ":" + strings.TrimSpace(payload.ChangeReason), IsRework: isRework, CountsAsOutput: !isRework, DurationSeconds: payload.DurationSeconds, CreatedAt: now}
|
||||
materialization, materializationErr := server.materializeCombinedSliceSubmission(request.Context(), *task, *humanItem, payload, revisionNumber)
|
||||
if request.Context().Err() != nil {
|
||||
renderAnnotationSubmissionError(writer, http.StatusConflict, request.Context().Err())
|
||||
@ -3141,7 +3145,7 @@ func (server *server) handleSliceRevisionCreate(writer http.ResponseWriter, requ
|
||||
return
|
||||
}
|
||||
revision := domain.SliceRevisionRecord{
|
||||
ID: domain.NewID("slice-revision"), OrganizationID: task.OrganizationID, TaskID: task.ID, Revision: len(existing) + 1,
|
||||
ID: domain.NewID("slice-revision"), OrganizationID: task.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch, Revision: max(len(existing)+1, task.AnnotationRevisionFloor+1),
|
||||
Segments: payload.Segments, Operations: payload.Operations, AuthorUserID: principal.User.ID,
|
||||
ChangeReason: strings.TrimSpace(payload.ChangeReason), DurationSeconds: payload.DurationSeconds, CreatedAt: nowISO(),
|
||||
}
|
||||
@ -3303,9 +3307,9 @@ func (server *server) handleAnnotationRevisionCreate(writer http.ResponseWriter,
|
||||
return
|
||||
}
|
||||
isRework := len(existingRevisions) > 0 || task.Status == domain.AnnotationTaskStatusRework
|
||||
revisionNumber := len(existingRevisions) + 1
|
||||
revisionNumber := max(len(existingRevisions)+1, task.AnnotationRevisionFloor+1)
|
||||
revision := domain.AnnotationRevisionRecord{
|
||||
ID: domain.NewID("annotation-revision"), OrganizationID: task.OrganizationID, TaskID: task.ID,
|
||||
ID: domain.NewID("annotation-revision"), OrganizationID: task.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch,
|
||||
Revision: revisionNumber, TemplateVersion: task.TemplateVersion, Segments: payload.Segments, ObjectHints: payload.ObjectHints,
|
||||
AuthorUserID: principal.User.ID, ChangeReason: strings.TrimSpace(payload.ChangeReason),
|
||||
IsRework: isRework, CountsAsOutput: !isRework, DurationSeconds: payload.DurationSeconds, CreatedAt: nowISO(),
|
||||
@ -3727,6 +3731,8 @@ func (server *server) handleFanyuV5CombinedQualityReview(writer http.ResponseWri
|
||||
if payload.Decision == domain.AnnotationReviewRejected {
|
||||
item.ReviewRound++
|
||||
item.CurrentStageID = "annotation_combined"
|
||||
item.ReworkRoutingPending = accountHandoffPolicyEnabled() || item.InheritedRework
|
||||
item.InheritedRework = false
|
||||
if item.ReviewRound > item.MaxReworkRounds {
|
||||
item.Status = domain.HumanItemEscalated
|
||||
task.Status = domain.AnnotationTaskStatusRejected
|
||||
|
||||
@ -233,7 +233,7 @@ func (s *server) handleAnnotationRandomClaim(w http.ResponseWriter, r *http.Requ
|
||||
// 挤出窗口而静默失去优先权(评审轮 c3);个人名下条目量级小,全扫可接受。
|
||||
if mine, me := s.store.ListHumanItems(ctx, p.User.OrganizationID, domain.HumanItemQuery{ProjectID: project.ID, ActorUserID: p.User.ID, CurrentStageID: domain.FanyuV58StageAnnotationCombined}); me == nil {
|
||||
for _, item := range mine {
|
||||
if item.ReviewRound > 0 && item.AssignedUserID == p.User.ID && (item.Status == domain.HumanItemRework || item.Status == domain.HumanItemClaimed) {
|
||||
if !item.InheritedRework && item.ReviewRound > 0 && item.AssignedUserID == p.User.ID && (item.Status == domain.HumanItemRework || item.Status == domain.HumanItemClaimed) {
|
||||
s.renderRandomClaimResult(ctx, w, r, item.ID, false)
|
||||
return
|
||||
}
|
||||
|
||||
@ -58,6 +58,7 @@ func (c *workBufferDisplayCache) set(key string, display *domain.HumanTaskDispla
|
||||
// annotationBufferSummary is a polling-only wire type. It is deliberately
|
||||
// separate from HumanItemRecord and must never replace a workbench detail.
|
||||
type annotationBufferSummary struct {
|
||||
InheritedRework bool `json:"inheritedRework,omitempty"`
|
||||
SourceValidationStatus string `json:"sourceValidationStatus,omitempty"`
|
||||
ID string `json:"id"`
|
||||
ProjectID string `json:"projectId"`
|
||||
@ -78,7 +79,7 @@ func summarizeAnnotationBuffer(items []domain.HumanItemRecord) []annotationBuffe
|
||||
out := make([]annotationBufferSummary, 0, len(items))
|
||||
for _, item := range items {
|
||||
phase := mapValue(item.Subject.Payload["sliceWorkflow"])
|
||||
out = append(out, annotationBufferSummary{SourceValidationStatus: item.SourceValidationStatus, ID: item.ID, ProjectID: item.ProjectID, LegacyTaskID: item.LegacyTaskID, AssignedUserID: item.AssignedUserID, CurrentStageID: item.CurrentStageID, Status: item.Status, Version: item.Version, ReviewRound: item.ReviewRound, ClaimedAt: item.ClaimedAt, PendingCombinedSubmissionID: item.PendingCombinedSubmissionID, SliceStatus: stringValue(phase["status"]), SlicePhase: stringValue(phase["phase"]), Display: item.Display})
|
||||
out = append(out, annotationBufferSummary{InheritedRework: item.InheritedRework, SourceValidationStatus: item.SourceValidationStatus, ID: item.ID, ProjectID: item.ProjectID, LegacyTaskID: item.LegacyTaskID, AssignedUserID: item.AssignedUserID, CurrentStageID: item.CurrentStageID, Status: item.Status, Version: item.Version, ReviewRound: item.ReviewRound, ClaimedAt: item.ClaimedAt, PendingCombinedSubmissionID: item.PendingCombinedSubmissionID, SliceStatus: stringValue(phase["status"]), SlicePhase: stringValue(phase["phase"]), Display: item.Display})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
279
internal/api/deleted_annotator.go
Normal file
279
internal/api/deleted_annotator.go
Normal file
@ -0,0 +1,279 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"droboticflow/internal/domain"
|
||||
)
|
||||
|
||||
func accountHandoffPolicyEnabled() bool {
|
||||
return strings.EqualFold(strings.TrimSpace(os.Getenv("DELETED_ANNOTATOR_HANDOFF")), "true")
|
||||
}
|
||||
|
||||
// The request is durable before sessions/claims are removed. Only new explicit
|
||||
// deletions carry this marker; a restart does not bulk-rewrite historical users.
|
||||
func (s *server) finishAccountDeletion(ctx context.Context, user domain.UserRecord) error {
|
||||
if err := s.store.DeleteSessionsByUser(ctx, user.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
if user.Role == domain.UserRoleAnnotator && user.DeletionRequestedAt != "" {
|
||||
if err := s.releaseDeletedAnnotatorTasks(ctx, user); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if _, err := s.releaseUserClaimedTasks(ctx, user.OrganizationID, user.ID, user.DeletionActorID); err != nil {
|
||||
return err
|
||||
}
|
||||
members, err := s.store.ListProjectMembers(ctx, user.OrganizationID, "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, member := range members {
|
||||
if member.UserID == user.ID {
|
||||
if _, err := s.store.DeleteProjectMember(ctx, member.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
current, err := s.store.GetUser(ctx, user.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if current == nil || current.DeletedAt != "" {
|
||||
return nil
|
||||
}
|
||||
before := *current
|
||||
current.DeletedAt, current.DeletedBy, current.PasswordHash = nowISO(), user.DeletionActorID, ""
|
||||
current.UpdatedAt = current.DeletedAt
|
||||
return s.store.UpdateUser(ctx, *current, before)
|
||||
}
|
||||
|
||||
func (s *server) releaseDeletedAnnotatorTasks(ctx context.Context, user domain.UserRecord) error {
|
||||
items, err := s.store.ListHumanItems(ctx, user.OrganizationID, domain.HumanItemQuery{ActorUserID: user.ID})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var pending error
|
||||
for _, item := range items {
|
||||
if item.AssignedUserID != user.ID || !humanItemPending(item.Status) {
|
||||
continue
|
||||
}
|
||||
if item.LegacyTaskID == "" {
|
||||
previous := item.Version
|
||||
item.AssignedUserID = ""
|
||||
item.ClaimedAt = ""
|
||||
if item.Status == domain.HumanItemClaimed || item.Status == domain.HumanItemInProgress {
|
||||
item.Status = domain.HumanItemReady
|
||||
}
|
||||
item.Version++
|
||||
ok, err := s.store.UpdateHumanItemVersioned(ctx, item, previous)
|
||||
if err != nil {
|
||||
pending = errors.Join(pending, err)
|
||||
continue
|
||||
}
|
||||
if !ok {
|
||||
pending = errors.Join(pending, fmt.Errorf("任务状态已变化,请重试"))
|
||||
}
|
||||
continue
|
||||
}
|
||||
if item.CurrentStageID != domain.FanyuV58StageAnnotationCombined {
|
||||
switch item.CurrentStageID {
|
||||
case "split_annotation", "cut_adjustment", "semantic_annotation":
|
||||
pending = errors.Join(pending, fmt.Errorf("历史标注阶段需要核对交接,内容已保留"))
|
||||
continue
|
||||
default:
|
||||
// Never turn an existing review-stage task into annotation.
|
||||
continue
|
||||
}
|
||||
}
|
||||
if err := s.handoffDeletedAnnotatorTask(ctx, item.ID, user); err != nil {
|
||||
pending = errors.Join(pending, err)
|
||||
}
|
||||
}
|
||||
return pending
|
||||
}
|
||||
|
||||
func (s *server) handoffDeletedAnnotatorTask(ctx context.Context, id string, user domain.UserRecord) error {
|
||||
for attempt := 0; attempt < 8; attempt++ {
|
||||
item, err := s.store.GetHumanItem(ctx, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if item == nil || item.AssignedUserID != user.ID || item.CurrentStageID != domain.FanyuV58StageAnnotationCombined || !humanItemPending(item.Status) {
|
||||
return nil
|
||||
}
|
||||
task, err := s.store.GetAnnotationTask(ctx, item.LegacyTaskID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if task == nil {
|
||||
return fmt.Errorf("删除交接的标注任务缺失")
|
||||
}
|
||||
if task.OrganizationID != item.OrganizationID || task.ProjectID != item.ProjectID || (task.SupplierID != "" && item.SupplierID != "" && task.SupplierID != item.SupplierID) {
|
||||
return fmt.Errorf("删除交接的任务范围不一致,内容已保留")
|
||||
}
|
||||
if item.AccountHandoff == nil || !item.AccountHandoff.Pending {
|
||||
// Successful submission/decision evidence wins over the current page.
|
||||
reviewed := item.SubmittedAt != "" || item.ReviewRound > 0 || item.LatestAuthorUserID != ""
|
||||
reviews, err := s.store.ListAnnotationReviews(ctx, item.OrganizationID, task.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
reviewed = reviewed || len(reviews) > 0
|
||||
attempts, err := s.store.ListHumanStageAttempts(ctx, item.OrganizationID, item.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, attempt := range attempts {
|
||||
if attempt.StageID == domain.FanyuV58StageQualityReview || attempt.StageID == "acceptance" || attempt.StageID == "prepackage_acceptance" {
|
||||
reviewed = true
|
||||
}
|
||||
}
|
||||
|
||||
annotations, err := s.store.ListAnnotationRevisions(ctx, item.OrganizationID, task.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// A legacy published revision without a trustworthy handoff marker is
|
||||
// ambiguous. Preserve it instead of guessing that it is safe to erase.
|
||||
ambiguousRevision := task.CurrentRevision > 0
|
||||
for _, revision := range annotations {
|
||||
if revision.Revision > 0 && revision.AuthorUserID != "system" {
|
||||
ambiguousRevision = true
|
||||
}
|
||||
}
|
||||
knownSubmissionAttempt := item.ProcessingJob != nil && item.ProcessingJob.Kind == "submit"
|
||||
if !reviewed && ambiguousRevision && item.PendingCombinedSubmissionID == "" && !knownSubmissionAttempt {
|
||||
return fmt.Errorf("历史任务的送检阶段无法确认,已保留内容")
|
||||
}
|
||||
floor := max(task.AnnotationRevisionFloor, task.CurrentRevision, item.CurrentRevision, intValue(mapValue(item.Subject.Payload["sliceWorkflow"])["revision"]))
|
||||
for _, revision := range annotations {
|
||||
floor = max(floor, revision.Revision)
|
||||
}
|
||||
slices, err := s.store.ListSliceRevisions(ctx, item.OrganizationID, task.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, revision := range slices {
|
||||
floor = max(floor, revision.Revision)
|
||||
}
|
||||
if item.ProcessingJob != nil {
|
||||
floor = max(floor, item.ProcessingJob.Revision)
|
||||
}
|
||||
previous := item.Version
|
||||
item.AccountHandoff = &domain.AccountTaskHandoff{UserID: user.ID, ActorID: user.DeletionActorID, Reset: !reviewed, Pending: true, RevisionFloor: floor, At: nowISO()}
|
||||
if !reviewed {
|
||||
item.AnnotationEpoch++
|
||||
// Source identity/configuration is retained; only editable results,
|
||||
// local archives and classification choices are reset.
|
||||
for _, key := range []string{"annotationWorkspace", "annotationDraft", "materialization", "sliceResetHistory", "sliceDispositionHistory", sliceRestoreEvidenceKey, sliceArchiveEditsKey, sliceMergeEvidenceKey, "dictionarySelectionHistory", "unconfirmedSliceNames", "lastSliceEditApplication", "preserveAnnotationRevisionHistory"} {
|
||||
delete(item.Subject.Payload, key)
|
||||
}
|
||||
if item.Subject.Payload == nil {
|
||||
item.Subject.Payload = map[string]any{}
|
||||
}
|
||||
item.Subject.Payload["sliceWorkflow"] = map[string]any{"status": "editing", "phase": "slice", "resetAt": item.AccountHandoff.At, "discardedWorkspace": true, "revision": floor}
|
||||
item.CurrentRevision = 0
|
||||
} else {
|
||||
item.InheritedRework = true
|
||||
}
|
||||
// Removing the job ID fences heartbeat, completion and queued retries.
|
||||
item.ProcessingJob = nil
|
||||
item.PendingCombinedSubmissionID = ""
|
||||
item.PendingCombinedPayloadSHA = ""
|
||||
item.ClaimRequestID = ""
|
||||
item.Version++
|
||||
item.UpdatedAt = nowISO()
|
||||
ok, err := s.store.UpdateHumanItemVersioned(ctx, *item, previous)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
}
|
||||
h := item.AccountHandoff
|
||||
if h.UserID != user.ID {
|
||||
return fmt.Errorf("删除交接归属已变化")
|
||||
}
|
||||
if h.Reset {
|
||||
task.AnnotationEpoch = item.AnnotationEpoch
|
||||
task.AnnotationRevisionFloor = h.RevisionFloor
|
||||
task.CurrentRevision = 0
|
||||
task.AssignedUserID = ""
|
||||
task.Status = domain.AnnotationTaskStatusPendingAnnotation
|
||||
if err := s.store.ResetDeletedAnnotatorTask(ctx, *task); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// A deterministic audit key makes crash/retry idempotent.
|
||||
if err := s.store.AppendHumanEvent(ctx, domain.HumanEventRecord{ID: fmt.Sprintf("deleted-annotator:%s:%s:%d", user.ID, id, item.AnnotationEpoch), OrganizationID: item.OrganizationID, BatchID: item.BatchID, ItemID: id, StageID: item.CurrentStageID, Round: item.ReviewRound, EventType: "deleted_annotator_handoff", ActorUserID: user.DeletionActorID, ItemVersion: item.Version, CreatedAt: h.At, Payload: map[string]any{"reset": h.Reset, "deletedUserId": user.ID}}); err != nil {
|
||||
return err
|
||||
}
|
||||
previous := item.Version
|
||||
item.AssignedUserID = ""
|
||||
item.ClaimedAt = ""
|
||||
item.AccountHandoff.Pending = false
|
||||
if h.Reset {
|
||||
item.Status = domain.HumanItemReady
|
||||
} else {
|
||||
item.Status = domain.HumanItemRework
|
||||
}
|
||||
item.Version++
|
||||
item.UpdatedAt = nowISO()
|
||||
ok, err := s.store.UpdateHumanItemVersioned(ctx, *item, previous)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if ok {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("删除交接任务状态持续变化,请重试")
|
||||
}
|
||||
|
||||
func (s *server) recoverAccountHandoffs(ctx context.Context) error {
|
||||
users, err := s.store.ListUsers(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var first error
|
||||
for _, user := range users {
|
||||
if user.DeletionRequestedAt != "" && user.DeletedAt == "" && user.Status == domain.UserStatusDisabled {
|
||||
if err := s.finishAccountDeletion(ctx, user); err != nil && first == nil {
|
||||
first = err
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := s.dispatchDeletedAuthorReworks(ctx); err != nil && first == nil {
|
||||
first = err
|
||||
}
|
||||
return first
|
||||
}
|
||||
|
||||
var accountHandoffRecoveryOnce sync.Once
|
||||
|
||||
func (s *server) startAccountHandoffRecovery() {
|
||||
if !accountHandoffPolicyEnabled() || strings.HasSuffix(os.Args[0], ".test") {
|
||||
return
|
||||
}
|
||||
accountHandoffRecoveryOnce.Do(func() {
|
||||
go func() {
|
||||
ticker := time.NewTicker(30 * time.Second)
|
||||
defer ticker.Stop()
|
||||
for range ticker.C {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
if s.recoverAccountHandoffs(ctx) != nil {
|
||||
log.Printf("[account-handoff] recovery incomplete")
|
||||
}
|
||||
cancel()
|
||||
}
|
||||
}()
|
||||
})
|
||||
}
|
||||
119
internal/api/deleted_annotator_mongo_test.go
Normal file
119
internal/api/deleted_annotator_mongo_test.go
Normal file
@ -0,0 +1,119 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"droboticflow/internal/domain"
|
||||
"droboticflow/internal/store"
|
||||
"go.mongodb.org/mongo-driver/bson"
|
||||
"go.mongodb.org/mongo-driver/mongo"
|
||||
"go.mongodb.org/mongo-driver/mongo/options"
|
||||
)
|
||||
|
||||
func TestDeletedAnnotatorMongoRestartAndHistory(t *testing.T) {
|
||||
if os.Getenv("DELETED_ANNOTATOR_LOCAL_MONGO") != "1" {
|
||||
t.Skip("isolated local Mongo opt-in")
|
||||
}
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
ctx, cancel := context.WithTimeout(t.Context(), 90*time.Second)
|
||||
defer cancel()
|
||||
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://127.0.0.1:27018/?directConnection=true"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Disconnect(context.Background())
|
||||
db := client.Database(fmt.Sprintf("deleted_account_api_test_%d", time.Now().UnixNano()))
|
||||
defer db.Drop(context.Background())
|
||||
for _, name := range []string{"organizations", "projects", "workflows", "users", "suppliers", "projectMembers", "annotationTasks", "annotationRevisions", "sliceRevisions", "humanItems", "humanBatches", "humanEvents", "humanStageAttempts", "humanClaimRequests"} {
|
||||
if _, err := db.Collection(name).Indexes().CreateOne(ctx, mongo.IndexModel{Keys: bson.D{{Key: "id", Value: 1}}, Options: options.Index().SetUnique(true)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, index := range []mongo.IndexModel{
|
||||
{Keys: bson.D{{Key: "organizationId", Value: 1}, {Key: "currentStageId", Value: 1}, {Key: "reworkRoutingPending", Value: 1}, {Key: "status", Value: 1}}, Options: options.Index().SetName("deleted_author_routing_v1")},
|
||||
{Keys: bson.D{{Key: "organizationId", Value: 1}, {Key: "currentStageId", Value: 1}, {Key: "inheritedRework", Value: 1}, {Key: "assignedUserId", Value: 1}, {Key: "status", Value: 1}}, Options: options.Index().SetName("deleted_author_unassigned_v1")},
|
||||
} {
|
||||
if _, err := db.Collection("humanItems").Indexes().CreateOne(ctx, index); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
f := newCombinedDraftFixture(t)
|
||||
ds := store.NewMongoStore(db)
|
||||
must := func(err error) {
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
must(ds.UpsertOrganization(ctx, domain.OrganizationRecord{ID: f.item.OrganizationID, Status: domain.OrganizationStatusActive}))
|
||||
supplier, _ := f.store.GetSupplier(ctx, f.annotator.SupplierID)
|
||||
must(ds.UpsertSupplier(ctx, *supplier))
|
||||
project, _ := f.store.GetProject(ctx, f.item.ProjectID)
|
||||
must(ds.UpsertProject(ctx, *project))
|
||||
for _, user := range []domain.UserRecord{f.admin, f.annotator, f.reviewer} {
|
||||
must(ds.UpsertUser(ctx, user))
|
||||
}
|
||||
members, _ := f.store.ListProjectMembers(ctx, f.item.OrganizationID, f.item.ProjectID)
|
||||
for _, member := range members {
|
||||
must(ds.UpsertProjectMember(ctx, member))
|
||||
}
|
||||
batch, _ := f.store.GetHumanBatch(ctx, f.item.BatchID)
|
||||
must(ds.UpsertHumanBatch(ctx, *batch))
|
||||
must(ds.UpsertAnnotationTask(ctx, f.task))
|
||||
f.item.ProcessingJob = &domain.AnnotationProcessingJob{ID: "old-job", TaskID: f.task.ID, ActorID: f.annotator.ID, Kind: "slice", Revision: 2, Status: "running", Owner: "old-process"}
|
||||
f.item.Subject.Payload["annotationDraft"] = map[string]any{"marker": "private work"}
|
||||
must(ds.UpsertHumanItem(ctx, f.item))
|
||||
// Already-reviewed material remains unrelated to the pre-quality reset.
|
||||
protected := f.item
|
||||
protected.ID = "reviewed-source"
|
||||
protected.AssignedUserID = ""
|
||||
protected.CurrentStageID = domain.FanyuV58StageQualityReview
|
||||
protected.SubmittedAt = nowISO()
|
||||
protected.LatestAuthorUserID = f.annotator.ID
|
||||
protected.ProcessingJob = nil
|
||||
protected.Subject.Payload = map[string]any{"annotationWorkspace": map[string]any{"marker": "formal work"}}
|
||||
must(ds.UpsertHumanItem(ctx, protected))
|
||||
must(ds.UpsertHumanStageAttempt(ctx, domain.HumanStageAttemptRecord{ID: "formal-review", OrganizationID: protected.OrganizationID, ItemID: protected.ID, StageID: domain.FanyuV58StageQualityReview, Status: domain.HumanItemAccepted, Comment: "正式质检记录"}))
|
||||
before, _ := ds.GetHumanItem(ctx, protected.ID)
|
||||
reviewsBefore, _ := ds.ListHumanStageAttempts(ctx, protected.OrganizationID, protected.ID)
|
||||
probe := &handoffFailureStore{Store: ds, failReset: true}
|
||||
f.handler = NewHandler(probe, nil)
|
||||
if r := f.request("DELETE", "/api/users/"+f.annotator.ID, f.admin, nil); r.Code != 500 {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
pending, _ := ds.GetHumanItem(ctx, f.item.ID)
|
||||
user, _ := ds.GetUser(ctx, f.annotator.ID)
|
||||
if pending.AccountHandoff == nil || !pending.AccountHandoff.Pending || pending.AnnotationEpoch != 1 || user.DeletionRequestedAt == "" {
|
||||
t.Fatal("Mongo lost durable reset intent")
|
||||
}
|
||||
// Reconstruct both handler state and Store; no process-local recovery data.
|
||||
restartedStore := store.NewMongoStore(db)
|
||||
restarted := &server{store: restartedStore}
|
||||
if err := restarted.recoverAccountHandoffs(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
item, _ := restartedStore.GetHumanItem(ctx, f.item.ID)
|
||||
user, _ = restartedStore.GetUser(ctx, f.annotator.ID)
|
||||
if item.AnnotationEpoch != 1 || item.AccountHandoff.Pending || item.AssignedUserID != "" || item.ProcessingJob != nil || user.DeletedAt == "" {
|
||||
t.Fatal("Mongo recovery incomplete")
|
||||
}
|
||||
if ok, err := restartedStore.UpdateHumanItemVersioned(ctx, f.item, f.item.Version); err != nil || ok {
|
||||
t.Fatal("old submit/save won after persisted reset", err)
|
||||
}
|
||||
after, _ := restartedStore.GetHumanItem(ctx, protected.ID)
|
||||
reviewsAfter, _ := restartedStore.ListHumanStageAttempts(ctx, protected.OrganizationID, protected.ID)
|
||||
if !reflect.DeepEqual(before, after) || !reflect.DeepEqual(reviewsBefore, reviewsAfter) {
|
||||
t.Fatal("quality history changed during account deletion")
|
||||
}
|
||||
if err := restarted.recoverAccountHandoffs(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rows, err := ds.ListHumanEvents(ctx, f.item.OrganizationID, f.item.ID); err != nil || len(rows) != 1 {
|
||||
t.Fatal("restart duplicated reset audit", err)
|
||||
}
|
||||
}
|
||||
218
internal/api/deleted_annotator_test.go
Normal file
218
internal/api/deleted_annotator_test.go
Normal file
@ -0,0 +1,218 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"droboticflow/internal/domain"
|
||||
"droboticflow/internal/store"
|
||||
"errors"
|
||||
"net/http"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDeletedAnnotatorResetsOnlyUnsubmittedWork(t *testing.T) {
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
for _, submitted := range []bool{false, true} {
|
||||
t.Run(map[bool]string{false: "before-quality", true: "quality-history"}[submitted], func(t *testing.T) {
|
||||
f := newCombinedDraftFixture(t)
|
||||
f.item.Subject.Payload["annotationWorkspace"] = map[string]any{"userId": f.annotator.ID, "sliceSegments": []domain.SliceSegment{{ID: "old", Name: "旧切片", Start: 0, End: 10}}, "segments": []domain.AnnotationSegment{{ID: "old-action", Start: 0, End: 10, Description: "旧描述"}}, "savedAt": nowISO()}
|
||||
f.item.Subject.Payload["annotationDraft"] = f.item.Subject.Payload["annotationWorkspace"]
|
||||
if submitted {
|
||||
f.item.SubmittedAt = nowISO()
|
||||
f.item.ReviewRound = 1
|
||||
f.item.LatestAuthorUserID = f.annotator.ID
|
||||
f.item.Status = domain.HumanItemRework
|
||||
}
|
||||
if err := f.store.UpsertHumanItem(f.ctx, f.item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := f.request(http.MethodDelete, "/api/users/"+f.annotator.ID, f.admin, nil)
|
||||
if r.Code != 204 {
|
||||
t.Fatalf("delete: %d %s", r.Code, r.Body.String())
|
||||
}
|
||||
item, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
if submitted {
|
||||
if !item.InheritedRework || len(mapValue(item.Subject.Payload["annotationWorkspace"])) == 0 {
|
||||
t.Fatal("reviewed contents were erased or not routed as inherited rework")
|
||||
}
|
||||
} else {
|
||||
if item.AnnotationEpoch != 1 || item.ProcessingJob != nil || item.Subject.Payload["annotationDraft"] != nil || item.Subject.Payload["annotationWorkspace"] != nil {
|
||||
t.Fatalf("unsubmitted workspace survived reset: epoch=%d", item.AnnotationEpoch)
|
||||
}
|
||||
if item.AssignedUserID != "" || item.Status != domain.HumanItemReady {
|
||||
t.Fatal("reset task not returned to pool")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type handoffFailureStore struct {
|
||||
store.Store
|
||||
failReset bool
|
||||
beforeReset func(domain.HumanItemRecord)
|
||||
}
|
||||
|
||||
func (s *handoffFailureStore) ResetDeletedAnnotatorTask(ctx context.Context, task domain.AnnotationTaskRecord) error {
|
||||
if s.failReset {
|
||||
s.failReset = false
|
||||
return errors.New("synthetic restart boundary")
|
||||
}
|
||||
return s.Store.ResetDeletedAnnotatorTask(ctx, task)
|
||||
}
|
||||
func (s *handoffFailureStore) UpdateHumanItemVersioned(ctx context.Context, item domain.HumanItemRecord, v int) (bool, error) {
|
||||
if item.AccountHandoff != nil && item.AccountHandoff.Pending && s.beforeReset != nil {
|
||||
f := s.beforeReset
|
||||
s.beforeReset = nil
|
||||
f(item)
|
||||
}
|
||||
return s.Store.UpdateHumanItemVersioned(ctx, item, v)
|
||||
}
|
||||
|
||||
func TestDeletedAnnotatorRestartFinishesResetOnce(t *testing.T) {
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
f := newCombinedDraftFixture(t)
|
||||
f.item.Subject.Payload["annotationDraft"] = map[string]any{"segments": []domain.AnnotationSegment{{ID: "old", Description: "旧草稿"}}}
|
||||
if err := f.store.UpsertHumanItem(f.ctx, f.item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
probe := &handoffFailureStore{Store: f.store, failReset: true}
|
||||
f.handler = NewHandler(probe, nil)
|
||||
if r := f.request("DELETE", "/api/users/"+f.annotator.ID, f.admin, nil); r.Code != 500 {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
user, _ := f.store.GetUser(f.ctx, f.annotator.ID)
|
||||
item, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
if user.DeletionRequestedAt == "" || user.DeletedAt != "" || item.AccountHandoff == nil || !item.AccountHandoff.Pending {
|
||||
t.Fatal("restart intent not durable")
|
||||
}
|
||||
if r := f.request("PUT", "/api/users/"+f.annotator.ID, f.admin, map[string]any{"status": "active"}); r.Code == 200 {
|
||||
t.Fatal("deletion-in-progress account was reactivated")
|
||||
}
|
||||
// A new server has no memory of the request; only durable records are shared.
|
||||
restarted := &server{store: f.store}
|
||||
if err := restarted.recoverAccountHandoffs(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
item, _ = f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
user, _ = f.store.GetUser(f.ctx, f.annotator.ID)
|
||||
if item.AnnotationEpoch != 1 || item.AccountHandoff.Pending || item.AssignedUserID != "" || user.DeletedAt == "" {
|
||||
t.Fatal("reset did not resume")
|
||||
}
|
||||
// A repeated delete/recovery must never clear the next person's edits.
|
||||
item.AssignedUserID = "next-author"
|
||||
item.Status = domain.HumanItemClaimed
|
||||
item.Subject.Payload["annotationDraft"] = map[string]any{"marker": "new work"}
|
||||
item.Version++
|
||||
if err := f.store.UpsertHumanItem(f.ctx, *item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if err := restarted.recoverAccountHandoffs(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r := f.request("DELETE", "/api/users/"+f.annotator.ID, f.admin, nil); r.Code != 204 {
|
||||
t.Fatal(r.Code)
|
||||
}
|
||||
after, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if !reflect.DeepEqual(before, after) {
|
||||
t.Fatal("repeated deletion erased next author work")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAnnotatorSubmissionWinsCASRace(t *testing.T) {
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
f := newCombinedDraftFixture(t)
|
||||
f.item.Subject.Payload["annotationWorkspace"] = map[string]any{"marker": "submitted work"}
|
||||
if err := f.store.UpsertHumanItem(f.ctx, f.item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
probe := &handoffFailureStore{Store: f.store}
|
||||
probe.beforeReset = func(_ domain.HumanItemRecord) {
|
||||
published := f.item
|
||||
published.CurrentStageID = domain.FanyuV58StageQualityReview
|
||||
published.Status = domain.HumanItemReady
|
||||
published.AssignedUserID = ""
|
||||
published.SubmittedAt = nowISO()
|
||||
published.LatestAuthorUserID = f.annotator.ID
|
||||
published.CurrentRevision = 1
|
||||
published.Version++
|
||||
ok, err := f.store.UpdateHumanItemVersioned(f.ctx, published, f.item.Version)
|
||||
if err != nil || !ok {
|
||||
t.Fatal("submission fixture lost", err)
|
||||
}
|
||||
}
|
||||
f.handler = NewHandler(probe, nil)
|
||||
if r := f.request("DELETE", "/api/users/"+f.annotator.ID, f.admin, nil); r.Code != 204 {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
item, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
if item.CurrentStageID != domain.FanyuV58StageQualityReview || item.AnnotationEpoch != 0 || item.Subject.Payload["annotationWorkspace"] == nil {
|
||||
t.Fatal("deletion rewound a winning quality handoff")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAnnotatorOldWritesCannotReviveReset(t *testing.T) {
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
f := newCombinedDraftFixture(t)
|
||||
f.item.ProcessingJob = &domain.AnnotationProcessingJob{ID: "old-job", TaskID: f.task.ID, Kind: "slice", Status: "running", Owner: "old-worker", ActorID: f.annotator.ID, Revision: 3}
|
||||
if err := f.store.UpsertHumanItem(f.ctx, f.item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
oldItem, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
oldTask, _ := f.store.GetAnnotationTask(f.ctx, f.task.ID)
|
||||
if r := f.request("DELETE", "/api/users/"+f.annotator.ID, f.admin, nil); r.Code != 204 {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
if ok, err := f.store.UpdateHumanItemVersioned(f.ctx, *oldItem, oldItem.Version); err != nil || ok {
|
||||
t.Fatal("old edit won after reset", err)
|
||||
}
|
||||
if ok, _ := f.store.TouchAnnotationProcessing(f.ctx, f.item.ID, "old-job", "old-worker", nowISO(), "publishing", 1, 1); ok {
|
||||
t.Fatal("old heartbeat revived")
|
||||
}
|
||||
if ok, _ := f.store.SaveAnnotationProcessing(f.ctx, f.item.ID, *oldItem.ProcessingJob); ok {
|
||||
t.Fatal("old job revived")
|
||||
}
|
||||
if err := f.store.UpsertAnnotationTask(f.ctx, *oldTask); !errors.Is(err, store.ErrHumanClaimConflict) {
|
||||
t.Fatal("old task publication admitted", err)
|
||||
}
|
||||
if err := f.store.UpsertSliceRevision(f.ctx, domain.SliceRevisionRecord{ID: "late", TaskID: f.task.ID, OrganizationID: f.task.OrganizationID, Revision: 3}); !errors.Is(err, store.ErrHumanClaimConflict) {
|
||||
t.Fatal("late generation admitted", err)
|
||||
}
|
||||
processor := &server{store: f.store}
|
||||
if err := processor.processAnnotationJob(f.ctx, *oldItem); err == nil {
|
||||
t.Fatal("deleted actor job resumed")
|
||||
}
|
||||
processor.finishSliceConfirmation(f.ctx, f.task.ID, f.item.ID, "old-job", sliceConfirmationInput{}, 3, f.annotator.ID)
|
||||
current, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
if current.AnnotationEpoch != 1 || current.ProcessingJob != nil || current.CurrentRevision != 0 {
|
||||
t.Fatal("old writes changed reset")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAnnotatorRejectsMismatchedTaskScope(t *testing.T) {
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
f := newCombinedDraftFixture(t)
|
||||
foreign := f.task
|
||||
foreign.ID = "foreign-task"
|
||||
foreign.OrganizationID = "another-organization"
|
||||
foreign.ProjectID = "another-project"
|
||||
if err := f.store.UpsertAnnotationTask(f.ctx, foreign); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.item.LegacyTaskID = foreign.ID
|
||||
if err := f.store.UpsertHumanItem(f.ctx, f.item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r := f.request("DELETE", "/api/users/"+f.annotator.ID, f.admin, nil); r.Code != 500 {
|
||||
t.Fatal("mismatched scope was reset", r.Code)
|
||||
}
|
||||
after, _ := f.store.GetAnnotationTask(f.ctx, foreign.ID)
|
||||
if after.AnnotationEpoch != 0 || after.CurrentRevision != foreign.CurrentRevision {
|
||||
t.Fatal("foreign task changed")
|
||||
}
|
||||
item, _ := f.store.GetHumanItem(f.ctx, f.item.ID)
|
||||
if item.AnnotationEpoch != 0 || item.AccountHandoff != nil {
|
||||
t.Fatal("ambiguous source was reset")
|
||||
}
|
||||
}
|
||||
215
internal/api/deleted_author_rework.go
Normal file
215
internal/api/deleted_author_rework.go
Normal file
@ -0,0 +1,215 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/rand/v2"
|
||||
"sort"
|
||||
|
||||
"droboticflow/internal/domain"
|
||||
"droboticflow/internal/store"
|
||||
)
|
||||
|
||||
// Recheck after the formal decision is committed. If deletion won while the
|
||||
// reviewer was submitting, its durable routing marker still finds this task.
|
||||
func (s *server) resolveReworkAuthor(ctx context.Context, item domain.HumanItemRecord) error {
|
||||
user, err := s.store.GetUser(ctx, item.LatestAuthorUserID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
deleted := user != nil && (user.DeletedAt != "" || user.DeletionRequestedAt != "")
|
||||
if !deleted {
|
||||
ok, err := s.store.ClearReworkRoutingPending(ctx, item.ID, item.Version)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !ok {
|
||||
return store.ErrHumanClaimConflict
|
||||
}
|
||||
return nil
|
||||
}
|
||||
previous := item.Version
|
||||
item.ReworkRoutingPending = false
|
||||
if deleted {
|
||||
item.InheritedRework = true
|
||||
if item.AssignedUserID == user.ID {
|
||||
item.AssignedUserID = ""
|
||||
item.ClaimedAt = ""
|
||||
item.Status = domain.HumanItemRework
|
||||
}
|
||||
}
|
||||
item.Version++
|
||||
item.UpdatedAt = nowISO()
|
||||
ok, err := s.store.UpdateHumanItemVersioned(ctx, item, previous)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !ok {
|
||||
return store.ErrHumanClaimConflict
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type reworkCandidate struct {
|
||||
user domain.UserRecord
|
||||
load int
|
||||
}
|
||||
type reworkCandidateCacheKey struct{}
|
||||
type reworkCandidateCache struct {
|
||||
byProject map[string][]reworkCandidate
|
||||
reserved map[string]bool
|
||||
}
|
||||
|
||||
func (s *server) dispatchDeletedAuthorReworks(ctx context.Context) error {
|
||||
ctx = context.WithValue(ctx, reworkCandidateCacheKey{}, &reworkCandidateCache{byProject: map[string][]reworkCandidate{}, reserved: map[string]bool{}})
|
||||
orgs, err := s.store.ListOrganizations(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var first error
|
||||
for _, org := range orgs {
|
||||
if org.Status != domain.OrganizationStatusActive {
|
||||
continue
|
||||
}
|
||||
items, err := s.store.ListHumanItems(ctx, org.ID, domain.HumanItemQuery{NeedsReworkRouting: true})
|
||||
if err != nil {
|
||||
if first == nil {
|
||||
first = err
|
||||
}
|
||||
continue
|
||||
}
|
||||
for _, item := range items {
|
||||
if item.AccountHandoff != nil && item.AccountHandoff.Pending {
|
||||
continue
|
||||
}
|
||||
if item.ReworkRoutingPending {
|
||||
if err := s.resolveReworkAuthor(ctx, item); err != nil {
|
||||
if first == nil {
|
||||
first = err
|
||||
}
|
||||
continue
|
||||
}
|
||||
fresh, err := s.store.GetHumanItem(ctx, item.ID)
|
||||
if err != nil || fresh == nil {
|
||||
continue
|
||||
}
|
||||
item = *fresh
|
||||
}
|
||||
if !item.InheritedRework || item.AssignedUserID != "" || item.Status != domain.HumanItemRework {
|
||||
continue
|
||||
}
|
||||
if err := s.assignDeletedAuthorRework(ctx, item); err != nil && first == nil {
|
||||
first = err
|
||||
}
|
||||
}
|
||||
}
|
||||
return first
|
||||
}
|
||||
|
||||
func (s *server) assignDeletedAuthorRework(ctx context.Context, item domain.HumanItemRecord) error {
|
||||
if item.SupplierID == "" {
|
||||
return nil
|
||||
} // Never guess a supplier for legacy data.
|
||||
supplier, err := s.store.GetSupplier(ctx, item.SupplierID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if supplier == nil || supplier.OrganizationID != item.OrganizationID || supplier.Status != "active" {
|
||||
return nil
|
||||
}
|
||||
users, err := s.store.ListUsersByOrganization(ctx, item.OrganizationID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
batch, err := s.store.GetHumanBatch(ctx, item.BatchID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if batch == nil || batch.Status == domain.HumanBatchCompleted {
|
||||
return nil
|
||||
}
|
||||
baseline, err := s.store.LoadInheritedReworkBaseline(ctx, item)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if baseline == nil || len(baseline.Slices) == 0 {
|
||||
return nil
|
||||
}
|
||||
legacy := domain.AnnotationTaskRecord{ID: item.LegacyTaskID, OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, SupplierID: item.SupplierID, ProcessingScope: item.ProcessingScope, ProcessingProfile: item.ProcessingProfile}
|
||||
legacy.SupplierPool = batch.ProcessingProfile == domain.ProcessingProfileRawHuman && domain.IsFanyuV58ThreeRoleHumanLoop(batch.Stages)
|
||||
eligible := []reworkCandidate{}
|
||||
memo, _ := ctx.Value(reworkCandidateCacheKey{}).(*reworkCandidateCache)
|
||||
cacheKey := item.OrganizationID + "/" + item.ProjectID + "/" + item.SupplierID
|
||||
cached := false
|
||||
if memo != nil {
|
||||
eligible, cached = memo.byProject[cacheKey]
|
||||
}
|
||||
if !cached {
|
||||
for _, user := range users {
|
||||
if user.Role != domain.UserRoleAnnotator || user.Status != domain.UserStatusActive || user.DeletedAt != "" || user.DeletionRequestedAt != "" || user.SupplierID != item.SupplierID {
|
||||
continue
|
||||
}
|
||||
members, err := s.annotationMembershipsForUser(ctx, item.OrganizationID, item.ProjectID, user.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !membershipHasRole(members, legacy, domain.AnnotationRoleAnnotator) {
|
||||
continue
|
||||
}
|
||||
load, inherited := 0, 0
|
||||
err = s.store.VisitHumanPoolCountCandidates(ctx, item.OrganizationID, domain.HumanItemQuery{ActorUserID: user.ID, MinePoolOnly: true, CurrentStageID: domain.FanyuV58StageAnnotationCombined}, func(page []domain.HumanItemRecord) error {
|
||||
for _, owned := range page {
|
||||
if owned.AssignedUserID == user.ID && humanItemPending(owned.Status) {
|
||||
load++
|
||||
if owned.InheritedRework {
|
||||
inherited++
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if inherited > 0 {
|
||||
continue
|
||||
}
|
||||
eligible = append(eligible, reworkCandidate{user, load})
|
||||
}
|
||||
|
||||
if memo != nil {
|
||||
memo.byProject[cacheKey] = eligible
|
||||
}
|
||||
}
|
||||
rand.Shuffle(len(eligible), func(i, j int) { eligible[i], eligible[j] = eligible[j], eligible[i] })
|
||||
sort.SliceStable(eligible, func(i, j int) bool { return eligible[i].load < eligible[j].load })
|
||||
for _, candidate := range eligible {
|
||||
if memo != nil && memo.reserved[candidate.user.ID] {
|
||||
continue
|
||||
}
|
||||
next := item
|
||||
next.Subject = item.Subject
|
||||
next.AssignedUserID = candidate.user.ID
|
||||
next.Status = domain.HumanItemClaimed
|
||||
next.ClaimedAt = nowISO()
|
||||
next.UpdatedAt = next.ClaimedAt
|
||||
next.Version++
|
||||
id := fmt.Sprintf("deleted-author:%s:%d:%s", item.ID, item.Version, candidate.user.ID)
|
||||
_, err := s.store.CommitHumanClaim(ctx, store.HumanClaimCommit{EnforceBuffer: legacy.SupplierPool, Receipt: store.HumanClaimReceipt{ID: id, ItemID: item.ID, UserID: candidate.user.ID, ProjectID: item.ProjectID, OrganizationID: item.OrganizationID}, Item: next, ExpectedVersion: item.Version, TaskStatus: domain.AnnotationTaskStatusRework, Attempt: domain.HumanStageAttemptRecord{OrganizationID: item.OrganizationID, ItemID: item.ID, StageID: item.CurrentStageID, Round: item.ReviewRound, Status: domain.HumanItemClaimed, AssigneeUserID: candidate.user.ID}, Event: domain.HumanEventRecord{ID: id, OrganizationID: item.OrganizationID, BatchID: item.BatchID, ItemID: item.ID, StageID: item.CurrentStageID, Round: item.ReviewRound, EventType: "deleted_author_rework_assigned", ActorUserID: candidate.user.ID, ItemVersion: next.Version, CreatedAt: next.ClaimedAt}})
|
||||
if err == nil {
|
||||
if memo != nil {
|
||||
memo.reserved[candidate.user.ID] = true
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if errors.Is(err, store.ErrAnnotationBufferFull) {
|
||||
if memo != nil {
|
||||
memo.reserved[candidate.user.ID] = true
|
||||
}
|
||||
continue
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
373
internal/api/deleted_author_rework_test.go
Normal file
373
internal/api/deleted_author_rework_test.go
Normal file
@ -0,0 +1,373 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"droboticflow/internal/domain"
|
||||
"droboticflow/internal/store"
|
||||
"fmt"
|
||||
"net/http/httptest"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func inheritedReworkFixture(t *testing.T, n int) (combinedDraftFixture, []string) {
|
||||
t.Helper()
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
f := newCombinedDraftFixture(t)
|
||||
if err := f.store.UpsertOrganization(f.ctx, domain.OrganizationRecord{ID: f.item.OrganizationID, Name: "Test org", Status: domain.OrganizationStatusActive}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other := f.annotator
|
||||
other.ID = "new-worker"
|
||||
other.CreatedAt = nowISO()
|
||||
if err := f.store.UpsertUser(f.ctx, other); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := f.store.UpsertProjectMember(f.ctx, domain.ProjectMemberRecord{ID: "member-new", OrganizationID: f.item.OrganizationID, ProjectID: f.item.ProjectID, UserID: other.ID, SupplierID: other.SupplierID, Roles: []domain.AnnotationAssignmentRole{domain.AnnotationRoleAnnotator}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
old := f.annotator
|
||||
old.ID = "deleted-author"
|
||||
old.Status = domain.UserStatusDisabled
|
||||
old.DeletedAt = nowISO()
|
||||
if err := f.store.UpsertUser(f.ctx, old); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ids := []string{}
|
||||
for i := 0; i < n; i++ {
|
||||
task := f.task
|
||||
task.ID = fmt.Sprintf("inherited-task-%d", i)
|
||||
task.SupplierID = f.annotator.SupplierID
|
||||
task.CurrentRevision = 1
|
||||
task.Status = domain.AnnotationTaskStatusRework
|
||||
if err := f.store.UpsertAnnotationTask(f.ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
item := f.item
|
||||
item.ID = fmt.Sprintf("inherited-item-%d", i)
|
||||
item.LegacyTaskID = task.ID
|
||||
item.Status = domain.HumanItemRework
|
||||
item.AssignedUserID = ""
|
||||
item.SupplierID = f.annotator.SupplierID
|
||||
item.InheritedRework = true
|
||||
item.ReviewRound = 1
|
||||
item.CurrentRevision = 1
|
||||
item.LatestAuthorUserID = old.ID
|
||||
item.Subject.Payload = map[string]any{"source": "retained", "annotationDraft": map[string]any{"userId": old.ID, "marker": "never adopt this private draft"}}
|
||||
if err := f.store.UpsertHumanItem(f.ctx, item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
baseline := domain.ReworkBaselineRecord{ID: store.ReworkBaselineID(item.ID, 1), OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, ItemID: item.ID, Round: 1, Slices: []domain.SliceSegment{{ID: "s", Name: "正式切片", Start: 0, End: 10}}, Actions: []domain.AnnotationSegment{{ID: "a", Start: 0, End: 10, Description: "正式描述", Status: "described"}}, Problems: []domain.ReviewProblem{{ID: "p", Scope: "slice", SliceID: "s", Text: "原质检意见"}}}
|
||||
if err := f.store.UpsertReworkBaseline(f.ctx, baseline); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ids = append(ids, item.ID)
|
||||
}
|
||||
return f, ids
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkFairCapacityAndFrozenContents(t *testing.T) {
|
||||
f, ids := inheritedReworkFixture(t, 5)
|
||||
s := &server{store: f.store}
|
||||
before, _ := f.store.GetReworkBaseline(f.ctx, store.ReworkBaselineID(ids[0], 1))
|
||||
for i := 0; i < 3; i++ {
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
counts := map[string]int{}
|
||||
pending := 0
|
||||
for _, id := range ids {
|
||||
item, _ := f.store.GetHumanItem(f.ctx, id)
|
||||
if item.AssignedUserID == "" {
|
||||
pending++
|
||||
continue
|
||||
}
|
||||
counts[item.AssignedUserID]++
|
||||
if item.AssignedUserID != f.annotator.ID && item.AssignedUserID != "new-worker" {
|
||||
t.Fatal("wrong role/supplier assignment")
|
||||
}
|
||||
if item.Subject.Payload["annotationDraft"] != nil {
|
||||
t.Fatal("inherited another person's private draft")
|
||||
}
|
||||
workspace := mapValue(item.Subject.Payload["annotationWorkspace"])
|
||||
if workspace["userId"] != item.AssignedUserID {
|
||||
t.Fatal("formal workspace not handed over")
|
||||
}
|
||||
actions := decodeMaterializationValue[domain.AnnotationSegment](workspace["segments"])
|
||||
if len(actions) != 1 || actions[0].Description != "正式描述" {
|
||||
t.Fatal("formal annotations lost")
|
||||
}
|
||||
}
|
||||
if counts[f.annotator.ID] != 1 || counts["new-worker"] != 1 || pending != 3 {
|
||||
t.Fatalf("tasks concentrated or lost: %#v pending=%d", counts, pending)
|
||||
}
|
||||
after, _ := f.store.GetReworkBaseline(f.ctx, store.ReworkBaselineID(ids[0], 1))
|
||||
if !reflect.DeepEqual(before, after) {
|
||||
t.Fatal("formal quality comments were changed")
|
||||
}
|
||||
// Finishing one handoff releases only that person's inherited-work slot.
|
||||
for _, id := range ids {
|
||||
item, _ := f.store.GetHumanItem(f.ctx, id)
|
||||
if item.AssignedUserID == f.annotator.ID {
|
||||
item.AssignedUserID = ""
|
||||
item.CurrentStageID = domain.FanyuV58StageQualityReview
|
||||
item.Status = domain.HumanItemReady
|
||||
item.Version++
|
||||
if err := f.store.UpsertHumanItem(f.ctx, *item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assigned := 0
|
||||
for _, id := range ids {
|
||||
item, _ := f.store.GetHumanItem(f.ctx, id)
|
||||
if item.AssignedUserID == f.annotator.ID {
|
||||
assigned++
|
||||
}
|
||||
}
|
||||
if assigned != 1 {
|
||||
t.Fatal("newly available worker was not reused fairly")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkDoesNotBlockOrdinaryClaim(t *testing.T) {
|
||||
f, _ := inheritedReworkFixture(t, 2)
|
||||
s := &server{store: f.store}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
task := f.task
|
||||
task.ID = "fresh-task"
|
||||
if err := f.store.UpsertAnnotationTask(f.ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
item := f.item
|
||||
item.ID = "fresh-item"
|
||||
item.LegacyTaskID = task.ID
|
||||
item.AssignedUserID = ""
|
||||
item.Status = domain.HumanItemReady
|
||||
if err := f.store.UpsertHumanItem(f.ctx, item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := f.request("POST", "/api/plugins/annotation-qa/random-claims", f.annotator, map[string]any{"projectId": f.item.ProjectID, "requestId": "fresh-after-inherited"})
|
||||
if r.Code != 200 {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
got, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if got.AssignedUserID != f.annotator.ID {
|
||||
t.Fatal("inherited task was returned instead of new work")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkDecisionDeletionRaceAndActiveAuthor(t *testing.T) {
|
||||
for _, deleted := range []bool{false, true} {
|
||||
t.Run(fmt.Sprint(deleted), func(t *testing.T) {
|
||||
f, ids := inheritedReworkFixture(t, 1)
|
||||
item, _ := f.store.GetHumanItem(f.ctx, ids[0])
|
||||
item.InheritedRework = false
|
||||
item.ReworkRoutingPending = true
|
||||
item.LatestAuthorUserID = f.annotator.ID
|
||||
item.AssignedUserID = f.annotator.ID
|
||||
item.Status = domain.HumanItemClaimed
|
||||
if err := f.store.UpsertHumanItem(f.ctx, *item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if deleted {
|
||||
user := f.annotator
|
||||
user.Status = domain.UserStatusDisabled
|
||||
user.DeletedAt = nowISO()
|
||||
if err := f.store.UpsertUser(f.ctx, user); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
s := &server{store: f.store}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if deleted {
|
||||
if after.AssignedUserID != "new-worker" || !after.InheritedRework {
|
||||
t.Fatal("late deletion left rework on deleted account")
|
||||
}
|
||||
} else {
|
||||
if after.AssignedUserID != f.annotator.ID || after.InheritedRework || after.Version != item.Version {
|
||||
t.Fatal("normal directed rework behavior changed")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkLegacyFormalRevisionFallback(t *testing.T) {
|
||||
f, _ := inheritedReworkFixture(t, 0)
|
||||
task := f.task
|
||||
task.ID = "legacy-reviewed-task"
|
||||
task.CurrentRevision = 1
|
||||
task.SupplierID = f.annotator.SupplierID
|
||||
task.Status = domain.AnnotationTaskStatusRework
|
||||
if err := f.store.UpsertAnnotationTask(f.ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
item := f.item
|
||||
item.ID = "legacy-reviewed-item"
|
||||
item.LegacyTaskID = task.ID
|
||||
item.CurrentRevision = 1
|
||||
item.ReviewRound = 1
|
||||
item.InheritedRework = true
|
||||
item.AssignedUserID = ""
|
||||
item.SupplierID = task.SupplierID
|
||||
item.Status = domain.HumanItemRework
|
||||
item.LatestAuthorUserID = "deleted-author"
|
||||
if err := f.store.UpsertHumanItem(f.ctx, item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := f.store.UpsertAnnotationRevision(f.ctx, domain.AnnotationRevisionRecord{ID: "legacy-formal", OrganizationID: item.OrganizationID, TaskID: task.ID, Revision: 1, Segments: []domain.AnnotationSegment{{ID: "formal", Start: 0, End: 10, Description: "旧正式描述"}}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := f.store.UpsertSliceRevision(f.ctx, domain.SliceRevisionRecord{ID: "legacy-slice", OrganizationID: item.OrganizationID, TaskID: task.ID, Revision: 1, Segments: []domain.SliceSegment{{ID: "formal-slice", Start: 0, End: 10}}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := &server{store: f.store}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if after.AssignedUserID == "" {
|
||||
t.Fatal("compatible formal history was not assigned")
|
||||
}
|
||||
actions := decodeMaterializationValue[domain.AnnotationSegment](mapValue(after.Subject.Payload["annotationWorkspace"])["segments"])
|
||||
if len(actions) != 1 || actions[0].Description != "旧正式描述" {
|
||||
t.Fatal("wrong legacy snapshot adopted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkNoEligibleRecipientRetainsContents(t *testing.T) {
|
||||
f, ids := inheritedReworkFixture(t, 1)
|
||||
for _, id := range []string{f.annotator.ID, "new-worker"} {
|
||||
user, _ := f.store.GetUser(f.ctx, id)
|
||||
user.Status = domain.UserStatusDisabled
|
||||
if err := f.store.UpsertUser(f.ctx, *user); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
before, _ := f.store.GetHumanItem(f.ctx, ids[0])
|
||||
s := &server{store: f.store}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after, _ := f.store.GetHumanItem(f.ctx, ids[0])
|
||||
if !reflect.DeepEqual(before, after) {
|
||||
t.Fatal("unassignable task was cleared or forced onto another role")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkKeepsActiveAuthorWithLegacyPoolSwitch(t *testing.T) {
|
||||
t.Setenv("DELETED_ANNOTATOR_HANDOFF", "true")
|
||||
t.Setenv("FANYU_REWORK_RETURN", "pool")
|
||||
if reworkReturnMode() != "directed" {
|
||||
t.Fatal("new policy allowed ordinary returns into public pool")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkHonorsOrdinaryBufferCapacity(t *testing.T) {
|
||||
f, ids := inheritedReworkFixture(t, 1)
|
||||
batch, _ := f.store.GetHumanBatch(f.ctx, f.item.BatchID)
|
||||
batch.ProcessingProfile = domain.ProcessingProfileRawHuman
|
||||
if err := f.store.UpsertHumanBatch(f.ctx, *batch); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.store.SetSupplierProjectAuthorization(f.ctx, f.item.OrganizationID, f.item.ProjectID, f.annotator.SupplierID, true, f.admin.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, actor := range []string{f.annotator.ID, "new-worker"} {
|
||||
owned := f.item
|
||||
owned.ID = "capacity-" + actor
|
||||
owned.AssignedUserID = actor
|
||||
owned.SupplierID = f.annotator.SupplierID
|
||||
if err := f.store.UpsertHumanItem(f.ctx, owned); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.store.AnnotationWorkBuffer(f.ctx, owned.OrganizationID, actor, owned.ProjectID, 1); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
s := &server{store: f.store}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pending, _ := f.store.GetHumanItem(f.ctx, ids[0])
|
||||
if pending.AssignedUserID != "" {
|
||||
t.Fatal("inherited assignment exceeded ordinary pool capacity")
|
||||
}
|
||||
// Free the first worker's ordinary tasks; the second worker stays full.
|
||||
owned, _ := f.store.ListHumanItems(f.ctx, f.item.OrganizationID, domain.HumanItemQuery{ActorUserID: f.annotator.ID})
|
||||
for _, item := range owned {
|
||||
if item.AssignedUserID == f.annotator.ID {
|
||||
item.AssignedUserID = ""
|
||||
item.CurrentStageID = domain.FanyuV58StageQualityReview
|
||||
item.Version++
|
||||
if err := f.store.UpsertHumanItem(f.ctx, item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := s.dispatchDeletedAuthorReworks(f.ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assigned, _ := f.store.GetHumanItem(f.ctx, ids[0])
|
||||
if assigned.AssignedUserID != f.annotator.ID {
|
||||
t.Fatal("available qualified worker did not receive inherited rework", assigned.AssignedUserID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletedAuthorReworkRemainsClaimableAfterSubmission(t *testing.T) {
|
||||
f, ids := inheritedReworkFixture(t, 1)
|
||||
s := &server{store: f.store}
|
||||
item, _ := f.store.GetHumanItem(f.ctx, ids[0])
|
||||
annotationRequest := httptest.NewRequest("GET", "/", nil)
|
||||
annotationRequest = annotationRequest.WithContext(context.WithValue(f.ctx, authContextKey{}, &authenticatedPrincipal{User: f.annotator, Organization: domain.OrganizationRecord{ID: item.OrganizationID, Status: domain.OrganizationStatusActive}}))
|
||||
item.InheritedRework = false
|
||||
item.ReworkRoutingPending = true
|
||||
if s.canClaimHumanItem(f.ctx, annotationRequest, *item, domain.AnnotationRoleAnnotator) {
|
||||
t.Fatal("pending routing advertised a manual claim")
|
||||
}
|
||||
item.ReworkRoutingPending = false
|
||||
item.InheritedRework = true
|
||||
acceptor := f.reviewer
|
||||
acceptor.ID = "inherited-acceptor"
|
||||
acceptor.Role = domain.UserRoleAcceptanceReviewer
|
||||
if err := f.store.UpsertUser(f.ctx, acceptor); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := f.store.UpsertProjectMember(f.ctx, domain.ProjectMemberRecord{ID: "inherited-acceptor-member", OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, UserID: acceptor.ID, SupplierID: acceptor.SupplierID, Roles: []domain.AnnotationAssignmentRole{domain.AnnotationRoleAcceptanceReviewer}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, scenario := range []struct {
|
||||
stage string
|
||||
role domain.AnnotationAssignmentRole
|
||||
actor domain.UserRecord
|
||||
}{
|
||||
{domain.FanyuV58StageQualityReview, domain.AnnotationRoleQualityReviewer, f.reviewer},
|
||||
{"acceptance", domain.AnnotationRoleAcceptanceReviewer, acceptor},
|
||||
} {
|
||||
item.CurrentStageID = scenario.stage
|
||||
item.Status = domain.HumanItemReady
|
||||
r := httptest.NewRequest("GET", "/", nil)
|
||||
r = r.WithContext(context.WithValue(f.ctx, authContextKey{}, &authenticatedPrincipal{User: scenario.actor, Organization: domain.OrganizationRecord{ID: item.OrganizationID, Status: domain.OrganizationStatusActive}}))
|
||||
if !s.canClaimHumanItem(f.ctx, r, *item, scenario.role) {
|
||||
t.Fatal("inherited marker blocked downstream reviewer", scenario.stage)
|
||||
}
|
||||
if scenario.role == domain.AnnotationRoleQualityReviewer {
|
||||
item.LatestAuthorUserID = scenario.actor.ID
|
||||
if s.canClaimHumanItem(f.ctx, r, *item, scenario.role) {
|
||||
t.Fatal("self-review was allowed")
|
||||
}
|
||||
item.LatestAuthorUserID = "deleted-author"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1046,6 +1046,12 @@ func (server *server) canAccessHumanItem(ctx context.Context, request *http.Requ
|
||||
}
|
||||
|
||||
func (server *server) canClaimHumanItem(ctx context.Context, request *http.Request, item domain.HumanItemRecord, requiredRole domain.AnnotationAssignmentRole) bool {
|
||||
if item.CurrentStageID == domain.FanyuV58StageAnnotationCombined && (item.ReworkRoutingPending || (item.InheritedRework && item.AssignedUserID == "")) {
|
||||
return false
|
||||
}
|
||||
if item.AccountHandoff != nil && item.AccountHandoff.Pending {
|
||||
return false
|
||||
}
|
||||
principal, ok := currentPrincipal(request)
|
||||
if !ok || principal == nil {
|
||||
return false
|
||||
|
||||
@ -25,6 +25,7 @@ func poolReadMode(r *http.Request) humanPoolReadMode {
|
||||
// Summary is an explicit opt-in protocol. Legacy clients retain their item
|
||||
// response; workbenches must fetch detail before opening a summary.
|
||||
type humanPoolSummary struct {
|
||||
InheritedRework bool `json:"inheritedRework,omitempty"`
|
||||
Kind string `json:"kind"`
|
||||
DurationSeconds float64 `json:"durationSeconds,omitempty"`
|
||||
SourceValidationStatus string `json:"sourceValidationStatus,omitempty"`
|
||||
@ -87,7 +88,7 @@ func summarizeHumanPoolItem(i domain.HumanItemRecord) humanPoolSummary {
|
||||
}
|
||||
subject := i.Subject
|
||||
subject.Payload = nil
|
||||
return humanPoolSummary{Kind: "pool-summary", DurationSeconds: seconds,
|
||||
return humanPoolSummary{InheritedRework: i.InheritedRework, Kind: "pool-summary", DurationSeconds: seconds,
|
||||
SourceValidationStatus: i.SourceValidationStatus,
|
||||
TraceSummary: i.TraceSummary,
|
||||
ReviewHistory: i.ReviewHistory,
|
||||
|
||||
@ -84,6 +84,8 @@ func (s *server) planQualityReviewDecision(w http.ResponseWriter, r *http.Reques
|
||||
task.Status = domain.AnnotationTaskStatusRework
|
||||
item.ReviewRound++
|
||||
item.CurrentStageID = "annotation_combined"
|
||||
item.ReworkRoutingPending = accountHandoffPolicyEnabled() || item.InheritedRework
|
||||
item.InheritedRework = false
|
||||
item.Status = domain.HumanItemRework
|
||||
item.AssignedUserID, task.AssignedUserID = returnAuthor, returnAuthor
|
||||
if returnAuthor != "" {
|
||||
|
||||
@ -7,11 +7,12 @@ import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// 新删除交接规则启用时,仍在职的原标注员保持定向返工。
|
||||
// reworkReturnMode 报告正式退回的路由策略(B01 回退开关):
|
||||
// - "directed"(默认):退回定向归还原标注员(spec §10 行为变更裁定,2026-10-12 用户确认);
|
||||
// - "pool":回退开关打开(FANYU_REWORK_RETURN=pool),恢复退回公共池的历史行为。
|
||||
func reworkReturnMode() string {
|
||||
if strings.EqualFold(strings.TrimSpace(os.Getenv("FANYU_REWORK_RETURN")), "pool") {
|
||||
if !accountHandoffPolicyEnabled() && strings.EqualFold(strings.TrimSpace(os.Getenv("FANYU_REWORK_RETURN")), "pool") {
|
||||
return "pool"
|
||||
}
|
||||
return "directed"
|
||||
|
||||
@ -1426,6 +1426,7 @@ func NewHandlerWithConfig(dataStore store.Store, aiService ai.Service, appConfig
|
||||
image: appConfig.Image,
|
||||
}
|
||||
instance.startHumanReleasePartitionSweeper()
|
||||
instance.startAccountHandoffRecovery()
|
||||
return instance.routes()
|
||||
}
|
||||
|
||||
@ -8193,6 +8194,12 @@ func (server *server) handleUsersDelete(writer http.ResponseWriter, request *htt
|
||||
}
|
||||
// Disable first so automatic project enrollment cannot re-add this account
|
||||
// while its existing memberships are being removed.
|
||||
if item.DeletionRequestedAt == "" && accountHandoffPolicyEnabled() {
|
||||
item.DeletionRequestedAt = nowISO()
|
||||
}
|
||||
if item.DeletionActorID == "" {
|
||||
item.DeletionActorID = actorID
|
||||
}
|
||||
item.Status = domain.UserStatusDisabled
|
||||
item.UpdatedAt = nowISO()
|
||||
if err := server.store.UpdateUser(request.Context(), *item, expected); err != nil {
|
||||
@ -8201,37 +8208,8 @@ func (server *server) handleUsersDelete(writer http.ResponseWriter, request *htt
|
||||
}
|
||||
// The deleted account's cached principal (if any) must not survive.
|
||||
invalidateAuthCacheForUser(item.ID)
|
||||
if err := server.store.DeleteSessionsByUser(request.Context(), item.ID); err != nil {
|
||||
renderError(writer, http.StatusInternalServerError, err)
|
||||
return
|
||||
}
|
||||
if _, err := server.releaseUserClaimedTasks(request.Context(), item.OrganizationID, item.ID, actorID); err != nil {
|
||||
renderJSON(writer, http.StatusInternalServerError, map[string]string{"error": "账号已停用,但任务释放未完成,请重试删除", "userId": item.ID})
|
||||
return
|
||||
}
|
||||
projectMembers, err := server.store.ListProjectMembers(request.Context(), item.OrganizationID, "")
|
||||
if err != nil {
|
||||
renderError(writer, http.StatusInternalServerError, err)
|
||||
return
|
||||
}
|
||||
for _, pm := range projectMembers {
|
||||
if pm.UserID == item.ID {
|
||||
if _, err := server.store.DeleteProjectMember(request.Context(), pm.ID); err != nil {
|
||||
renderError(writer, http.StatusInternalServerError, err)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A disabled account remains manageable; a completed deletion is an
|
||||
// immutable audit identity and must never return in directory listings.
|
||||
beforeDeletion := *item
|
||||
item.DeletedAt = nowISO()
|
||||
item.DeletedBy = actorID
|
||||
item.PasswordHash = ""
|
||||
item.UpdatedAt = item.DeletedAt
|
||||
if err := server.store.UpdateUser(request.Context(), *item, beforeDeletion); err != nil {
|
||||
renderUserMutationError(writer, err)
|
||||
if err := server.finishAccountDeletion(request.Context(), *item); err != nil {
|
||||
renderJSON(writer, http.StatusInternalServerError, map[string]string{"error": "账号已停用,任务交接尚未完成,请重试删除", "userId": item.ID})
|
||||
return
|
||||
}
|
||||
writer.WriteHeader(http.StatusNoContent)
|
||||
|
||||
@ -91,7 +91,48 @@ func TestSlicePhaseBrowserReplica(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
return f.store.UpsertHumanItem(f.ctx, item)
|
||||
if err := f.store.UpsertHumanItem(f.ctx, item); err != nil {
|
||||
return err
|
||||
}
|
||||
if status == "deleted-reset" {
|
||||
// An actual deletion/reset followed by a different account opening
|
||||
// the same source. Never synthesize the resulting editor response.
|
||||
if err := f.store.UpsertSliceRevision(f.ctx, domain.SliceRevisionRecord{ID: "old-user-cut", OrganizationID: item.OrganizationID, TaskID: f.task.ID, Revision: 1, AuthorUserID: f.annotator.ID, Segments: before}); err != nil {
|
||||
return err
|
||||
}
|
||||
deleted := f.annotator
|
||||
deleted.Status = domain.UserStatusDisabled
|
||||
deleted.DeletionRequestedAt = nowISO()
|
||||
deleted.DeletionActorID = f.admin.ID
|
||||
if err := f.store.UpsertUser(f.ctx, deleted); err != nil {
|
||||
return err
|
||||
}
|
||||
deletion := &server{store: f.store}
|
||||
if err := deletion.finishAccountDeletion(f.ctx, deleted); err != nil {
|
||||
return err
|
||||
}
|
||||
recipient := f.annotator
|
||||
recipient.ID = "replacement-annotator"
|
||||
if err := f.store.UpsertUser(f.ctx, recipient); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := f.store.UpsertProjectMember(f.ctx, domain.ProjectMemberRecord{ID: "replacement-member", OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, UserID: recipient.ID, SupplierID: recipient.SupplierID, Roles: []domain.AnnotationAssignmentRole{domain.AnnotationRoleAnnotator}}); err != nil {
|
||||
return err
|
||||
}
|
||||
fresh, err := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
version := fresh.Version
|
||||
fresh.AssignedUserID = recipient.ID
|
||||
fresh.Status = domain.HumanItemClaimed
|
||||
fresh.Version++
|
||||
if ok, err := f.store.UpdateHumanItemVersioned(f.ctx, *fresh, version); err != nil || !ok {
|
||||
return fmt.Errorf("synthetic replacement claim: %v", err)
|
||||
}
|
||||
f.annotator = recipient
|
||||
}
|
||||
return nil
|
||||
}
|
||||
root := t.TempDir()
|
||||
f.task.SourceVideo = "/api/runs/" + f.task.RunID + "/artifacts/left-video/content"
|
||||
@ -232,7 +273,7 @@ func TestInitialSliceConfirmationBrowserReplica(t *testing.T) {
|
||||
t.Skip("local browser opt-in")
|
||||
}
|
||||
t.Setenv("REWORK_BOUNDARY_BROWSER_TEST", "1")
|
||||
for _, scenario := range []string{"pending", "edited", "legacy", "failed", "save-retry", "save-throttle", "save-timeout", "save-dictionary", "malformed"} {
|
||||
for _, scenario := range []string{"pending", "edited", "legacy", "failed", "save-retry", "save-throttle", "save-timeout", "save-dictionary", "deleted-reset", "malformed"} {
|
||||
t.Run(scenario, func(t *testing.T) {
|
||||
t.Setenv("INITIAL_SLICE_CONFIRM_SCENARIO", scenario)
|
||||
TestSlicePhaseBrowserReplica(t)
|
||||
|
||||
@ -173,16 +173,18 @@ type StructureChangeSummary struct {
|
||||
}
|
||||
|
||||
type SliceRevisionRecord struct {
|
||||
ID string `json:"id" bson:"id"`
|
||||
OrganizationID string `json:"organizationId" bson:"organizationId"`
|
||||
TaskID string `json:"taskId" bson:"taskId"`
|
||||
Revision int `json:"revision" bson:"revision"`
|
||||
Segments []SliceSegment `json:"segments" bson:"segments"`
|
||||
Operations []SliceOperation `json:"operations,omitempty" bson:"operations,omitempty"`
|
||||
AuthorUserID string `json:"authorUserId" bson:"authorUserId"`
|
||||
ChangeReason string `json:"changeReason,omitempty" bson:"changeReason,omitempty"`
|
||||
DurationSeconds int64 `json:"durationSeconds,omitempty" bson:"durationSeconds,omitempty"`
|
||||
CreatedAt string `json:"createdAt" bson:"createdAt"`
|
||||
AnnotationEpoch int `json:"-" bson:"annotationEpoch,omitempty"`
|
||||
SupersededByAccountDeletion bool `json:"-" bson:"supersededByAccountDeletion,omitempty"`
|
||||
ID string `json:"id" bson:"id"`
|
||||
OrganizationID string `json:"organizationId" bson:"organizationId"`
|
||||
TaskID string `json:"taskId" bson:"taskId"`
|
||||
Revision int `json:"revision" bson:"revision"`
|
||||
Segments []SliceSegment `json:"segments" bson:"segments"`
|
||||
Operations []SliceOperation `json:"operations,omitempty" bson:"operations,omitempty"`
|
||||
AuthorUserID string `json:"authorUserId" bson:"authorUserId"`
|
||||
ChangeReason string `json:"changeReason,omitempty" bson:"changeReason,omitempty"`
|
||||
DurationSeconds int64 `json:"durationSeconds,omitempty" bson:"durationSeconds,omitempty"`
|
||||
CreatedAt string `json:"createdAt" bson:"createdAt"`
|
||||
// StructureChangeSummary is attached by the server on rework submissions
|
||||
// only (ticket 07): net slice-structure difference against the return
|
||||
// baseline. Nil on first submissions and legacy records.
|
||||
@ -192,35 +194,37 @@ type SliceRevisionRecord struct {
|
||||
type SemanticAnnotationRevisionRecord = AnnotationRevisionRecord
|
||||
|
||||
type AnnotationTaskRecord struct {
|
||||
ReviewOperationID string `json:"reviewOperationId,omitempty" bson:"reviewOperationId,omitempty"`
|
||||
SupplierPool bool `json:"-" bson:"-"`
|
||||
SupplierPoolClosed bool `json:"-" bson:"-"`
|
||||
ContinuousActions bool `json:"continuousActions,omitempty" bson:"-"`
|
||||
ProcessingScope string `json:"processingScope,omitempty" bson:"processingScope,omitempty"`
|
||||
ProcessingProfile string `json:"processingProfile,omitempty" bson:"processingProfile,omitempty"`
|
||||
ID string `json:"id" bson:"id"`
|
||||
OrganizationID string `json:"organizationId,omitempty" bson:"organizationId,omitempty"`
|
||||
ProjectID string `json:"projectId" bson:"projectId"`
|
||||
DataTaskID string `json:"dataTaskId,omitempty" bson:"dataTaskId,omitempty"`
|
||||
WorkflowID string `json:"workflowId,omitempty" bson:"workflowId,omitempty"`
|
||||
RunID string `json:"runId,omitempty" bson:"runId,omitempty"`
|
||||
NodeID string `json:"nodeId,omitempty" bson:"nodeId,omitempty"`
|
||||
ArtifactID string `json:"artifactId,omitempty" bson:"artifactId,omitempty"`
|
||||
SourceVideo string `json:"sourceVideo" bson:"sourceVideo"`
|
||||
RightArtifactID string `json:"rightArtifactId,omitempty" bson:"rightArtifactId,omitempty"`
|
||||
RightSourceVideo string `json:"rightSourceVideo,omitempty" bson:"rightSourceVideo,omitempty"`
|
||||
ClipStart float64 `json:"clipStart" bson:"clipStart"`
|
||||
ClipEnd float64 `json:"clipEnd" bson:"clipEnd"`
|
||||
SourceIsFullVideo bool `json:"sourceIsFullVideo,omitempty" bson:"sourceIsFullVideo,omitempty"`
|
||||
SourceDurationSeconds float64 `json:"sourceDurationSeconds,omitempty" bson:"sourceDurationSeconds,omitempty"`
|
||||
CoverageStartSecond float64 `json:"coverageStartSecond,omitempty" bson:"coverageStartSecond,omitempty"`
|
||||
CoverageEndSecond float64 `json:"coverageEndSecond,omitempty" bson:"coverageEndSecond,omitempty"`
|
||||
CoverageMode string `json:"coverageMode,omitempty" bson:"coverageMode,omitempty"`
|
||||
TemplateID string `json:"templateId" bson:"templateId"`
|
||||
TemplateVersion int `json:"templateVersion" bson:"templateVersion"`
|
||||
LabelSnapshot []SemanticLabelDefinition `json:"labelSnapshot,omitempty" bson:"labelSnapshot,omitempty"`
|
||||
SupplierID string `json:"supplierId,omitempty" bson:"supplierId,omitempty"`
|
||||
LegacyAssignment map[string]string `json:"legacyAssignment,omitempty" bson:"legacyAssignment,omitempty"`
|
||||
AnnotationEpoch int `json:"-" bson:"annotationEpoch,omitempty"`
|
||||
AnnotationRevisionFloor int `json:"-" bson:"annotationRevisionFloor,omitempty"`
|
||||
ReviewOperationID string `json:"reviewOperationId,omitempty" bson:"reviewOperationId,omitempty"`
|
||||
SupplierPool bool `json:"-" bson:"-"`
|
||||
SupplierPoolClosed bool `json:"-" bson:"-"`
|
||||
ContinuousActions bool `json:"continuousActions,omitempty" bson:"-"`
|
||||
ProcessingScope string `json:"processingScope,omitempty" bson:"processingScope,omitempty"`
|
||||
ProcessingProfile string `json:"processingProfile,omitempty" bson:"processingProfile,omitempty"`
|
||||
ID string `json:"id" bson:"id"`
|
||||
OrganizationID string `json:"organizationId,omitempty" bson:"organizationId,omitempty"`
|
||||
ProjectID string `json:"projectId" bson:"projectId"`
|
||||
DataTaskID string `json:"dataTaskId,omitempty" bson:"dataTaskId,omitempty"`
|
||||
WorkflowID string `json:"workflowId,omitempty" bson:"workflowId,omitempty"`
|
||||
RunID string `json:"runId,omitempty" bson:"runId,omitempty"`
|
||||
NodeID string `json:"nodeId,omitempty" bson:"nodeId,omitempty"`
|
||||
ArtifactID string `json:"artifactId,omitempty" bson:"artifactId,omitempty"`
|
||||
SourceVideo string `json:"sourceVideo" bson:"sourceVideo"`
|
||||
RightArtifactID string `json:"rightArtifactId,omitempty" bson:"rightArtifactId,omitempty"`
|
||||
RightSourceVideo string `json:"rightSourceVideo,omitempty" bson:"rightSourceVideo,omitempty"`
|
||||
ClipStart float64 `json:"clipStart" bson:"clipStart"`
|
||||
ClipEnd float64 `json:"clipEnd" bson:"clipEnd"`
|
||||
SourceIsFullVideo bool `json:"sourceIsFullVideo,omitempty" bson:"sourceIsFullVideo,omitempty"`
|
||||
SourceDurationSeconds float64 `json:"sourceDurationSeconds,omitempty" bson:"sourceDurationSeconds,omitempty"`
|
||||
CoverageStartSecond float64 `json:"coverageStartSecond,omitempty" bson:"coverageStartSecond,omitempty"`
|
||||
CoverageEndSecond float64 `json:"coverageEndSecond,omitempty" bson:"coverageEndSecond,omitempty"`
|
||||
CoverageMode string `json:"coverageMode,omitempty" bson:"coverageMode,omitempty"`
|
||||
TemplateID string `json:"templateId" bson:"templateId"`
|
||||
TemplateVersion int `json:"templateVersion" bson:"templateVersion"`
|
||||
LabelSnapshot []SemanticLabelDefinition `json:"labelSnapshot,omitempty" bson:"labelSnapshot,omitempty"`
|
||||
SupplierID string `json:"supplierId,omitempty" bson:"supplierId,omitempty"`
|
||||
LegacyAssignment map[string]string `json:"legacyAssignment,omitempty" bson:"legacyAssignment,omitempty"`
|
||||
// Deprecated compatibility fields. Mongo migration copies their historical
|
||||
// value into LegacyAssignment and unsets them for production records.
|
||||
WorkgroupID string `json:"-" bson:"workgroupId,omitempty"`
|
||||
@ -237,19 +241,21 @@ type AnnotationTaskRecord struct {
|
||||
}
|
||||
|
||||
type AnnotationRevisionRecord struct {
|
||||
ID string `json:"id" bson:"id"`
|
||||
OrganizationID string `json:"organizationId,omitempty" bson:"organizationId,omitempty"`
|
||||
TaskID string `json:"taskId" bson:"taskId"`
|
||||
Revision int `json:"revision" bson:"revision"`
|
||||
TemplateVersion int `json:"templateVersion" bson:"templateVersion"`
|
||||
Segments []AnnotationSegment `json:"segments" bson:"segments"`
|
||||
ObjectHints []string `json:"objectHints,omitempty" bson:"objectHints,omitempty"`
|
||||
AuthorUserID string `json:"authorUserId" bson:"authorUserId"`
|
||||
ChangeReason string `json:"changeReason,omitempty" bson:"changeReason,omitempty"`
|
||||
IsRework bool `json:"isRework" bson:"isRework"`
|
||||
CountsAsOutput bool `json:"countsAsOutput" bson:"countsAsOutput"`
|
||||
DurationSeconds int64 `json:"durationSeconds,omitempty" bson:"durationSeconds,omitempty"`
|
||||
CreatedAt string `json:"createdAt" bson:"createdAt"`
|
||||
AnnotationEpoch int `json:"-" bson:"annotationEpoch,omitempty"`
|
||||
SupersededByAccountDeletion bool `json:"-" bson:"supersededByAccountDeletion,omitempty"`
|
||||
ID string `json:"id" bson:"id"`
|
||||
OrganizationID string `json:"organizationId,omitempty" bson:"organizationId,omitempty"`
|
||||
TaskID string `json:"taskId" bson:"taskId"`
|
||||
Revision int `json:"revision" bson:"revision"`
|
||||
TemplateVersion int `json:"templateVersion" bson:"templateVersion"`
|
||||
Segments []AnnotationSegment `json:"segments" bson:"segments"`
|
||||
ObjectHints []string `json:"objectHints,omitempty" bson:"objectHints,omitempty"`
|
||||
AuthorUserID string `json:"authorUserId" bson:"authorUserId"`
|
||||
ChangeReason string `json:"changeReason,omitempty" bson:"changeReason,omitempty"`
|
||||
IsRework bool `json:"isRework" bson:"isRework"`
|
||||
CountsAsOutput bool `json:"countsAsOutput" bson:"countsAsOutput"`
|
||||
DurationSeconds int64 `json:"durationSeconds,omitempty" bson:"durationSeconds,omitempty"`
|
||||
CreatedAt string `json:"createdAt" bson:"createdAt"`
|
||||
}
|
||||
|
||||
type AnnotationReviewRecord struct {
|
||||
|
||||
@ -164,7 +164,20 @@ type HumanPoolSearchMetadata struct {
|
||||
PayloadDirectories []string `bson:"payloadDirectories"`
|
||||
}
|
||||
|
||||
type AccountTaskHandoff struct {
|
||||
UserID string `json:"-" bson:"userId"`
|
||||
ActorID string `json:"-" bson:"actorId"`
|
||||
Reset bool `json:"-" bson:"reset"`
|
||||
Pending bool `json:"-" bson:"pending"`
|
||||
RevisionFloor int `json:"-" bson:"revisionFloor"`
|
||||
At string `json:"-" bson:"at"`
|
||||
}
|
||||
|
||||
type HumanItemRecord struct {
|
||||
AnnotationEpoch int `json:"-" bson:"annotationEpoch,omitempty"`
|
||||
AccountHandoff *AccountTaskHandoff `json:"-" bson:"accountHandoff,omitempty"`
|
||||
ReworkRoutingPending bool `json:"-" bson:"reworkRoutingPending,omitempty"`
|
||||
InheritedRework bool `json:"inheritedRework,omitempty" bson:"inheritedRework,omitempty"`
|
||||
PoolSearch *HumanPoolSearchMetadata `json:"-" bson:"poolSearch,omitempty"`
|
||||
SourceValidationStatus string `json:"sourceValidationStatus,omitempty" bson:"-"`
|
||||
TraceSummary *HumanTraceSummary `json:"traceSummary,omitempty" bson:"-"`
|
||||
@ -345,6 +358,9 @@ type HumanItemCursor struct {
|
||||
}
|
||||
|
||||
type HumanItemQuery struct {
|
||||
InheritedUnassigned bool
|
||||
NeedsReworkRouting bool
|
||||
|
||||
// Server-derived necessary QC scope; nil is unrestricted, empty matches nothing.
|
||||
AvailableProjectIDs []string
|
||||
AvailableSupplierID *string
|
||||
|
||||
@ -986,6 +986,8 @@ type AuditTaskRecord struct {
|
||||
}
|
||||
|
||||
type UserRecord struct {
|
||||
DeletionRequestedAt string `json:"-" bson:"deletionRequestedAt,omitempty"`
|
||||
DeletionActorID string `json:"-" bson:"deletionActorId,omitempty"`
|
||||
// Empty/legacy is grandfathered; every new account and role change is managed.
|
||||
AcceptanceAccessMode string `json:"acceptanceAccessMode,omitempty" bson:"acceptanceAccessMode,omitempty"`
|
||||
DeletedAt string `json:"deletedAt,omitempty" bson:"deletedAt,omitempty"`
|
||||
|
||||
@ -92,12 +92,12 @@ func (store *MongoStore) updateAdministrator(ctx context.Context, item domain.Us
|
||||
|
||||
// Compare security and profile fields that may be overwritten by a stale form.
|
||||
func sameUserMutationSnapshot(a, b domain.UserRecord) bool {
|
||||
return a.AcceptanceAccessMode == b.AcceptanceAccessMode && a.DeletedAt == b.DeletedAt && a.DeletedBy == b.DeletedBy && a.ID == b.ID && a.OrganizationID == b.OrganizationID && a.SupplierID == b.SupplierID && a.Role == b.Role && a.IsSuperAdmin == b.IsSuperAdmin && a.Status == b.Status && a.PasswordHash == b.PasswordHash && a.UpdatedAt == b.UpdatedAt && a.Name == b.Name && a.Phone == b.Phone && reflect.DeepEqual(a.DefaultProjectRoles, b.DefaultProjectRoles)
|
||||
return a.DeletionRequestedAt == b.DeletionRequestedAt && a.DeletionActorID == b.DeletionActorID && a.AcceptanceAccessMode == b.AcceptanceAccessMode && a.DeletedAt == b.DeletedAt && a.DeletedBy == b.DeletedBy && a.ID == b.ID && a.OrganizationID == b.OrganizationID && a.SupplierID == b.SupplierID && a.Role == b.Role && a.IsSuperAdmin == b.IsSuperAdmin && a.Status == b.Status && a.PasswordHash == b.PasswordHash && a.UpdatedAt == b.UpdatedAt && a.Name == b.Name && a.Phone == b.Phone && reflect.DeepEqual(a.DefaultProjectRoles, b.DefaultProjectRoles)
|
||||
}
|
||||
|
||||
func userMutationFilter(old domain.UserRecord) bson.M {
|
||||
filter := bson.M{"id": old.ID}
|
||||
for key, value := range map[string]string{"acceptanceAccessMode": old.AcceptanceAccessMode, "deletedAt": old.DeletedAt, "organizationId": old.OrganizationID, "supplierId": old.SupplierID, "role": string(old.Role), "status": string(old.Status), "passwordHash": old.PasswordHash, "updatedAt": old.UpdatedAt, "name": old.Name, "phone": old.Phone} {
|
||||
for key, value := range map[string]string{"deletionRequestedAt": old.DeletionRequestedAt, "deletionActorId": old.DeletionActorID, "acceptanceAccessMode": old.AcceptanceAccessMode, "deletedAt": old.DeletedAt, "organizationId": old.OrganizationID, "supplierId": old.SupplierID, "role": string(old.Role), "status": string(old.Status), "passwordHash": old.PasswordHash, "updatedAt": old.UpdatedAt, "name": old.Name, "phone": old.Phone} {
|
||||
if value == "" {
|
||||
filter[key] = bson.M{"$in": bson.A{"", nil}}
|
||||
} else {
|
||||
|
||||
@ -67,7 +67,7 @@ func (s *MemoryStore) ClaimHumanItemVersioned(_ context.Context, item domain.Hum
|
||||
if !ok || old.Version != version {
|
||||
return false, nil
|
||||
}
|
||||
if role == domain.UserRoleAnnotator && (!sameAnnotationClaimSourceSnapshot(old, item) || !s.annotationSourceReadyLocked(old)) {
|
||||
if role == domain.UserRoleAnnotator && (old.InheritedRework || old.ReworkRoutingPending || (old.AccountHandoff != nil && old.AccountHandoff.Pending) || old.AnnotationEpoch != item.AnnotationEpoch || !sameAnnotationClaimSourceSnapshot(old, item) || !s.annotationSourceReadyLocked(old)) {
|
||||
return false, ErrHumanClaimConflict
|
||||
}
|
||||
if batch, ok := s.humanBatches[item.BatchID]; !shared && ok && batch.ProcessingProfile == domain.ProcessingProfileRawHuman && domain.IsFanyuV58ThreeRoleHumanLoop(batch.Stages) {
|
||||
@ -123,6 +123,9 @@ func (s *MongoStore) claimHumanItemSupplierGuarded(ctx context.Context, item dom
|
||||
if e != nil {
|
||||
return false, e
|
||||
}
|
||||
if current != nil && (current.InheritedRework || current.ReworkRoutingPending || (current.AccountHandoff != nil && current.AccountHandoff.Pending)) {
|
||||
return false, ErrHumanClaimConflict
|
||||
}
|
||||
if current == nil || current.Version != version {
|
||||
return false, nil
|
||||
}
|
||||
|
||||
@ -27,7 +27,7 @@ func (s *MemoryStore) VisitAnnotationClaimCandidates(ctx context.Context, org, p
|
||||
return false, nil
|
||||
}
|
||||
func (s *MongoStore) VisitAnnotationClaimCandidates(ctx context.Context, org, project string, visit func(domain.HumanItemRecord) (bool, error)) (bool, error) {
|
||||
filter := bson.M{"organizationId": org, "projectId": project, "pluginId": "annotation-qa", "legacyTaskId": bson.M{"$exists": true, "$ne": ""}, "assignedUserId": bson.M{"$in": bson.A{"", nil}}, "status": bson.M{"$in": bson.A{domain.HumanItemReady, domain.HumanItemRework, domain.HumanItemSubmitted}}}
|
||||
filter := bson.M{"reworkRoutingPending": bson.M{"$ne": true}, "inheritedRework": bson.M{"$ne": true}, "organizationId": org, "projectId": project, "pluginId": "annotation-qa", "legacyTaskId": bson.M{"$exists": true, "$ne": ""}, "assignedUserId": bson.M{"$in": bson.A{"", nil}}, "status": bson.M{"$in": bson.A{domain.HumanItemReady, domain.HumanItemRework, domain.HumanItemSubmitted}}}
|
||||
projection := annotationSourceAdmissionProjection()
|
||||
projection["_id"] = 0
|
||||
pipeline := mongo.Pipeline{{{Key: "$sample", Value: bson.M{"size": 64}}}, {{Key: "$match", Value: filter}}, {{Key: "$project", Value: projection}}}
|
||||
|
||||
@ -290,6 +290,13 @@ func (store *MemoryStore) GetAnnotationTask(_ context.Context, id string) (*doma
|
||||
func (store *MemoryStore) UpsertAnnotationTask(_ context.Context, item domain.AnnotationTaskRecord) error {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
return store.upsertAnnotationTaskLocked(item)
|
||||
}
|
||||
|
||||
func (store *MemoryStore) upsertAnnotationTaskLocked(item domain.AnnotationTaskRecord) error {
|
||||
if old, ok := store.annotationTasks[item.ID]; ok && old.AnnotationEpoch > item.AnnotationEpoch {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
if store.supplierRecordBlockedLocked("annotationTasks", item.ID, item) {
|
||||
return nil
|
||||
}
|
||||
@ -327,7 +334,7 @@ func (store *MemoryStore) ListAnnotationRevisions(_ context.Context, organizatio
|
||||
defer store.mu.Unlock()
|
||||
items := make([]domain.AnnotationRevisionRecord, 0)
|
||||
for _, item := range store.annotationRevisions {
|
||||
if annotationOrganizationMatches(item.OrganizationID, organizationID) && (taskID == "" || item.TaskID == taskID) {
|
||||
if !item.SupersededByAccountDeletion && annotationOrganizationMatches(item.OrganizationID, organizationID) && (taskID == "" || item.TaskID == taskID) {
|
||||
items = append(items, item)
|
||||
}
|
||||
}
|
||||
@ -349,7 +356,7 @@ func (store *MemoryStore) ListAnnotationRevisionsByTaskIDs(_ context.Context, or
|
||||
defer store.mu.Unlock()
|
||||
items := make([]domain.AnnotationRevisionRecord, 0, len(allowed))
|
||||
for _, item := range store.annotationRevisions {
|
||||
if annotationOrganizationMatches(item.OrganizationID, organizationID) && allowed[item.TaskID] {
|
||||
if !item.SupersededByAccountDeletion && annotationOrganizationMatches(item.OrganizationID, organizationID) && allowed[item.TaskID] {
|
||||
items = append(items, item)
|
||||
}
|
||||
}
|
||||
@ -375,6 +382,12 @@ func annotationTaskIDSet(taskIDs []string) map[string]bool {
|
||||
func (store *MemoryStore) UpsertAnnotationRevision(_ context.Context, item domain.AnnotationRevisionRecord) error {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
if task, ok := store.annotationTasks[item.TaskID]; ok && (item.AnnotationEpoch != task.AnnotationEpoch || (task.AnnotationEpoch > 0 && item.Revision <= task.AnnotationRevisionFloor)) {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
if old, ok := store.annotationRevisions[item.ID]; ok && old.SupersededByAccountDeletion {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
if store.supplierRecordBlockedLocked("annotationRevisions", item.ID, item) {
|
||||
return nil
|
||||
}
|
||||
@ -405,7 +418,7 @@ func (store *MemoryStore) ListSliceRevisions(_ context.Context, organizationID s
|
||||
defer store.mu.Unlock()
|
||||
items := make([]domain.SliceRevisionRecord, 0)
|
||||
for _, item := range store.sliceRevisions {
|
||||
if annotationOrganizationMatches(item.OrganizationID, organizationID) && (taskID == "" || item.TaskID == taskID) {
|
||||
if !item.SupersededByAccountDeletion && annotationOrganizationMatches(item.OrganizationID, organizationID) && (taskID == "" || item.TaskID == taskID) {
|
||||
items = append(items, clone(item))
|
||||
}
|
||||
}
|
||||
@ -422,7 +435,7 @@ func (store *MemoryStore) ListSliceRevisionsByTaskIDs(_ context.Context, organiz
|
||||
defer store.mu.Unlock()
|
||||
items := make([]domain.SliceRevisionRecord, 0, len(allowed))
|
||||
for _, item := range store.sliceRevisions {
|
||||
if annotationOrganizationMatches(item.OrganizationID, organizationID) && allowed[item.TaskID] {
|
||||
if !item.SupersededByAccountDeletion && annotationOrganizationMatches(item.OrganizationID, organizationID) && allowed[item.TaskID] {
|
||||
items = append(items, clone(item))
|
||||
}
|
||||
}
|
||||
@ -433,6 +446,12 @@ func (store *MemoryStore) ListSliceRevisionsByTaskIDs(_ context.Context, organiz
|
||||
func (store *MemoryStore) UpsertSliceRevision(_ context.Context, item domain.SliceRevisionRecord) error {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
if task, ok := store.annotationTasks[item.TaskID]; ok && (item.AnnotationEpoch != task.AnnotationEpoch || (task.AnnotationEpoch > 0 && item.Revision <= task.AnnotationRevisionFloor)) {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
if old, ok := store.sliceRevisions[item.ID]; ok && old.SupersededByAccountDeletion {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
if store.supplierRecordBlockedLocked("sliceRevisions", item.ID, item) {
|
||||
return nil
|
||||
}
|
||||
@ -851,7 +870,7 @@ func (store *MongoStore) UpsertAnnotationTask(ctx context.Context, item domain.A
|
||||
item.CreatedAt = now
|
||||
}
|
||||
item.UpdatedAt = now
|
||||
if err := mongoUpsert(ctx, store.annotationTasks, item.ID, item); err != nil {
|
||||
if err := upsertAnnotationEpoch(ctx, store.annotationTasks, item.ID, item, item.AnnotationEpoch); err != nil {
|
||||
return err
|
||||
}
|
||||
if item.SourceIsFullVideo && item.TemplateVersion >= 5 && item.ArtifactID != "" {
|
||||
@ -863,7 +882,7 @@ func (store *MongoStore) UpsertAnnotationTask(ctx context.Context, item domain.A
|
||||
}
|
||||
|
||||
func (store *MongoStore) ListAnnotationRevisions(ctx context.Context, organizationID string, taskID string) ([]domain.AnnotationRevisionRecord, error) {
|
||||
extra := bson.M{}
|
||||
extra := bson.M{"supersededByAccountDeletion": bson.M{"$ne": true}}
|
||||
if taskID != "" {
|
||||
extra["taskId"] = taskID
|
||||
}
|
||||
@ -884,7 +903,7 @@ func (store *MongoStore) ListAnnotationRevisionsByTaskIDs(ctx context.Context, o
|
||||
for start := 0; start < len(taskIDs); start += mongoTaskIDBatchSize {
|
||||
end := min(start+mongoTaskIDBatchSize, len(taskIDs))
|
||||
batch, err := mongoList[domain.AnnotationRevisionRecord](ctx, store.annotationRevisions,
|
||||
mongoAnnotationFilter(organizationID, bson.M{"taskId": bson.M{"$in": taskIDs[start:end]}}),
|
||||
mongoAnnotationFilter(organizationID, bson.M{"taskId": bson.M{"$in": taskIDs[start:end]}, "supersededByAccountDeletion": bson.M{"$ne": true}}),
|
||||
bson.D{{Key: "taskId", Value: 1}, {Key: "revision", Value: 1}})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@ -902,7 +921,7 @@ func (store *MongoStore) UpsertAnnotationRevision(ctx context.Context, item doma
|
||||
if item.CreatedAt == "" {
|
||||
item.CreatedAt = nowISO()
|
||||
}
|
||||
return mongoUpsert(ctx, store.annotationRevisions, item.ID, item)
|
||||
return store.upsertActiveAnnotationRevision(ctx, store.annotationRevisions, item.ID, item.TaskID, item.Revision, item.AnnotationEpoch, item)
|
||||
}
|
||||
|
||||
func (store *MongoStore) DeleteAnnotationRevisions(ctx context.Context, organizationID string, taskID string) error {
|
||||
@ -915,7 +934,7 @@ func (store *MongoStore) DeleteAnnotationRevisions(ctx context.Context, organiza
|
||||
}
|
||||
|
||||
func (store *MongoStore) ListSliceRevisions(ctx context.Context, organizationID string, taskID string) ([]domain.SliceRevisionRecord, error) {
|
||||
filter := bson.M{"organizationId": normalizedOrganizationID(organizationID)}
|
||||
filter := bson.M{"organizationId": normalizedOrganizationID(organizationID), "supersededByAccountDeletion": bson.M{"$ne": true}}
|
||||
if taskID != "" {
|
||||
filter["taskId"] = taskID
|
||||
}
|
||||
@ -930,7 +949,7 @@ func (store *MongoStore) ListSliceRevisionsByTaskIDs(ctx context.Context, organi
|
||||
for start := 0; start < len(taskIDs); start += mongoTaskIDBatchSize {
|
||||
end := min(start+mongoTaskIDBatchSize, len(taskIDs))
|
||||
batch, err := mongoList[domain.SliceRevisionRecord](ctx, store.sliceRevisions,
|
||||
mongoAnnotationFilter(organizationID, bson.M{"taskId": bson.M{"$in": taskIDs[start:end]}}),
|
||||
mongoAnnotationFilter(organizationID, bson.M{"taskId": bson.M{"$in": taskIDs[start:end]}, "supersededByAccountDeletion": bson.M{"$ne": true}}),
|
||||
bson.D{{Key: "revision", Value: 1}})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@ -948,7 +967,7 @@ func (store *MongoStore) UpsertSliceRevision(ctx context.Context, item domain.Sl
|
||||
if item.CreatedAt == "" {
|
||||
item.CreatedAt = nowISO()
|
||||
}
|
||||
return mongoUpsert(ctx, store.sliceRevisions, item.ID, item)
|
||||
return store.upsertActiveAnnotationRevision(ctx, store.sliceRevisions, item.ID, item.TaskID, item.Revision, item.AnnotationEpoch, item)
|
||||
}
|
||||
|
||||
func (store *MongoStore) DeleteSliceRevisions(ctx context.Context, organizationID string, taskID string) error {
|
||||
|
||||
@ -32,7 +32,7 @@ func (s *MemoryStore) ListLatestSliceProgressByTaskIDs(_ context.Context, org st
|
||||
defer s.mu.Unlock()
|
||||
latest := map[string]domain.SliceRevisionRecord{}
|
||||
for _, r := range s.sliceRevisions {
|
||||
if annotationOrganizationMatches(r.OrganizationID, org) && allowed[r.TaskID] {
|
||||
if !r.SupersededByAccountDeletion && annotationOrganizationMatches(r.OrganizationID, org) && allowed[r.TaskID] {
|
||||
if old, ok := latest[r.TaskID]; !ok || r.Revision > old.Revision {
|
||||
latest[r.TaskID] = r
|
||||
}
|
||||
@ -51,7 +51,7 @@ func (s *MemoryStore) ListLatestAnnotationProgressByTaskIDs(_ context.Context, o
|
||||
defer s.mu.Unlock()
|
||||
latest := map[string]domain.AnnotationRevisionRecord{}
|
||||
for _, r := range s.annotationRevisions {
|
||||
if annotationOrganizationMatches(r.OrganizationID, org) && allowed[r.TaskID] && r.Revision > 0 && strings.TrimSpace(r.AuthorUserID) != "system" {
|
||||
if !r.SupersededByAccountDeletion && annotationOrganizationMatches(r.OrganizationID, org) && allowed[r.TaskID] && r.Revision > 0 && strings.TrimSpace(r.AuthorUserID) != "system" {
|
||||
if old, ok := latest[r.TaskID]; !ok || r.Revision > old.Revision {
|
||||
latest[r.TaskID] = r
|
||||
}
|
||||
@ -66,7 +66,7 @@ func (s *MemoryStore) ListLatestAnnotationProgressByTaskIDs(_ context.Context, o
|
||||
}
|
||||
|
||||
func latestProgressPipeline(org string, ids []string, annotation bool) mongo.Pipeline {
|
||||
filter := bson.M{"taskId": bson.M{"$in": ids}}
|
||||
filter := bson.M{"taskId": bson.M{"$in": ids}, "supersededByAccountDeletion": bson.M{"$ne": true}}
|
||||
projection := bson.M{"_id": 0, "taskId": 1, "revision": 1, "segments.start": 1, "segments.end": 1}
|
||||
if annotation {
|
||||
filter["revision"] = bson.M{"$gt": 0}
|
||||
|
||||
@ -244,7 +244,7 @@ func (s *MemoryStore) AnnotationWorkBufferSummary(ctx context.Context, org, user
|
||||
return out, err
|
||||
}
|
||||
func annotationBufferProjectionClone(item domain.HumanItemRecord) domain.HumanItemRecord {
|
||||
result := domain.HumanItemRecord{ID: item.ID, OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, WorkflowID: item.WorkflowID, RunID: item.RunID, BatchID: item.BatchID, PluginID: item.PluginID, LegacyTaskID: item.LegacyTaskID, AssignedUserID: item.AssignedUserID, CurrentStageID: item.CurrentStageID, Status: item.Status, Version: item.Version, ReviewRound: item.ReviewRound, ClaimedAt: item.ClaimedAt, PendingCombinedSubmissionID: item.PendingCombinedSubmissionID}
|
||||
result := domain.HumanItemRecord{InheritedRework: item.InheritedRework, ID: item.ID, OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, WorkflowID: item.WorkflowID, RunID: item.RunID, BatchID: item.BatchID, PluginID: item.PluginID, LegacyTaskID: item.LegacyTaskID, AssignedUserID: item.AssignedUserID, CurrentStageID: item.CurrentStageID, Status: item.Status, Version: item.Version, ReviewRound: item.ReviewRound, ClaimedAt: item.ClaimedAt, PendingCombinedSubmissionID: item.PendingCombinedSubmissionID}
|
||||
if item.ProcessingJob != nil {
|
||||
result.ProcessingJob = &domain.AnnotationProcessingJob{SourceFingerprint: item.ProcessingJob.SourceFingerprint, SourceIndex: item.ProcessingJob.SourceIndex, Status: item.ProcessingJob.Status, Kind: item.ProcessingJob.Kind, Error: item.ProcessingJob.Error}
|
||||
}
|
||||
|
||||
272
internal/store/deleted_annotator.go
Normal file
272
internal/store/deleted_annotator.go
Normal file
@ -0,0 +1,272 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"droboticflow/internal/domain"
|
||||
"fmt"
|
||||
"go.mongodb.org/mongo-driver/bson"
|
||||
"go.mongodb.org/mongo-driver/mongo"
|
||||
"go.mongodb.org/mongo-driver/mongo/options"
|
||||
"reflect"
|
||||
)
|
||||
|
||||
func (s *MemoryStore) ClearReworkRoutingPending(_ context.Context, id string, version int) (bool, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
item, ok := s.humanItems[id]
|
||||
if !ok || item.Version != version {
|
||||
return false, nil
|
||||
}
|
||||
item.ReworkRoutingPending = false
|
||||
s.humanItems[id] = item
|
||||
return true, nil
|
||||
}
|
||||
func (s *MongoStore) ClearReworkRoutingPending(ctx context.Context, id string, version int) (bool, error) {
|
||||
r, err := s.humanItems.UpdateOne(ctx, bson.M{"id": id, "version": version}, bson.M{"$set": bson.M{"reworkRoutingPending": false}})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return r.MatchedCount == 1, nil
|
||||
}
|
||||
|
||||
// A reset never reuses a generation number or deletes formal audit history.
|
||||
// Old jobs may finish, but cannot publish a task from an earlier editing epoch.
|
||||
func (s *MemoryStore) ResetDeletedAnnotatorTask(_ context.Context, task domain.AnnotationTaskRecord) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, exists := s.annotationTasks[task.ID]
|
||||
if exists {
|
||||
if current.AnnotationEpoch > task.AnnotationEpoch {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
task.AnnotationRevisionFloor = max(task.AnnotationRevisionFloor, current.AnnotationRevisionFloor, current.CurrentRevision)
|
||||
}
|
||||
for _, revision := range s.annotationRevisions {
|
||||
if revision.TaskID == task.ID {
|
||||
task.AnnotationRevisionFloor = max(task.AnnotationRevisionFloor, revision.Revision)
|
||||
}
|
||||
}
|
||||
for _, revision := range s.sliceRevisions {
|
||||
if revision.TaskID == task.ID {
|
||||
task.AnnotationRevisionFloor = max(task.AnnotationRevisionFloor, revision.Revision)
|
||||
}
|
||||
}
|
||||
if !exists || current.AnnotationEpoch < task.AnnotationEpoch {
|
||||
if err := s.upsertAnnotationTaskLocked(task); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
for id, revision := range s.annotationRevisions {
|
||||
if revision.TaskID == task.ID && revision.AnnotationEpoch < task.AnnotationEpoch {
|
||||
revision.SupersededByAccountDeletion = true
|
||||
s.annotationRevisions[id] = revision
|
||||
}
|
||||
}
|
||||
for id, revision := range s.sliceRevisions {
|
||||
if revision.TaskID == task.ID && revision.AnnotationEpoch < task.AnnotationEpoch {
|
||||
revision.SupersededByAccountDeletion = true
|
||||
s.sliceRevisions[id] = revision
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *MongoStore) ResetDeletedAnnotatorTask(ctx context.Context, task domain.AnnotationTaskRecord) error {
|
||||
_, err := s.withSupplierRecordWriteLock(ctx, task.ProjectID, "annotationTasks", task.ID, func(ctx context.Context) (any, error) {
|
||||
current, err := s.GetAnnotationTask(ctx, task.ID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if current != nil {
|
||||
if current.AnnotationEpoch > task.AnnotationEpoch {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
task.AnnotationRevisionFloor = max(task.AnnotationRevisionFloor, current.AnnotationRevisionFloor, current.CurrentRevision)
|
||||
}
|
||||
filter := bson.M{"organizationId": normalizedOrganizationID(task.OrganizationID), "taskId": task.ID}
|
||||
for _, col := range []*mongo.Collection{s.annotationRevisions, s.sliceRevisions} {
|
||||
var last struct {
|
||||
Revision int `bson:"revision"`
|
||||
}
|
||||
err := col.FindOne(ctx, filter, options.FindOne().SetSort(bson.D{{Key: "revision", Value: -1}}).SetProjection(bson.M{"revision": 1})).Decode(&last)
|
||||
if err != nil && err != mongo.ErrNoDocuments {
|
||||
return nil, err
|
||||
}
|
||||
task.AnnotationRevisionFloor = max(task.AnnotationRevisionFloor, last.Revision)
|
||||
}
|
||||
// A second recoverer may arrive after release and after the next
|
||||
// annotator has saved. Replaying this epoch must not rewrite their task.
|
||||
if current == nil || current.AnnotationEpoch < task.AnnotationEpoch {
|
||||
task.OrganizationID = normalizedOrganizationID(task.OrganizationID)
|
||||
task.UpdatedAt = nowISO()
|
||||
if err := upsertAnnotationEpoch(ctx, s.annotationTasks, task.ID, task, task.AnnotationEpoch); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
filter["$or"] = bson.A{bson.M{"annotationEpoch": bson.M{"$lt": task.AnnotationEpoch}}, bson.M{"annotationEpoch": bson.M{"$exists": false}}}
|
||||
for _, col := range []*mongo.Collection{s.annotationRevisions, s.sliceRevisions} {
|
||||
if _, err := col.UpdateMany(ctx, filter, bson.M{"$set": bson.M{"supersededByAccountDeletion": true}}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return nil, nil
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func upsertAnnotationEpoch(ctx context.Context, col *mongo.Collection, id string, value any, epoch int) error {
|
||||
return supplierGuardedMongoWrite(ctx, col, id, value, func(ctx context.Context, d bson.M) error {
|
||||
_, err := col.ReplaceOne(ctx, bson.M{"id": id, "$or": bson.A{bson.M{"annotationEpoch": bson.M{"$lte": epoch}}, bson.M{"annotationEpoch": bson.M{"$exists": false}}}}, d, options.Replace().SetUpsert(true))
|
||||
if mongo.IsDuplicateKeyError(err) {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
func (s *MongoStore) upsertActiveAnnotationRevision(ctx context.Context, col *mongo.Collection, id, taskID string, revision, epoch int, value any) error {
|
||||
return supplierGuardedMongoWrite(ctx, col, id, value, func(ctx context.Context, d bson.M) error {
|
||||
// This read and write hold the same task stripe as reset; a pre-lock
|
||||
// parent read alone would leave a check/write race.
|
||||
task, err := s.GetAnnotationTask(ctx, taskID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if task != nil && (epoch != task.AnnotationEpoch || (task.AnnotationEpoch > 0 && revision <= task.AnnotationRevisionFloor)) {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
_, err = col.ReplaceOne(ctx, bson.M{"id": id, "supersededByAccountDeletion": bson.M{"$ne": true}}, d, options.Replace().SetUpsert(true))
|
||||
if mongo.IsDuplicateKeyError(err) {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
// Start an inherited rework from the frozen reviewed version, never another
|
||||
// account's private, unfinished draft. The baseline and review records remain.
|
||||
func applyInheritedReworkWorkspace(item *domain.HumanItemRecord, baseline domain.ReworkBaselineRecord, user string) {
|
||||
item.Subject = clone(item.Subject)
|
||||
if item.Subject.Payload == nil {
|
||||
item.Subject.Payload = map[string]any{}
|
||||
}
|
||||
delete(item.Subject.Payload, "annotationDraft")
|
||||
statuses := map[string]domain.SliceAnnotationStatus{}
|
||||
for _, slice := range baseline.Slices {
|
||||
status := slice.AnnotationStatus
|
||||
if status == "" {
|
||||
status = "pending"
|
||||
}
|
||||
statuses[slice.ID] = status
|
||||
}
|
||||
item.Subject.Payload["annotationWorkspace"] = map[string]any{"sliceSegments": baseline.Slices, "segments": baseline.Actions, "operations": []domain.SliceOperation{}, "statuses": statuses, "userId": user, "savedAt": nowISO()}
|
||||
}
|
||||
|
||||
func inheritedBaselineFromRecords(item domain.HumanItemRecord, saved *domain.ReworkBaselineRecord, annotations []domain.AnnotationRevisionRecord, slices []domain.SliceRevisionRecord) (*domain.ReworkBaselineRecord, error) {
|
||||
if saved != nil {
|
||||
if saved.OrganizationID != item.OrganizationID || saved.ProjectID != item.ProjectID || saved.ItemID != item.ID || saved.Round != item.ReviewRound || len(saved.Slices) == 0 {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
return saved, nil
|
||||
}
|
||||
// Older reviewed tasks predate independent baselines. Recover only their
|
||||
// exact submitted revision, never a newer private workspace or draft.
|
||||
if item.CurrentRevision <= 0 {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
result := &domain.ReworkBaselineRecord{OrganizationID: item.OrganizationID, ProjectID: item.ProjectID, ItemID: item.ID, Round: item.ReviewRound}
|
||||
foundAnnotation, foundSlices := false, false
|
||||
for _, rev := range annotations {
|
||||
if !rev.SupersededByAccountDeletion && rev.OrganizationID == item.OrganizationID && rev.TaskID == item.LegacyTaskID && rev.Revision == item.CurrentRevision {
|
||||
result.Actions = rev.Segments
|
||||
foundAnnotation = true
|
||||
}
|
||||
}
|
||||
for _, rev := range slices {
|
||||
if !rev.SupersededByAccountDeletion && rev.OrganizationID == item.OrganizationID && rev.TaskID == item.LegacyTaskID && rev.Revision == item.CurrentRevision {
|
||||
result.Slices = rev.Segments
|
||||
foundSlices = true
|
||||
}
|
||||
}
|
||||
if !foundAnnotation || !foundSlices || len(result.Slices) == 0 {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func (s *MemoryStore) inheritedReworkBaselineLocked(item domain.HumanItemRecord) (*domain.ReworkBaselineRecord, error) {
|
||||
var saved *domain.ReworkBaselineRecord
|
||||
if v, ok := s.reworkBaselines[ReworkBaselineID(item.ID, item.ReviewRound)]; ok {
|
||||
saved = &v
|
||||
}
|
||||
annotations := []domain.AnnotationRevisionRecord{}
|
||||
slices := []domain.SliceRevisionRecord{}
|
||||
if saved == nil {
|
||||
for _, rev := range s.annotationRevisions {
|
||||
if rev.TaskID == item.LegacyTaskID {
|
||||
annotations = append(annotations, rev)
|
||||
}
|
||||
}
|
||||
for _, rev := range s.sliceRevisions {
|
||||
if rev.TaskID == item.LegacyTaskID {
|
||||
slices = append(slices, rev)
|
||||
}
|
||||
}
|
||||
}
|
||||
return inheritedBaselineFromRecords(item, saved, annotations, slices)
|
||||
}
|
||||
func (s *MemoryStore) LoadInheritedReworkBaseline(_ context.Context, item domain.HumanItemRecord) (*domain.ReworkBaselineRecord, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return s.inheritedReworkBaselineLocked(item)
|
||||
}
|
||||
func (s *MongoStore) LoadInheritedReworkBaseline(ctx context.Context, item domain.HumanItemRecord) (*domain.ReworkBaselineRecord, error) {
|
||||
saved, err := s.GetReworkBaseline(ctx, ReworkBaselineID(item.ID, item.ReviewRound))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if saved != nil {
|
||||
return inheritedBaselineFromRecords(item, saved, nil, nil)
|
||||
}
|
||||
annotations, err := s.ListAnnotationRevisions(ctx, item.OrganizationID, item.LegacyTaskID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
slices, err := s.ListSliceRevisions(ctx, item.OrganizationID, item.LegacyTaskID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return inheritedBaselineFromRecords(item, nil, annotations, slices)
|
||||
}
|
||||
|
||||
// Refuse the periodic routing scan if its bounded lookup indexes are absent.
|
||||
// Index creation is an explicitly authorized operation, never API startup work.
|
||||
func (s *MongoStore) validateDeletedAuthorRoutingIndexes(ctx context.Context) error {
|
||||
cur, err := s.humanItems.Indexes().List(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer cur.Close(ctx)
|
||||
wanted := map[string]bson.D{
|
||||
"deleted_author_routing_v1": {{Key: "organizationId", Value: int32(1)}, {Key: "currentStageId", Value: int32(1)}, {Key: "reworkRoutingPending", Value: int32(1)}, {Key: "status", Value: int32(1)}},
|
||||
"deleted_author_unassigned_v1": {{Key: "organizationId", Value: int32(1)}, {Key: "currentStageId", Value: int32(1)}, {Key: "inheritedRework", Value: int32(1)}, {Key: "assignedUserId", Value: int32(1)}, {Key: "status", Value: int32(1)}},
|
||||
}
|
||||
for cur.Next(ctx) {
|
||||
var index struct {
|
||||
Name string `bson:"name"`
|
||||
Key bson.D `bson:"key"`
|
||||
}
|
||||
if err := cur.Decode(&index); err != nil {
|
||||
return err
|
||||
}
|
||||
if key, ok := wanted[index.Name]; ok && reflect.DeepEqual(key, index.Key) {
|
||||
delete(wanted, index.Name)
|
||||
}
|
||||
}
|
||||
if err := cur.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(wanted) > 0 {
|
||||
return fmt.Errorf("deleted author routing indexes are not ready")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
253
internal/store/deleted_annotator_test.go
Normal file
253
internal/store/deleted_annotator_test.go
Normal file
@ -0,0 +1,253 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"droboticflow/internal/domain"
|
||||
"go.mongodb.org/mongo-driver/bson"
|
||||
"go.mongodb.org/mongo-driver/mongo"
|
||||
"go.mongodb.org/mongo-driver/mongo/options"
|
||||
)
|
||||
|
||||
func deletedAnnotatorStoreContract(t *testing.T, s Store) {
|
||||
t.Helper()
|
||||
ctx := t.Context()
|
||||
org, supplier, actor := "deletion-test-org", "deletion-test-supplier", "recipient"
|
||||
must := func(err error) {
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
must(s.UpsertOrganization(ctx, domain.OrganizationRecord{ID: org, Status: domain.OrganizationStatusActive}))
|
||||
must(s.UpsertSupplier(ctx, domain.SupplierRecord{ID: supplier, OrganizationID: org, Status: "active"}))
|
||||
must(s.UpsertUser(ctx, domain.UserRecord{ID: actor, OrganizationID: org, SupplierID: supplier, Role: domain.UserRoleAnnotator, Status: domain.UserStatusActive}))
|
||||
var commits []HumanClaimCommit
|
||||
for i := 0; i < 2; i++ {
|
||||
project := fmt.Sprintf("project-%d", i)
|
||||
taskID := fmt.Sprintf("task-%d", i)
|
||||
id := fmt.Sprintf("item-%d", i)
|
||||
must(s.UpsertProject(ctx, domain.ProjectRecord{ID: project, OrganizationID: org}))
|
||||
must(s.UpsertProjectMember(ctx, domain.ProjectMemberRecord{ID: "member-" + project, OrganizationID: org, ProjectID: project, UserID: actor, SupplierID: supplier, Roles: []domain.AnnotationAssignmentRole{domain.AnnotationRoleAnnotator}}))
|
||||
batch := domain.HumanBatchRecord{ID: "batch-" + project, OrganizationID: org, ProjectID: project, PluginID: "annotation-qa", Status: domain.HumanBatchActive, Stages: domain.DefaultFanyuV58HumanStages()}
|
||||
must(s.UpsertHumanBatch(ctx, batch))
|
||||
must(s.UpsertAnnotationTask(ctx, domain.AnnotationTaskRecord{ID: taskID, OrganizationID: org, ProjectID: project, SupplierID: supplier, Status: domain.AnnotationTaskStatusRework, CurrentRevision: 1}))
|
||||
item := domain.HumanItemRecord{ID: id, OrganizationID: org, ProjectID: project, SupplierID: supplier, BatchID: batch.ID, LegacyTaskID: taskID, PluginID: "annotation-qa", Status: domain.HumanItemRework, CurrentStageID: domain.FanyuV58StageAnnotationCombined, Version: 1, ReviewRound: 1, CurrentRevision: 1, InheritedRework: true, Subject: domain.HumanSubjectRef{ItemKey: id, Payload: map[string]any{"annotationDraft": map[string]any{"userId": "deleted", "marker": "old private draft"}}}}
|
||||
must(s.UpsertHumanItem(ctx, item))
|
||||
must(s.UpsertReworkBaseline(ctx, domain.ReworkBaselineRecord{ID: ReworkBaselineID(id, 1), OrganizationID: org, ProjectID: project, ItemID: id, Round: 1, Slices: []domain.SliceSegment{{ID: "s", Start: 0, End: 10}}, Actions: []domain.AnnotationSegment{{ID: "a", Start: 0, End: 10, Description: "正式内容"}}, Problems: []domain.ReviewProblem{{ID: "opinion", Text: "保留质检意见"}}}))
|
||||
item.AssignedUserID = actor
|
||||
item.Status = domain.HumanItemClaimed
|
||||
item.Version++
|
||||
commits = append(commits, HumanClaimCommit{Receipt: HumanClaimReceipt{ID: "claim-" + id, ItemID: id, UserID: actor, ProjectID: project, OrganizationID: org}, Item: item, ExpectedVersion: 1, TaskStatus: domain.AnnotationTaskStatusRework, Attempt: domain.HumanStageAttemptRecord{OrganizationID: org, ItemID: id, StageID: item.CurrentStageID, Round: 1, Status: domain.HumanItemClaimed, AssigneeUserID: actor}, Event: domain.HumanEventRecord{ID: "event-" + id, OrganizationID: org, ItemID: id, EventType: "test-handoff"}})
|
||||
}
|
||||
start := make(chan struct{})
|
||||
var wg sync.WaitGroup
|
||||
errs := make([]error, 2)
|
||||
for i := range commits {
|
||||
wg.Add(1)
|
||||
go func(i int) { defer wg.Done(); <-start; _, errs[i] = s.CommitHumanClaim(ctx, commits[i]) }(i)
|
||||
}
|
||||
close(start)
|
||||
wg.Wait()
|
||||
winner, loser := -1, -1
|
||||
for i, err := range errs {
|
||||
if err == nil {
|
||||
winner = i
|
||||
} else if errors.Is(err, ErrAnnotationBufferFull) {
|
||||
loser = i
|
||||
} else {
|
||||
t.Fatal("unexpected concurrent claim", err)
|
||||
}
|
||||
}
|
||||
if winner < 0 || loser < 0 {
|
||||
t.Fatalf("global cross-project inherited cap failed: %v", errs)
|
||||
}
|
||||
item, err := s.GetHumanItem(ctx, commits[winner].Item.ID)
|
||||
must(err)
|
||||
if item.Subject.Payload["annotationDraft"] != nil {
|
||||
t.Fatal("private old draft carried into claim")
|
||||
}
|
||||
if _, err := s.CommitHumanClaim(ctx, commits[winner]); err != nil {
|
||||
t.Fatal("restart/replay allocated or rejected completed assignment", err)
|
||||
}
|
||||
item.CurrentStageID = domain.FanyuV58StageQualityReview
|
||||
item.AssignedUserID = ""
|
||||
item.Version++
|
||||
must(s.UpsertHumanItem(ctx, *item))
|
||||
if _, err := s.CommitHumanClaim(ctx, commits[loser]); err != nil {
|
||||
t.Fatal("released inherited slot unavailable", err)
|
||||
}
|
||||
|
||||
// Race the publication itself, not merely the final HumanItem CAS. Either
|
||||
// the old write precedes reset and is superseded, or its old epoch is refused.
|
||||
for i := 0; i < 4; i++ {
|
||||
task := domain.AnnotationTaskRecord{ID: fmt.Sprintf("race-task-%d", i), OrganizationID: org, ProjectID: "project-0", SupplierID: supplier}
|
||||
must(s.UpsertAnnotationTask(ctx, task))
|
||||
reset := task
|
||||
reset.AnnotationEpoch = 1
|
||||
start := make(chan struct{})
|
||||
var wg sync.WaitGroup
|
||||
var writeErr, resetErr error
|
||||
wg.Add(2)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
<-start
|
||||
writeErr = s.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: "late-" + task.ID, OrganizationID: org, TaskID: task.ID, Revision: 99})
|
||||
}()
|
||||
go func() { defer wg.Done(); <-start; resetErr = s.ResetDeletedAnnotatorTask(ctx, reset) }()
|
||||
close(start)
|
||||
wg.Wait()
|
||||
must(resetErr)
|
||||
if writeErr != nil && !errors.Is(writeErr, ErrHumanClaimConflict) {
|
||||
t.Fatal(writeErr)
|
||||
}
|
||||
visible, err := s.ListAnnotationRevisions(ctx, org, task.ID)
|
||||
must(err)
|
||||
if len(visible) != 0 {
|
||||
t.Fatal("racing old write survived reset", visible)
|
||||
}
|
||||
current, err := s.GetAnnotationTask(ctx, task.ID)
|
||||
must(err)
|
||||
if writeErr == nil && current.AnnotationRevisionFloor < 99 {
|
||||
t.Fatal("reset reused a published revision number")
|
||||
}
|
||||
must(s.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: "new-" + task.ID, OrganizationID: org, TaskID: task.ID, AnnotationEpoch: 1, Revision: current.AnnotationRevisionFloor + 1}))
|
||||
}
|
||||
|
||||
old := domain.AnnotationTaskRecord{ID: "reset-task", OrganizationID: org, ProjectID: "project-0", SupplierID: supplier}
|
||||
must(s.UpsertAnnotationTask(ctx, old))
|
||||
must(s.UpsertSliceRevision(ctx, domain.SliceRevisionRecord{ID: "old-slice", OrganizationID: org, TaskID: old.ID, Revision: 3}))
|
||||
must(s.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: "old-annotation", OrganizationID: org, TaskID: old.ID, Revision: 3}))
|
||||
reset := old
|
||||
reset.AnnotationEpoch = 1
|
||||
reset.AnnotationRevisionFloor = 3
|
||||
must(s.ResetDeletedAnnotatorTask(ctx, reset))
|
||||
must(s.ResetDeletedAnnotatorTask(ctx, reset))
|
||||
if err := s.UpsertAnnotationTask(ctx, old); !errors.Is(err, ErrHumanClaimConflict) {
|
||||
t.Fatal("old task epoch admitted", err)
|
||||
}
|
||||
if err := s.UpsertSliceRevision(ctx, domain.SliceRevisionRecord{ID: "late-old", OrganizationID: org, TaskID: old.ID, Revision: 3}); !errors.Is(err, ErrHumanClaimConflict) {
|
||||
t.Fatal("late old slice admitted", err)
|
||||
}
|
||||
if err := s.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: "old-annotation", OrganizationID: org, TaskID: old.ID, Revision: 3}); !errors.Is(err, ErrHumanClaimConflict) {
|
||||
t.Fatal("old annotation resurrected", err)
|
||||
}
|
||||
if err := s.UpsertSliceRevision(ctx, domain.SliceRevisionRecord{ID: "late-higher-slice", OrganizationID: org, TaskID: old.ID, Revision: 99}); !errors.Is(err, ErrHumanClaimConflict) {
|
||||
t.Fatal("late higher revision from old epoch admitted", err)
|
||||
}
|
||||
if err := s.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: "late-higher-annotation", OrganizationID: org, TaskID: old.ID, Revision: 99}); !errors.Is(err, ErrHumanClaimConflict) {
|
||||
t.Fatal("late higher annotation from old epoch admitted", err)
|
||||
}
|
||||
slices, err := s.ListSliceRevisions(ctx, org, old.ID)
|
||||
must(err)
|
||||
annotations, err := s.ListAnnotationRevisions(ctx, org, old.ID)
|
||||
must(err)
|
||||
if len(slices) != 0 || len(annotations) != 0 {
|
||||
t.Fatal("reset revisions visible to next worker")
|
||||
}
|
||||
progressSlices, err := s.ListLatestSliceProgressByTaskIDs(ctx, org, []string{old.ID})
|
||||
must(err)
|
||||
progressActions, err := s.ListLatestAnnotationProgressByTaskIDs(ctx, org, []string{old.ID})
|
||||
must(err)
|
||||
if len(progressSlices) != 0 || len(progressActions) != 0 {
|
||||
t.Fatal("reset history leaked into progress summaries")
|
||||
}
|
||||
must(s.UpsertSliceRevision(ctx, domain.SliceRevisionRecord{ID: "new-slice", OrganizationID: org, TaskID: old.ID, AnnotationEpoch: 1, Revision: 4}))
|
||||
slices, err = s.ListSliceRevisions(ctx, org, old.ID)
|
||||
must(err)
|
||||
if len(slices) != 1 || slices[0].Revision != 4 {
|
||||
t.Fatal("new epoch cannot publish")
|
||||
}
|
||||
newTask := reset
|
||||
newTask.AssignedUserID = "next-annotator"
|
||||
newTask.CurrentRevision = 4
|
||||
must(s.UpsertAnnotationTask(ctx, newTask))
|
||||
// Simulate a slow second replica resuming the old reset after release.
|
||||
must(s.ResetDeletedAnnotatorTask(ctx, reset))
|
||||
afterReplay, err := s.GetAnnotationTask(ctx, old.ID)
|
||||
must(err)
|
||||
if afterReplay.AssignedUserID != newTask.AssignedUserID || afterReplay.CurrentRevision != 4 {
|
||||
t.Fatal("late recovery rewrote new task projection")
|
||||
}
|
||||
slices, err = s.ListSliceRevisions(ctx, org, old.ID)
|
||||
must(err)
|
||||
if len(slices) != 1 || slices[0].Revision != 4 {
|
||||
t.Fatal("late recovery superseded next annotator's revision")
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func TestDeletedAnnotatorStoreMemory(t *testing.T) {
|
||||
deletedAnnotatorStoreContract(t, NewMemoryStore())
|
||||
}
|
||||
|
||||
func TestDeletedAnnotatorStoreMongo(t *testing.T) {
|
||||
if os.Getenv("DELETED_ANNOTATOR_LOCAL_MONGO") != "1" {
|
||||
t.Skip("isolated local Mongo opt-in")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(t.Context(), 90*time.Second)
|
||||
defer cancel()
|
||||
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://127.0.0.1:27018/?directConnection=true"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Disconnect(context.Background())
|
||||
db := client.Database(fmt.Sprintf("deleted_annotator_test_%d", time.Now().UnixNano()))
|
||||
defer db.Drop(context.Background())
|
||||
s := NewMongoStore(db)
|
||||
for _, name := range []string{"organizations", "projects", "users", "suppliers", "projectMembers", "annotationTasks", "annotationRevisions", "sliceRevisions", "humanItems", "humanBatches", "humanEvents", "humanStageAttempts", "humanClaimRequests"} {
|
||||
if _, err := db.Collection(name).Indexes().CreateOne(ctx, uniqueIndex("id_unique", bson.D{{Key: "id", Value: 1}})); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
t.Setenv("DROBOTICFLOW_PROJECT_WRITE_LOCK_MODE", "striped")
|
||||
deletedAnnotatorStoreContract(t, s)
|
||||
q := domain.HumanItemQuery{NeedsReworkRouting: true}
|
||||
if _, err := s.ListHumanItems(ctx, "deletion-test-org", q); err == nil {
|
||||
t.Fatal("background routing scanned without its indexes")
|
||||
}
|
||||
for _, index := range []mongo.IndexModel{
|
||||
{Keys: bson.D{{Key: "organizationId", Value: 1}, {Key: "currentStageId", Value: 1}, {Key: "reworkRoutingPending", Value: 1}, {Key: "status", Value: 1}}, Options: options.Index().SetName("deleted_author_routing_v1")},
|
||||
{Keys: bson.D{{Key: "organizationId", Value: 1}, {Key: "currentStageId", Value: 1}, {Key: "inheritedRework", Value: 1}, {Key: "assignedUserId", Value: 1}, {Key: "status", Value: 1}}, Options: options.Index().SetName("deleted_author_unassigned_v1")},
|
||||
} {
|
||||
if _, err := s.humanItems.Indexes().CreateOne(ctx, index); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
noise := make([]any, 1000)
|
||||
for i := range noise {
|
||||
noise[i] = bson.M{"id": fmt.Sprintf("noise-%d", i), "organizationId": "deletion-test-org", "currentStageId": domain.FanyuV58StageAnnotationCombined, "status": domain.HumanItemClaimed, "assignedUserId": "busy-worker", "inheritedRework": false, "reworkRoutingPending": false}
|
||||
}
|
||||
if _, err := s.humanItems.InsertMany(ctx, noise); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.humanItems.InsertOne(ctx, bson.M{"id": "routing-pending", "organizationId": "deletion-test-org", "currentStageId": domain.FanyuV58StageAnnotationCombined, "status": domain.HumanItemRework, "reworkRoutingPending": true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rows, err := s.ListHumanItems(ctx, "deletion-test-org", q)
|
||||
if err != nil || len(rows) != 1 {
|
||||
t.Fatal("indexed routing inventory changed", err)
|
||||
}
|
||||
var explain struct {
|
||||
ExecutionStats struct {
|
||||
NReturned int `bson:"nReturned"`
|
||||
Documents int `bson:"totalDocsExamined"`
|
||||
Keys int `bson:"totalKeysExamined"`
|
||||
} `bson:"executionStats"`
|
||||
}
|
||||
if err := db.RunCommand(ctx, bson.D{{Key: "explain", Value: bson.D{{Key: "find", Value: "humanItems"}, {Key: "filter", Value: humanItemMongoFilter("deletion-test-org", q)}, {Key: "sort", Value: bson.D{{Key: "createdAt", Value: -1}, {Key: "id", Value: -1}}}}}, {Key: "verbosity", Value: "executionStats"}}).Decode(&explain); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if explain.ExecutionStats.NReturned != 1 || explain.ExecutionStats.Documents > 4 || explain.ExecutionStats.Keys > 20 {
|
||||
t.Fatalf("routing lookup scanned unrelated annotation inventory: %+v", explain.ExecutionStats)
|
||||
}
|
||||
if n, err := s.annotationRevisions.CountDocuments(ctx, bson.M{"id": "old-annotation", "supersededByAccountDeletion": true}); err != nil || n != 1 {
|
||||
t.Fatal("logical reset deleted retained evidence", err)
|
||||
}
|
||||
}
|
||||
@ -227,19 +227,48 @@ func clone[T any](value T) T {
|
||||
}
|
||||
// The durable processing outbox is intentionally absent from public JSON.
|
||||
// Preserve it when cloning private in-memory records for store operations.
|
||||
if item, ok := any(value).(domain.HumanItemRecord); ok && item.ProcessingJob != nil {
|
||||
if item, ok := any(value).(domain.HumanItemRecord); ok {
|
||||
copied := any(out).(domain.HumanItemRecord)
|
||||
job := *item.ProcessingJob
|
||||
job.Input = clone(job.Input)
|
||||
copied.ProcessingJob = &job
|
||||
copied.ReworkRoutingPending = item.ReworkRoutingPending
|
||||
copied.AnnotationEpoch = item.AnnotationEpoch
|
||||
if item.AccountHandoff != nil {
|
||||
handoff := *item.AccountHandoff
|
||||
copied.AccountHandoff = &handoff
|
||||
}
|
||||
if item.ProcessingJob != nil {
|
||||
job := *item.ProcessingJob
|
||||
job.Input = clone(job.Input)
|
||||
copied.ProcessingJob = &job
|
||||
}
|
||||
return any(copied).(T)
|
||||
}
|
||||
if item, ok := any(value).(domain.SliceRevisionRecord); ok {
|
||||
copied := any(out).(domain.SliceRevisionRecord)
|
||||
copied.AnnotationEpoch = item.AnnotationEpoch
|
||||
copied.SupersededByAccountDeletion = item.SupersededByAccountDeletion
|
||||
out = any(copied).(T)
|
||||
}
|
||||
if item, ok := any(value).(domain.AnnotationRevisionRecord); ok {
|
||||
copied := any(out).(domain.AnnotationRevisionRecord)
|
||||
copied.AnnotationEpoch = item.AnnotationEpoch
|
||||
copied.SupersededByAccountDeletion = item.SupersededByAccountDeletion
|
||||
out = any(copied).(T)
|
||||
}
|
||||
if item, ok := any(value).(domain.AnnotationTaskRecord); ok {
|
||||
copied := any(out).(domain.AnnotationTaskRecord)
|
||||
copied.AnnotationEpoch = item.AnnotationEpoch
|
||||
copied.AnnotationRevisionFloor = item.AnnotationRevisionFloor
|
||||
return any(copied).(T)
|
||||
}
|
||||
|
||||
// LegacyEmail is deliberately hidden from JSON responses, but it must
|
||||
// survive in-memory store clones so the pre-phone login compatibility path
|
||||
// can exercise the same data shape as Mongo.
|
||||
if item, ok := any(value).(domain.UserRecord); ok {
|
||||
copied := any(out).(domain.UserRecord)
|
||||
copied.LegacyEmail = item.LegacyEmail
|
||||
copied.DeletionRequestedAt = item.DeletionRequestedAt
|
||||
copied.DeletionActorID = item.DeletionActorID
|
||||
return any(copied).(T)
|
||||
}
|
||||
return out
|
||||
|
||||
@ -38,7 +38,7 @@ type HumanClaimCommit struct {
|
||||
}
|
||||
|
||||
func annotationClaimCandidate(item domain.HumanItemRecord) bool {
|
||||
return item.PluginID == "annotation-qa" && item.LegacyTaskID != "" && item.AssignedUserID == "" &&
|
||||
return !item.ReworkRoutingPending && (item.AccountHandoff == nil || !item.AccountHandoff.Pending) && item.PluginID == "annotation-qa" && item.LegacyTaskID != "" && item.AssignedUserID == "" &&
|
||||
(item.Status == domain.HumanItemReady || item.Status == domain.HumanItemRework || item.Status == domain.HumanItemSubmitted)
|
||||
}
|
||||
|
||||
@ -47,7 +47,7 @@ func (s *MemoryStore) ListAnnotationClaimCandidates(_ context.Context, org, proj
|
||||
defer s.mu.Unlock()
|
||||
out := []domain.HumanItemRecord{}
|
||||
for _, item := range s.humanItems {
|
||||
if humanOrganizationMatches(item.OrganizationID, org) && item.ProjectID == project && annotationClaimCandidate(item) && s.annotationSourceReadyLocked(item) {
|
||||
if !item.InheritedRework && humanOrganizationMatches(item.OrganizationID, org) && item.ProjectID == project && annotationClaimCandidate(item) && s.annotationSourceReadyLocked(item) {
|
||||
item.Subject = domain.HumanSubjectRef{}
|
||||
out = append(out, clone(item))
|
||||
}
|
||||
@ -55,7 +55,7 @@ func (s *MemoryStore) ListAnnotationClaimCandidates(_ context.Context, org, proj
|
||||
return out, nil
|
||||
}
|
||||
func (s *MongoStore) ListAnnotationClaimCandidates(ctx context.Context, org, project string) ([]domain.HumanItemRecord, error) {
|
||||
filter := bson.M{"organizationId": org, "projectId": project, "pluginId": "annotation-qa", "legacyTaskId": bson.M{"$exists": true, "$ne": ""}, "assignedUserId": bson.M{"$in": bson.A{"", nil}}, "status": bson.M{"$in": bson.A{domain.HumanItemReady, domain.HumanItemRework, domain.HumanItemSubmitted}}}
|
||||
filter := bson.M{"reworkRoutingPending": bson.M{"$ne": true}, "inheritedRework": bson.M{"$ne": true}, "organizationId": org, "projectId": project, "pluginId": "annotation-qa", "legacyTaskId": bson.M{"$exists": true, "$ne": ""}, "assignedUserId": bson.M{"$in": bson.A{"", nil}}, "status": bson.M{"$in": bson.A{domain.HumanItemReady, domain.HumanItemRework, domain.HumanItemSubmitted}}}
|
||||
return s.listSourceReadyClaimCandidates(ctx, filter)
|
||||
}
|
||||
func (s *MemoryStore) GetHumanClaimReceipt(_ context.Context, key string) (*HumanClaimReceipt, error) {
|
||||
@ -77,7 +77,7 @@ func (s *MongoStore) GetHumanClaimReceipt(ctx context.Context, key string) (*Hum
|
||||
}
|
||||
|
||||
func validClaimSnapshot(old domain.HumanItemRecord, c HumanClaimCommit) bool {
|
||||
return annotationClaimCandidate(old) && old.Version == c.ExpectedVersion && c.Item.Version == old.Version+1 && (old.SupplierID == "" || old.SupplierID == c.Item.SupplierID) && old.ProjectID == c.Receipt.ProjectID && old.OrganizationID == c.Receipt.OrganizationID && old.CurrentStageID == c.Item.CurrentStageID && old.BatchID == c.Item.BatchID && old.LegacyTaskID == c.Item.LegacyTaskID && c.Item.AssignedUserID == c.Receipt.UserID
|
||||
return (old.AccountHandoff == nil || !old.AccountHandoff.Pending) && old.InheritedRework == c.Item.InheritedRework && old.AnnotationEpoch == c.Item.AnnotationEpoch && annotationClaimCandidate(old) && old.Version == c.ExpectedVersion && c.Item.Version == old.Version+1 && (old.SupplierID == "" || old.SupplierID == c.Item.SupplierID) && old.ProjectID == c.Receipt.ProjectID && old.OrganizationID == c.Receipt.OrganizationID && old.CurrentStageID == c.Item.CurrentStageID && old.BatchID == c.Item.BatchID && old.LegacyTaskID == c.Item.LegacyTaskID && c.Item.AssignedUserID == c.Receipt.UserID
|
||||
}
|
||||
func claimTaskFields(task *domain.AnnotationTaskRecord, c HumanClaimCommit) {
|
||||
task.AssignedUserID = c.Item.AssignedUserID
|
||||
@ -113,6 +113,13 @@ func (s *MemoryStore) CommitHumanClaim(_ context.Context, c HumanClaimCommit) (*
|
||||
v = clone(v)
|
||||
return &v, nil
|
||||
}
|
||||
if c.Item.InheritedRework {
|
||||
for _, owned := range s.humanItems {
|
||||
if owned.ID != c.Item.ID && owned.InheritedRework && owned.AssignedUserID == c.Receipt.UserID && owned.OrganizationID == c.Item.OrganizationID && pendingAccountItem(owned.Status) && owned.CurrentStageID == domain.FanyuV58StageAnnotationCombined {
|
||||
return nil, ErrAnnotationBufferFull
|
||||
}
|
||||
}
|
||||
}
|
||||
if c.EnforceBuffer {
|
||||
capacity := s.annotationBufferCapacities[bufferKey(c.Receipt.OrganizationID, c.Receipt.UserID, c.Receipt.ProjectID)]
|
||||
if capacity == 0 {
|
||||
@ -162,6 +169,13 @@ func (s *MemoryStore) CommitHumanClaim(_ context.Context, c HumanClaimCommit) (*
|
||||
if !claimActorAllowed(&user, &supplier, members, grants, authorizationCommit) {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
if c.Item.InheritedRework {
|
||||
baseline, err := s.inheritedReworkBaselineLocked(c.Item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
applyInheritedReworkWorkspace(&c.Item, *baseline, c.Receipt.UserID)
|
||||
}
|
||||
if c.BindSupplier {
|
||||
batch.SupplierID = c.Item.SupplierID
|
||||
batch.UpdatedAt = c.Item.UpdatedAt
|
||||
@ -192,6 +206,17 @@ func (s *MongoStore) CommitHumanClaim(ctx context.Context, c HumanClaimCommit) (
|
||||
}
|
||||
|
||||
func (s *MongoStore) commitHumanClaimWithAccountLock(ctx context.Context, c HumanClaimCommit) (*domain.HumanItemRecord, error) {
|
||||
if c.Item.InheritedRework {
|
||||
owned, err := s.ListHumanItems(ctx, c.Item.OrganizationID, domain.HumanItemQuery{ActorUserID: c.Receipt.UserID})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, item := range owned {
|
||||
if item.ID != c.Item.ID && item.InheritedRework && item.AssignedUserID == c.Receipt.UserID && pendingAccountItem(item.Status) && item.CurrentStageID == domain.FanyuV58StageAnnotationCombined {
|
||||
return nil, ErrAnnotationBufferFull
|
||||
}
|
||||
}
|
||||
}
|
||||
pending, err := s.supplierPoolConversionPending(ctx, c.Item.BatchID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@ -287,6 +312,16 @@ func (s *MongoStore) commitHumanClaimTransaction(ctx context.Context, c HumanCla
|
||||
if _, e = s.users.UpdateOne(sc, bson.M{"id": user.ID}, bson.M{"$inc": bson.M{"claimVersion": 1}}); e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if c.Item.InheritedRework {
|
||||
baseline, e := s.LoadInheritedReworkBaseline(sc, c.Item)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if baseline == nil {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
applyInheritedReworkWorkspace(&c.Item, *baseline, c.Receipt.UserID)
|
||||
}
|
||||
updated, e := s.UpdateHumanItemVersioned(sc, c.Item, c.ExpectedVersion)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
@ -354,7 +389,7 @@ type AnnotationClaimCountQuery struct {
|
||||
}
|
||||
|
||||
func annotationClaimCountMatches(item domain.HumanItemRecord, batch domain.HumanBatchRecord, org string, q AnnotationClaimCountQuery) bool {
|
||||
if !humanOrganizationMatches(item.OrganizationID, org) || item.ProjectID != q.ProjectID || !annotationClaimCandidate(item) || !annotationClaimCountStageMatches(batch, org, q.ProjectID, item.CurrentStageID) {
|
||||
if item.InheritedRework || !humanOrganizationMatches(item.OrganizationID, org) || item.ProjectID != q.ProjectID || !annotationClaimCandidate(item) || !annotationClaimCountStageMatches(batch, org, q.ProjectID, item.CurrentStageID) {
|
||||
return false
|
||||
}
|
||||
if (item.SupplierID != "" && !containsHumanItemID(q.ActiveSupplierIDs, item.SupplierID)) || (item.ProcessingScope == "internal" && item.SupplierID == "") {
|
||||
@ -530,7 +565,7 @@ func annotationClaimCountCandidateFilter(org string, q AnnotationClaimCountQuery
|
||||
for _, id := range q.ActiveSupplierIDs {
|
||||
suppliers = append(suppliers, id)
|
||||
}
|
||||
return bson.M{"organizationId": normalizedOrganizationID(org), "projectId": q.ProjectID, "pluginId": "annotation-qa", "legacyTaskId": bson.M{"$type": "string", "$ne": ""}, "assignedUserId": bson.M{"$in": bson.A{"", nil}}, "status": bson.M{"$in": bson.A{domain.HumanItemReady, domain.HumanItemRework, domain.HumanItemSubmitted}}, "supplierId": bson.M{"$in": suppliers}, "$nor": bson.A{bson.M{"processingScope": "internal", "supplierId": bson.M{"$in": bson.A{"", nil}}}}}
|
||||
return bson.M{"reworkRoutingPending": bson.M{"$ne": true}, "inheritedRework": bson.M{"$ne": true}, "organizationId": normalizedOrganizationID(org), "projectId": q.ProjectID, "pluginId": "annotation-qa", "legacyTaskId": bson.M{"$type": "string", "$ne": ""}, "assignedUserId": bson.M{"$in": bson.A{"", nil}}, "status": bson.M{"$in": bson.A{domain.HumanItemReady, domain.HumanItemRework, domain.HumanItemSubmitted}}, "supplierId": bson.M{"$in": suppliers}, "$nor": bson.A{bson.M{"processingScope": "internal", "supplierId": bson.M{"$in": bson.A{"", nil}}}}}
|
||||
}
|
||||
|
||||
func (s *MongoStore) CountAnnotationClaimCandidates(ctx context.Context, org string, q AnnotationClaimCountQuery) (int, error) {
|
||||
|
||||
@ -131,6 +131,16 @@ func (s *MongoStore) commitHumanClaimStandalone(ctx context.Context, proposed Hu
|
||||
}
|
||||
claimed := c.Item
|
||||
claimed.Subject, claimed.ClaimRequestID = item.Subject, r.ID
|
||||
if claimed.InheritedRework {
|
||||
baseline, e := s.LoadInheritedReworkBaseline(ctx, claimed)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if baseline == nil {
|
||||
return nil, ErrHumanClaimConflict
|
||||
}
|
||||
applyInheritedReworkWorkspace(&claimed, *baseline, r.UserID)
|
||||
}
|
||||
updated, e := s.UpdateHumanItemVersioned(ctx, claimed, c.ExpectedVersion)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
|
||||
@ -45,7 +45,7 @@ func matchesHumanMinePool(item domain.HumanItemRecord, query domain.HumanItemQue
|
||||
}
|
||||
|
||||
func humanItemMatches(item domain.HumanItemRecord, query domain.HumanItemQuery) bool {
|
||||
return (query.AvailableSupplierID == nil || item.SupplierID == "" || item.SupplierID == *query.AvailableSupplierID) && (query.AvailableProjectIDs == nil || containsHumanItemID(query.AvailableProjectIDs, item.ProjectID)) && matchesHumanMinePool(item, query) && matchesAnnotationView(item, query) && (query.IDs == nil || containsHumanItemID(query.IDs, item.ID)) && (query.LegacyTaskID == "" || item.LegacyTaskID == query.LegacyTaskID) &&
|
||||
return (!query.NeedsReworkRouting || (item.CurrentStageID == domain.FanyuV58StageAnnotationCombined && (item.Status == domain.HumanItemRework || item.Status == domain.HumanItemClaimed || item.Status == domain.HumanItemInProgress) && (item.ReworkRoutingPending || (item.InheritedRework && item.AssignedUserID == "")))) && (!query.InheritedUnassigned || (item.InheritedRework && item.AssignedUserID == "" && item.CurrentStageID == domain.FanyuV58StageAnnotationCombined && item.Status == domain.HumanItemRework)) && (query.AvailableSupplierID == nil || item.SupplierID == "" || item.SupplierID == *query.AvailableSupplierID) && (query.AvailableProjectIDs == nil || containsHumanItemID(query.AvailableProjectIDs, item.ProjectID)) && matchesHumanMinePool(item, query) && matchesAnnotationView(item, query) && (query.IDs == nil || containsHumanItemID(query.IDs, item.ID)) && (query.LegacyTaskID == "" || item.LegacyTaskID == query.LegacyTaskID) &&
|
||||
(query.ActorUserID == "" || item.AssignedUserID == query.ActorUserID || item.LatestAuthorUserID == query.ActorUserID) &&
|
||||
(query.PluginID == "" || item.PluginID == query.PluginID) &&
|
||||
(query.BatchID == "" || item.BatchID == query.BatchID) &&
|
||||
@ -658,6 +658,17 @@ func humanItemMongoFilter(organizationID string, query domain.HumanItemQuery) bs
|
||||
if query.LegacyTaskID != "" {
|
||||
filter["legacyTaskId"] = query.LegacyTaskID
|
||||
}
|
||||
if query.NeedsReworkRouting {
|
||||
filter["status"] = bson.M{"$in": bson.A{domain.HumanItemRework, domain.HumanItemClaimed, domain.HumanItemInProgress}}
|
||||
filter["currentStageId"] = domain.FanyuV58StageAnnotationCombined
|
||||
filter["$or"] = bson.A{bson.M{"reworkRoutingPending": true}, bson.M{"inheritedRework": true, "assignedUserId": ""}}
|
||||
}
|
||||
if query.InheritedUnassigned {
|
||||
filter["inheritedRework"] = true
|
||||
filter["assignedUserId"] = ""
|
||||
filter["currentStageId"] = domain.FanyuV58StageAnnotationCombined
|
||||
filter["status"] = domain.HumanItemRework
|
||||
}
|
||||
if query.ActorUserID != "" && query.AnnotationView == "" {
|
||||
filter["$or"] = bson.A{bson.M{"assignedUserId": query.ActorUserID}, bson.M{"latestAuthorUserId": query.ActorUserID}}
|
||||
if query.MinePoolOnly {
|
||||
@ -745,7 +756,7 @@ func humanItemListProjection(query domain.HumanItemQuery) bson.M {
|
||||
return bson.M{"_id": 0, "id": 1}
|
||||
}
|
||||
if query.BufferSummaryProjection {
|
||||
return bson.M{"_id": 0, "id": 1, "organizationId": 1, "projectId": 1, "workflowId": 1, "runId": 1, "batchId": 1, "pluginId": 1, "legacyTaskId": 1, "assignedUserId": 1, "currentStageId": 1, "status": 1, "version": 1, "reviewRound": 1, "claimedAt": 1, "pendingCombinedSubmissionId": 1, "processingJob.sourceFingerprint": 1, "processingJob.sourceIndex": 1, "subject.payload.materialization.sourceIndex": 1, "processingJob.status": 1, "processingJob.kind": 1, "processingJob.error": 1, "subject.payload.sliceWorkflow.status": 1, "subject.payload.sliceWorkflow.phase": 1}
|
||||
return bson.M{"_id": 0, "inheritedRework": 1, "id": 1, "organizationId": 1, "projectId": 1, "workflowId": 1, "runId": 1, "batchId": 1, "pluginId": 1, "legacyTaskId": 1, "assignedUserId": 1, "currentStageId": 1, "status": 1, "version": 1, "reviewRound": 1, "claimedAt": 1, "pendingCombinedSubmissionId": 1, "processingJob.sourceFingerprint": 1, "processingJob.sourceIndex": 1, "subject.payload.materialization.sourceIndex": 1, "processingJob.status": 1, "processingJob.kind": 1, "processingJob.error": 1, "subject.payload.sliceWorkflow.status": 1, "subject.payload.sliceWorkflow.phase": 1}
|
||||
}
|
||||
projection := bson.M{"processingJob.input": 0, "poolSearch.text": 0, "poolSearch.source": 0}
|
||||
for _, key := range humanItemPayloadKeysRemoved(query) {
|
||||
@ -850,6 +861,11 @@ func (store *MongoStore) UpsertHumanBatch(ctx context.Context, item domain.Human
|
||||
}
|
||||
|
||||
func (store *MongoStore) ListHumanItems(ctx context.Context, organizationID string, query domain.HumanItemQuery) ([]domain.HumanItemRecord, error) {
|
||||
if query.NeedsReworkRouting {
|
||||
if err := store.validateDeletedAuthorRoutingIndexes(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if humanPoolFiltered(query) {
|
||||
return store.listHumanPoolSearch(ctx, organizationID, query)
|
||||
}
|
||||
|
||||
@ -29,7 +29,7 @@ func (s *MongoStore) VisitHumanPoolCountCandidates(ctx context.Context, org stri
|
||||
filter = bson.M{"$and": bson.A{filter, visibility}}
|
||||
}
|
||||
projection := bson.M{"_id": 0}
|
||||
for _, field := range []string{"id", "organizationId", "pluginId", "projectId", "batchId", "legacyTaskId", "supplierId", "processingScope", "processingProfile", "workgroupId", "vendorId", "currentStageId", "status", "assignedUserId", "latestAuthorUserId", "acceptanceClaim", "reviewRound", "claimedAt"} {
|
||||
for _, field := range []string{"inheritedRework", "id", "organizationId", "pluginId", "projectId", "batchId", "legacyTaskId", "supplierId", "processingScope", "processingProfile", "workgroupId", "vendorId", "currentStageId", "status", "assignedUserId", "latestAuthorUserId", "acceptanceClaim", "reviewRound", "claimedAt"} {
|
||||
projection[field] = 1
|
||||
}
|
||||
cur, err := s.humanItems.Find(ctx, filter, options.Find().SetProjection(projection).SetBatchSize(humanPoolCountBatchSize).SetMaxTime(8*time.Second))
|
||||
|
||||
@ -2380,6 +2380,9 @@ func (store *MemoryStore) upsertUser(_ context.Context, item domain.UserRecord,
|
||||
if supplier, ok := store.suppliers[item.SupplierID]; ok && (supplier.Status == "deleting" || supplier.Status == "deleted") {
|
||||
return ErrSupplierDeleting
|
||||
}
|
||||
if current, ok := store.users[item.ID]; ok && current.DeletionRequestedAt != "" && (item.DeletionRequestedAt != current.DeletionRequestedAt || item.Status != domain.UserStatusDisabled) {
|
||||
return ErrAccountDeleted
|
||||
}
|
||||
if current, ok := store.users[item.ID]; ok && current.DeletedAt != "" {
|
||||
return ErrAccountDeleted
|
||||
}
|
||||
|
||||
@ -2363,6 +2363,9 @@ func (store *MongoStore) upsertUser(ctx context.Context, item domain.UserRecord,
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if existing != nil && existing.DeletionRequestedAt != "" && (item.DeletionRequestedAt != existing.DeletionRequestedAt || item.Status != domain.UserStatusDisabled) {
|
||||
return ErrAccountDeleted
|
||||
}
|
||||
if existing != nil && existing.DeletedAt != "" {
|
||||
return ErrAccountDeleted
|
||||
}
|
||||
|
||||
@ -324,6 +324,9 @@ type Store interface {
|
||||
AnnotationWorkBuffer(context.Context, string, string, string, int) (AnnotationWorkBuffer, error)
|
||||
EnrichAnnotationSourceStatuses(context.Context, []domain.HumanItemRecord) error
|
||||
AnnotationWorkBufferSummary(context.Context, string, string, string, int) (AnnotationWorkBuffer, error)
|
||||
ResetDeletedAnnotatorTask(context.Context, domain.AnnotationTaskRecord) error
|
||||
ClearReworkRoutingPending(context.Context, string, int) (bool, error)
|
||||
LoadInheritedReworkBaseline(context.Context, domain.HumanItemRecord) (*domain.ReworkBaselineRecord, error)
|
||||
CommitHumanClaim(context.Context, HumanClaimCommit) (*domain.HumanItemRecord, error)
|
||||
GetHumanItem(context.Context, string) (*domain.HumanItemRecord, error)
|
||||
UpsertHumanItem(context.Context, domain.HumanItemRecord) error
|
||||
|
||||
@ -193,7 +193,15 @@ func supplierGuardedMongoWrite(ctx context.Context, col *mongo.Collection, id st
|
||||
}
|
||||
}
|
||||
}
|
||||
_, e := s.withSupplierRecordWriteLock(ctx, pid, col.Name(), id, func(ctx context.Context) (any, error) {
|
||||
// Revisions and their task reset share one stripe. A checked old job
|
||||
// cannot publish between the reset fence and revision supersession.
|
||||
lockCollection, lockID := col.Name(), id
|
||||
if col.Name() == "sliceRevisions" || col.Name() == "annotationRevisions" {
|
||||
if taskID := deletionString(d, "taskId"); taskID != "" {
|
||||
lockCollection, lockID = "annotationTasks", taskID
|
||||
}
|
||||
}
|
||||
_, e := s.withSupplierRecordWriteLock(ctx, pid, lockCollection, lockID, func(ctx context.Context) (any, error) {
|
||||
jobs, e := s.supplierRecordJobs(ctx, col.Name(), id, d)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
|
||||
@ -837,11 +837,11 @@ func (runtime *Runtime) createAnnotationGateTasks(
|
||||
confidence, _ := annotationNumber(window["confidence"])
|
||||
segments = append(segments, domain.AnnotationSegment{ID: "prelabel-" + windowID, LabelID: labelID, Label: label, Start: start, End: end, Confidence: confidence, Source: "qwen"})
|
||||
}
|
||||
if err := runtime.store.UpsertSliceRevision(ctx, domain.SliceRevisionRecord{ID: domain.NewID("slice-revision"), OrganizationID: run.OrganizationID, TaskID: task.ID, Revision: 0, Segments: slices, AuthorUserID: "system", ChangeReason: sliceReason, CreatedAt: nowISO()}); err != nil {
|
||||
if err := runtime.store.UpsertSliceRevision(ctx, domain.SliceRevisionRecord{ID: domain.NewID("slice-revision"), OrganizationID: run.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch, Revision: 0, Segments: slices, AuthorUserID: "system", ChangeReason: sliceReason, CreatedAt: nowISO()}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(segments) > 0 {
|
||||
if err := runtime.store.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: domain.NewID("annotation-revision"), OrganizationID: run.OrganizationID, TaskID: task.ID, Revision: 0, TemplateVersion: template.Version, Segments: segments, AuthorUserID: "system", ChangeReason: "Qwen timestamp-window suggestions", CountsAsOutput: false, CreatedAt: nowISO()}); err != nil {
|
||||
if err := runtime.store.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: domain.NewID("annotation-revision"), OrganizationID: run.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch, Revision: 0, TemplateVersion: template.Version, Segments: segments, AuthorUserID: "system", ChangeReason: "Qwen timestamp-window suggestions", CountsAsOutput: false, CreatedAt: nowISO()}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
@ -908,7 +908,7 @@ func (runtime *Runtime) createAnnotationGateTasks(
|
||||
segments := prelabels[clipID]
|
||||
if len(segments) > 0 {
|
||||
if err := runtime.store.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{
|
||||
ID: domain.NewID("annotation-revision"), OrganizationID: run.OrganizationID, TaskID: task.ID,
|
||||
ID: domain.NewID("annotation-revision"), OrganizationID: run.OrganizationID, TaskID: task.ID, AnnotationEpoch: task.AnnotationEpoch,
|
||||
Revision: 0, TemplateVersion: template.Version, Segments: segments,
|
||||
AuthorUserID: "system", ChangeReason: "workflow prelabel", IsRework: false, CountsAsOutput: false, CreatedAt: nowISO(),
|
||||
}); err != nil {
|
||||
|
||||
@ -673,6 +673,67 @@
|
||||
"tests": true
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "deleted-annotator-handoffs",
|
||||
"paths": [
|
||||
"^internal/api/server\\.go$",
|
||||
"^internal/api/deleted_(annotator|author_rework).*\\.go$",
|
||||
"^internal/store/deleted_annotator.*\\.go$",
|
||||
"^docs/operations/deleted-annotator.*$",
|
||||
"^src/lib/annotation-work-mode\\.js$",
|
||||
"^tests/frontend/annotation-work-mode\\.test\\.mjs$"
|
||||
],
|
||||
"frontend": ["tests/frontend/annotation-work-mode.test.mjs"],
|
||||
"go": {
|
||||
"./internal/api": [
|
||||
"^TestDeletedAnnotator",
|
||||
"^TestDeletedAuthorRework",
|
||||
"^TestAccountDeleteDirectoryAndAudit$",
|
||||
"^TestAccountDeleteFailureCanBeRetried$",
|
||||
"^TestQualityRejectionReturnsToEligibleAuthor$"
|
||||
],
|
||||
"./internal/store": ["^TestDeletedAnnotatorStoreMemory$"]
|
||||
},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"./internal/store",
|
||||
"-run",
|
||||
"^TestDeletedAnnotator|^TestDeletedAuthorRework",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"DELETED_ANNOTATOR_LOCAL_MONGO": "1"
|
||||
},
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"-run",
|
||||
"^TestInitialSliceConfirmationBrowserReplica$/deleted-reset$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"INITIAL_SLICE_CONFIRM_BROWSER_TEST": "1",
|
||||
"DELETED_ANNOTATOR_HANDOFF": "true"
|
||||
},
|
||||
"tests": true
|
||||
}
|
||||
],
|
||||
"warnings": [
|
||||
"新规则默认关闭;启用前须完成所有写入组件兼容升级及单独授权的路由索引建设。生产数据补处理不包含在本地验证中。"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@ -30,6 +30,7 @@ export function hasDirectedAnnotationRework(items) {
|
||||
return items.some(
|
||||
(item) =>
|
||||
item.currentStageId === "annotation_combined" &&
|
||||
!item.inheritedRework &&
|
||||
item.reviewRound > 0 &&
|
||||
["claimed", "rework"].includes(item.status),
|
||||
);
|
||||
|
||||
@ -48,6 +48,7 @@ export type AnnotationBufferSummary = Pick<
|
||||
| "currentStageId"
|
||||
| "status"
|
||||
| "version"
|
||||
| "inheritedRework"
|
||||
| "reviewRound"
|
||||
| "claimedAt"
|
||||
| "sourceValidationStatus"
|
||||
@ -69,6 +70,7 @@ export function summarizeWorkBuffer(buffer: AnnotationWorkBuffer): AnnotationBuf
|
||||
currentStageId: item.currentStageId,
|
||||
status: item.status,
|
||||
version: item.version,
|
||||
inheritedRework: item.inheritedRework,
|
||||
reviewRound: item.reviewRound,
|
||||
claimedAt: item.claimedAt,
|
||||
sourceValidationStatus: item.sourceValidationStatus,
|
||||
|
||||
@ -2216,6 +2216,7 @@ export interface HumanTaskDisplayContext {
|
||||
}
|
||||
|
||||
export interface HumanItem {
|
||||
inheritedRework?: boolean;
|
||||
sourceValidationStatus?: "failed" | "pending" | "unverified";
|
||||
traceSummary?: { author: string; supplier: string; rejections: number | null };
|
||||
/** Server-computed acceptance claim capability; never used as a write credential. */
|
||||
|
||||
@ -140,10 +140,22 @@ try {
|
||||
}),
|
||||
);
|
||||
const workspace = item.subject.payload.annotationWorkspace;
|
||||
assert.deepEqual(
|
||||
workspace.segments.map((a) => a.id),
|
||||
["pending-first"],
|
||||
);
|
||||
if (status === "deleted-reset") {
|
||||
assert.ok(
|
||||
workspace.segments.every((action) => action.id !== "pending-first"),
|
||||
"old user's actions must not return",
|
||||
);
|
||||
assert.ok(
|
||||
workspace.sliceSegments.every((slice) => slice.id !== "first"),
|
||||
"old user's cuts must not return",
|
||||
);
|
||||
assert.ok(workspace.segments.length > 0, "new annotation remains editable");
|
||||
} else {
|
||||
assert.deepEqual(
|
||||
workspace.segments.map((a) => a.id),
|
||||
["pending-first"],
|
||||
);
|
||||
}
|
||||
assert.equal(workspace.sliceSegments[0].name, status !== "pending" ? "重新选择切片" : "测试切片");
|
||||
await page.reload();
|
||||
await page.getByRole("button", { name: "继续工作", exact: true }).click();
|
||||
|
||||
@ -164,3 +164,32 @@ test("summary buffer preserves directed rework priority without requiring task p
|
||||
assert.equal(claims, 0);
|
||||
assert.equal(result.summaries[0].id, "rework");
|
||||
});
|
||||
|
||||
test("inherited rework leaves ordinary refill available in item and summary modes", async () => {
|
||||
for (const field of ["items", "summaries"]) {
|
||||
let claims = 0;
|
||||
const rows = [
|
||||
{
|
||||
id: "inherited",
|
||||
inheritedRework: true,
|
||||
reviewRound: 2,
|
||||
status: "claimed",
|
||||
currentStageId: "annotation_combined",
|
||||
},
|
||||
];
|
||||
const result = await refillAnnotationWorkBuffer({
|
||||
refill: true,
|
||||
readBuffer: async () => ({ enabled: true, occupied: rows.length, capacity: 2, [field]: [...rows] }),
|
||||
shouldContinue: () => true,
|
||||
onClaim: () => {},
|
||||
claim: async () => {
|
||||
claims++;
|
||||
const item = { id: "new-task" };
|
||||
rows.push(item);
|
||||
return { item, task: { id: "new-task" } };
|
||||
},
|
||||
});
|
||||
assert.equal(claims, 1);
|
||||
assert.equal(result.occupied, 2);
|
||||
}
|
||||
});
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user