🛡️ Bind admission to current proof and isolate legacy revalidation
This commit is contained in:
parent
b3322680d9
commit
5106abd8b3
@ -26,6 +26,10 @@
|
||||
|
||||
旧任务缺少校验记录或完整证据时显示待校验,不能领取。新增任务自动排入完整校验;已经排队的旧时间轴作业也必须完整校验后才能发布成功证据。既有 `completed` 记录不会在读取时自动重置、补造指纹或发起大规模回填,需通过明确的重校验入口补验。只有历史 `materialization.sourceIndex` 标志、没有 outbox 指纹的任务,明确重校验成功且记录当前源指纹后才能恢复;后续出现新 outbox 指纹仍必须核对其校验状态。
|
||||
|
||||
自动排队以实际插入新任务为界,Mongo 使用写入结果判定,不先做无锁存在性猜测。已有任务的普通投影更新(包括详情 GET 恢复已完成送检)、状态/历史读取及重启读取,均不为缺失的校验记录补队,不重置已有排队、运行、完成或失败记录。若新任务落库后排队失败,创建操作返回错误,任务保持未核验、不可领取;通过明确的重校验入口恢复,普通读取或投影重试不冒充创建或补验授权。
|
||||
|
||||
当前 outbox 带源指纹时,物理切片与源索引模式都必须找到同指纹的完整成功证据:可由任务校验记录或对应的源证书作业提供;源索引模式原有证书要求继续生效。任务的 A 证据不能放行后来 B 的 outbox,明确记录的 `supersededSourceFingerprint` 仅替代其确指的旧版本,不能延伸至 C。Go 准入、Mongo 聚合、批量状态及详情/时间轴响应使用相同规则;当前证据正在排队或运行时保持待校验,完成但缺少当前版本证据时为未核验,已知失败仍显示失败。
|
||||
|
||||
只读委托映射仍可解析已有时间轴,但若没有完整校验所需的合法固定源输出位置,不得标记成功或领取;需要由管理员明确配置输出权限,不能擅自扩大源目录权限。
|
||||
|
||||
上线须同时升级 API 与校验器;旧校验器只能产生时间轴成功时,新 API 仍会阻断。历史补验会增加视频解码、源文件读取及固定源副本成本,应先盘点待补验任务,再按现有校验并发分批执行,生产补验需另行授权。严格准入启用后,可领取数可能减少,这是排除未完整校验任务后的结果,不应通过忽略缺证据记录恢复数量。已领取任务保持占位和已保存内容,补验通过后继续原任务,不重跑整条工作流、不自动重放失败送检。
|
||||
|
||||
@ -13,12 +13,32 @@
|
||||
- 已过期校验器、迟到的读取失败不能覆盖新的校验代次。
|
||||
- 未要求重验的旧任务实际发现非法 IMU 后会保存异常;无旧记录的失败插入不能覆盖并发新校验。旧任务修复、payload-only 源索引记录补证均经过真实校验器恢复;草稿与归属不变,跨组织读取和非管理员重验拒绝。
|
||||
- 直接领取的存储边界加固:同版本内源作业变更时,陈旧快照不能覆盖当前缺证、失败或有效的新作业。内存与 Mongo 使用同一回归;这不是已观察到的线上生产竞态。
|
||||
- 已完成时间轴仍受当前源证书失败约束;新源索引 outbox 缺证书拒绝,旧物理 outbox 不强制新增协议。
|
||||
- 已完成时间轴仍受当前源证书失败约束;新源索引 outbox 缺证书拒绝,旧物理 outbox 也核对当前源指纹,但不要求改成源索引修订。
|
||||
- 原指纹 A 失败,修复 B 成功,仅替代 A;之后 C 缺证书/失败继续阻断。内存和真实 Mongo 覆盖等价。
|
||||
- 真实合成双目视频/IMU:末行 gyro 改成 gyr,真实解析拒绝;恢复文件并调用管理员重校验,真实后台校验器建立当前指纹证书,旧失败作业与已保存内容不变。
|
||||
- 原来校验成功的文件再次损坏,工作台检测后撤销准入并给出可执行重校验入口。取消或超时的读取请求不会把有效源数据误标为异常;新增取消请求回归先复现误标,再通过。
|
||||
- 现有随机领取、源索引、时间轴与源校验等待回归通过。委托只读对象存储用真实校验器完成验证后,左右时间轴正常读取,全程仅 HEAD/GET;未校验时明确返回等待状态。
|
||||
|
||||
## 2026-11-11:修复 b3322680 审查发现的两个 P1
|
||||
|
||||
- Go 准入、Mongo 聚合、批量状态、详情状态和时间轴阻断共同覆盖:物理/源索引模式的同指纹证明、A 到 B 的源变化、B 作业缺失/排队/失败/完成、仅时间轴的旧记录,以及修复仅替代 A、不能放行后来 C 的边界。等待完整证据时不可领取,已排队/运行保持 `pending`,没有当前版本证据则为 `unverified`。
|
||||
- 真实存储回归覆盖同版本物理 outbox 换源:列表和数量排除,最终指定领取拒绝,草稿、当前作业和归属不被失败领取覆盖。
|
||||
- 真实 HTTP/Mongo 用旧模板、无校验记录、正式送检修订已落盘的任务核验状态、摘要、历史、详情读取以及服务重启。详情仍能恢复已完成送检的正常流转,但不新增完整校验作业,原工作区、草稿和正式修订保留。
|
||||
- 已有任务投影更新不重置或创建校验记录;显式补验入口及新建旧格式任务的自动排队继续工作。Mongo 通过实际写入的插入结果判断新建,保留原项目写保护与编辑代次冲突检查。
|
||||
|
||||
新增专项入口(仅隔离本地 Mongo):
|
||||
|
||||
```sh
|
||||
POOL_LOCAL_MONGO_URI=mongodb://127.0.0.1:27018 SOURCE_ADMISSION_LOCAL_MONGO=1 \
|
||||
go test ./internal/store -run '^TestSourceAdmission' -count=1 -v
|
||||
SOURCE_ADMISSION_LOCAL_MONGO=1 \
|
||||
go test ./internal/api -run '^TestSourceValidation(CurrentOutboxStatusAndTimeline|LegacyDetailsGETDoesNotEnqueueMongo)$' -count=1 -v
|
||||
```
|
||||
|
||||
反例、原提交 overlay 对照和专项日志位于 `local-data/diagnostics/fix-b332/`。没有对生产任务、校验队列或源文件执行写入;生产补验和上线仍需单独授权。
|
||||
|
||||
本节基线为 `b3322680`。上述 Go/Mongo、HTTP、竞态、编辑代次及删除恢复专项通过;源校验、任务池和删除后重新切片的浏览器专项通过。官方 Go 门禁为 68 项既有失败、无新增失败,未扩大已知失败基线。统一任务报告 `local-data/test-results/verify-1794408009785993000/report.json` 保留 `incomplete`:混合工作树内已有的生产专用只读诊断及全包可选环境检查被跳过,不表示整套发布或生产验收已通过。另一项管理员诊断任务的两个文件未修改。
|
||||
|
||||
可复跑:
|
||||
|
||||
```sh
|
||||
|
||||
@ -119,14 +119,22 @@ func (s *server) annotationSourceValidationForTask(ctx context.Context, task dom
|
||||
if len(jobs) > 1 {
|
||||
source = jobs[1]
|
||||
}
|
||||
ready := store.AnnotationSourceJobsReady(task, base, source)
|
||||
var current domain.HumanItemRecord
|
||||
if item != nil {
|
||||
ready = store.AnnotationItemSourceJobsReady(task, *item, base, source)
|
||||
current = *item
|
||||
}
|
||||
if ready && base != nil {
|
||||
// The detail/status projection uses the same decision as pool eligibility.
|
||||
// A completed proof for another physical outbox must not appear "valid".
|
||||
switch store.AnnotationSourceAdmissionStatus(task, current, base, source) {
|
||||
case "":
|
||||
v.Status, v.Code, v.Message = "valid", "", ""
|
||||
}
|
||||
if !ready && v.Status == "unverified" {
|
||||
case "pending":
|
||||
v.Status, v.Code, v.Message = "pending", "source_validation_pending", "源数据校验中"
|
||||
case "failed":
|
||||
// Keep the safe, specific error selected from the failed job above.
|
||||
v.Status, v.Code = "failed", "source_validation_failed"
|
||||
default:
|
||||
v.Status = "unverified"
|
||||
v.Required = true
|
||||
v.Code, v.Message = "source_validation_unverified", "源数据尚未完成校验"
|
||||
}
|
||||
|
||||
@ -306,8 +306,15 @@ func TestRequiredSourceCertificateCannotBeMissing(t *testing.T) {
|
||||
}
|
||||
item.ProcessingJob.SourceIndex = false
|
||||
f.store.UpsertHumanItem(f.ctx, *item)
|
||||
if got := claimRandom(t, f, "legacy-physical-001"); got.ID != "random-item-0" {
|
||||
t.Fatal("legacy physical outbox incorrectly required source certificate")
|
||||
if got := claimRandom(t, f, "legacy-physical-001"); got.ID != "" {
|
||||
t.Fatal("legacy physical outbox borrowed another source version's proof")
|
||||
}
|
||||
item.ProcessingJob.SourceFingerprint = "validated-source"
|
||||
if err := f.store.UpsertHumanItem(f.ctx, *item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := claimRandom(t, f, "legacy-physical-002"); got.ID != "random-item-0" {
|
||||
t.Fatal("matching complete task proof did not admit physical outbox")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
204
internal/api/annotation_source_proof_test.go
Normal file
204
internal/api/annotation_source_proof_test.go
Normal file
@ -0,0 +1,204 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"droboticflow/internal/domain"
|
||||
"droboticflow/internal/store"
|
||||
"go.mongodb.org/mongo-driver/mongo"
|
||||
"go.mongodb.org/mongo-driver/mongo/options"
|
||||
)
|
||||
|
||||
type sourceProofReadStore struct {
|
||||
*store.MemoryStore
|
||||
timeline, source *domain.AnnotationIndexJob
|
||||
taskWrites int
|
||||
}
|
||||
|
||||
func (s *sourceProofReadStore) GetAnnotationIndexJob(_ context.Context, id string) (*domain.AnnotationIndexJob, error) {
|
||||
for _, j := range []*domain.AnnotationIndexJob{s.timeline, s.source} {
|
||||
if j != nil && j.ID == id {
|
||||
return j, nil
|
||||
}
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
func (s *sourceProofReadStore) UpsertAnnotationTask(ctx context.Context, v domain.AnnotationTaskRecord) error {
|
||||
s.taskWrites++
|
||||
return s.MemoryStore.UpsertAnnotationTask(ctx, v)
|
||||
}
|
||||
|
||||
func TestSourceValidationCurrentOutboxStatusAndTimeline(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name, timelineFP, timelineState, sourceState, sourceFP, superseded, current, want string
|
||||
indexed bool
|
||||
}{
|
||||
{name: "physical-changed", timelineFP: "A", current: "B", want: "unverified"},
|
||||
{name: "physical-same", timelineFP: "A", current: "A", want: "valid"},
|
||||
{name: "physical-source-completed", timelineFP: "A", current: "B", sourceState: "completed", sourceFP: "B", want: "valid"},
|
||||
{name: "physical-source-pending", timelineFP: "A", current: "B", sourceState: "queued", want: "pending"},
|
||||
{name: "physical-source-failed", timelineFP: "A", current: "B", sourceState: "failed", want: "failed"},
|
||||
{name: "indexed-changed", indexed: true, timelineFP: "A", current: "B", want: "unverified"},
|
||||
{name: "indexed-task-pending", indexed: true, timelineState: "queued", current: "B", want: "pending"},
|
||||
{name: "explicit-repair", indexed: true, timelineFP: "B", current: "A", superseded: "A", want: "valid"},
|
||||
{name: "repair-not-for-C", timelineFP: "B", current: "C", superseded: "A", want: "unverified"},
|
||||
} {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
f := newCombinedDraftFixture(t)
|
||||
item := f.item
|
||||
item.ProcessingJob = &domain.AnnotationProcessingJob{Kind: "slice", Status: "failed", SourceFingerprint: c.current, SourceIndex: c.indexed}
|
||||
item.Subject.Payload["annotationWorkspace"] = map[string]any{"saved": "keep reviewed contents"}
|
||||
item.Subject.Payload["annotationDraft"] = map[string]any{"saved": "keep newer edits"}
|
||||
if err := f.store.UpsertHumanItem(f.ctx, item); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
job := domain.AnnotationIndexJob{ID: f.task.ID, OrganizationID: f.task.OrganizationID, ProjectID: f.task.ProjectID, RunID: f.task.RunID, Status: "completed", RevalidateSource: true, SourceFingerprint: c.timelineFP, SupersededSourceFingerprint: c.superseded}
|
||||
if c.timelineState != "" {
|
||||
job.Status = c.timelineState
|
||||
}
|
||||
probe := &sourceProofReadStore{MemoryStore: f.store, timeline: &job}
|
||||
if c.sourceState != "" {
|
||||
source := job
|
||||
source.ID = store.AnnotationSourceJobID(job.OrganizationID, c.current)
|
||||
source.Status = c.sourceState
|
||||
source.SourceFingerprint = c.sourceFP
|
||||
source.SupersededSourceFingerprint = ""
|
||||
probe.source = &source
|
||||
}
|
||||
f.handler = NewHandler(probe, nil)
|
||||
check := func(status annotationSourceValidation) {
|
||||
t.Helper()
|
||||
if status.Status != c.want || !status.Required || status.RetryAllowed != (c.want == "unverified" || c.want == "failed") {
|
||||
t.Fatalf("status=%+v want=%s", status, c.want)
|
||||
}
|
||||
}
|
||||
r := f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks/"+f.task.ID+"?view=status", f.admin, nil)
|
||||
var response struct {
|
||||
SourceValidation annotationSourceValidation `json:"sourceValidation"`
|
||||
}
|
||||
if r.Code != 200 || json.Unmarshal(r.Body.Bytes(), &response) != nil {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
check(response.SourceValidation)
|
||||
if c.want != "valid" {
|
||||
r = f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks/"+f.task.ID+"/frame-timeline", f.admin, nil)
|
||||
var blocked struct {
|
||||
Details struct {
|
||||
SourceValidation annotationSourceValidation `json:"sourceValidation"`
|
||||
} `json:"details"`
|
||||
}
|
||||
if r.Code != 422 || json.Unmarshal(r.Body.Bytes(), &blocked) != nil {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
check(blocked.Details.SourceValidation)
|
||||
}
|
||||
after, _ := f.store.GetHumanItem(f.ctx, item.ID)
|
||||
if probe.taskWrites != 0 || !reflect.DeepEqual(before, after) {
|
||||
t.Fatal("source reads changed saved work or task projection")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceValidationLegacyDetailsGETDoesNotEnqueueMongo(t *testing.T) {
|
||||
if os.Getenv("SOURCE_ADMISSION_LOCAL_MONGO") != "1" {
|
||||
t.Skip("isolated local Mongo opt-in")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(t.Context(), 30*time.Second)
|
||||
defer cancel()
|
||||
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://127.0.0.1:27018/?directConnection=true").SetServerSelectionTimeout(3*time.Second))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db := client.Database(fmt.Sprintf("source_read_boundary_%d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() {
|
||||
c, stop := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer stop()
|
||||
if err := db.Drop(c); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
_ = client.Disconnect(c)
|
||||
})
|
||||
ds := store.NewMongoStore(db)
|
||||
f := newCombinedDraftFixture(t)
|
||||
must := func(err error) {
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
project, err := f.store.GetProject(ctx, f.task.ProjectID)
|
||||
must(err)
|
||||
must(ds.UpsertProject(ctx, *project))
|
||||
supplier, err := f.store.GetSupplier(ctx, f.annotator.SupplierID)
|
||||
must(err)
|
||||
must(ds.UpsertSupplier(ctx, *supplier))
|
||||
template, err := f.store.GetAnnotationTemplate(ctx, f.task.TemplateID)
|
||||
must(err)
|
||||
must(ds.UpsertAnnotationTemplate(ctx, *template))
|
||||
batch, err := f.store.GetHumanBatch(ctx, f.item.BatchID)
|
||||
must(err)
|
||||
must(ds.UpsertHumanBatch(ctx, *batch))
|
||||
for _, user := range []domain.UserRecord{f.admin, f.annotator, f.reviewer} {
|
||||
must(ds.UpsertUser(ctx, user))
|
||||
}
|
||||
task := f.task
|
||||
task.TemplateVersion = 4
|
||||
task.SourceIsFullVideo = false
|
||||
task.ArtifactID = ""
|
||||
_, err = db.Collection("annotationTasks").InsertOne(ctx, task)
|
||||
must(err) // A persisted legacy record with no validation job.
|
||||
item := f.item
|
||||
item.PendingCombinedSubmissionID = "old-submit"
|
||||
item.PendingCombinedPayloadSHA = "old-content"
|
||||
item.Subject.Payload["annotationWorkspace"] = map[string]any{"saved": "reviewed contents"}
|
||||
item.Subject.Payload["annotationDraft"] = map[string]any{"saved": "newer edits"}
|
||||
must(ds.UpsertHumanItem(ctx, item))
|
||||
before, err := ds.GetHumanItem(ctx, item.ID)
|
||||
must(err)
|
||||
must(ds.UpsertAnnotationRevision(ctx, domain.AnnotationRevisionRecord{ID: "annotation-old-submit", OrganizationID: task.OrganizationID, TaskID: task.ID, Revision: 1, Segments: []domain.AnnotationSegment{{ID: "saved-action", Start: 0, End: 5, Description: "preserve formal result"}}}))
|
||||
f.handler = NewHandler(ds, nil)
|
||||
for _, view := range []string{"?view=status", "?view=summary", "?view=history", ""} {
|
||||
r := f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks/"+task.ID+view, f.admin, nil)
|
||||
if r.Code != 200 {
|
||||
t.Fatalf("view=%q code=%d body=%s", view, r.Code, r.Body.String())
|
||||
}
|
||||
job, err := ds.GetAnnotationIndexJob(ctx, task.ID)
|
||||
must(err)
|
||||
if job != nil {
|
||||
t.Fatalf("GET %q queued full source validation without confirmation", view)
|
||||
}
|
||||
}
|
||||
after, err := ds.GetHumanItem(ctx, item.ID)
|
||||
must(err)
|
||||
if after.CurrentStageID != "quality_review" || after.PendingCombinedSubmissionID != "" {
|
||||
t.Fatal("GET did not exercise interrupted submission recovery")
|
||||
}
|
||||
if !reflect.DeepEqual(before.Subject, after.Subject) {
|
||||
t.Fatal("recovery discarded saved workspace/draft")
|
||||
}
|
||||
revisions, err := ds.ListAnnotationRevisions(ctx, task.OrganizationID, task.ID)
|
||||
must(err)
|
||||
if len(revisions) != 1 || len(revisions[0].Segments) != 1 || revisions[0].Segments[0].Description != "preserve formal result" {
|
||||
t.Fatal("formal annotation was altered")
|
||||
}
|
||||
// A new server instance reading the same database also must not enqueue.
|
||||
f.handler = NewHandler(ds, nil)
|
||||
r := f.request(http.MethodGet, "/api/plugins/annotation-qa/tasks/"+task.ID, f.admin, nil)
|
||||
if r.Code != 200 {
|
||||
t.Fatal(r.Code, r.Body.String())
|
||||
}
|
||||
job, err := ds.GetAnnotationIndexJob(ctx, task.ID)
|
||||
must(err)
|
||||
if job != nil {
|
||||
t.Fatal("restart/details read queued historical validation")
|
||||
}
|
||||
}
|
||||
@ -317,8 +317,9 @@ func (store *MemoryStore) upsertAnnotationTaskLocked(item domain.AnnotationTaskR
|
||||
item.CreatedAt = now
|
||||
}
|
||||
item.UpdatedAt = now
|
||||
_, existed := store.annotationTasks[item.ID]
|
||||
store.annotationTasks[item.ID] = item
|
||||
if AnnotationSourceValidationRequired(item) {
|
||||
if !existed && AnnotationSourceValidationRequired(item) {
|
||||
if store.annotationIndexJobs == nil {
|
||||
store.annotationIndexJobs = map[string]domain.AnnotationIndexJob{}
|
||||
}
|
||||
@ -870,10 +871,14 @@ func (store *MongoStore) UpsertAnnotationTask(ctx context.Context, item domain.A
|
||||
item.CreatedAt = now
|
||||
}
|
||||
item.UpdatedAt = now
|
||||
if err := upsertAnnotationEpoch(ctx, store.annotationTasks, item.ID, item, item.AnnotationEpoch); err != nil {
|
||||
inserted, err := upsertAnnotationEpochInserted(ctx, store.annotationTasks, item.ID, item, item.AnnotationEpoch)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if AnnotationSourceValidationRequired(item) {
|
||||
// Existing task projections are also updated during detail GET recovery.
|
||||
// Missing historical evidence requires explicit repair, never a read-triggered
|
||||
// full decode/source copy. New task creation alone bootstraps validation.
|
||||
if inserted && AnnotationSourceValidationRequired(item) {
|
||||
if err := store.EnqueueAnnotationIndex(ctx, item); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@ -54,6 +54,23 @@ func annotationSourceHasCurrentProof(item domain.HumanItemRecord, timeline *doma
|
||||
return materialization["sourceIndex"] == true && timeline.SupersededSourceFingerprint == ""
|
||||
}
|
||||
|
||||
// Physical outboxes also record the current source identity. Absence of the
|
||||
// source-index protocol is not permission to borrow another version's proof.
|
||||
// The explicit superseded-source recovery is handled before this predicate.
|
||||
func annotationOutboxProofMatches(item domain.HumanItemRecord, timeline, source *domain.AnnotationIndexJob) bool {
|
||||
if item.ProcessingJob == nil {
|
||||
return true
|
||||
}
|
||||
fingerprint := item.ProcessingJob.SourceFingerprint
|
||||
if source != nil {
|
||||
return source.SourceFingerprint == fingerprint
|
||||
}
|
||||
if fingerprint == "" {
|
||||
return true
|
||||
}
|
||||
return timeline != nil && timeline.SourceFingerprint == fingerprint
|
||||
}
|
||||
|
||||
func AnnotationItemSourceJobsReady(task domain.AnnotationTaskRecord, item domain.HumanItemRecord, timeline, source *domain.AnnotationIndexJob) bool {
|
||||
if timeline != nil && timeline.RevalidateSource {
|
||||
if timeline.SourceFingerprint == "" || !AnnotationSourceJobsReady(task, timeline, nil) {
|
||||
@ -63,7 +80,7 @@ func AnnotationItemSourceJobsReady(task domain.AnnotationTaskRecord, item domain
|
||||
return true
|
||||
}
|
||||
}
|
||||
if source != nil && item.ProcessingJob != nil && source.SourceFingerprint != item.ProcessingJob.SourceFingerprint {
|
||||
if !annotationOutboxProofMatches(item, timeline, source) {
|
||||
return false
|
||||
}
|
||||
if AnnotationSourceCertificateRequired(item) && source == nil {
|
||||
@ -131,11 +148,17 @@ func AnnotationSourceAdmissionStatus(task domain.AnnotationTaskRecord, item doma
|
||||
return "unverified"
|
||||
}
|
||||
}
|
||||
if source != nil && item.ProcessingJob != nil && source.SourceFingerprint != item.ProcessingJob.SourceFingerprint {
|
||||
currentProof := annotationSourceHasCurrentProof(item, timeline)
|
||||
timelineActive := timeline != nil && (timeline.Status == "queued" || timeline.Status == "running")
|
||||
if timeline == nil || timeline.Status == "completed" && timeline.SourceFingerprint == "" || !timelineActive && !currentProof && !AnnotationSourceJobSuperseded(item, timeline) && AnnotationSourceCertificateRequired(item) && source == nil {
|
||||
return "unverified"
|
||||
}
|
||||
currentProof := annotationSourceHasCurrentProof(item, timeline)
|
||||
if timeline == nil || timeline.Status == "completed" && timeline.SourceFingerprint == "" || !currentProof && !AnnotationSourceJobSuperseded(item, timeline) && AnnotationSourceCertificateRequired(item) && source == nil {
|
||||
for _, job := range []*domain.AnnotationIndexJob{timeline, source} {
|
||||
if job != nil && (job.Status == "queued" || job.Status == "running") {
|
||||
return "pending"
|
||||
}
|
||||
}
|
||||
if !AnnotationSourceJobSuperseded(item, timeline) && !annotationOutboxProofMatches(item, timeline, source) {
|
||||
return "unverified"
|
||||
}
|
||||
return "pending"
|
||||
@ -158,13 +181,18 @@ func annotationSourceAdmissionStatusPipeline() mongo.Pipeline {
|
||||
taskPresent := bson.M{"$gt": bson.A{bson.M{"$size": "$_sourceTask"}, 0}}
|
||||
certificatePresent := bson.M{"$gt": bson.A{bson.M{"$size": "$_sourcePrepared"}, 0}}
|
||||
sourceVersionMatches := bson.M{"$or": bson.A{bson.M{"$not": bson.A{certificatePresent}}, bson.M{"$eq": bson.A{first("$_sourcePrepared.sourceFingerprint"), "$processingJob.sourceFingerprint"}}}}
|
||||
outboxProofMatches := bson.M{"$or": bson.A{
|
||||
bson.M{"$not": bson.A{nonempty("$processingJob.sourceFingerprint")}}, superseded,
|
||||
bson.M{"$and": bson.A{certificatePresent, sourceVersionMatches}},
|
||||
bson.M{"$and": bson.A{bson.M{"$not": bson.A{certificatePresent}}, bson.M{"$eq": bson.A{first("$_sourceTimeline.sourceFingerprint"), "$processingJob.sourceFingerprint"}}}},
|
||||
}}
|
||||
certificateSatisfied := bson.M{"$or": bson.A{bson.M{"$not": bson.A{needsCertificate}}, certificatePresent, superseded, currentProof}}
|
||||
timelineActive := bson.M{"$in": bson.A{first("$_sourceTimeline.status"), bson.A{"queued", "running"}}}
|
||||
missing := bson.M{"$or": bson.A{
|
||||
bson.M{"$not": bson.A{taskPresent}},
|
||||
bson.M{"$not": bson.A{timelinePresent}},
|
||||
bson.M{"$and": bson.A{bson.M{"$eq": bson.A{first("$_sourceTimeline.status"), "completed"}}, bson.M{"$not": bson.A{nonempty(first("$_sourceTimeline.sourceFingerprint"))}}}},
|
||||
bson.M{"$not": bson.A{certificateSatisfied}},
|
||||
bson.M{"$not": bson.A{sourceVersionMatches}},
|
||||
bson.M{"$and": bson.A{bson.M{"$not": bson.A{certificateSatisfied}}, bson.M{"$not": bson.A{timelineActive}}}},
|
||||
bson.M{"$anyElementTrue": bson.A{bson.M{"$map": bson.M{"input": jobs, "as": "job", "in": bson.M{"$and": bson.A{bson.M{"$eq": bson.A{"$$job.status", "completed"}}, bson.M{"$not": bson.A{nonempty("$$job.sourceFingerprint")}}}}}}}},
|
||||
}}
|
||||
ready := bson.M{"$and": bson.A{
|
||||
@ -173,8 +201,10 @@ func annotationSourceAdmissionStatusPipeline() mongo.Pipeline {
|
||||
nonempty(first("$_sourceTimeline.sourceFingerprint")),
|
||||
certificateSatisfied,
|
||||
sourceVersionMatches,
|
||||
outboxProofMatches,
|
||||
bson.M{"$allElementsTrue": bson.A{bson.M{"$map": bson.M{"input": jobs, "as": "job", "in": safeJob}}}},
|
||||
}}
|
||||
active := bson.M{"$anyElementTrue": bson.A{bson.M{"$map": bson.M{"input": jobs, "as": "job", "in": bson.M{"$in": bson.A{"$$job.status", bson.A{"queued", "running"}}}}}}}
|
||||
failed := bson.M{"$anyElementTrue": bson.A{bson.M{"$map": bson.M{"input": jobs, "as": "job", "in": bson.M{"$or": bson.A{bson.M{"$eq": bson.A{"$$job.status", "failed"}}, bson.M{"$not": bson.A{matchingScope}}}}}}}}
|
||||
jobProjection := bson.M{"status": 1, "organizationId": 1, "projectId": 1, "runId": 1, "revalidateSource": 1, "sourceFingerprint": 1, "supersededSourceFingerprint": 1}
|
||||
lookup := func(collection, local, as string, projection bson.M) bson.D {
|
||||
@ -185,7 +215,7 @@ func annotationSourceAdmissionStatusPipeline() mongo.Pipeline {
|
||||
lookup("annotationIndexJobs", "legacyTaskId", "_sourceTimeline", jobProjection),
|
||||
{{Key: "$set", Value: bson.M{"_sourceJobID": bson.M{"$cond": bson.A{bson.M{"$and": bson.A{nonempty("$processingJob.sourceFingerprint"), bson.M{"$not": bson.A{superseded}}}}, bson.M{"$concat": bson.A{"source-v1:", "$organizationId", ":", "$processingJob.sourceFingerprint"}}, ""}}}}},
|
||||
lookup("annotationIndexJobs", "_sourceJobID", "_sourcePrepared", jobProjection),
|
||||
{{Key: "$set", Value: bson.M{"_sourceReady": ready, "_sourceAdmissionStatus": bson.M{"$switch": bson.M{"branches": bson.A{bson.M{"case": failed, "then": "failed"}, bson.M{"case": missing, "then": "unverified"}, bson.M{"case": ready, "then": ""}}, "default": "pending"}}}}},
|
||||
{{Key: "$set", Value: bson.M{"_sourceReady": ready, "_sourceAdmissionStatus": bson.M{"$switch": bson.M{"branches": bson.A{bson.M{"case": failed, "then": "failed"}, bson.M{"case": missing, "then": "unverified"}, bson.M{"case": ready, "then": ""}, bson.M{"case": active, "then": "pending"}, bson.M{"case": bson.M{"$or": bson.A{bson.M{"$not": bson.A{outboxProofMatches}}, bson.M{"$not": bson.A{sourceVersionMatches}}}}, "then": "unverified"}}, "default": "pending"}}}}},
|
||||
{{Key: "$unset", Value: bson.A{"_sourceTask", "_sourceTimeline", "_sourcePrepared", "_sourceJobID"}}},
|
||||
}
|
||||
}
|
||||
|
||||
@ -7,6 +7,7 @@ import (
|
||||
"fmt"
|
||||
"go.mongodb.org/mongo-driver/event"
|
||||
"os"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
@ -94,6 +95,46 @@ func testSourceAdmissionAndClaimFence(t *testing.T, s Store) {
|
||||
if e != nil || count != 2 {
|
||||
t.Fatalf("count=%d err=%v", count, e)
|
||||
}
|
||||
// A physical outbox can change source identity without bumping workflow
|
||||
// version. Lists, counts and the final claim must all reject the old proof.
|
||||
validBefore, e := s.GetHumanItem(ctx, tasks["valid"].ID)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
changed := *validBefore
|
||||
changed.ProcessingJob = &domain.AnnotationProcessingJob{Kind: "slice", Status: "failed", SourceFingerprint: "changed-physical-source", SourceIndex: false}
|
||||
if e := s.UpsertHumanItem(ctx, changed); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
changedBefore, e := s.GetHumanItem(ctx, changed.ID)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if n, e := s.CountAnnotationClaimCandidates(ctx, org, q); e != nil || n != 1 {
|
||||
t.Fatalf("old proof counted changed physical outbox: %d %v", n, e)
|
||||
}
|
||||
rows, e := s.ListAnnotationClaimCandidates(ctx, org, project)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
for _, v := range rows {
|
||||
if v.ID == changed.ID {
|
||||
t.Fatal("old proof exposed changed physical outbox")
|
||||
}
|
||||
}
|
||||
claimChanged := *changedBefore
|
||||
claimChanged.AssignedUserID = "source-actor"
|
||||
claimChanged.Version++
|
||||
if ok, e := s.ClaimHumanItemVersioned(ctx, claimChanged, changedBefore.Version, domain.UserRoleAnnotator); ok || !errors.Is(e, ErrHumanClaimConflict) {
|
||||
t.Fatalf("changed physical source directly claimed: %t %v", ok, e)
|
||||
}
|
||||
unchanged, e := s.GetHumanItem(ctx, changed.ID)
|
||||
if e != nil || !reflect.DeepEqual(changedBefore, unchanged) {
|
||||
t.Fatal("failed claim changed saved work or source outbox", e)
|
||||
}
|
||||
if e := s.UpsertHumanItem(ctx, *validBefore); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
// Selection completed before administrator revalidation: commit must recheck.
|
||||
current, e := s.GetHumanItem(ctx, tasks["valid"].ID)
|
||||
if e != nil {
|
||||
@ -112,9 +153,18 @@ func testSourceAdmissionAndClaimFence(t *testing.T, s Store) {
|
||||
t.Fatal(e)
|
||||
}
|
||||
physical, e := s.CountAnnotationClaimCandidates(ctx, org, q)
|
||||
if e != nil || physical != 2 {
|
||||
if e != nil || physical != 1 {
|
||||
t.Fatalf("legacy physical fingerprint count=%d err=%v", physical, e)
|
||||
}
|
||||
// A same-version full task proof still admits a physical outbox without
|
||||
// requiring it to adopt the source-index materialization protocol.
|
||||
current.ProcessingJob.SourceFingerprint = "validated-source"
|
||||
if e := s.UpsertHumanItem(ctx, *current); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if n, e := s.CountAnnotationClaimCandidates(ctx, org, q); e != nil || n != 2 {
|
||||
t.Fatalf("matching physical proof not admitted: %d %v", n, e)
|
||||
}
|
||||
previous := *current
|
||||
current.Version++
|
||||
current.Status = domain.HumanItemClaimed
|
||||
|
||||
192
internal/store/annotation_source_proof_test.go
Normal file
192
internal/store/annotation_source_proof_test.go
Normal file
@ -0,0 +1,192 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"droboticflow/internal/domain"
|
||||
"go.mongodb.org/mongo-driver/bson"
|
||||
)
|
||||
|
||||
type sourceProofCase struct {
|
||||
name, timelineState, timelineFP, sourceState, sourceFP, superseded, currentFP, want string
|
||||
indexed bool
|
||||
}
|
||||
|
||||
func sourceProofCases() []sourceProofCase {
|
||||
return []sourceProofCase{
|
||||
{name: "physical-changed-missing-proof", timelineState: "completed", timelineFP: "A", currentFP: "B", want: "unverified"},
|
||||
{name: "physical-current-task-proof", timelineState: "completed", timelineFP: "A", currentFP: "A"},
|
||||
{name: "physical-current-certificate", timelineState: "completed", timelineFP: "A", currentFP: "B", sourceState: "completed", sourceFP: "B"},
|
||||
{name: "physical-wrong-certificate", timelineState: "completed", timelineFP: "A", currentFP: "B", sourceState: "completed", sourceFP: "A", want: "unverified"},
|
||||
{name: "physical-current-pending", timelineState: "completed", timelineFP: "A", currentFP: "B", sourceState: "queued", want: "pending"},
|
||||
{name: "physical-current-failed", timelineState: "completed", timelineFP: "A", currentFP: "B", sourceState: "failed", sourceFP: "B", want: "failed"},
|
||||
{name: "indexed-changed-missing-proof", indexed: true, timelineState: "completed", timelineFP: "A", currentFP: "B", want: "unverified"},
|
||||
{name: "indexed-current-task-proof", indexed: true, timelineState: "completed", timelineFP: "A", currentFP: "A"},
|
||||
{name: "indexed-normal-pending-task-proof", indexed: true, timelineState: "queued", currentFP: "A", want: "pending"},
|
||||
{name: "indexed-pending-task-proof", indexed: true, timelineState: "queued", currentFP: "A", superseded: "A", want: "pending"},
|
||||
{name: "legacy-timeline-only", timelineState: "completed", currentFP: "A", want: "unverified"},
|
||||
{name: "legacy-no-record", currentFP: "A", want: "unverified"},
|
||||
{name: "controlled-repair", indexed: true, timelineState: "completed", timelineFP: "B", currentFP: "A", superseded: "A"},
|
||||
{name: "repair-not-for-later-source", timelineState: "completed", timelineFP: "B", currentFP: "C", superseded: "A", want: "unverified"},
|
||||
{name: "repair-later-proof-pending", timelineState: "completed", timelineFP: "B", currentFP: "C", superseded: "A", sourceState: "running", want: "pending"},
|
||||
{name: "repair-later-proof-failed", timelineState: "completed", timelineFP: "B", currentFP: "C", superseded: "A", sourceState: "failed", want: "failed"},
|
||||
}
|
||||
}
|
||||
|
||||
func (c sourceProofCase) records(id string) (domain.AnnotationTaskRecord, domain.HumanItemRecord, *domain.AnnotationIndexJob, *domain.AnnotationIndexJob) {
|
||||
// Real fingerprints include the Run identity. Keep fixture jobs unique
|
||||
// across cases too, so a B proof for another Run cannot alias this one.
|
||||
fp := func(v string) string {
|
||||
if v == "" {
|
||||
return ""
|
||||
}
|
||||
return id + "-" + v
|
||||
}
|
||||
task := domain.AnnotationTaskRecord{ID: id, OrganizationID: "proof-org", ProjectID: "proof-project", RunID: id, TemplateVersion: 4}
|
||||
item := domain.HumanItemRecord{ID: id, OrganizationID: task.OrganizationID, ProjectID: task.ProjectID, RunID: task.RunID, LegacyTaskID: id, Version: 7, ProcessingJob: &domain.AnnotationProcessingJob{Kind: "slice", Status: "failed", SourceIndex: c.indexed, SourceFingerprint: fp(c.currentFP)}, Subject: domain.HumanSubjectRef{Payload: map[string]any{"annotationWorkspace": map[string]any{"preserve": "reviewed contents"}, "annotationDraft": map[string]any{"preserve": "latest edits"}}}}
|
||||
var timeline, source *domain.AnnotationIndexJob
|
||||
job := domain.AnnotationIndexJob{OrganizationID: task.OrganizationID, ProjectID: task.ProjectID, RunID: task.RunID}
|
||||
if c.timelineState != "" {
|
||||
v := job
|
||||
v.ID = id
|
||||
v.Status = c.timelineState
|
||||
v.SourceFingerprint = fp(c.timelineFP)
|
||||
v.RevalidateSource = true
|
||||
v.SupersededSourceFingerprint = fp(c.superseded)
|
||||
timeline = &v
|
||||
}
|
||||
if c.sourceState != "" {
|
||||
v := job
|
||||
v.ID = AnnotationSourceJobID(task.OrganizationID, fp(c.currentFP))
|
||||
v.Status = c.sourceState
|
||||
v.SourceFingerprint = fp(c.sourceFP)
|
||||
source = &v
|
||||
}
|
||||
return task, item, timeline, source
|
||||
}
|
||||
|
||||
func TestSourceAdmissionCurrentOutboxProof(t *testing.T) {
|
||||
for i, c := range sourceProofCases() {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
task, item, timeline, source := c.records(fmt.Sprint("proof-", i))
|
||||
before := clone(item)
|
||||
if got := AnnotationItemSourceJobsReady(task, item, timeline, source); got != (c.want == "") {
|
||||
t.Errorf("ready=%t want status=%q", got, c.want)
|
||||
}
|
||||
if got := AnnotationSourceAdmissionStatus(task, item, timeline, source); got != c.want {
|
||||
t.Errorf("status=%q want=%q", got, c.want)
|
||||
}
|
||||
if !reflect.DeepEqual(item, before) {
|
||||
t.Fatal("admission changed saved annotation")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceAdmissionCurrentOutboxProofMongo(t *testing.T) {
|
||||
ctx, s := poolLocalMongo(t)
|
||||
for i, c := range sourceProofCases() {
|
||||
task, item, timeline, source := c.records(fmt.Sprint("proof-", i))
|
||||
for name, value := range map[string]any{"annotationTasks": task, "humanItems": item} {
|
||||
if _, err := s.db.Collection(name).InsertOne(ctx, value); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, job := range []*domain.AnnotationIndexJob{timeline, source} {
|
||||
if job != nil {
|
||||
if _, err := s.db.Collection("annotationIndexJobs").InsertOne(ctx, job); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
var projected struct {
|
||||
Ready bool `bson:"_sourceReady"`
|
||||
Status string `bson:"_sourceAdmissionStatus"`
|
||||
}
|
||||
pipeline := append([]bson.D{{{Key: "$match", Value: bson.M{"id": item.ID}}}}, annotationSourceAdmissionStatusPipeline()...)
|
||||
cur, err := s.humanItems.Aggregate(ctx, pipeline)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !cur.Next(ctx) {
|
||||
t.Fatal("missing projection", cur.Err())
|
||||
}
|
||||
err = cur.Decode(&projected)
|
||||
cur.Close(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if projected.Ready != (c.want == "") || projected.Status != c.want {
|
||||
t.Errorf("%s Mongo ready=%t status=%q want=%q", c.name, projected.Ready, projected.Status, c.want)
|
||||
}
|
||||
rows := []domain.HumanItemRecord{item}
|
||||
if err := s.EnrichAnnotationSourceStatuses(ctx, rows); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rows[0].SourceValidationStatus != c.want {
|
||||
t.Errorf("%s batch status=%q want=%q", c.name, rows[0].SourceValidationStatus, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceAdmissionExistingTaskUpdateDoesNotEnqueue(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
s := NewMemoryStore()
|
||||
task := domain.AnnotationTaskRecord{ID: "legacy-no-job", OrganizationID: "proof-org", ProjectID: "proof-project", RunID: "run", TemplateVersion: 4}
|
||||
s.annotationTasks[task.ID] = task
|
||||
task.Status = domain.AnnotationTaskStatusPendingQuality
|
||||
if err := s.UpsertAnnotationTask(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job, _ := s.GetAnnotationIndexJob(ctx, task.ID); job != nil {
|
||||
t.Fatal("existing projection update started full validation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceAdmissionExistingTaskUpdateDoesNotEnqueueMongo(t *testing.T) {
|
||||
ctx, s := poolLocalMongo(t)
|
||||
task := domain.AnnotationTaskRecord{ID: "legacy-no-job", OrganizationID: "proof-org", ProjectID: "proof-project", RunID: "run", TemplateVersion: 4}
|
||||
if _, err := s.annotationTasks.InsertOne(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
task.Status = domain.AnnotationTaskStatusPendingQuality
|
||||
if err := s.UpsertAnnotationTask(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job, _ := s.GetAnnotationIndexJob(ctx, task.ID); job != nil {
|
||||
t.Fatal("existing projection update started full validation")
|
||||
}
|
||||
// Explicit repair remains restart-safe and idempotent; ordinary updates
|
||||
// must not overwrite its queued/running/completed/failed evidence.
|
||||
if err := s.RetryAnnotationSourceValidation(ctx, task, true, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
job, err := s.GetAnnotationIndexJob(ctx, task.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, state := range []string{"queued", "running", "completed", "failed"} {
|
||||
job.Status = state
|
||||
job.SourceFingerprint = "A"
|
||||
if err := mongoUpsert(ctx, s.db.Collection("annotationIndexJobs"), job.ID, job); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.UpsertAnnotationTask(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after, err := s.GetAnnotationIndexJob(ctx, task.ID)
|
||||
if err != nil || !reflect.DeepEqual(job, after) {
|
||||
t.Fatal("projection update replaced repair evidence", state, err)
|
||||
}
|
||||
}
|
||||
// A genuinely new legacy-format task still enters full validation.
|
||||
task.ID = "new-legacy-task"
|
||||
if err := s.UpsertAnnotationTask(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job, _ := s.GetAnnotationIndexJob(ctx, task.ID); job == nil || !job.RevalidateSource || job.Status != "queued" {
|
||||
t.Fatal("new task lost automatic validation")
|
||||
}
|
||||
}
|
||||
@ -115,13 +115,25 @@ func (s *MongoStore) ResetDeletedAnnotatorTask(ctx context.Context, task domain.
|
||||
}
|
||||
|
||||
func upsertAnnotationEpoch(ctx context.Context, col *mongo.Collection, id string, value any, epoch int) error {
|
||||
return supplierGuardedMongoWrite(ctx, col, id, value, func(ctx context.Context, d bson.M) error {
|
||||
_, err := col.ReplaceOne(ctx, bson.M{"id": id, "$or": bson.A{bson.M{"annotationEpoch": bson.M{"$lte": epoch}}, bson.M{"annotationEpoch": bson.M{"$exists": false}}}}, d, options.Replace().SetUpsert(true))
|
||||
_, err := upsertAnnotationEpochInserted(ctx, col, id, value, epoch)
|
||||
return err
|
||||
}
|
||||
|
||||
// Use the write result, rather than an unlocked existence read: concurrent
|
||||
// projection updates must not be mistaken for task creation.
|
||||
func upsertAnnotationEpochInserted(ctx context.Context, col *mongo.Collection, id string, value any, epoch int) (bool, error) {
|
||||
inserted := false
|
||||
err := supplierGuardedMongoWrite(ctx, col, id, value, func(ctx context.Context, d bson.M) error {
|
||||
result, err := col.ReplaceOne(ctx, bson.M{"id": id, "$or": bson.A{bson.M{"annotationEpoch": bson.M{"$lte": epoch}}, bson.M{"annotationEpoch": bson.M{"$exists": false}}}}, d, options.Replace().SetUpsert(true))
|
||||
if mongo.IsDuplicateKeyError(err) {
|
||||
return ErrHumanClaimConflict
|
||||
}
|
||||
if err == nil {
|
||||
inserted = result.UpsertedCount == 1
|
||||
}
|
||||
return err
|
||||
})
|
||||
return inserted, err
|
||||
}
|
||||
|
||||
func (s *MongoStore) upsertActiveAnnotationRevision(ctx context.Context, col *mongo.Collection, id, taskID string, revision, epoch int, value any) error {
|
||||
|
||||
@ -202,13 +202,14 @@
|
||||
"test",
|
||||
"./internal/store",
|
||||
"-run",
|
||||
"^TestSourceAdmissionMongo(ClaimFence|BatchMatchesLegacyPipeline|ClaimReadPlan|LargePool)$|^TestAdminSourceExceptionsMongo$",
|
||||
"^TestSourceAdmission(Mongo(ClaimFence|BatchMatchesLegacyPipeline|ClaimReadPlan|LargePool)|CurrentOutboxProofMongo|ExistingTaskUpdateDoesNotEnqueueMongo)$|^TestAdminSourceExceptionsMongo$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"SOURCE_ADMISSION_LOCAL_MONGO": "1"
|
||||
"SOURCE_ADMISSION_LOCAL_MONGO": "1",
|
||||
"POOL_LOCAL_MONGO_URI": "mongodb://127.0.0.1:27018"
|
||||
},
|
||||
"tests": true
|
||||
},
|
||||
@ -228,6 +229,19 @@
|
||||
},
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"-run",
|
||||
"^TestSourceValidationLegacyDetailsGETDoesNotEnqueueMongo$",
|
||||
"-count=1",
|
||||
"-v"
|
||||
],
|
||||
"env": { "SOURCE_ADMISSION_LOCAL_MONGO": "1" },
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user