⚡ Scope project/workflow deletion run lookups instead of org-wide scan #9

Merged
longtao.wu merged 1 commits from fix/project-deletion-org-wide-scan-perf into dev 2026-09-22 19:30:45 +08:00
Owner

修复今晚在 Fanyu 上删除 SL-data/BLF-data 时发现的删除预览挂起问题:buildProjectDeletionPlan/workflowRunsForDeletion 都是拉整个组织的全部 Run 再在内存过滤,真实数据量下会挂起数分钟。runs 集合已有 organizationId_projectId_createdAt_id/organizationId_workflowId_createdAt_id 索引但代码没用上。新增 ListRunArtifactsForProject 按项目/工作流收窄查询,替换全部调用点。已跑通相关测试,全量套件无新增失败(75 项,与基线一致)。

修复今晚在 Fanyu 上删除 SL-data/BLF-data 时发现的删除预览挂起问题:`buildProjectDeletionPlan`/`workflowRunsForDeletion` 都是拉整个组织的全部 Run 再在内存过滤,真实数据量下会挂起数分钟。runs 集合已有 `organizationId_projectId_createdAt_id`/`organizationId_workflowId_createdAt_id` 索引但代码没用上。新增 `ListRunArtifactsForProject` 按项目/工作流收窄查询,替换全部调用点。已跑通相关测试,全量套件无新增失败(75 项,与基线一致)。
longtao.wu added 1 commit 2026-09-22 19:30:40 +08:00
buildProjectDeletionPlan and workflowRunsForDeletion both fetched EVERY
run in the organization via ListRunArtifactsByOrganization, then
filtered in Go by projectId/workflowId. For orgs with light test data
this is instant; for an org with real production history (thousands
of runs, 1000+ jobs/day) it made the project-deletion preview hang for
minutes against the shared external MongoDB — discovered live on
Fanyu trying to delete SL-data/BLF-data.

The runs collection already has organizationId_projectId_createdAt_id
and organizationId_workflowId_createdAt_id indexes (mongo_indexes.go)
that this code never used. Added ListRunArtifactsForProject, which
queries by projectId-or-workflowId-in-list directly, and switched both
call sites (plus buildWorkflowDeletionPreview, which was called once
per workflow inside buildProjectDeletionPlan — compounding the same
scan N times) to use it. The mid-deletion re-verification call site
(project_deletion.go ~669) keeps its exact prior semantics
(projectId-only match, no workflowId fallback) by passing nil
workflowIDs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
longtao.wu merged commit e188e13124 into dev 2026-09-22 19:30:45 +08:00
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: longtao.wu/droboticflow#9
No description provided.