eust-w 2864d2aa11 ⚡ Process deletion targets concurrently within each phase
executeProjectDeletion processed every target (runtime, then workspace,
then bos, then records) strictly one at a time — for a project with
thousands of real runs, that meant thousands of sequential K8s/BOS/Mongo
round trips. Observed live on Fanyu: BLF-data's runtime phase (stopping
4,641 runs) took tens of minutes at that pace.

Replaced the per-target sequential loop with a bounded worker pool
(PROJECT_DELETION_TARGET_CONCURRENCY, default 12) applied within each
phase — targets of the same kind now run concurrently, but phases still
run strictly in order (runtime, then workspace, then bos, then records),
preserving the "stop/verify runtime, then remove files, then DB"
ordering the plan is built with. The one-time phase-transition safety
checks (wait for in-flight human work, re-verify runtime absence,
revalidate references) still run exactly once, at the same point,
between the runtime and workspace phases.

Every read-modify-write of shared task state (a target's Status/Error,
and every persist() call) is guarded by a mutex; the actual I/O
(deleteProjectTarget) runs unlocked so workers genuinely overlap. Added
TestProjectDeletionExecutesTargetsConcurrentlyAndCompletes (30 runs) to
lock in that concurrent execution reaches the same end state — every
target done, every workspace directory removed, project record gone —
as the old sequential loop. Full suite passes under -race with no
warnings; 75 pre-existing unrelated failures unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 04:30:13 +08:00
..
2026-08-22 19:46:05 +08:00
2026-08-22 06:02:27 +08:00