openmaic-custom/tests/utils/concurrency.test.ts
ly-wang19 3ccd5da6da
feat(generation): opt-in parallel scene-content generation (#660)
* feat(generation): opt-in parallel scene-content generation

Scene generation was strictly serial — content -> actions -> TTS, one
outline at a time — so an N-scene classroom paid N x (content + actions
+ TTS) of wall-clock latency, which dominates the post-outline wait.

Add an opt-in hybrid two-phase path (#572):

- Phase 1 fetches scene *content* concurrently (bounded). Content is the
  only per-scene step independent of cross-scene state, so it is safe to
  parallelise; a content failure marks just that outline and does not
  pause the batch.
- Phase 2 keeps the existing in-order actions + TTS loop, so
  previousSpeeches threading, ordered addScene, the abort/epoch guards,
  and the pause-on-failure UX are all unchanged.

Gated by a server-side PARALLEL_SCENE_CONCURRENCY (default 0 = off,
clamped to 10), surfaced to the client through the existing
/api/server-providers response. With it unset the content map is null
and the loop is byte-for-byte the original serial path, so out-of-box
behaviour is unchanged.

The bounded worker pool is extracted to lib/utils/concurrency.ts
(mapWithConcurrency, with an early-stop hook for abort/epoch) and
unit-tested; getParallelSceneConcurrency env parsing/clamping is tested
too.

Closes #572

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(generation): pipeline content fetches instead of a barrier

Address @wyuc's review on #660. The previous version awaited the whole
bounded content pool before Phase 2, so for N > C scenes the user saw the
first scene, then a stall on a single page for the entire content phase,
then a burst — perceived as worse than serial despite the wall-clock win
(the gap scales with ceil((N-1)/C)).

Pipeline instead: lazyBoundedMap starts the content fetches (<= C in
flight) and returns one promise per outline immediately; the serial loop
awaits them in order, so each resolves as soon as its own content is ready
(usually already done, hidden behind the previous scene's actions/TTS). The
first scene now paints after content(1)+actions(1)+TTS(1) — same as serial
— with the full concurrency benefit and no stall.

Also from the review:
- a content fetch can no longer take its siblings down: an unexpected throw
  is caught and returned as a failure result, routed through the same
  mark-failed path as the serial loop (symmetric);
- drop the cosmetic Phase-1 setCurrentGeneratingOrder — the loop already
  sets it per outline;
- note the intentional belt-and-suspenders clamp;
- concurrency.test.ts asserts <= limit (not == limit), and gains
  lazyBoundedMap tests including the no-barrier property.

mapWithConcurrency is kept as a thin await-all wrapper over lazyBoundedMap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-06-08 07:44:33 -04:00

123 lines
3.5 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

import { describe, expect, it } from 'vitest';
import { lazyBoundedMap, mapWithConcurrency } from '@/lib/utils/concurrency';
const tick = (ms = 5) => new Promise((resolve) => setTimeout(resolve, ms));
describe('mapWithConcurrency', () => {
it('returns results in input order regardless of completion order', async () => {
// Later items resolve first, but results must stay aligned with input.
const out = await mapWithConcurrency([30, 10, 20], 3, async (ms, i) => {
await tick(ms);
return i;
});
expect(out).toEqual([0, 1, 2]);
});
it('never runs more than `limit` workers at once', async () => {
let active = 0;
let peak = 0;
await mapWithConcurrency(
Array.from({ length: 9 }, (_, i) => i),
3,
async (n) => {
active += 1;
peak = Math.max(peak, active);
await tick();
active -= 1;
return n;
},
);
expect(peak).toBeLessThanOrEqual(3); // never exceeds the pool
});
it('clamps the limit to the item count (no over-spawn)', async () => {
let active = 0;
let peak = 0;
const out = await mapWithConcurrency([1, 2], 100, async (n) => {
active += 1;
peak = Math.max(peak, active);
await tick();
active -= 1;
return n;
});
expect(peak).toBeLessThanOrEqual(2);
expect(out).toEqual([1, 2]);
});
it('stops pulling new items once shouldContinue() turns false', async () => {
const processed: number[] = [];
let done = 0;
await mapWithConcurrency(
[1, 2, 3, 4, 5, 6],
1,
async (n) => {
processed.push(n);
done += 1;
return n;
},
{ shouldContinue: () => done < 3 },
);
// limit 1 + stop after 3 ⇒ items 4–6 are never started.
expect(processed).toEqual([1, 2, 3]);
});
it('handles an empty list without spawning workers', async () => {
expect(await mapWithConcurrency([], 4, async (n) => n)).toEqual([]);
});
});
describe('lazyBoundedMap', () => {
it('returns promises immediately and resolves them without a barrier', async () => {
const started: number[] = [];
const promises = lazyBoundedMap([0, 1, 2], 1, async (n) => {
started.push(n);
await tick();
return n * 10;
});
expect(promises).toHaveLength(3); // the array of promises exists synchronously
expect(await promises[0]).toBe(0); // the first resolves on its own…
expect(started.length).toBeLessThan(3); // …without forcing the last item to run (no barrier)
expect(await Promise.all(promises)).toEqual([0, 10, 20]); // order + values preserved
});
it('caps in-flight work at `limit`', async () => {
let active = 0;
let peak = 0;
await Promise.all(
lazyBoundedMap(
Array.from({ length: 8 }, (_, i) => i),
3,
async (n) => {
active += 1;
peak = Math.max(peak, active);
await tick();
active -= 1;
return n;
},
),
);
expect(peak).toBeLessThanOrEqual(3);
expect(peak).toBeGreaterThan(1);
});
it('skips items via shouldContinue without running fn', async () => {
const ran: number[] = [];
let done = 0;
const out = await Promise.all(
lazyBoundedMap(
[1, 2, 3, 4, 5],
1,
async (n) => {
ran.push(n);
done += 1;
return n;
},
{ shouldContinue: () => done < 2 },
),
);
expect(ran).toEqual([1, 2]); // fn ran only twice
expect(out).toEqual([1, 2, undefined, undefined, undefined]); // skipped → undefined
});
});