* feat(generation): opt-in parallel scene-content generation Scene generation was strictly serial — content -> actions -> TTS, one outline at a time — so an N-scene classroom paid N x (content + actions + TTS) of wall-clock latency, which dominates the post-outline wait. Add an opt-in hybrid two-phase path (#572): - Phase 1 fetches scene *content* concurrently (bounded). Content is the only per-scene step independent of cross-scene state, so it is safe to parallelise; a content failure marks just that outline and does not pause the batch. - Phase 2 keeps the existing in-order actions + TTS loop, so previousSpeeches threading, ordered addScene, the abort/epoch guards, and the pause-on-failure UX are all unchanged. Gated by a server-side PARALLEL_SCENE_CONCURRENCY (default 0 = off, clamped to 10), surfaced to the client through the existing /api/server-providers response. With it unset the content map is null and the loop is byte-for-byte the original serial path, so out-of-box behaviour is unchanged. The bounded worker pool is extracted to lib/utils/concurrency.ts (mapWithConcurrency, with an early-stop hook for abort/epoch) and unit-tested; getParallelSceneConcurrency env parsing/clamping is tested too. Closes #572 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(generation): pipeline content fetches instead of a barrier Address @wyuc's review on #660. The previous version awaited the whole bounded content pool before Phase 2, so for N > C scenes the user saw the first scene, then a stall on a single page for the entire content phase, then a burst — perceived as worse than serial despite the wall-clock win (the gap scales with ceil((N-1)/C)). Pipeline instead: lazyBoundedMap starts the content fetches (<= C in flight) and returns one promise per outline immediately; the serial loop awaits them in order, so each resolves as soon as its own content is ready (usually already done, hidden behind the previous scene's actions/TTS). The first scene now paints after content(1)+actions(1)+TTS(1) — same as serial — with the full concurrency benefit and no stall. Also from the review: - a content fetch can no longer take its siblings down: an unexpected throw is caught and returned as a failure result, routed through the same mark-failed path as the serial loop (symmetric); - drop the cosmetic Phase-1 setCurrentGeneratingOrder — the loop already sets it per outline; - note the intentional belt-and-suspenders clamp; - concurrency.test.ts asserts <= limit (not == limit), and gains lazyBoundedMap tests including the no-barrier property. mapWithConcurrency is kept as a thin await-all wrapper over lazyBoundedMap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
76 lines
2.6 KiB
TypeScript
76 lines
2.6 KiB
TypeScript
/** A FIFO counting semaphore: at most `size` `run()` callbacks execute at once. */
|
|
function createSemaphore(size: number) {
|
|
const max = Math.max(1, Math.floor(size));
|
|
let active = 0;
|
|
const queue: Array<() => void> = [];
|
|
|
|
const pump = () => {
|
|
while (active < max && queue.length > 0) {
|
|
active += 1;
|
|
const start = queue.shift()!;
|
|
start();
|
|
}
|
|
};
|
|
|
|
return {
|
|
run<R>(fn: () => Promise<R>): Promise<R> {
|
|
return new Promise<R>((resolve, reject) => {
|
|
queue.push(() => {
|
|
fn()
|
|
.then(resolve, reject)
|
|
.finally(() => {
|
|
active -= 1;
|
|
pump();
|
|
});
|
|
});
|
|
pump();
|
|
});
|
|
},
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Start `fn` over every item with at most `limit` calls in flight at once, and
|
|
* return one promise per item **immediately**, in input order — without awaiting
|
|
* them. Each item acquires a `limit`-sized semaphore slot before `fn` runs, so
|
|
* all the promises exist up front but only `limit` execute concurrently.
|
|
*
|
|
* This is the no-barrier primitive: the caller can `await` the promises in any
|
|
* order (e.g. sequentially) and each resolves as soon as *its* work is done,
|
|
* while later items keep running in the background. `shouldContinue` is checked
|
|
* when an item reaches the front of the queue; once it returns false, the
|
|
* remaining items resolve to `undefined` without running `fn`.
|
|
*
|
|
* `limit` is clamped to `[1, items.length]`, so a raw/too-large concurrency is
|
|
* safe to pass.
|
|
*/
|
|
export function lazyBoundedMap<T, R>(
|
|
items: readonly T[],
|
|
limit: number,
|
|
fn: (item: T, index: number) => Promise<R>,
|
|
options?: { shouldContinue?: () => boolean },
|
|
): Array<Promise<R | undefined>> {
|
|
const shouldContinue = options?.shouldContinue ?? (() => true);
|
|
const semaphore = createSemaphore(Math.min(Math.floor(limit), items.length || 1));
|
|
return items.map((item, index) =>
|
|
semaphore.run(async () => (shouldContinue() ? fn(item, index) : undefined)),
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Run `fn` over `items` with at most `limit` calls in flight at once and await
|
|
* them all (a barrier). Results are returned in input order; a slot is
|
|
* `undefined` if its item was skipped because `shouldContinue` turned false.
|
|
*
|
|
* Prefer {@link lazyBoundedMap} when you can consume results incrementally —
|
|
* this wrapper waits for every item before returning.
|
|
*/
|
|
export async function mapWithConcurrency<T, R>(
|
|
items: readonly T[],
|
|
limit: number,
|
|
fn: (item: T, index: number) => Promise<R>,
|
|
options?: { shouldContinue?: () => boolean },
|
|
): Promise<Array<R | undefined>> {
|
|
return Promise.all(lazyBoundedMap(items, limit, fn, options));
|
|
}
|