openmaic-custom/app/api/server-providers
ly-wang19 3ccd5da6da
feat(generation): opt-in parallel scene-content generation (#660)
* feat(generation): opt-in parallel scene-content generation

Scene generation was strictly serial — content -> actions -> TTS, one
outline at a time — so an N-scene classroom paid N x (content + actions
+ TTS) of wall-clock latency, which dominates the post-outline wait.

Add an opt-in hybrid two-phase path (#572):

- Phase 1 fetches scene *content* concurrently (bounded). Content is the
  only per-scene step independent of cross-scene state, so it is safe to
  parallelise; a content failure marks just that outline and does not
  pause the batch.
- Phase 2 keeps the existing in-order actions + TTS loop, so
  previousSpeeches threading, ordered addScene, the abort/epoch guards,
  and the pause-on-failure UX are all unchanged.

Gated by a server-side PARALLEL_SCENE_CONCURRENCY (default 0 = off,
clamped to 10), surfaced to the client through the existing
/api/server-providers response. With it unset the content map is null
and the loop is byte-for-byte the original serial path, so out-of-box
behaviour is unchanged.

The bounded worker pool is extracted to lib/utils/concurrency.ts
(mapWithConcurrency, with an early-stop hook for abort/epoch) and
unit-tested; getParallelSceneConcurrency env parsing/clamping is tested
too.

Closes #572

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(generation): pipeline content fetches instead of a barrier

Address @wyuc's review on #660. The previous version awaited the whole
bounded content pool before Phase 2, so for N > C scenes the user saw the
first scene, then a stall on a single page for the entire content phase,
then a burst — perceived as worse than serial despite the wall-clock win
(the gap scales with ceil((N-1)/C)).

Pipeline instead: lazyBoundedMap starts the content fetches (<= C in
flight) and returns one promise per outline immediately; the serial loop
awaits them in order, so each resolves as soon as its own content is ready
(usually already done, hidden behind the previous scene's actions/TTS). The
first scene now paints after content(1)+actions(1)+TTS(1) — same as serial
— with the full concurrency benefit and no stall.

Also from the review:
- a content fetch can no longer take its siblings down: an unexpected throw
  is caught and returned as a failure result, routed through the same
  mark-failed path as the serial loop (symmetric);
- drop the cosmetic Phase-1 setCurrentGeneratingOrder — the loop already
  sets it per outline;
- note the intentional belt-and-suspenders clamp;
- concurrency.test.ts asserts <= limit (not == limit), and gains
  lazyBoundedMap tests including the no-barrier property.

mapWithConcurrency is kept as a thin await-all wrapper over lazyBoundedMap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-06-08 07:44:33 -04:00
..