* refactor: align widget actions with scene pipeline
* feat(edit): migrate legacy teacherActions off interactive content on load
The widget-actions pipeline no longer persists a teacherActions authoring
layer; playback reads only scene.actions. Add migrateInteractiveContent to
strip the dead field from existing documents as they load through migrateScene
(wired into all store/classroom load paths). Satisfies the #794 forward-migration
acceptance item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(generation): default widget_setState.state to {} when the LLM omits it
The removed TeacherAction->Action converter always defaulted state to {}.
Restore that guard in the structured-output parser so an action missing
state can't forward state: undefined into the widget iframe, where
SET_WIDGET_STATE handlers dereference it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(edit): also strip nested widgetConfig.teacherActions in migration
Legacy WidgetConfig variants each declared their own teacherActions, so existing
documents can carry the dead field nested inside content.widgetConfig as well as
at the top level. Extend migrateInteractiveContent to drop both, preserving all
other widgetConfig fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(prompts): document stable widget-action selectors for diagram/game/viz3d
The interactive-actions prompt documented stable target selectors only for
procedural-skill and simulation widgets. Add the canonical selectors the
diagram / game / visualization3d content prompts actually emit (node IDs via
revealOrder; #zoom-in-btn / #canvas-container / #speed-slider / ...; #start-btn),
plus a rule to use widget_setState or a speech-only beat instead of guessing a
target when no stable selector is known (e.g. free-form code widgets). Hardens
against widget actions silently no-oping on non-existent selectors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Adds automatic retry with exponential backoff for scene generation (content/actions/TTS): shared withGenerationRetry helper at the fetch boundary, smart error classification (retries 429/5xx/timeouts/network, fails fast on 401/403/404), AI SDK RetryError unwrapping, abort-aware backoff, per-route thinking config preserved, and upstream HTTP status surfaced for client retry classification. Fixes#755.
Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777.
* feat(generation): infer concise courseTitle from outlines for readable course names
Generated courses were named after the raw user prompt (truncated to 500 chars
client-side, or 50 chars / first outline title server-side), which is long and
unreadable. The outline-generation step already understands the full course
semantics, so have it emit a concise `courseTitle` alongside `languageDirective`
and `outlines`, and use it as the stage name.
- Prompts (requirements-to-outlines, interactive-outlines): top-level JSON gains
a required `courseTitle` (≤30 chars, in the teaching language, noun phrase).
- Streaming route: extract `courseTitle` head-bound like `languageDirective`,
emit a `courseTitle` SSE event, and include it in the `done` event.
- Non-streaming outline-generator: parse and return optional `courseTitle`
(defensive trim + 120-char cap).
- Naming: stage.name = courseTitle || outlines[0]?.title || requirement fallback
(client preview path backfills stage.name after outlines resolve; server
classroom-generation uses it directly).
- Session: persist `courseTitle` on GenerationSessionState for reload safety.
- Tests: add courseTitle parsing coverage; e2e mock done event carries it.
Fully optional in the pipeline — every consumer falls back to the previous
behavior when the field is absent, so legacy/LLM-skipped cases are unaffected.
* fix(generation): make courseTitle propagation robust across all outline paths
Address review feedback on the courseTitle threading.
Prompts — align every outline-prompt schema statement to the 3-key shape.
The user-prompt "final reminder" / top-level shape blocks for
requirements-to-outlines, interactive-outlines, and task-engine-outlines still
demanded the old two-key {languageDirective, outlines} object; as the last
instruction the model reads, that caused it to omit courseTitle and the stage
name to fall back to the raw requirement. Task Engine prompts now emit
courseTitle as well.
Streaming route — normalize the streamed courseTitle (trim, ignore
whitespace-only, cap length) to match the non-streaming parser, and add a
full-buffer fallback so a title emitted after the outlines array or beyond the
head-scan window is still recovered before the done event.
Client — reset latched languageDirective/title on outline retry so a
succeeding attempt that omits them falls back instead of inheriting a failed
attempt's stale value; also carry courseTitle in the stream-end resolve path
that fires when the stream closes without an explicit done event.
courseTitle remains optional on every consumer path.
---------
Co-authored-by: wyuc <zdq1204@gmail.com>
* feat(model): optional per-stage LLM model routing (#745)
Add a config-only stage -> model map consulted during model resolution,
falling back to DEFAULT_MODEL when unset (zero behavior change unless opted in).
- lib/server/model-routes.ts: parse/validate/cache MODEL_ROUTES env JSON;
LLM_STAGES registry of routable stages; getStageModel(stage).
- resolveModel: resolution order x-model > stage route > DEFAULT_MODEL > builtin;
thread optional `stage` through resolveModelFromHeaders/FromRequest.
- Wire each route's resolveModel call site to its stage; classroom-generation
resolves generate-classroom and web-search-query-rewrite independently.
- .env.example: document MODEL_ROUTES; unit tests for both modules.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(model): address cross-review findings for per-stage routing (#745)
- classroom-generation: resolve the web-search-query-rewrite model lazily,
only when explicitly routed and inside the web-search branch, with try/catch
fallback to the classroom model. Fixes a regression where a misconfigured
optional route (keyless/invalid provider) threw and aborted ALL classroom
generation even with web search disabled; also avoids wasted resolution on the
common no-search path and preserves classroom-model inheritance when unrouted.
- resolve-model: type the stage param as LlmStage across resolveModel/
resolveModelFromHeaders/resolveModelFromRequest so a mistyped stage literal is
a compile error instead of silently falling through to DEFAULT_MODEL.
- model-routes: simplify the cache to a process singleton (matching
provider-config), dropping the raw-string-keyed cache and redundant STAGE_SET.
- .env.example: clarify x-model precedence and that a route to an unconfigured
provider fails at request time (no startup validation), like a bad DEFAULT_MODEL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(model): stage route takes precedence over client x-model (#745)
Cross-review (codex P2 + Claude tracer) found that the browser UI always sends
its saved model as x-model, so the previous x-model > stage order made
MODEL_ROUTES inert for normal UI traffic on exactly the heavy stages the RFC
targets (scene-content, quiz, pbl-chat, chat). Flip resolution to
stage route > x-model > DEFAULT_MODEL: a configured route is the operator's
deliberate per-stage choice and wins, while unrouted stages still honor the
client's x-model. Update docs and tests accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(model): prettier-format changed files for #745 CI
Run the repo formatter (pnpm check / prettier) on the changed routes, the new
model-routes module, and the new tests so `prettier . --check` (CI) passes;
also correct a stale precedence comment in the resolve-model test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(model): isolate routed model from client connection params (#745)
When a stage route overrides the client's x-model and points at a different
provider, the client-sent apiKey/baseUrl/providerType (for the client's model)
must not bleed onto the routed provider — otherwise a routed Anthropic model
would be constructed with the client's OpenAI providerType/key and fail. A
routed model now resolves its connection params purely from server config, as
if no x-model was sent. Unrouted stages still honor the client params. Adds
tests covering both the drop (routed) and keep (unrouted) cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(model): correct graceful-degradation comment in classroom (#745)
resolveModel does not throw on a missing key; clarify that a keyless route
surfaces later in callLLM (degraded by the outer try/catch), while only a
resolution-time failure is caught at the rewrite re-resolve. Comment-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(model): route scene-content per scene type (#745)
Extend MODEL_ROUTES with composite keys scene-content:<type> for the four core
scene types (slide/quiz/interactive/pbl). getStageModel now resolves composite
`a:b` stages most-specific-first, trimming `:` segments, so scene-content:<type>
falls back to the base scene-content route, then x-model, then DEFAULT_MODEL.
The scene-content route derives its stage from outline.type. Backward
compatible: with no composite key configured, behavior is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(model): per-stage thinking effort + drop hardcoded gpt fallback (#745)
MODEL_ROUTES values may now be {model, effort} (not just a model string) to pin
the thinking effort per stage. Arbitration mirrors model routing: a routed stage
with effort set wins over the client's thinking (effort "none" disables); routed
with no effort uses the model's default and drops client thinking; unrouted
stages keep the client thinking. resolveModel is the single arbiter — body
thinking is threaded in via resolveModelFromRequest. classroom-generation passes
the resolved thinking into its callLLM calls so generate-classroom and
web-search-query-rewrite honor route effort too.
Also: removed the hardcoded `|| 'gpt-5.4-mini'` fallback in resolveModel — if no
model resolves (no route / x-model / DEFAULT_MODEL) it now throws instead of
silently picking a vendor default.
model-routes: getStageRoute returns {model, effort?}; getStageModel delegates to
it. Docs + tests updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(model): route value carries full ThinkingConfig (#745)
Per-stage routing accepts the full unified ThinkingConfig in the route value
({model, thinking:{mode,effort,level,enabled,budgetTokens,excludeReasoningOutput}})
instead of just an effort string. The route's thinking is passed through
resolveModel and normalized per the model's capability by callLLM, so
budgetTokens (qwen), level (Gemini), enabled/mode, etc. all work.
(qwen3.7-plus/max thinking capability now comes from upstream #753, so this no
longer registers them.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(model): honor routed thinking for chat-adapter; doc fixes (#745)
Cross-review (codex P2 + Claude) found chat-adapter was the only routable stage
that ignored the route's thinking: it took the routed model but built its own
thinkingConfig from the request body, so operator-pinned thinking never applied
and client thinking still leaked onto a routed model. Now pass the client
thinking into resolveModel and use the resolved thinkingConfig (route-pinned for
a routed stage, client otherwise; defaults to disabled for low-latency chat).
Also: clarify DEFAULT_MODEL is now required for server-side stages (the
hardcoded gpt-5.4-mini fallback was removed → resolveModel throws if nothing
resolves), and fix a stale {model,effort}→{model,thinking} doc comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
proxyFetch routed every request through the configured HTTP(S) proxy,
including loopback targets. A deployment with HTTP_PROXY set would send
a http://localhost:8888 SearXNG (or any self-hosted service) request to
the external proxy, where localhost resolves to the proxy machine and
the request fails.
- Always bypass loopback hosts (localhost, *.localhost, 127.0.0.0/8, ::1).
- Honor the standard no_proxy / NO_PROXY env var: comma-separated hosts,
`*` wildcard, optional `:port` (with protocol-default ports), and
curl-style domain-suffix matching (`example.com` matches
`api.example.com`; a leading dot is equivalent).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(generation): opt-in parallel scene-content generation
Scene generation was strictly serial — content -> actions -> TTS, one
outline at a time — so an N-scene classroom paid N x (content + actions
+ TTS) of wall-clock latency, which dominates the post-outline wait.
Add an opt-in hybrid two-phase path (#572):
- Phase 1 fetches scene *content* concurrently (bounded). Content is the
only per-scene step independent of cross-scene state, so it is safe to
parallelise; a content failure marks just that outline and does not
pause the batch.
- Phase 2 keeps the existing in-order actions + TTS loop, so
previousSpeeches threading, ordered addScene, the abort/epoch guards,
and the pause-on-failure UX are all unchanged.
Gated by a server-side PARALLEL_SCENE_CONCURRENCY (default 0 = off,
clamped to 10), surfaced to the client through the existing
/api/server-providers response. With it unset the content map is null
and the loop is byte-for-byte the original serial path, so out-of-box
behaviour is unchanged.
The bounded worker pool is extracted to lib/utils/concurrency.ts
(mapWithConcurrency, with an early-stop hook for abort/epoch) and
unit-tested; getParallelSceneConcurrency env parsing/clamping is tested
too.
Closes#572
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(generation): pipeline content fetches instead of a barrier
Address @wyuc's review on #660. The previous version awaited the whole
bounded content pool before Phase 2, so for N > C scenes the user saw the
first scene, then a stall on a single page for the entire content phase,
then a burst — perceived as worse than serial despite the wall-clock win
(the gap scales with ceil((N-1)/C)).
Pipeline instead: lazyBoundedMap starts the content fetches (<= C in
flight) and returns one promise per outline immediately; the serial loop
awaits them in order, so each resolves as soon as its own content is ready
(usually already done, hidden behind the previous scene's actions/TTS). The
first scene now paints after content(1)+actions(1)+TTS(1) — same as serial
— with the full concurrency benefit and no stall.
Also from the review:
- a content fetch can no longer take its siblings down: an unexpected throw
is caught and returned as a failure result, routed through the same
mark-failed path as the serial loop (symmetric);
- drop the cosmetic Phase-1 setCurrentGeneratingOrder — the loop already
sets it per outline;
- note the intentional belt-and-suspenders clamp;
- concurrency.test.ts asserts <= limit (not == limit), and gains
lazyBoundedMap tests including the no-barrier property.
mapWithConcurrency is kept as a thin await-all wrapper over lazyBoundedMap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(tts): voxcpm voice-design types + deterministic voice id helpers (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): emit + persist per-agent voiceDesign descriptor (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): voxcpm voice registration backend client (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): voxcpm-voice ensure/register endpoint (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): client auto-voice registration + reference-clip cache (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): reference registered voice id in vLLM-Omni speech (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): thread voiceDesign + backend through tts call sites (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): match vLLM-Omni voice registration contract (#670)
Live e2e against the real backend revealed the assumed multipart contract was
wrong: POST /v1/audio/voices needs name + consent + audio_sample (not
voice_id/file), and there is no per-name GET (405) — existence must list
/v1/audio/voices and check membership.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(tts): make auto-voice register-once pattern provider-neutral (#670)
The descriptor + register-once/reference-by-id pattern is not VoxCPM-specific.
De-couple it into a provider-neutral seam so other registration-capable
providers (ElevenLabs/MiniMax/Doubao voice cloning, …) can plug in:
- lib/audio/voice-design.ts: VoiceDesign + buildVoiceDesignPrompt /
normalizeVoiceDesign / getDeterministicVoiceId (neutral 'auto-<hash>' id,
namespaced by providerId).
- lib/audio/voice-registration.ts: VoiceRegistrationAdapter interface +
providerId->adapter registry + supportsVoiceRegistration.
- lib/audio/voice-registration-client.ts: neutral ensureRegisteredVoice() +
IndexedDB clip cache.
- app/api/generate/voice (replaces .../voxcpm-voice): dispatches by providerId.
- voxcpm-registration.ts becomes the VoxCPM adapter (sole registered provider).
- AgentConfig.voiceDesign + DB cache table renamed neutral.
Behavior-preserving; VoxCPM-specific bits (inline (prompt)text, backend kinds,
capability gate) stay in the voxcpm modules. Full suite green (674).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): build the voice clip Blob from an inline Uint8Array (#670)
A factored-out helper returning Uint8Array widened to Uint8Array<ArrayBufferLike>,
which next build (stricter than bare tsc) rejects as a BlobPart. Inline the
buffer like the rest of the repo does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): managed TTS providers resolve model server-side (#670)
A server-managed TTS provider's model was still client-driven, but the
managed-provider settings UI hides the model field — so a backend whose model
id isn't the client default (e.g. VoxCPM/vLLM-Omni serving the full model path)
always 500'd. resolveTTSModel() makes the model authoritative from server
config (${PREFIX}_MODELS, first entry) for managed providers, like key/baseUrl;
unmanaged/unconfigured providers keep the client model unchanged. Used by the
tts and voice routes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): single agent-voice resolver + stable teacher narration voice (#670)
Route all TTS voice resolution (narration, discussion, preview) through one
resolveAgentVoiceOptions(agent, ...) that reads the agent profile and registers
+ references the voice by id. Eagerly warm up generated agents' voices on save.
Critically fixes the teacher narration drifting (male/female jumps): the registry
is always seeded with DEFAULT_AGENTS, so the old narration lookup
find(role==='teacher') returned the default teacher (no voiceDesign) instead of
the generated one — so narration never registered a voice and fell back to the
inline prompt. pickNarratorAgent() now prefers the teacher carrying a voiceDesign.
Also: drop language from the deterministic voice id (descriptor already encodes
it) so narration (directive) and discussion (locale) resolve the same id; log
the effective registeredVoiceId. Regression tests for pickNarratorAgent.
Verified e2e: agent-profiles -> 1 voice registration -> 16 narration TTS all
referencing the same registeredVoiceId; voice present on the backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(tts): fall back to persona as the voice seed for agents without a voiceDesign (#670)
Preset/default agents carry no LLM voiceDesign. Rather than add a static field,
derive the bootstrap descriptor from the agent's persona when voiceDesign is
absent, so they still register one stable reference voice (stable-but-generic;
persona is not a vocal spec). Generated agents keep their LLM voiceDesign.
Also hardens replay: stage snapshots drop voiceDesign but keep persona.
Unit-tested: resolveAgentVoiceOptions uses real voiceDesign when present, persona
otherwise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): match Auto Voice by its localized label in the agent voice picker (#670)
The picker filtered on voice.name ('Auto Voice'), but Auto Voice is shown via
its localized label (自动音色). Searching '自动' returned '没有匹配音色'. Match
the localized label for the Auto Voice option so it's findable in any language.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): address code-review round 1 (#670)
- strip parentheses in the voice-design prompt so a paren in the descriptor/
persona can't corrupt the (prompt)text bootstrap delimiter
- gate POST /api/generate/voice on isServerTTSProviderDisabled (#665): a
force-disabled provider was off for the tts route but not this sibling
- check voiceExists before re-registering a client-cached clip, so a cached
voice that's still live isn't needlessly re-uploaded every session
- dedup concurrent ensureRegisteredVoice calls via an in-flight promise map
(eager warm-up + first utterance no longer double bootstrap/register)
- warm up only the narrator (teacher), not every generated agent, to avoid
synthesizing voices at save time for agents that may never speak
- collapse the duplicated vLLM-Omni speech payload into one object
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(tts): prettier formatting (#670)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): key auto-voice memo by (voiceId, backend) so a base-URL change re-registers (#670)
Addresses review (cosarah): registeredThisSession/inFlight were keyed by voiceId
alone, so switching the VoxCPM base URL mid-session made ensureRegisteredVoice
short-circuit and return an id registered only on the old backend — TTS then sent
that stale registeredVoiceId and skipped the inline fallback, failing with
voice-not-found on the new backend. Memo key now includes the base URL; the
IndexedDB clip cache stays keyed by voiceId (the reference clip is
backend-independent and reused to re-register elsewhere). Regression test added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tts): include API key in the auto-voice memo key (#670)
Addresses review (cosarah, round 2): memoKeyFor was keyed by (voiceId, baseUrl)
but not the API key. Registration/existence checks and speech calls are
auth-scoped, so switching account/key on the same base URL could reuse a
registeredVoiceId from the old credentials and skip re-validation. Memo key now
includes the API key (in-memory only, never persisted/logged). Regression test
covers the key-switch case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
A single TTS provider-enablement model honored everywhere (picker,
auto-assignment, fallback, lecture/playback, server).
Model
- lib/audio/provider-enablement.ts: isTTSProviderConfigured /
isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the
registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices),
honors the per-provider `enabled` flag, server-disable overrides (server
precedence).
- voice-resolver: getEnabledProvidersWithVoices (enabled server/custom
providers) and getSelectableProvidersWithVoices (the ONE client source of
truth = enabled providers + opt-in browser-native when the browser exposes
voices). resolveAgentVoice picks deterministically among the given providers,
honors a voiceConfig only while its provider is present.
- Picker + discussion build their list from getSelectableProvidersWithVoices, so
the AgentBar and the live discussion agree on selectable voices (fixes
student agents going silent in auto mode). Students resolve via
resolveAgentVoice; the teacher uses the global lecture voice verbatim (same
voice the pre-generated lecture sends), so lecture and discussion never
diverge.
- Voice assignment vs fallback: an agent's voice is bound at generation (the
agent-profile LLM picks from availableVoices; preset uses the user's pick).
The fallback for a missing/invalid/unavailable voice is applied only at the
live TTS calls, not baked into the saved agent — so it reflects current
provider availability.
Defaults / opt-in
- Browser-native: first-class, default OFF, opt-in. Built-in providers default
enabled:true (only surface once configured).
- TTS master toggle defaults OFF; first server-sync auto-enables it when a
provider is configured (mirrors image/video).
- Persist v3->v4 migration normalizes the never-user-editable flags.
Server/admin disable (server precedence): server-providers.yml `enabled:false`
and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider
`disabled` flag on /api/server-providers; merged as serverDisabled; enforced at
the /api/generate/tts trust boundary, classroom media generation, and /api/health.
UI
- Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the
provider is unavailable, locked off (with a note) when server-disabled.
- AgentBar voice pills render disabled with a "No voice" placeholder when no
provider is enabled. The toolbar media-popover TTS tab keeps its existing
on/off toggle (no picker/CTA).
Tests: predicates, resolver determinism + unified selectable list, server
force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified
in a real browser (Playwright): settings toggle states + a full Gemini classroom
generation whose TTS requests used the picked voice.
Found on main @ 2586cb5.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
TTS providers throw a plain `Error` on any non-OK response, and the
`/api/generate/tts` route catches everything as `GENERATION_FAILED` (500),
so an upstream 429 (quota/concurrency) was indistinguishable from a real
failure — clients had no signal to retry/back off. `TTSRateLimitError`
already existed for exactly this (with a TODO noting the route collapses
everything uniformly), but only Doubao threw it and the route never mapped
it.
- Add a `RATE_LIMITED` API error code.
- Map upstream HTTP 429 to `TTSRateLimitError` at every provider's
`!response.ok` site via a shared `throwIfTtsRateLimited` helper.
- Return `429 RATE_LIMITED` from the TTS route when a `TTSRateLimitError`
is caught (all other errors unchanged).
Purely additive: non-429 responses keep their existing error messages.
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* refactor(providers): make server-configured providers admin-managed, not client-overridable
A server-configured provider is now authoritative: the operator owns its API
key and base URL, and the client cannot override either. This collapses the
tri-state (server key / client base URL / client key) that was the root of a
recurring "missing API key" bug class — when the settings UI echoed a base URL
with an empty key, routes that treated "any base URL ⇒ fully client-configured"
dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl;
this removes the guard entirely by fixing the design.
Server:
- resolve*ApiKey/BaseUrl ignore client values when the provider is server-
configured; add isServerConfiguredProvider(section, id); drop
canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl.
- getServer*Providers() no longer return base URLs — only allowed models
(LLM/image) and the managed flag (presence) — so an internal gateway/proxy
endpoint is never exposed to the client.
- routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers
purely server-side; SSRF validation now applies only to unmanaged (client)
base URLs.
Client:
- drop serverBaseUrl from ProviderSettings, the settings store, request-building
(TTS base URL no longer echoes the server URL) and voice resolution.
- settings UI renders managed providers read-only: the key/base-URL override
inputs and region toggles are hidden, leaving the "managed by server" notice.
Tradeoff: overriding a server provider's key with your own for the same provider
is removed; add a separate custom provider instead. Closes#620.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e
- serverConfiguredNotice now says the provider is managed and not editable here,
pointing users to add a separate provider for their own credentials (7 locales).
- delete components/settings/audio-settings.tsx — unreferenced dead code.
- add e2e/tests/managed-provider-620.spec.ts: managed provider hides the
key/base-URL inputs; unmanaged provider keeps them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(providers): lock the model catalog for managed providers with a pinned list
When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels),
the model catalog is admin-managed too: the LLM provider panel shows the pinned
models read-only with a "Server" badge and no add / reset / edit / delete
affordances. Without a pinned list the server manages only credentials and the
user keeps curating models.
This removes the prior inconsistency where the settings panel let you add models
to a pinned managed provider that the generation toolbar then filtered out.
e2e: the managed-openai case now also asserts the catalog is locked.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl
Addresses review feedback on #624:
1. Managed web-search providers now ignore a client-sent key/baseUrl instead of
validating and rejecting it. A server-configured provider is admin-owned, so
the server config is authoritative and a stale client base URL is dropped
rather than 400-ing the request. The allowlist validation still runs for
unmanaged providers, where the client base URL is actually used.
2. Drop the removed `serverBaseUrl` field from persisted localStorage on
rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so
old clients don't keep a stale server URL in client state.
Tests: web-search route test split into unmanaged-rejects-allowlist and
managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy
serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
* Fix server API key fallback for provider base URLs
* Format verify PDF provider route
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* add Azure ASR config and localization
* add transcribeAzureASR func
* fix(settings): prevent Azure STT crash in ASR modal
* feat(i18n): add Azure STT provider support in language locales
Add Alibaba Cloud DashScope HappyHorse text-to-video support as an OpenMAIC video generation provider.
Co-authored-by: xuruiray <15962843+xuruiray@users.noreply.github.com>
Summary:
- Enable the OpenAI image provider from standard OpenAI server env vars when image-specific env vars are absent.
- Add provider-config coverage for resolving the fallback API key and base URL.
Rationale:
- Allows gpt-image-2 image generation to work with common OpenAI proxy configuration.
Tests:
- pnpm vitest run tests/media/openai-image-adapter.test.ts tests/server/provider-config.test.ts
- pnpm prettier lib/server/provider-config.ts tests/server/provider-config.test.ts --check
- pnpm lint -- lib/server/provider-config.ts tests/server/provider-config.test.ts
Co-authored-by: Codex <codex@openai.com>
* fix(generation): unify languageDirective across outline + scene pipeline (#472)
Root issue: languageDirective was dropped, renamed, or hardcoded at several
points in the scene-generation pipeline, silently degrading prompt-level
language control. Slide paths survived because their fallback happened to
hint requirement-language; widget paths degraded to zh-CN default.
Fix plan (all code-only threading + prompt hygiene, no schema changes):
1. Thread languageDirective through every buildPrompt in scene-generator.ts:
- Add param to generateSlideContent / generateQuizContent / generatePBLSceneContent
- Forward from generateSceneContent's slide / quiz / pbl branches
- Include key in the four generateSceneActions buildPrompt calls
- Demote generatePBLSceneContent's hardcoded English to a fallback
- Drop the dishonest `as 'zh-CN' | 'en-US'` cast on the widget path
2. Unify variable naming across widget templates + callers:
- Rename {{language}} → {{languageDirective}} in 6 widget templates
(simulation / diagram / code / game / visualization3d / widget-teacher-actions)
- generateWidgetContent and generateWidgetTeacherActions now pass
`languageDirective: languageDirective || ''` instead of a locale-code shim
3. Make the outline LLM reliably emit the wrapper object:
- requirements-to-outlines/system.md + user.md restructured so the
"top-level JSON object with languageDirective + outlines" contract is
surfaced up-front, short, and unambiguous — drowning-by-scene-examples
removed. Important Reminders reordered: top-level shape first.
- interactive-outlines/system.md + user.md aligned with main outliner:
adds full Language Inference section, requires wrapper output, drops
the orphan {{language}} placeholder that never had a value source.
4. Remove lib/prompts/loader.ts prompt/snippet caches:
- Disk reads for prompt templates are not on any hot path (LLM call
dominates by 5 orders of magnitude). The cache's only observable
effect was blocking prompt edits from taking effect without restart
— which masked the wrapper bug during this investigation.
Verification:
- 7 new regression tests in tests/generation cover directive threading at
each buildPrompt site plus the PBL caller-forwarding guarantee.
- outline-language eval (50 scenarios, gemini-3-flash-preview inference,
qwen-3.5-flash judge): 48/50 (96%). The 2 persistent failures are in
the proxy-request category (teacher / tutor preparing lessons for
students) where the system prompt's rule 5 has genuine ambiguity —
orthogonal to this fix.
- Curl burst tests: 8/8 runs each for regular-mode and interactive-mode
outline endpoints emit the wrapper object and extract a detailed
inferred directive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: apply prettier formatting to loader.ts and new test file
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: thread languageDirective in scene-builder and classroom-generation
Code review caught two additional sites missing the directive:
- lib/server/classroom-generation.ts:371 — generateSceneActions call
was missing languageDirective, reproducing the issue #472 symptom
in the shareable-classroom server generation path.
- lib/generation/scene-builder.ts:114 — buildSceneFromOutline's
generateSceneActions call had the same miss; latent (no current
live caller) but wrong for consistency.
Also renumber duplicate `8.` in requirements-to-outlines/system.md
scene-level rules.
Extend regression tests with a buildSceneFromOutline end-to-end case
that captures both the content and actions prompts via a stub aiCall
and asserts the directive reaches both phases — this would have
caught the scene-builder miss.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* review: widget-path test + docs + fallback constant (PR #474)
Address @cosarah's review feedback:
- Add widget-path regression test (tests/generation/...): interactive
scene with widgetType=simulation captures both widget content and
widget-teacher-actions prompts, asserts the directive reaches each
and no `{{language}}` / `{{languageDirective}}` placeholder leaks.
This is exactly the path issue #472 was filed for.
- Rewrite lib/prompts/README.md "Caching" section: the cache was
removed in this PR; README still instructed callers to use a
clearPromptCache() function that no longer exists.
- Sync interactive-outlines/user.md with the main outliner's
"## Language Context" reminder block (line 13-19) so both outline
paths give the LLM the same language-inference reminders.
- Hoist 'Teach in the language that matches the user requirement.'
into DEFAULT_LANGUAGE_DIRECTIVE, exported from outline-generator.
Replaces 4 copies across outline-generator, scene-generator, and
the scene-outlines-stream route.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(prompts): move loader to lib/prompts/
Move file-based prompt loader from lib/generation/prompts/ to
project-level lib/prompts/ so it can be shared by orchestration
and PBL prompts in subsequent commits.
- Templates and snippets moved alongside the loader
- getPromptsDir() now points at lib/prompts/
- Generation pipeline imports updated to @/lib/prompts
(outline-generator, scene-generator, search-query-builder,
scene-outlines-stream route)
- Add tests/prompts/loader.test.ts as sanity gate
Pure file move — no behavior change.
* test(orchestration): add baseline snapshots for prompt builders
Snapshot tests cover variant matrix:
- buildStructuredPrompt: role × scene-type × peer/ledger/discussion/profile
- convertMessagesToOpenAI: mixed message kinds
- summarizeConversation: truncation
- buildDirectorPrompt: Q&A vs discussion, ledger, profile
- buildPBLSystemPrompt: default config
These lock current output so the body refactor in subsequent
commits can be verified byte-equal.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(orchestration): plug snapshot coverage gaps from review
- Split convertMessagesToOpenAI test: same-agent (existing) +
cross-agent (new). The original test passed currentAgentId
matching the message's agentId, so the cross-agent role-conversion
branch was never exercised.
- Add 'teacher / quiz scene' snapshot to lock the spotlight/laser
strip path in getEffectiveActions. The previous variant-7 test
(whiteboard-open) only triggered the mutual-exclusion warning,
not the strip — renamed it to reflect what it actually tests.
- Add assistant-role variant (was uncovered: ROLE_GUIDELINES,
buildLengthGuidelines, buildWhiteboardGuidelines all branch on
role).
* refactor(orchestration): extract summarizers from prompt-builder.ts
Move state-context, virtual whiteboard ledger replay, peer context,
message converter, and conversation summary out of the 890-line
prompt-builder.ts into focused modules under lib/orchestration/summarizers/.
prompt-builder.ts now only owns prompt assembly. director-graph
imports updated to the new locations.
Snapshot tests pass byte-equal — no behavior change.
* refactor(orchestration): move agent-system prompt to template file
The buildStructuredPrompt template literal is now
lib/prompts/templates/agent-system/system.md, assembled by a
thin variable-pass through the shared prompt loader.
Per-variant ternaries (format example, ordering, spotlight
examples, mutual-exclusion note) are kept as module-level
constants in prompt-builder.ts. ROLE_GUIDELINES,
buildLengthGuidelines, and buildWhiteboardGuidelines stay too
(they may further migrate to template partials in Phase 2).
Also: PROMPT_IDS gains a 'satisfies Record<string, PromptId>'
clause to prevent constant/union drift as more IDs are added.
Snapshot tests (12 prompt-builder + 4 director + 1 pbl) pass
byte-equal.
* refactor(prompts): align agent-system template with project conventions
- Rename agent-system placeholders snake_case → camelCase to match
the convention already used by slide-actions/user.md and other
generation templates. Settles convention before Task 5 lands two
more templates that would otherwise inherit drift.
- Extract '## Speech Guidelines (CRITICAL)' into
lib/prompts/snippets/speech-guidelines.md for reuse by director
and PBL prompts in Task 5.
Snapshot tests pass byte-equal — placeholder name changes are
invisible in rendered output, and snippet inclusion happens at
load time.
* refactor(orchestration,pbl): move director + PBL prompts to templates
director-prompt.ts and pbl-system-prompt.ts now use the shared
template loader. Bodies collapse to thin variable assembly.
Three orchestration-tier prompts now share one infrastructure:
agent-system, director, pbl-design.
Snapshot tests pass byte-equal.
* refactor(orchestration): extract WhiteboardActionRecord + AgentTurnSummary to types module
These two interfaces are imported by 6+ modules including
summarizers/ — having them live in director-prompt.ts created
an awkward upward import direction (summarizers reaching back
to a sibling prompt builder for types).
Move them to lib/orchestration/types.ts and update all callers.
director-prompt.ts now imports them too.
Pure type-location refactor — snapshots pass byte-equal.
* test: remove snapshot tests in favor of eval-as-integration-test
Per review feedback: byte-equal snapshots were the wrong invariant
for this refactor. The goal was a stable substrate for editing
prompts, not a frozen contract. Snapshots would have generated
maintenance churn (~100-200 line diffs per intentional Phase 2
prompt tweak) without providing value the eval doesn't already.
Test posture going forward:
- tests/prompts/loader.test.ts (3 tests) — bounds the loader
infrastructure (template load, snippet inclusion, variable
interpolation, missing-id behavior)
- pnpm eval:whiteboard — bounds end-to-end agent loop, template
composition, and integration with chat/director/state-manager
Removed: 1879 lines of .snap + 320 lines of test scaffolding +
fixtures. Net PR diff drops by ~2200 lines.
* docs(prompts): add README + structural assertion tests (review feedback)
Addresses two items flagged by PR #459 final review:
1. lib/prompts/README.md — conventions, syntax, gotchas, and local-testing
recipe for template authors. Previously there was no doc for the stated
goal of "non-engineers can edit prompts."
2. tests/prompts/templates.test.ts — 8 structural assertions covering:
- no surviving {{...}} placeholders in any rendered template
- role dispatch (teacher → LEAD TEACHER, student → not)
- scene-type action stripping (quiz scene has no spotlight/laser)
- director prompt output spec mentions next_agent
These replace the removed byte-equal snapshot suite at much lower
maintenance cost — they assert behaviors the refactor must preserve,
not exact bytes.
* chore(prompts): relocate templates added on main during rebase
PR #461 (interactive mode clean) added 7 new templates under
lib/generation/prompts/templates/ (the pre-refactor location).
Rebase onto origin/main landed them at the old path; git's rename
detection didn't forward them through the directory move in db56c40.
Manually move them to lib/prompts/templates/ to match the new
convention, and fix scene-generator.ts's relative import
('./prompts/types' → '@/lib/prompts/types') since the dir no
longer exists as a sibling.
No behavior change — templates read from PROMPT_IDS values via
getPromptsDir() which already points at lib/prompts/.
* refactor(prompts): address review feedback from @cosarah
- loadSnippet now throws on missing file instead of silently returning
a literal {{snippet:id}} string. A typo like {{snippet:speach-guidelines}}
now fails at load time instead of reaching the LLM.
- README gains a "Still in TypeScript" section listing the role-conditional
content that still lives in prompt-builder.ts (ROLE_GUIDELINES,
buildLengthGuidelines, buildWhiteboardGuidelines) so contributors
expecting a pure-markdown workflow know where to look.
- Expand tests/prompts/templates.test.ts to cover the conditional branches
Phase 2 is most likely to touch (9 new assertions):
- assistant role dispatch
- peer-context section toggles on agentResponses presence
- language constraint toggles on stage.languageDirective presence
- director Q&A vs discussion mode branching
- pbl-design {{issueCount}} substituted at all 3 occurrences
- placeholder-naming-convention lint scans every template for
non-camelCase placeholders (slide-content grandfathered)
- Comment above interpolateVariables regex documents why kebab-case
placeholders pass through silently (the lint test now catches them).
- New test in loader.test.ts locks the throw-on-missing-snippet behavior.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(pdf): add MinerU Cloud API as PDF parsing provider
Add MinerU Cloud (v4 API) as a new PDF provider alongside unpdf (built-in)
and MinerU (self-hosted). Users can now parse PDFs via the cloud API without
deploying a self-hosted MinerU instance.
- New provider `mineru-cloud` with API key auth and optional base URL
- Cloud flow: batch create → presigned upload → poll → ZIP download → parse
- Extract shared `mineru-parser.ts` from inline code (used by both paths)
- Settings UI adapted for cloud (API key required) vs self-hosted (base URL required)
- Server-side env var support: PDF_MINERU_CLOUD_API_KEY / PDF_MINERU_CLOUD_BASE_URL
- SSRF protection on cloud verification endpoint
- i18n translations for en-US, zh-CN, ja-JP, ru-RU
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(provider-config): revert PDF activation to requiresBaseUrl
Reverts the PDF section from keylessProviders back to requiresBaseUrl: true
to fix failing test: mineru with only apiKey (no baseUrl) should not activate.
mineru-cloud API key can be configured via browser settings UI instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(pdf-settings): enable test button when API key is server-configured
The test connection button was disabled when the API key was only
configured server-side (not entered by user in browser). Now checks
isServerConfigured in addition to user-entered values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: inline language inference in outline generation
Replace manual language selection with automatic inference from user
requirement text. The outline generator now produces a languageDirective
that propagates through the entire generation pipeline.
Key changes:
- Rewrite Language Inference section in outline prompt with decision
rules for foreign language learning, cross-language PDF, proxy
requests, and terminology handling
- Reorder pipeline: outlines before agents, so agent profiles can
use the inferred languageDirective
- Pass languageDirective through scene content/actions generation
- Add SSE streaming of languageDirective from outline generation
- Remove manual language selection UI
Eval test suite:
- 50 test cases covering 9 scenario types: single-language, language
learning, immersive, explicit instruction, code-switching, minimal
input, user profiles, cross-language PDF, locale mismatch
- LLM-as-judge evaluation with configurable inference/judge models
- 50/50 pass rate with gemini-3-flash-preview + gpt-4o judge
- Excluded from CI (requires LLM API keys)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove useless attributes in test case
* feat: pass languageDirective to PBL, remove hardcoded zh-CN/en-US
- Remove pblConfig.language field, use languageDirective from outline
inference instead
- Delete all Chinese prompt duplicates in pbl-system-prompt.ts,
agent-templates.ts; keep English templates with languageDirective
injection
- Remove zh-CN/en-US branches in generate-pbl.ts (initial prompt,
post-process context, welcome message)
- Pipe languageDirective through scene-generator → generatePBLContent
→ IssueboardMCP → agent templates
- Remove i18n template wrapping from PBL welcome messages in
pbl-renderer.tsx and use-pbl-chat.ts; use LLM-generated content
directly to avoid language mismatch when UI locale ≠ course language
- Update pblConfig schema in outline prompt (remove language field)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: clean up legacy language fields, pass languageDirective through buildSceneFromOutline
- Add languageDirective param to buildSceneFromOutline, combining with
outline.languageNote via buildLanguageText
- Remove dead types: AudienceProfile, StylePreferences, LegacyUserRequirements,
SceneOutline.language
- Remove Stage.language and all its propagation (storage, stageInfo, prompt-builder fallback)
- Remove GenerateClassroomInput.language and job store inputSummary.language
- Remove unused pdfLanguageSample from outline SSE route
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: persist languageDirective to IndexedDB, fix TTS preview and dead refs
- Add languageDirective to StageRecord and stage-storage write path
- Add DB v9 migration: convert legacy language locale codes to directives
- Use voice.language for TTS preview text instead of dead localStorage key
- Omit empty ## Language section in PBL agent prompts
- Use JSON.parse for extractLanguageDirective unescape (\uXXXX support)
- Remove dead i18n keys: toolbar.languageHint, pbl.chat.welcomeMessage
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(audio): extend TTSProviderId/ASRProviderId to support custom providers
* feat(audio): update provider registry helpers for custom provider support
* feat(audio): route custom providers to OpenAI-compatible implementations
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(audio): add custom TTS/ASR provider CRUD to settings store
* feat(audio): add dialog for creating custom audio providers
* feat(audio): add custom provider UI with voice management and delete
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(i18n): add custom audio provider translation strings
* fix(audio): resolve remaining type errors for custom provider support
Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now
include template literal types) index into the built-in-only registries
TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(audio): allow custom provider IDs to reach switch default branch
The early guard `if (!provider) throw` blocked custom provider IDs from
reaching the switch-case default branch where they get routed to
OpenAI-compatible implementations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(audio): add default model field to dialog and polish voice list UI
- Add "Default Model" input to custom audio provider dialog (fixes
Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts")
- Redesign voice list as aligned table with column headers, hover
reveal delete buttons, and empty state message
- Add Enter key support to voice add inputs
- Use consistent grid layout for add voice row
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider
The baseUrl field in provider config is empty by default (it's an
override field). Custom providers store their default URL in
customDefaultBaseUrl, which needs to be used as fallback for test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): auto-select first voice when adding to custom TTS provider
When a custom provider's voice is 'default' (initial state before any
voices are added), adding the first voice now automatically updates the
active voice selection so test TTS works immediately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): fallback to customDefaultBaseUrl in all TTS call sites
Scene generator, discussion TTS, and agent bar voice preview all read
providerConfig.baseUrl which is empty for custom providers (the default
URL is stored in customDefaultBaseUrl). Add fallback chain to all 4
call sites so custom providers work during actual playback, not just
in the settings test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): include custom provider voices in agent voice picker
getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS,
so custom providers never appeared in the agent bar voice dropdown.
Now also iterates ttsProvidersConfig for custom-tts-* entries and
includes their customVoices in the picker.
Also updated getServerVoiceList, findVoiceDisplayName, and
resolveAgentVoice to handle custom provider voice lookups.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): fallback to customDefaultBaseUrl in all ASR call sites
Same issue as TTS: audio-recorder, asr-settings, and audio-settings
all read providerConfig.baseUrl which is empty for custom providers.
Add customDefaultBaseUrl fallback to all 3 ASR call sites.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(audio): add model list management for custom ASR providers
Custom ASR providers now support adding/removing models with a table
UI (same design as TTS voice list). The model selector dropdown shows
custom models and auto-selects the first one.
Also adds custom ASR providers to the Toolbar's ASR provider dropdown
and fixes custom provider handling in audio-settings.tsx (baseUrl
fallback, request URL preview, API key section visibility).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): seed customModels from defaultModel and fix ASR model label
- When creating custom ASR provider with a defaultModel, auto-add it
to customModels so the model list isn't empty on first visit
- Fix ASR settings label from "TTS Model" to "Default Model"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): separate ASR model management from creation dialog
ASR models should be managed in provider settings (add/remove from
list, select default), not pre-filled at creation time. Unlike TTS
where a single default model suffices, ASR users need to manage
multiple models (e.g. whisper-small, whisper-large-v3).
- Remove Default Model field from ASR creation dialog (keep for TTS)
- Remove defaultModel parameter from addCustomASRProvider
- ASR models are added via the model list UI in provider settings
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): simplify ASR model management — first model is default
Remove the separate model selector dropdown for custom ASR providers.
The first model in the list is always used as default. Users manage
the model list only (add/remove), no separate "default" selection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): add loading state and detailed errors to ASR test
- Show spinner with "Processing..." while waiting for transcription
- Disable record button during processing
- Show specific error messages from API (e.g. "No transcription
generated") instead of generic failure text
- Handle empty transcription result as an error with helpful message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(i18n): use correct key for processing label in ASR test button
* fix(audio): polish custom provider UX and type safety
- Add proper type declarations for custom provider fields (customName,
customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts
- Show custom ASR providers in toolbar with default language list
- Hide custom ASR providers without models from toolbar selection
- Add no-models warning in ASR settings panel
- Add delete confirmation dialog for custom TTS/ASR providers
- Add duplicate voice/model ID validation with toast error
- Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel)
- Fix i18n key for ASR processing label (settings.asrProcessing)
- Add radio-button model selection in custom ASR provider settings
- Add visible OpenAI-compatible description in provider creation dialog
- Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock
The settings-server-sync test mocked @/lib/audio/types as an empty object,
but settings.ts now imports isCustomASRProvider for provider validation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites
When a custom provider is created, the actual URL is stored in
customDefaultBaseUrl while baseUrl is initialized empty. Several call
sites only checked baseUrl, causing fetch failures during course
generation. Add fallback to customDefaultBaseUrl in:
- generation-preview/page.tsx (course generation TTS)
- tts-providers.ts (getCurrentTTSConfig)
- asr-providers.ts (getCurrentASRConfig)
- media-popover.tsx (TTS preview)
- tts-config-popover.tsx (TTS preview)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: format generation-preview/page.tsx
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove unused ttsSpeedRange variable in media-popover
Flagged by CodeQL code quality check.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(#353,#354): conditionally set generatedAgentConfigs based on agentMode
#353: Only embed generatedAgentConfigs in the Stage when agentMode is
'generate'. For default mode, set agentIds instead. This prevents
default agents from being incorrectly marked as isGenerated on the
client-side registry.
Also reset agentMode to 'default' in the catch block when LLM agent
generation fails and falls back to defaults.
#354: Remove redundant setSelectedAgentIds call from Path 1 (server
hydration). Path 2 handles agent restoration uniformly for both
IndexedDB and server-loaded classrooms.
Includes unit tests covering all three scenarios (default, generate,
and LLM fallback).
* fix: replace `as any` with typed interfaces to satisfy eslint no-explicit-any
---------
- Make validateUrlForSSRF async; resolve hostnames via dns.promises.lookup
with { all: true, verbatim: true } and reject if any address is private
- Add isPrivateIP helper covering IPv4 private ranges, IPv6 loopback/ULA/
link-local, IPv4-mapped IPv6, deprecated fec0::/10 site-local, 6to4
tunnel (2002::/16), and Teredo tunnel (2001:0000::/32)
- Detect IP literals via net.isIP() and validate directly without DNS
- Fail closed on DNS resolution errors
- Propagate async to resolveModel, resolveModelFromHeaders, and all 20+
route-level callers; preserve existing production-only gating
- Add 14 unit tests covering public pass-through, private literals, DNS
rebinding, mixed DNS answers, tunnel addresses, and DNS failure
Closes#378
* feat: Ollama local LLM support
- Ollama as a built-in provider (OpenAI-compatible, no API key needed)
- 8 default models: llama3.3, llama3.2, qwen2.5, qwen2.5:32b, mistral, gemma3, deepseek-r1, phi4
- Keyless provider activation: providers with only a baseUrl (no API key) now activate correctly
- isProviderKeyRequired() helper for consistent key validation across routes
- i18n strings for Ollama (zh-CN + en-US) and missing Doubao strings
- SSRF guard documentation clarifying server-configured URLs bypass safely
Previously, clients could send `x-requires-api-key: false` (header) or
`requiresApiKey: false` (body) to skip API key validation for ANY provider,
including ones like OpenAI that require keys. This meant a client could
consume server-configured credentials without authorization.
Fix: the server now derives requiresApiKey exclusively from its own provider
registry. The client header and body field are removed from all server-side
parsing. For Ollama, the registry says `requiresApiKey: false` so it works
keyless. For OpenAI, the registry says `true` so the check is enforced
regardless of what a client sends.
* fix: address reviewer feedback on Ollama integration
- Fix env-var activation so OLLAMA_BASE_URL alone activates the provider
- Move SSRF bypass comment from ssrf-guard.ts to resolve-model.ts
- Add ollama and doubao provider names to i18n locale files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fixed issues from reviewer notes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: format provider-config.ts with Prettier
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Self-hosted users need to access local network services (Ollama, local
TTS/ASR endpoints, etc.) but the SSRF guard blocks all private addresses.
Add ALLOW_LOCAL_NETWORKS env var that skips private-IP checks when set
to "true" or "1", while still validating URL format and protocol.
Fixes#310Fixes#311
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
* fix: persist agent configs in server-generated classrooms (#316)
Server-generated classrooms via /api/generate-classroom silently skip all discussion actions because agent configurations are never embedded in the persisted Stage object.
Changes:
- Add optional generatedAgentConfigs field to Stage interface
- Inject agent configs (with avatar/color/priority) into Stage during server-side generation
- Hydrate agents into IndexedDB/registry when loading from server storage on client
Closes#316
* fix: extract shared agent constants and fix avatar paths
- Extract AGENT_COLOR_PALETTE and AGENT_DEFAULT_AVATARS into
lib/constants/agent-defaults.ts to eliminate duplication
- Fix invalid avatar paths (student.png -> assist.png, etc.)
to match actual files in public/avatars/
- Remove duplicated inline constants from classroom-generation.ts
and agent-profiles/route.ts
Addresses reviewer feedback on #336
---------
* fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#324)
- Move model selection from global ttsModelId to per-provider
ttsProvidersConfig[providerId].modelId
- Remove supportsModelSelection flag, use models.length > 0 instead
- Add defaultModelId to TTSProviderConfig and ASRProviderConfig
- Add providerOptions field to TTSModelConfig for future extensibility
- Fix ElevenLabs hardcoded model_id: 'eleven_multilingual_v2'
- Fix MiniMax TTS using config.model instead of config.modelId
- Update model lists from provider docs:
- Qwen: add qwen3-tts-instruct-flash, qwen-tts
- ElevenLabs: add eleven_flash_v2_5, eleven_flash_v2
- MiniMax: remove non-existent speech-2.8 series
- Replace custom model CRUD UI with simple Select dropdown
- Remove MiniMax-specific hardcoded model selector from audio-settings
- Add migration for old ttsModelId/asrModelId/model fields
* feat(tts): voice picker groups by provider:model, sets both model and voice
Voice selection in agent bar now groups voices by model. Picking a
voice from a model group sets both modelId and voiceId on the agent.
TTS settings shows available models as display-only chips.
- Add compatibleModels field to TTSVoiceInfo (OpenAI marin/cedar restricted to gpt-4o-mini-tts)
- Add modelId to agent voiceConfig and ResolvedVoice
- Add ModelVoiceGroup to ProviderWithVoices, build groups in getAvailableProvidersWithVoices
- Update AgentVoicePill and TeacherVoicePill to iterate modelGroups
- Thread modelId from voice selection through discussion TTS pipeline
- Replace model Select with display-only chips in tts-settings
* fix(tts): show provider name in voice picker group headers
* fix(tts): restore MiniMax speech-01 models per official API docs
* fix(tts): add MiniMax speech-2.8 models per official docs, default to speech-2.8-hd
* feat(tts): expand MiniMax voice list with recommended voices
* fix(tts): remove non-official MiniMax recommended voices, keep 10 from official list
* fix: address CR findings - align defaults, fix test mocks, ASR modelId fallback
* fix(tts): pass modelId in voice preview and classroom generation
* fix(tts): voice picker highlights only the exact model:voice match
---------
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Add MiniMax LLM models (M2–M2.7), TTS provider (speech-2.8/2.6/02/01),
image generation (image-01), and video generation (Hailuo 2.3/02,
T2V-01). Unify base URLs to api.minimaxi.com and add MiniMax-specific
base URL normalization for the Anthropic-compatible endpoint.
Co-authored-by: Hi-Jiajun <3138947285@qq.com>
* feat(tts): add Doubao TTS 2.0 (Volcengine Seed-TTS 2.0) provider
Add Doubao TTS as a new TTS provider with 17 voices (14 Chinese, 3 English),
streaming response parsing for Volcengine's chunked JSON format, rate-limit
error handling, and full integration across types, constants, provider
implementation, server config, settings store, i18n, and UI components.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ui(tts): split Doubao API key into separate App ID and Access Key fields
Show two input fields (App ID + Access Key) for Doubao TTS in settings,
instead of requiring the compound "appId:accessKey" format. The values
are combined internally so no changes are needed to the API layer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): address code review feedback for Doubao TTS provider
- Add TODO note explaining TTSRateLimitError's future use for retry/backoff
- Replace atob() + manual byte loop with Buffer.from() for server-side code
- Trim supportedFormats to ['mp3'] since only mp3 is used in requests
- Add i18n keys for Doubao App ID / Access Key labels (zh-CN + en-US)
- Add show/hide toggle to Access Key input field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add failing unit tests for isProviderUsable, validateProvider, validateModel
* test: add failing integration tests for all provider stale selection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: implement isProviderUsable, validateProvider, validateModel
* feat: validate all provider selections after server sync (#263)
Replace inline LLM-only validation from PR #232 with unified validation
for all provider types (LLM, TTS, ASR, PDF, Image, Video).
- Extract isProviderUsable, validateProvider, validateModel into
lib/store/settings-validation.ts
- Usability checks server config and client API key, not requiresApiKey
- Auto-select fallback to usable providers instead of clearing to empty
- TTS/ASR/PDF fall back to local defaults (browser-native, unpdf)
- Image/Video clear to empty when no server provider available
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove setup-needed prompt from header
Server-configured providers auto-select on first load, making the
manual configuration prompt unnecessary.
* fix: disable image/video generation when no provider is usable
Auto-close imageGenerationEnabled/videoGenerationEnabled when server
removes all providers. Prevent re-enabling until a provider becomes
available.
* fix: auto-recover image/video provider and enable on server restore
When server adds image/video providers after empty state, auto-select
the first available provider+model and enable generation. When server
removes all providers, disable generation and prevent re-enabling.
* fix: auto-enable image/video and select model when server has provider
Fix cases where provider matches default but generation stays disabled,
and where model is empty after provider recovery. Split provider and
model spreads so model updates independently of provider changes.
* fix: allow baseUrl-only server config for PDF providers (e.g. mineru)
loadEnvSection now accepts allowBaseUrlOnly option. PDF providers like
mineru that only need a baseUrl (no API key) are now correctly included
in the server provider response.
* fix: remove auto-enable logic, clean up dead CSS
Only auto-disable image/video generation when no provider is usable.
Never auto-enable — user controls when to turn it on. First-run
auto-enable via autoConfigApplied guard is preserved.
Also remove dead setup-needed CSS animations from globals.css.
* style: fix prettier formatting
* fix: unify fallback strategy to include client-key providers for all types
All provider types now fall back to client-key-only providers (not just
server-configured ones), matching the LLM fallback strategy. Extracted
buildFallback helper to reduce duplication. Added spread precedence
comment per review feedback.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use resolved API key in chat route instead of client-sent key (#220)
PR #114 refactored the chat route to use resolveModel() but replaced the
server-resolved API key with the raw client-sent body.apiKey for both the
401 check and the statelessGenerate call. This broke server-configured
providers (e.g. OPENAI_API_KEY env var) where the client sends no key.
- Add apiKey field to ResolvedModel interface so callers can access the
effective key after server-side fallback resolution
- Use the resolved apiKey for the 401 guard and pass it to statelessGenerate
Fixes#220
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: allow keyless providers (e.g. local Ollama) to bypass API key check
Add requiresApiKey !== false guard so providers that don't need an API
key are not rejected by the 401 check.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add Grok (xAI) provider support for LLM, Image and Video generation
- Add Grok as LLM provider (OpenAI-compatible, 11 models including grok-4.20, grok-4, grok-3)
- Add Grok Image generation provider (grok-imagine-image, grok-imagine-image-pro)
- Add Grok Video generation provider with async polling (grok-imagine-video)
- Add env vars: GROK_*, IMAGE_GROK_*, VIDEO_GROK_*
- Add web search note: Grok search is via chat completions, not standalone API
- Update README/README-zh with Grok in supported providers
Closes#87
* fix: ensure new image/video providers are auto-added to persisted settings
Add ensureBuiltInImageProviders and ensureBuiltInVideoProviders functions
that run on every store rehydrate, mirroring the existing LLM provider
sync logic. Without this, Grok Image/Video providers would not appear
for users with existing localStorage settings.
* fix: add missing English translation for providerGrokVideo
* style: fix prettier formatting for grok video provider
* fix: remove unsupported grok multi-agent model and improve video error
* feat(generate): server-side media & TTS generation in classroom pipeline
When `enableImageGeneration`, `enableVideoGeneration`, or `enableTTS` flags
are passed to /api/generate-classroom, the server now actually generates
media files and TTS audio, persists them to disk, and replaces placeholders
in the classroom JSON with serving URLs.
- Add media serving API route (GET /api/classroom-media/[id]/[...path])
- Add server-side media generation utility (image, video, TTS)
- Wire generation phases into classroom pipeline with progress steps
- Update AudioPlayer to support URL-based playback (server-generated TTS)
- Add audioUrl field to SpeechAction type
- Extend /api/health with capabilities detection
- Pass through feature flags in generate-classroom route
- Update skill docs for optional feature configuration
Closes#49
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(generate): add agentMode support for server-side classroom generation
Allow callers to choose between built-in default agents ('default') or
LLM-generated custom agent profiles ('generate') via a new agentMode
field on GenerateClassroomInput. The resolved agents are injected into
outline, scene content, and scene action prompts via teacherContext.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(media): harden download, parallelize media gen, fix TTS concatenation
- Add timeout (2min) and max size (100MB) guard to downloadToBuffer
- Run image and video generation in parallel via Promise.all
- Only split TTS text for MP3 (frame-based); skip for WAV/OGG/AAC
whose container headers break on naive byte concatenation
- Remove unnecessary type assertion for speechAction.audioUrl
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(media): harden media serving and TTS generation
- Add realpath check to prevent symlink-based path traversal in media route
- Stream files via createReadStream instead of readFile to avoid OOM on large videos
- Truncate TTS text for non-concatenable formats (WAV/OGG/AAC) when exceeding provider limit
- Add 'researching' step to distinguish web search from outline generation progress
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): split long speech actions instead of concatenating audio bytes
Adopt the same strategy as client-side: split a long speech action into
multiple shorter actions before TTS generation, so each sub-action gets
its own audio file. This avoids the broken byte-concatenation approach
that corrupted non-MP3 formats (WAV/OGG/AAC container headers).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): extract shared TTS splitting logic into lib/audio/tts-utils
Both client-side (use-scene-generator) and server-side (classroom-media-generation)
had duplicated TTS_MAX_TEXT_LENGTH, text splitting, and action splitting logic.
Extract into a shared module to keep them in sync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove unused TTS_MAX_TEXT_LENGTH import
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
- Fix Prettier formatting on 4 files failing CI
- Fix race condition in classroom page by using setStage() and adding
try/catch around server-side fetch fallback
- Consolidate duplicate classroom storage: api/classroom/route.ts now
uses shared persistClassroom/readClassroom from classroom-storage.ts
- Add per-job mutex (withJobLock) to prevent concurrent read-modify-write
on job files
- Add stale job detection: jobs stuck in "running" for >30 min are
auto-marked as failed when read
- Validate provider API key before starting generation pipeline to
fail fast with actionable error message
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>