wyuc 9a0060e2f0
feat(audio): add custom OpenAI-compatible TTS/ASR provider support (#357) (#409)
* feat(audio): extend TTSProviderId/ASRProviderId to support custom providers

* feat(audio): update provider registry helpers for custom provider support

* feat(audio): route custom providers to OpenAI-compatible implementations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(audio): add custom TTS/ASR provider CRUD to settings store

* feat(audio): add dialog for creating custom audio providers

* feat(audio): add custom provider UI with voice management and delete

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(i18n): add custom audio provider translation strings

* fix(audio): resolve remaining type errors for custom provider support

Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now
include template literal types) index into the built-in-only registries
TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(audio): allow custom provider IDs to reach switch default branch

The early guard `if (!provider) throw` blocked custom provider IDs from
reaching the switch-case default branch where they get routed to
OpenAI-compatible implementations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(audio): add default model field to dialog and polish voice list UI

- Add "Default Model" input to custom audio provider dialog (fixes
  Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts")
- Redesign voice list as aligned table with column headers, hover
  reveal delete buttons, and empty state message
- Add Enter key support to voice add inputs
- Use consistent grid layout for add voice row

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider

The baseUrl field in provider config is empty by default (it's an
override field). Custom providers store their default URL in
customDefaultBaseUrl, which needs to be used as fallback for test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): auto-select first voice when adding to custom TTS provider

When a custom provider's voice is 'default' (initial state before any
voices are added), adding the first voice now automatically updates the
active voice selection so test TTS works immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): fallback to customDefaultBaseUrl in all TTS call sites

Scene generator, discussion TTS, and agent bar voice preview all read
providerConfig.baseUrl which is empty for custom providers (the default
URL is stored in customDefaultBaseUrl). Add fallback chain to all 4
call sites so custom providers work during actual playback, not just
in the settings test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): include custom provider voices in agent voice picker

getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS,
so custom providers never appeared in the agent bar voice dropdown.
Now also iterates ttsProvidersConfig for custom-tts-* entries and
includes their customVoices in the picker.

Also updated getServerVoiceList, findVoiceDisplayName, and
resolveAgentVoice to handle custom provider voice lookups.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): fallback to customDefaultBaseUrl in all ASR call sites

Same issue as TTS: audio-recorder, asr-settings, and audio-settings
all read providerConfig.baseUrl which is empty for custom providers.
Add customDefaultBaseUrl fallback to all 3 ASR call sites.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(audio): add model list management for custom ASR providers

Custom ASR providers now support adding/removing models with a table
UI (same design as TTS voice list). The model selector dropdown shows
custom models and auto-selects the first one.

Also adds custom ASR providers to the Toolbar's ASR provider dropdown
and fixes custom provider handling in audio-settings.tsx (baseUrl
fallback, request URL preview, API key section visibility).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): seed customModels from defaultModel and fix ASR model label

- When creating custom ASR provider with a defaultModel, auto-add it
  to customModels so the model list isn't empty on first visit
- Fix ASR settings label from "TTS Model" to "Default Model"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): separate ASR model management from creation dialog

ASR models should be managed in provider settings (add/remove from
list, select default), not pre-filled at creation time. Unlike TTS
where a single default model suffices, ASR users need to manage
multiple models (e.g. whisper-small, whisper-large-v3).

- Remove Default Model field from ASR creation dialog (keep for TTS)
- Remove defaultModel parameter from addCustomASRProvider
- ASR models are added via the model list UI in provider settings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): simplify ASR model management — first model is default

Remove the separate model selector dropdown for custom ASR providers.
The first model in the list is always used as default. Users manage
the model list only (add/remove), no separate "default" selection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): add loading state and detailed errors to ASR test

- Show spinner with "Processing..." while waiting for transcription
- Disable record button during processing
- Show specific error messages from API (e.g. "No transcription
  generated") instead of generic failure text
- Handle empty transcription result as an error with helpful message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(i18n): use correct key for processing label in ASR test button

* fix(audio): polish custom provider UX and type safety

- Add proper type declarations for custom provider fields (customName,
  customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts
- Show custom ASR providers in toolbar with default language list
- Hide custom ASR providers without models from toolbar selection
- Add no-models warning in ASR settings panel
- Add delete confirmation dialog for custom TTS/ASR providers
- Add duplicate voice/model ID validation with toast error
- Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel)
- Fix i18n key for ASR processing label (settings.asrProcessing)
- Add radio-button model selection in custom ASR provider settings
- Add visible OpenAI-compatible description in provider creation dialog
- Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock

The settings-server-sync test mocked @/lib/audio/types as an empty object,
but settings.ts now imports isCustomASRProvider for provider validation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites

When a custom provider is created, the actual URL is stored in
customDefaultBaseUrl while baseUrl is initialized empty. Several call
sites only checked baseUrl, causing fetch failures during course
generation. Add fallback to customDefaultBaseUrl in:
- generation-preview/page.tsx (course generation TTS)
- tts-providers.ts (getCurrentTTSConfig)
- asr-providers.ts (getCurrentASRConfig)
- media-popover.tsx (TTS preview)
- tts-config-popover.tsx (TTS preview)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format generation-preview/page.tsx

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused ttsSpeedRange variable in media-popover

Flagged by CodeQL code quality check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 21:49:09 +08:00
..