* feat(model): optional per-stage LLM model routing (#745) Add a config-only stage -> model map consulted during model resolution, falling back to DEFAULT_MODEL when unset (zero behavior change unless opted in). - lib/server/model-routes.ts: parse/validate/cache MODEL_ROUTES env JSON; LLM_STAGES registry of routable stages; getStageModel(stage). - resolveModel: resolution order x-model > stage route > DEFAULT_MODEL > builtin; thread optional `stage` through resolveModelFromHeaders/FromRequest. - Wire each route's resolveModel call site to its stage; classroom-generation resolves generate-classroom and web-search-query-rewrite independently. - .env.example: document MODEL_ROUTES; unit tests for both modules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): address cross-review findings for per-stage routing (#745) - classroom-generation: resolve the web-search-query-rewrite model lazily, only when explicitly routed and inside the web-search branch, with try/catch fallback to the classroom model. Fixes a regression where a misconfigured optional route (keyless/invalid provider) threw and aborted ALL classroom generation even with web search disabled; also avoids wasted resolution on the common no-search path and preserves classroom-model inheritance when unrouted. - resolve-model: type the stage param as LlmStage across resolveModel/ resolveModelFromHeaders/resolveModelFromRequest so a mistyped stage literal is a compile error instead of silently falling through to DEFAULT_MODEL. - model-routes: simplify the cache to a process singleton (matching provider-config), dropping the raw-string-keyed cache and redundant STAGE_SET. - .env.example: clarify x-model precedence and that a route to an unconfigured provider fails at request time (no startup validation), like a bad DEFAULT_MODEL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): stage route takes precedence over client x-model (#745) Cross-review (codex P2 + Claude tracer) found that the browser UI always sends its saved model as x-model, so the previous x-model > stage order made MODEL_ROUTES inert for normal UI traffic on exactly the heavy stages the RFC targets (scene-content, quiz, pbl-chat, chat). Flip resolution to stage route > x-model > DEFAULT_MODEL: a configured route is the operator's deliberate per-stage choice and wins, while unrouted stages still honor the client's x-model. Update docs and tests accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(model): prettier-format changed files for #745 CI Run the repo formatter (pnpm check / prettier) on the changed routes, the new model-routes module, and the new tests so `prettier . --check` (CI) passes; also correct a stale precedence comment in the resolve-model test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): isolate routed model from client connection params (#745) When a stage route overrides the client's x-model and points at a different provider, the client-sent apiKey/baseUrl/providerType (for the client's model) must not bleed onto the routed provider — otherwise a routed Anthropic model would be constructed with the client's OpenAI providerType/key and fail. A routed model now resolves its connection params purely from server config, as if no x-model was sent. Unrouted stages still honor the client params. Adds tests covering both the drop (routed) and keep (unrouted) cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(model): correct graceful-degradation comment in classroom (#745) resolveModel does not throw on a missing key; clarify that a keyless route surfaces later in callLLM (degraded by the outer try/catch), while only a resolution-time failure is caught at the rewrite re-resolve. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model): route scene-content per scene type (#745) Extend MODEL_ROUTES with composite keys scene-content:<type> for the four core scene types (slide/quiz/interactive/pbl). getStageModel now resolves composite `a:b` stages most-specific-first, trimming `:` segments, so scene-content:<type> falls back to the base scene-content route, then x-model, then DEFAULT_MODEL. The scene-content route derives its stage from outline.type. Backward compatible: with no composite key configured, behavior is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model): per-stage thinking effort + drop hardcoded gpt fallback (#745) MODEL_ROUTES values may now be {model, effort} (not just a model string) to pin the thinking effort per stage. Arbitration mirrors model routing: a routed stage with effort set wins over the client's thinking (effort "none" disables); routed with no effort uses the model's default and drops client thinking; unrouted stages keep the client thinking. resolveModel is the single arbiter — body thinking is threaded in via resolveModelFromRequest. classroom-generation passes the resolved thinking into its callLLM calls so generate-classroom and web-search-query-rewrite honor route effort too. Also: removed the hardcoded `|| 'gpt-5.4-mini'` fallback in resolveModel — if no model resolves (no route / x-model / DEFAULT_MODEL) it now throws instead of silently picking a vendor default. model-routes: getStageRoute returns {model, effort?}; getStageModel delegates to it. Docs + tests updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(model): route value carries full ThinkingConfig (#745) Per-stage routing accepts the full unified ThinkingConfig in the route value ({model, thinking:{mode,effort,level,enabled,budgetTokens,excludeReasoningOutput}}) instead of just an effort string. The route's thinking is passed through resolveModel and normalized per the model's capability by callLLM, so budgetTokens (qwen), level (Gemini), enabled/mode, etc. all work. (qwen3.7-plus/max thinking capability now comes from upstream #753, so this no longer registers them.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): honor routed thinking for chat-adapter; doc fixes (#745) Cross-review (codex P2 + Claude) found chat-adapter was the only routable stage that ignored the route's thinking: it took the routed model but built its own thinkingConfig from the request body, so operator-pinned thinking never applied and client thinking still leaked onto a routed model. Now pass the client thinking into resolveModel and use the resolved thinkingConfig (route-pinned for a routed stage, client otherwise; defaults to disabled for low-latency chat). Also: clarify DEFAULT_MODEL is now required for server-side stages (the hardcoded gpt-5.4-mini fallback was removed → resolveModel throws if nothing resolves), and fix a stale {model,effort}→{model,thinking} doc comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
208 lines
9.2 KiB
TypeScript
208 lines
9.2 KiB
TypeScript
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
|
|
|
// Mock the heavy downstream of resolveModel so the test isolates the model
|
|
// string *resolution order*: stage route > x-model > DEFAULT_MODEL > builtin.
|
|
// model-routes is left real (it just reads MODEL_ROUTES) so we exercise the
|
|
// real integration point.
|
|
// Use the real parseModelString (canonical `provider:model` colon format) so
|
|
// the test exercises actual separator handling; only stub getModel (recording
|
|
// its args) so no real provider client is constructed. provider-config stubs
|
|
// echo the client-supplied key/baseUrl so a test can assert they are dropped
|
|
// when a stage route overrides the client model.
|
|
const mocks = vi.hoisted(() => ({ getModelCalls: [] as Array<Record<string, unknown>> }));
|
|
|
|
vi.mock('@/lib/ai/providers', async (importOriginal) => {
|
|
const actual = await importOriginal<typeof import('@/lib/ai/providers')>();
|
|
return {
|
|
...actual,
|
|
getModel: (args: Record<string, unknown>) => {
|
|
mocks.getModelCalls.push(args);
|
|
return { model: { id: args.modelId }, modelInfo: undefined };
|
|
},
|
|
};
|
|
});
|
|
|
|
vi.mock('@/lib/server/provider-config', () => ({
|
|
isServerConfiguredProvider: () => false,
|
|
resolveApiKey: (_id: string, clientKey: string) => clientKey || 'server-key',
|
|
resolveBaseUrl: (_id: string, clientBaseUrl?: string) => clientBaseUrl,
|
|
resolveProxy: () => undefined,
|
|
}));
|
|
|
|
vi.mock('@/lib/server/ssrf-guard', () => ({
|
|
validateUrlForSSRF: async () => null,
|
|
}));
|
|
|
|
describe('resolveModel — per-stage resolution order', () => {
|
|
beforeEach(() => {
|
|
vi.resetModules();
|
|
mocks.getModelCalls.length = 0;
|
|
delete process.env.MODEL_ROUTES;
|
|
delete process.env.DEFAULT_MODEL;
|
|
});
|
|
|
|
it('throws (no hardcoded fallback) when nothing is configured', async () => {
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
await expect(resolveModel({ stage: 'scene-content' })).rejects.toThrow(
|
|
/No model could be resolved/,
|
|
);
|
|
});
|
|
|
|
it('uses DEFAULT_MODEL when no stage route matches', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'scene-content' });
|
|
expect(r.modelString).toBe('openai:gpt-5.4-mini');
|
|
});
|
|
|
|
it('uses the stage route over DEFAULT_MODEL', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'scene-content' });
|
|
expect(r.modelString).toBe('openai:gpt-5.4');
|
|
});
|
|
|
|
it('uses DEFAULT_MODEL for stages not listed in MODEL_ROUTES', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'quiz-grade' });
|
|
expect(r.modelString).toBe('openai:gpt-5.4-mini');
|
|
});
|
|
|
|
it('lets a configured stage route win over an explicit modelString (x-model)', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({
|
|
stage: 'scene-content',
|
|
modelString: 'anthropic:claude-sonnet-4',
|
|
});
|
|
expect(r.modelString).toBe('openai:gpt-5.4');
|
|
});
|
|
|
|
it('falls back to x-model for a stage that is not routed', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'quiz-grade', modelString: 'anthropic:claude-sonnet-4' });
|
|
expect(r.modelString).toBe('anthropic:claude-sonnet-4');
|
|
});
|
|
|
|
it('drops client apiKey/baseUrl/providerType when a stage route overrides the client model', async () => {
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'pbl-chat': 'anthropic:claude-sonnet-4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
await resolveModel({
|
|
stage: 'pbl-chat',
|
|
modelString: 'openai:gpt-5.4-mini',
|
|
apiKey: 'client-openai-key',
|
|
baseUrl: 'https://client.example/v1',
|
|
providerType: 'openai',
|
|
});
|
|
const call = mocks.getModelCalls.at(-1)!;
|
|
expect(call.providerId).toBe('anthropic');
|
|
expect(call.modelId).toBe('claude-sonnet-4');
|
|
// None of the client-sent connection params for the OLD provider leak onto
|
|
// the routed provider — they resolve from server config instead.
|
|
expect(call.providerType).toBeUndefined();
|
|
expect(call.baseUrl).toBeUndefined();
|
|
expect(call.apiKey).toBe('server-key');
|
|
});
|
|
|
|
it('keeps client apiKey/baseUrl/providerType for an unrouted stage (x-model honored)', async () => {
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
await resolveModel({
|
|
stage: 'quiz-grade',
|
|
modelString: 'openai:gpt-5.4-mini',
|
|
apiKey: 'client-key',
|
|
baseUrl: 'https://client.example/v1',
|
|
providerType: 'openai',
|
|
});
|
|
const call = mocks.getModelCalls.at(-1)!;
|
|
expect(call.providerType).toBe('openai');
|
|
expect(call.baseUrl).toBe('https://client.example/v1');
|
|
expect(call.apiKey).toBe('client-key');
|
|
});
|
|
|
|
it('uses a scene-content:<type> route over the base route and x-model', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({
|
|
'scene-content': 'openai:gpt-5.4-mini',
|
|
'scene-content:quiz': 'openai:gpt-5.4',
|
|
});
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({
|
|
stage: 'scene-content:quiz',
|
|
modelString: 'anthropic:claude-sonnet-4',
|
|
});
|
|
expect(r.modelString).toBe('openai:gpt-5.4');
|
|
});
|
|
|
|
it('falls back to the base scene-content route for an unrouted type', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'scene-content:slide' });
|
|
expect(r.modelString).toBe('openai:gpt-5.4');
|
|
});
|
|
|
|
it('resolves the stage route provider for cross-provider routing', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'pbl-chat': 'anthropic:claude-sonnet-4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'pbl-chat' });
|
|
expect(r.modelString).toBe('anthropic:claude-sonnet-4');
|
|
expect(r.providerId).toBe('anthropic');
|
|
expect(r.modelId).toBe('claude-sonnet-4');
|
|
});
|
|
|
|
it('route thinking wins over client thinking when the stage is routed', async () => {
|
|
process.env.MODEL_ROUTES = JSON.stringify({
|
|
'pbl-chat': { model: 'anthropic:claude-sonnet-4', thinking: { effort: 'high' } },
|
|
});
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'pbl-chat', thinkingConfig: { effort: 'low' } });
|
|
expect(r.thinkingConfig).toEqual({ effort: 'high' });
|
|
});
|
|
|
|
it('route can pass a full thinking config (enabled + budgetTokens)', async () => {
|
|
process.env.MODEL_ROUTES = JSON.stringify({
|
|
'scene-content:interactive': {
|
|
model: 'qwen:qwen3.7-plus',
|
|
thinking: { enabled: true, budgetTokens: 8000 },
|
|
},
|
|
});
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'scene-content:interactive' });
|
|
expect(r.thinkingConfig).toEqual({ enabled: true, budgetTokens: 8000 });
|
|
});
|
|
|
|
it('routed-without-thinking drops client thinking (routed model uses its default)', async () => {
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'deepseek:deepseek-v4-pro' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({ stage: 'scene-content', thinkingConfig: { effort: 'high' } });
|
|
expect(r.thinkingConfig).toBeUndefined();
|
|
});
|
|
|
|
it('unrouted stage keeps the client thinking config', async () => {
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'deepseek:deepseek-v4-pro' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({
|
|
stage: 'quiz-grade',
|
|
modelString: 'openai:gpt-5.4-mini',
|
|
thinkingConfig: { effort: 'medium' },
|
|
});
|
|
expect(r.thinkingConfig).toEqual({ effort: 'medium' });
|
|
});
|
|
|
|
it('ignores stage routing entirely when no stage is passed', async () => {
|
|
process.env.DEFAULT_MODEL = 'openai:gpt-5.4-mini';
|
|
process.env.MODEL_ROUTES = JSON.stringify({ 'scene-content': 'openai:gpt-5.4' });
|
|
const { resolveModel } = await import('@/lib/server/resolve-model');
|
|
const r = await resolveModel({});
|
|
expect(r.modelString).toBe('openai:gpt-5.4-mini');
|
|
});
|
|
});
|