openmaic-custom/lib/server/resolve-model.ts
wyuc 25cf58d19f
feat(model): optional per-stage LLM model routing (#745) (#747)
* feat(model): optional per-stage LLM model routing (#745)

Add a config-only stage -> model map consulted during model resolution,
falling back to DEFAULT_MODEL when unset (zero behavior change unless opted in).

- lib/server/model-routes.ts: parse/validate/cache MODEL_ROUTES env JSON;
  LLM_STAGES registry of routable stages; getStageModel(stage).
- resolveModel: resolution order x-model > stage route > DEFAULT_MODEL > builtin;
  thread optional `stage` through resolveModelFromHeaders/FromRequest.
- Wire each route's resolveModel call site to its stage; classroom-generation
  resolves generate-classroom and web-search-query-rewrite independently.
- .env.example: document MODEL_ROUTES; unit tests for both modules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(model): address cross-review findings for per-stage routing (#745)

- classroom-generation: resolve the web-search-query-rewrite model lazily,
  only when explicitly routed and inside the web-search branch, with try/catch
  fallback to the classroom model. Fixes a regression where a misconfigured
  optional route (keyless/invalid provider) threw and aborted ALL classroom
  generation even with web search disabled; also avoids wasted resolution on the
  common no-search path and preserves classroom-model inheritance when unrouted.
- resolve-model: type the stage param as LlmStage across resolveModel/
  resolveModelFromHeaders/resolveModelFromRequest so a mistyped stage literal is
  a compile error instead of silently falling through to DEFAULT_MODEL.
- model-routes: simplify the cache to a process singleton (matching
  provider-config), dropping the raw-string-keyed cache and redundant STAGE_SET.
- .env.example: clarify x-model precedence and that a route to an unconfigured
  provider fails at request time (no startup validation), like a bad DEFAULT_MODEL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(model): stage route takes precedence over client x-model (#745)

Cross-review (codex P2 + Claude tracer) found that the browser UI always sends
its saved model as x-model, so the previous x-model > stage order made
MODEL_ROUTES inert for normal UI traffic on exactly the heavy stages the RFC
targets (scene-content, quiz, pbl-chat, chat). Flip resolution to
stage route > x-model > DEFAULT_MODEL: a configured route is the operator's
deliberate per-stage choice and wins, while unrouted stages still honor the
client's x-model. Update docs and tests accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(model): prettier-format changed files for #745 CI

Run the repo formatter (pnpm check / prettier) on the changed routes, the new
model-routes module, and the new tests so `prettier . --check` (CI) passes;
also correct a stale precedence comment in the resolve-model test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(model): isolate routed model from client connection params (#745)

When a stage route overrides the client's x-model and points at a different
provider, the client-sent apiKey/baseUrl/providerType (for the client's model)
must not bleed onto the routed provider — otherwise a routed Anthropic model
would be constructed with the client's OpenAI providerType/key and fail. A
routed model now resolves its connection params purely from server config, as
if no x-model was sent. Unrouted stages still honor the client params. Adds
tests covering both the drop (routed) and keep (unrouted) cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(model): correct graceful-degradation comment in classroom (#745)

resolveModel does not throw on a missing key; clarify that a keyless route
surfaces later in callLLM (degraded by the outer try/catch), while only a
resolution-time failure is caught at the rewrite re-resolve. Comment-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(model): route scene-content per scene type (#745)

Extend MODEL_ROUTES with composite keys scene-content:<type> for the four core
scene types (slide/quiz/interactive/pbl). getStageModel now resolves composite
`a:b` stages most-specific-first, trimming `:` segments, so scene-content:<type>
falls back to the base scene-content route, then x-model, then DEFAULT_MODEL.
The scene-content route derives its stage from outline.type. Backward
compatible: with no composite key configured, behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(model): per-stage thinking effort + drop hardcoded gpt fallback (#745)

MODEL_ROUTES values may now be {model, effort} (not just a model string) to pin
the thinking effort per stage. Arbitration mirrors model routing: a routed stage
with effort set wins over the client's thinking (effort "none" disables); routed
with no effort uses the model's default and drops client thinking; unrouted
stages keep the client thinking. resolveModel is the single arbiter — body
thinking is threaded in via resolveModelFromRequest. classroom-generation passes
the resolved thinking into its callLLM calls so generate-classroom and
web-search-query-rewrite honor route effort too.

Also: removed the hardcoded `|| 'gpt-5.4-mini'` fallback in resolveModel — if no
model resolves (no route / x-model / DEFAULT_MODEL) it now throws instead of
silently picking a vendor default.

model-routes: getStageRoute returns {model, effort?}; getStageModel delegates to
it. Docs + tests updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(model): route value carries full ThinkingConfig (#745)

Per-stage routing accepts the full unified ThinkingConfig in the route value
({model, thinking:{mode,effort,level,enabled,budgetTokens,excludeReasoningOutput}})
instead of just an effort string. The route's thinking is passed through
resolveModel and normalized per the model's capability by callLLM, so
budgetTokens (qwen), level (Gemini), enabled/mode, etc. all work.

(qwen3.7-plus/max thinking capability now comes from upstream #753, so this no
longer registers them.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(model): honor routed thinking for chat-adapter; doc fixes (#745)

Cross-review (codex P2 + Claude) found chat-adapter was the only routable stage
that ignored the route's thinking: it took the routed model but built its own
thinkingConfig from the request body, so operator-pinned thinking never applied
and client thinking still leaked onto a routed model. Now pass the client
thinking into resolveModel and use the resolved thinkingConfig (route-pinned for
a routed stage, client otherwise; defaults to disabled for low-latency chat).

Also: clarify DEFAULT_MODEL is now required for server-side stages (the
hardcoded gpt-5.4-mini fallback was removed → resolveModel throws if nothing
resolves), and fix a stale {model,effort}→{model,thinking} doc comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 22:26:37 +08:00

174 lines
6.8 KiB
TypeScript

/**
* Shared model resolution utilities for API routes.
*
* Extracts the repeated parseModelString → resolveApiKey → resolveBaseUrl →
* resolveProxy → getModel boilerplate into a single call.
*/
import type { NextRequest } from 'next/server';
import { getModel, parseModelString, type ModelWithInfo } from '@/lib/ai/providers';
import type { ThinkingConfig } from '@/lib/types/provider';
import {
isServerConfiguredProvider,
resolveApiKey,
resolveBaseUrl,
resolveProxy,
} from '@/lib/server/provider-config';
import { validateUrlForSSRF } from '@/lib/server/ssrf-guard';
import { getStageRoute, type LlmStage } from '@/lib/server/model-routes';
export interface ResolvedModel extends ModelWithInfo {
/** Original model string (e.g. "openai/gpt-4o-mini") */
modelString: string;
/** Resolved provider ID (e.g. "openai", "ollama") */
providerId: string;
/** Resolved model ID (e.g. "gpt-4o-mini") */
modelId: string;
/** Effective API key after server-side fallback resolution */
apiKey: string;
/** Effective base URL after server/client resolution */
baseUrl?: string;
/** Optional per-request thinking configuration from the client. */
thinkingConfig?: ThinkingConfig;
}
/**
* Resolve a language model from explicit parameters.
*
* Use this when model config comes from the request body.
*/
export async function resolveModel(params: {
modelString?: string;
/**
* Optional generation stage (a `callLLM` source label, e.g. 'scene-content').
* When set and a route is configured via `MODEL_ROUTES`, the route wins for
* this call — even over a client-sent `modelString` (x-model). Unrouted
* stages fall back to `modelString` then `DEFAULT_MODEL`. See
* lib/server/model-routes.ts.
*/
stage?: LlmStage;
apiKey?: string;
baseUrl?: string;
providerType?: string;
thinkingConfig?: ThinkingConfig;
}): Promise<ResolvedModel> {
// Resolution order: stage route > x-model > DEFAULT_MODEL.
// A configured stage route is the operator's deliberate per-stage choice and
// wins even over a client-sent x-model (otherwise the browser UI, which always
// sends its saved model, would shadow every route). Unrouted stages fall back
// to the client x-model, then DEFAULT_MODEL. There is intentionally no hardcoded
// model fallback — if nothing resolves we fail loud rather than silently pick a
// vendor default.
const stageRoute = getStageRoute(params.stage);
const stageModel = stageRoute?.model;
const modelString = stageModel || params.modelString || process.env.DEFAULT_MODEL;
if (!modelString) {
throw new Error(
'No model could be resolved. Configure DEFAULT_MODEL (and/or a MODEL_ROUTES entry for this stage), or send a model via x-model.',
);
}
const { providerId, modelId } = parseModelString(modelString);
// When a stage route overrides the client's model, the client-sent connection
// params (apiKey/baseUrl/providerType) belong to the client's *other* model
// and must not bleed onto the routed provider — otherwise e.g. a routed
// Anthropic model would be built with the client's OpenAI providerType/key.
// A routed model resolves purely from server config, as if no x-model was sent.
const routed = Boolean(stageModel);
const clientApiKey = routed ? undefined : params.apiKey;
const clientProviderType = routed ? undefined : params.providerType;
const clientBaseUrlParam = routed ? undefined : params.baseUrl;
// Server-managed providers are admin-owned: the operator's key and base URL
// are authoritative and any client-sent override is ignored. SSRF validation
// therefore applies only to unmanaged providers, where the base URL really is
// client-supplied. (Server-configured URLs are trusted by the operator.)
const managed = isServerConfiguredProvider('providers', providerId);
const clientBaseUrl = managed ? undefined : clientBaseUrlParam || undefined;
if (clientBaseUrl && process.env.NODE_ENV === 'production') {
const ssrfError = await validateUrlForSSRF(clientBaseUrl);
if (ssrfError) {
throw new Error(ssrfError);
}
}
const apiKey = resolveApiKey(providerId, clientApiKey || '');
const baseUrl = resolveBaseUrl(providerId, clientBaseUrl);
const proxy = resolveProxy(providerId);
const { model, modelInfo } = getModel({
providerId,
modelId,
apiKey,
baseUrl,
proxy,
providerType: clientProviderType as 'openai' | 'anthropic' | 'google' | undefined,
});
// Thinking arbitration mirrors model routing — the route carries a full
// ThinkingConfig (mode/effort/level/enabled/budgetTokens/…) which callLLM
// normalizes against the model's capability:
// - routed + thinking set → the route's thinking wins (over client thinking).
// - routed + no thinking → routed model uses its own default; client thinking
// is dropped (it belonged to the client's other model).
// - unrouted → honor the client's thinking config.
const thinkingConfig: ThinkingConfig | undefined = routed
? stageRoute?.thinking
: params.thinkingConfig;
return {
model,
modelInfo,
modelString,
providerId,
modelId,
apiKey,
baseUrl,
thinkingConfig,
};
}
function getThinkingConfigFromBody(body: unknown): ThinkingConfig | undefined {
if (!body || typeof body !== 'object') return undefined;
const record = body as { thinkingConfig?: unknown; thinking?: unknown };
const config = record.thinkingConfig ?? record.thinking;
return config && typeof config === 'object' ? (config as ThinkingConfig) : undefined;
}
/**
* Resolve a language model from standard request headers.
*
* Reads: x-model, x-api-key, x-base-url, x-provider-type
* Note: requiresApiKey is derived server-side from the provider registry,
* never from client headers, to prevent auth bypass.
*/
export async function resolveModelFromHeaders(
req: NextRequest,
stage?: LlmStage,
thinkingConfig?: ThinkingConfig,
): Promise<ResolvedModel> {
return resolveModel({
modelString: req.headers.get('x-model') || undefined,
stage,
apiKey: req.headers.get('x-api-key') || undefined,
baseUrl: req.headers.get('x-base-url') || undefined,
providerType: req.headers.get('x-provider-type') || undefined,
thinkingConfig,
});
}
/**
* Resolve a language model from standard request headers plus body fields.
*
* Reads model credentials from headers and per-request thinking config from
* the JSON body field `thinkingConfig` (or legacy/eval field `thinking`).
*/
export async function resolveModelFromRequest(
req: NextRequest,
body: unknown,
stage?: LlmStage,
): Promise<ResolvedModel> {
// Pass the client's body thinking into resolveModel so the single arbiter
// there decides (a routed stage may override or drop it). See resolveModel.
return resolveModelFromHeaders(req, stage, getThinkingConfigFromBody(body));
}