feat: inline language inference for outline and PBL generation (#412)

* feat: inline language inference in outline generation

Replace manual language selection with automatic inference from user
requirement text. The outline generator now produces a languageDirective
that propagates through the entire generation pipeline.

Key changes:
- Rewrite Language Inference section in outline prompt with decision
  rules for foreign language learning, cross-language PDF, proxy
  requests, and terminology handling
- Reorder pipeline: outlines before agents, so agent profiles can
  use the inferred languageDirective
- Pass languageDirective through scene content/actions generation
- Add SSE streaming of languageDirective from outline generation
- Remove manual language selection UI

Eval test suite:
- 50 test cases covering 9 scenario types: single-language, language
  learning, immersive, explicit instruction, code-switching, minimal
  input, user profiles, cross-language PDF, locale mismatch
- LLM-as-judge evaluation with configurable inference/judge models
- 50/50 pass rate with gemini-3-flash-preview + gpt-4o judge
- Excluded from CI (requires LLM API keys)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove useless attributes in test case

* feat: pass languageDirective to PBL, remove hardcoded zh-CN/en-US

- Remove pblConfig.language field, use languageDirective from outline
  inference instead
- Delete all Chinese prompt duplicates in pbl-system-prompt.ts,
  agent-templates.ts; keep English templates with languageDirective
  injection
- Remove zh-CN/en-US branches in generate-pbl.ts (initial prompt,
  post-process context, welcome message)
- Pipe languageDirective through scene-generator → generatePBLContent
  → IssueboardMCP → agent templates
- Remove i18n template wrapping from PBL welcome messages in
  pbl-renderer.tsx and use-pbl-chat.ts; use LLM-generated content
  directly to avoid language mismatch when UI locale ≠ course language
- Update pblConfig schema in outline prompt (remove language field)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: clean up legacy language fields, pass languageDirective through buildSceneFromOutline

- Add languageDirective param to buildSceneFromOutline, combining with
  outline.languageNote via buildLanguageText
- Remove dead types: AudienceProfile, StylePreferences, LegacyUserRequirements,
  SceneOutline.language
- Remove Stage.language and all its propagation (storage, stageInfo, prompt-builder fallback)
- Remove GenerateClassroomInput.language and job store inputSummary.language
- Remove unused pdfLanguageSample from outline SSE route

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: persist languageDirective to IndexedDB, fix TTS preview and dead refs

- Add languageDirective to StageRecord and stage-storage write path
- Add DB v9 migration: convert legacy language locale codes to directives
- Use voice.language for TTS preview text instead of dead localStorage key
- Omit empty ## Language section in PBL agent prompts
- Use JSON.parse for extractLanguageDirective unescape (\uXXXX support)
- Remove dead i18n keys: toolbar.languageHint, pbl.chat.welcomeMessage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
杨慎 2026-04-13 21:49:56 +08:00 committed by GitHub
parent c071e02d83
commit 470aa5e42e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
52 changed files with 1288 additions and 660 deletions

1
.gitignore vendored
View File

@ -71,3 +71,4 @@ next-env.d.ts
# docs
/docs
tests/generation/*.eval.result.md

View File

@ -19,7 +19,7 @@ export async function POST(req: NextRequest) {
const body: GenerateClassroomInput = {
requirement: rawBody.requirement || '',
...(rawBody.pdfContent ? { pdfContent: rawBody.pdfContent } : {}),
...(rawBody.language ? { language: rawBody.language } : {}),
...(rawBody.enableWebSearch != null ? { enableWebSearch: rawBody.enableWebSearch } : {}),
...(rawBody.enableImageGeneration != null
? { enableImageGeneration: rawBody.enableImageGeneration }

View File

@ -20,7 +20,7 @@ export const maxDuration = 120;
interface RequestBody {
stageInfo: { name: string; description?: string };
sceneOutlines?: { title: string; description?: string }[];
language: string;
languageDirective: string;
availableAvatars: string[];
avatarDescriptions?: Array<{ path: string; desc: string }>;
availableVoices?: Array<{ providerId: string; voiceId: string; voiceName: string }>;
@ -43,7 +43,7 @@ export async function POST(req: NextRequest) {
const {
stageInfo,
sceneOutlines,
language,
languageDirective,
availableAvatars,
avatarDescriptions,
availableVoices,
@ -54,8 +54,8 @@ export async function POST(req: NextRequest) {
if (!stageInfo?.name) {
return apiError('MISSING_REQUIRED_FIELD', 400, 'stageInfo.name is required');
}
if (!language) {
return apiError('MISSING_REQUIRED_FIELD', 400, 'language is required');
if (!languageDirective) {
return apiError('MISSING_REQUIRED_FIELD', 400, 'languageDirective is required');
}
if (!availableAvatars || availableAvatars.length === 0) {
return apiError(
@ -109,7 +109,8 @@ Requirements:
- Exactly 1 agent must have role "teacher", the rest can be "assistant" or "student"
- Priority values: teacher=10 (highest), assistant=7, student=4-6
- Each agent needs: name, role, persona (2-3 sentences describing personality and teaching/learning style)
- Names and personas must be in language: ${language}
- Language directive for this course: ${languageDirective}
Agent names and personas must follow this language directive.
- Each agent must be assigned one avatar from this list: ${JSON.stringify(avatarDescriptions && avatarDescriptions.length > 0 ? avatarDescriptions.map((a) => ({ path: a.path, description: a.desc })) : availableAvatars)}
- Pick an avatar that visually matches the agent's personality and role
- Try to use different avatars for each agent

View File

@ -44,6 +44,7 @@ export async function POST(req: NextRequest) {
agents,
previousSpeeches: incomingPreviousSpeeches,
userProfile,
languageDirective,
} = body as {
outline: SceneOutline;
allOutlines: SceneOutline[];
@ -56,6 +57,7 @@ export async function POST(req: NextRequest) {
agents?: AgentInfo[];
previousSpeeches?: string[];
userProfile?: string;
languageDirective?: string;
};
// Validate required fields
@ -132,7 +134,12 @@ export async function POST(req: NextRequest) {
// ── Generate actions ──
log.info(`Generating actions: "${outline.title}" (${outline.type}) [model=${modelString}]`);
const actions = await generateSceneActions(outline, content, aiCall, ctx, agents, userProfile);
const actions = await generateSceneActions(outline, content, aiCall, {
ctx,
agents,
userProfile,
languageDirective,
});
log.info(`Generated ${actions.length} actions for: "${outline.title}"`);

View File

@ -33,9 +33,10 @@ export async function POST(req: NextRequest) {
allOutlines,
pdfImages,
imageMapping,
stageInfo,
stageInfo: _stageInfo,
stageId,
agents,
languageDirective,
} = body as {
outline: SceneOutline;
allOutlines: SceneOutline[];
@ -44,11 +45,11 @@ export async function POST(req: NextRequest) {
stageInfo: {
name: string;
description?: string;
language?: string;
style?: string;
};
stageId: string;
agents?: AgentInfo[];
languageDirective?: string;
};
// Validate required fields
@ -66,11 +67,7 @@ export async function POST(req: NextRequest) {
return apiError('MISSING_REQUIRED_FIELD', 400, 'stageId is required');
}
// Ensure outline has language from stageInfo (fallback for older outlines)
const outline: SceneOutline = {
...rawOutline,
language: rawOutline.language || (stageInfo?.language as 'zh-CN' | 'en-US') || 'zh-CN',
};
const outline: SceneOutline = { ...rawOutline };
// ── Model resolution from request headers ──
const { model: languageModel, modelInfo, modelString } = await resolveModelFromHeaders(req);
@ -140,16 +137,15 @@ export async function POST(req: NextRequest) {
`Generating content: "${effectiveOutline.title}" (${effectiveOutline.type}) [model=${modelString}]`,
);
const content = await generateSceneContent(
effectiveOutline,
aiCall,
const content = await generateSceneContent(effectiveOutline, aiCall, {
assignedImages,
imageMapping,
effectiveOutline.type === 'pbl' ? languageModel : undefined,
hasVision,
languageModel: effectiveOutline.type === 'pbl' ? languageModel : undefined,
visionEnabled: hasVision,
generatedMediaMapping,
agents,
);
languageDirective,
});
if (!content) {
log.error(`Failed to generate content for: "${effectiveOutline.title}"`);

View File

@ -6,8 +6,9 @@
* so the frontend can display them incrementally.
*
* SSE events:
* { type: 'languageDirective', data: string }
* { type: 'outline', data: SceneOutline, index: number }
* { type: 'done', outlines: SceneOutline[] }
* { type: 'done', outlines: SceneOutline[], languageDirective: string }
* { type: 'error', error: string }
*/
@ -37,17 +38,45 @@ const log = createLogger('Outlines Stream');
export const maxDuration = 300;
/**
* Extract the languageDirective from the streamed wrapper JSON.
* Matches `"languageDirective":"<value>"` in partial JSON like:
* {"languageDirective":"用中文授课...","outlines":[...
*/
function extractLanguageDirective(buffer: string): string | null {
const match = buffer.match(/"languageDirective"\s*:\s*"((?:[^"\\]|\\.)*)"/);
if (!match) return null;
try {
return JSON.parse(`"${match[1]}"`);
} catch {
return match[1];
}
}
/**
* Incremental JSON array parser.
* Extracts complete top-level objects from a partially-streamed JSON array.
* Supports both a flat array `[{...},{...}]` and a wrapper object
* `{"languageDirective":"...","outlines":[{...},{...}]}`.
* Returns newly found objects (skipping `alreadyParsed` count).
*/
function extractNewOutlines(buffer: string, alreadyParsed: number): SceneOutline[] {
const results: SceneOutline[] = [];
// Find the start of the JSON array (skip any markdown fencing)
const stripped = buffer.replace(/^[\s\S]*?(?=\[)/, '');
const arrayStart = stripped.indexOf('[');
// Strip markdown fencing if present
const stripped = buffer.replace(/^[\s\S]*?(?=[\[{])/, '');
// Find the outlines array — either nested in {"outlines": [...]} or a flat array
let arrayStart = -1;
const outlinesKeyIdx = stripped.indexOf('"outlines"');
if (outlinesKeyIdx >= 0) {
// Wrapper format: find [ after "outlines":
arrayStart = stripped.indexOf('[', outlinesKeyIdx);
} else {
// Flat array fallback
arrayStart = stripped.indexOf('[');
}
if (arrayStart === -1) return results;
let depth = 0;
@ -120,12 +149,17 @@ export async function POST(req: NextRequest) {
};
requirementSnippet = requirements?.requirement?.substring(0, 60);
// Build user profile string for language inference context
const userProfileText =
requirements.userNickname || requirements.userBio
? `## Student Profile\n\nStudent: ${requirements.userNickname || 'Unknown'}${requirements.userBio ? ` — ${requirements.userBio}` : ''}\n\nConsider this student's background when designing the course. Adapt difficulty, examples, and teaching approach accordingly.\n\n---`
: '';
// Detect vision capability
const hasVision = !!modelInfo?.capabilities?.vision;
// Build prompt (same logic as generateSceneOutlinesFromRequirements)
let availableImagesText =
requirements.language === 'zh-CN' ? '无可用图片' : 'No images available';
let availableImagesText = 'No images available';
let visionImages: Array<{ id: string; src: string }> | undefined;
if (pdfImages && pdfImages.length > 0) {
@ -136,11 +170,9 @@ export async function POST(req: NextRequest) {
const textOnlySlice = allWithSrc.slice(MAX_VISION_IMAGES);
const noSrcImages = pdfImages.filter((img) => !imageMapping[img.id]);
const visionDescriptions = visionSlice.map((img) =>
formatImagePlaceholder(img, requirements.language),
);
const visionDescriptions = visionSlice.map((img) => formatImagePlaceholder(img));
const textDescriptions = [...textOnlySlice, ...noSrcImages].map((img) =>
formatImageDescription(img, requirements.language),
formatImageDescription(img),
);
availableImagesText = [...visionDescriptions, ...textDescriptions].join('\n');
@ -152,9 +184,7 @@ export async function POST(req: NextRequest) {
}));
} else {
// Text-only mode: full descriptions
availableImagesText = pdfImages
.map((img) => formatImageDescription(img, requirements.language))
.join('\n');
availableImagesText = pdfImages.map((img) => formatImageDescription(img)).join('\n');
}
}
@ -178,16 +208,12 @@ export async function POST(req: NextRequest) {
const prompts = buildPrompt(PROMPT_IDS.REQUIREMENTS_TO_OUTLINES, {
requirement: requirements.requirement,
language: requirements.language,
pdfContent: pdfText
? pdfText.substring(0, MAX_PDF_CONTENT_CHARS)
: requirements.language === 'zh-CN'
? '无'
: 'None',
pdfContent: pdfText ? pdfText.substring(0, MAX_PDF_CONTENT_CHARS) : 'None',
availableImages: availableImagesText,
researchContext: researchContext || (requirements.language === 'zh-CN' ? '无' : 'None'),
researchContext: researchContext || 'None',
mediaGenerationPolicy,
teacherContext,
userProfile: userProfileText,
});
if (!prompts) {
@ -247,6 +273,7 @@ export async function POST(req: NextRequest) {
};
let parsedOutlines: SceneOutline[] = [];
let languageDirective: string | null = null;
let lastError: string | undefined;
for (let attempt = 1; attempt <= MAX_STREAM_RETRIES + 1; attempt++) {
@ -255,10 +282,23 @@ export async function POST(req: NextRequest) {
let fullText = '';
parsedOutlines = [];
languageDirective = null;
for await (const chunk of result.textStream) {
fullText += chunk;
// Try to extract language directive early
if (!languageDirective) {
languageDirective = extractLanguageDirective(fullText);
if (languageDirective) {
const ldEvent = JSON.stringify({
type: 'languageDirective',
data: languageDirective,
});
controller.enqueue(encoder.encode(`data: ${ldEvent}\n\n`));
}
}
// Try to extract new outlines from the accumulated text
const newOutlines = extractNewOutlines(fullText, parsedOutlines.length);
for (const outline of newOutlines) {
@ -325,6 +365,8 @@ export async function POST(req: NextRequest) {
const doneEvent = JSON.stringify({
type: 'done',
outlines: uniquifiedOutlines,
languageDirective:
languageDirective || 'Teach in the language that matches the user requirement.',
});
controller.enqueue(encoder.encode(`data: ${doneEvent}\n\n`));
} else {

View File

@ -159,11 +159,11 @@ export default function ClassroomDetailPage() {
stageInfo: {
name: stage.name || '',
description: stage.description,
language: stage.language,
style: stage.style,
},
agents: params.agents,
userProfile: params.userProfile,
languageDirective: params.languageDirective || stage.languageDirective,
});
});
} else if (outlines.length > 0 && stage) {

View File

@ -349,7 +349,146 @@ function GenerationPreviewContent() {
imageMapping = currentSession.imageMapping;
}
// ── Agent generation (before outlines so persona can influence structure) ──
// Create stage client-side
const stageId = nanoid(10);
const stage: Stage = {
id: stageId,
name: extractTopicFromRequirement(currentSession.requirements.requirement),
description: '',
style: 'professional',
createdAt: Date.now(),
updatedAt: Date.now(),
};
// ── Generate outlines first (infers languageDirective) ──
let outlines = currentSession.sceneOutlines;
let languageDirective: string | undefined;
const outlineStepIdx = activeSteps.findIndex((s) => s.id === 'outline');
setCurrentStepIndex(outlineStepIdx >= 0 ? outlineStepIdx : 0);
if (!outlines || outlines.length === 0) {
log.debug('=== Generating outlines (SSE) ===');
setStreamingOutlines([]);
const outlineResult = await new Promise<{
outlines: SceneOutline[];
languageDirective: string;
}>((resolve, reject) => {
const collected: SceneOutline[] = [];
let directive: string | undefined;
fetch('/api/generate/scene-outlines-stream', {
method: 'POST',
headers: getApiHeaders(),
body: JSON.stringify({
requirements: currentSession.requirements,
pdfText: currentSession.pdfText,
pdfImages: currentSession.pdfImages,
imageMapping,
researchContext: currentSession.researchContext,
}),
signal,
})
.then((res) => {
if (!res.ok) {
return res.json().then((d) => {
reject(new Error(d.error || t('generation.outlineGenerateFailed')));
});
}
const reader = res.body?.getReader();
if (!reader) {
reject(new Error(t('generation.streamNotReadable')));
return;
}
const decoder = new TextDecoder();
let sseBuffer = '';
const pump = (): Promise<void> =>
reader.read().then(({ done, value }) => {
if (value) {
sseBuffer += decoder.decode(value, { stream: !done });
const lines = sseBuffer.split('\n');
sseBuffer = lines.pop() || '';
for (const line of lines) {
if (!line.startsWith('data: ')) continue;
try {
const evt = JSON.parse(line.slice(6));
if (evt.type === 'languageDirective') {
directive = evt.data;
} else if (evt.type === 'outline') {
collected.push(evt.data);
setStreamingOutlines([...collected]);
} else if (evt.type === 'retry') {
collected.length = 0;
setStreamingOutlines([]);
setStatusMessage(t('generation.outlineRetrying'));
} else if (evt.type === 'done') {
directive = evt.languageDirective || directive;
resolve({
outlines: evt.outlines || collected,
languageDirective:
directive ||
'Teach in the language that matches the user requirement.',
});
return;
} else if (evt.type === 'error') {
reject(new Error(evt.error));
return;
}
} catch (e) {
log.error('Failed to parse outline SSE:', line, e);
}
}
}
if (done) {
if (collected.length > 0) {
resolve({
outlines: collected,
languageDirective:
directive || 'Teach in the language that matches the user requirement.',
});
} else {
reject(new Error(t('generation.outlineEmptyResponse')));
}
return;
}
return pump();
});
pump().catch(reject);
})
.catch(reject);
});
outlines = outlineResult.outlines;
languageDirective = outlineResult.languageDirective;
// Store languageDirective on the stage
stage.languageDirective = languageDirective;
const updatedSession = {
...currentSession,
sceneOutlines: outlines,
languageDirective,
};
setSession(updatedSession);
sessionStorage.setItem('generationSession', JSON.stringify(updatedSession));
// Outline generation succeeded — clear homepage draft cache
try {
localStorage.removeItem('requirementDraft');
} catch {
/* ignore */
}
// Brief pause to let user see the final outline state
await new Promise((resolve) => setTimeout(resolve, 800));
}
// ── Agent generation (after outlines — uses languageDirective + outlines) ──
const settings = useSettingsStore.getState();
let agents: Array<{
id: string;
@ -358,18 +497,6 @@ function GenerationPreviewContent() {
persona?: string;
}> = [];
// Create stage client-side (needed for agent generation stageId)
const stageId = nanoid(10);
const stage: Stage = {
id: stageId,
name: extractTopicFromRequirement(currentSession.requirements.requirement),
description: '',
language: currentSession.requirements.language || 'zh-CN',
style: 'professional',
createdAt: Date.now(),
updatedAt: Date.now(),
};
if (settings.agentMode === 'auto') {
const agentStepIdx = activeSteps.findIndex((s) => s.id === 'agent-generation');
if (agentStepIdx >= 0) setCurrentStepIndex(agentStepIdx);
@ -437,13 +564,13 @@ function GenerationPreviewContent() {
);
};
// No outlines yet — agent generation uses only stage name + description
const agentResp = await fetch('/api/generate/agent-profiles', {
method: 'POST',
headers: getApiHeaders(),
body: JSON.stringify({
stageInfo: { name: stage.name, description: stage.description },
language: currentSession.requirements.language || 'zh-CN',
sceneOutlines: outlines.map((o) => ({ title: o.title, description: o.description })),
languageDirective,
availableAvatars: allAvatars.map((a) => a.path),
avatarDescriptions: allAvatars.map((a) => ({ path: a.path, desc: a.desc })),
availableVoices: getAvailableVoicesForGeneration(),
@ -515,108 +642,6 @@ function GenerationPreviewContent() {
stage.agentIds = presetAgentIds;
}
// ── Generate outlines (with agent personas for teacher context) ──
let outlines = currentSession.sceneOutlines;
const outlineStepIdx = activeSteps.findIndex((s) => s.id === 'outline');
setCurrentStepIndex(outlineStepIdx >= 0 ? outlineStepIdx : 0);
if (!outlines || outlines.length === 0) {
log.debug('=== Generating outlines (SSE) ===');
setStreamingOutlines([]);
outlines = await new Promise<SceneOutline[]>((resolve, reject) => {
const collected: SceneOutline[] = [];
fetch('/api/generate/scene-outlines-stream', {
method: 'POST',
headers: getApiHeaders(),
body: JSON.stringify({
requirements: currentSession.requirements,
pdfText: currentSession.pdfText,
pdfImages: currentSession.pdfImages,
imageMapping,
researchContext: currentSession.researchContext,
agents,
}),
signal,
})
.then((res) => {
if (!res.ok) {
return res.json().then((d) => {
reject(new Error(d.error || t('generation.outlineGenerateFailed')));
});
}
const reader = res.body?.getReader();
if (!reader) {
reject(new Error(t('generation.streamNotReadable')));
return;
}
const decoder = new TextDecoder();
let sseBuffer = '';
const pump = (): Promise<void> =>
reader.read().then(({ done, value }) => {
if (value) {
sseBuffer += decoder.decode(value, { stream: !done });
const lines = sseBuffer.split('\n');
sseBuffer = lines.pop() || '';
for (const line of lines) {
if (!line.startsWith('data: ')) continue;
try {
const evt = JSON.parse(line.slice(6));
if (evt.type === 'outline') {
collected.push(evt.data);
setStreamingOutlines([...collected]);
} else if (evt.type === 'retry') {
collected.length = 0;
setStreamingOutlines([]);
setStatusMessage(t('generation.outlineRetrying'));
} else if (evt.type === 'done') {
resolve(evt.outlines || collected);
return;
} else if (evt.type === 'error') {
reject(new Error(evt.error));
return;
}
} catch (e) {
log.error('Failed to parse outline SSE:', line, e);
}
}
}
if (done) {
if (collected.length > 0) {
resolve(collected);
} else {
reject(new Error(t('generation.outlineEmptyResponse')));
}
return;
}
return pump();
});
pump().catch(reject);
})
.catch(reject);
});
const updatedSession = { ...currentSession, sceneOutlines: outlines };
setSession(updatedSession);
sessionStorage.setItem('generationSession', JSON.stringify(updatedSession));
// Outline generation succeeded — clear homepage draft cache
try {
localStorage.removeItem('requirementDraft');
} catch {
/* ignore */
}
// Brief pause to let user see the final outline state
await new Promise((resolve) => setTimeout(resolve, 800));
}
// Move to scene generation step
setStatusMessage('');
if (!outlines || outlines.length === 0) {
@ -636,7 +661,6 @@ function GenerationPreviewContent() {
const stageInfo = {
name: stage.name,
description: stage.description,
language: stage.language,
style: stage.style,
};
@ -662,6 +686,7 @@ function GenerationPreviewContent() {
stageInfo,
stageId: stage.id,
agents,
languageDirective,
}),
signal,
});
@ -691,6 +716,7 @@ function GenerationPreviewContent() {
agents,
previousSpeeches: [],
userProfile,
languageDirective,
}),
signal,
});
@ -780,6 +806,7 @@ function GenerationPreviewContent() {
pdfImages: currentSession.pdfImages,
agents,
userProfile,
languageDirective,
}),
);

View File

@ -25,6 +25,8 @@ export interface GenerationSessionState {
// Web search context
researchContext?: string;
researchSources?: Array<{ title: string; url: string }>;
// Language directive inferred from outline generation
languageDirective?: string;
}
export type GenerationStep = {
@ -50,13 +52,6 @@ export const ALL_STEPS: GenerationStep[] = [
icon: Search,
type: 'analysis',
},
{
id: 'agent-generation',
title: 'generation.agentGeneration',
description: 'generation.agentGenerationDesc',
icon: Bot,
type: 'writing',
},
{
id: 'outline',
title: 'generation.generatingOutlines',
@ -64,6 +59,13 @@ export const ALL_STEPS: GenerationStep[] = [
icon: FileText,
type: 'writing',
},
{
id: 'agent-generation',
title: 'generation.agentGeneration',
description: 'generation.agentGenerationDesc',
icon: Bot,
type: 'writing',
},
{
id: 'slide-content',
title: 'generation.generatingSlideContent',

View File

@ -52,20 +52,17 @@ import { SpeechButton } from '@/components/audio/speech-button';
const log = createLogger('Home');
const WEB_SEARCH_STORAGE_KEY = 'webSearchEnabled';
const LANGUAGE_STORAGE_KEY = 'generationLanguage';
const RECENT_OPEN_STORAGE_KEY = 'recentClassroomsOpen';
interface FormState {
pdfFile: File | null;
requirement: string;
language: 'zh-CN' | 'en-US';
webSearch: boolean;
}
const initialFormState: FormState = {
pdfFile: null,
requirement: '',
language: 'zh-CN',
webSearch: false,
};
@ -98,15 +95,8 @@ function HomePage() {
}
try {
const savedWebSearch = localStorage.getItem(WEB_SEARCH_STORAGE_KEY);
const savedLanguage = localStorage.getItem(LANGUAGE_STORAGE_KEY);
const updates: Partial<FormState> = {};
if (savedWebSearch === 'true') updates.webSearch = true;
if (savedLanguage === 'zh-CN' || savedLanguage === 'en-US') {
updates.language = savedLanguage;
} else {
const detected = navigator.language?.startsWith('zh') ? 'zh-CN' : 'en-US';
updates.language = detected;
}
if (Object.keys(updates).length > 0) {
setForm((prev) => ({ ...prev, ...updates }));
}
@ -200,7 +190,6 @@ function HomePage() {
setForm((prev) => ({ ...prev, [field]: value }));
try {
if (field === 'webSearch') localStorage.setItem(WEB_SEARCH_STORAGE_KEY, String(value));
if (field === 'language') localStorage.setItem(LANGUAGE_STORAGE_KEY, String(value));
if (field === 'requirement') updateRequirementCache(value as string);
} catch {
/* ignore */
@ -260,7 +249,6 @@ function HomePage() {
const userProfile = useUserProfileStore.getState();
const requirements: UserRequirements = {
requirement: form.requirement,
language: form.language,
userNickname: userProfile.nickname || undefined,
userBio: userProfile.bio || undefined,
webSearch: form.webSearch || undefined,
@ -500,8 +488,6 @@ function HomePage() {
<div className="px-3 pb-3 flex items-end gap-2">
<div className="flex-1 min-w-0">
<GenerationToolbar
language={form.language}
onLanguageChange={(lang) => updateForm('language', lang)}
webSearch={form.webSearch}
onWebSearchChange={(v) => updateForm('webSearch', v)}
onSettingsOpen={(section) => {

View File

@ -71,7 +71,12 @@ function AgentVoicePill({
}, []);
const handlePreview = useCallback(
async (providerId: TTSProviderId, voiceId: string, modelId?: string) => {
async (
providerId: TTSProviderId,
voiceId: string,
modelId?: string,
voiceLanguage?: string,
) => {
const key = `${providerId}::${voiceId}`;
if (previewingId === key) {
stopPreview();
@ -80,10 +85,8 @@ function AgentVoicePill({
stopPreview();
setPreviewingId(key);
const courseLanguage =
(typeof localStorage !== 'undefined' && localStorage.getItem('generationLanguage')) ||
'zh-CN';
const previewText = courseLanguage === 'en-US' ? 'Welcome to AI Classroom' : '欢迎来到AI课堂';
const isEnglish = voiceLanguage?.startsWith('en') ?? false;
const previewText = isEnglish ? 'Welcome to AI Classroom' : '欢迎来到AI课堂';
if (providerId === 'browser-native-tts') {
const { promise, cancel } = playBrowserTTSPreview({ text: previewText, voice: voiceId });
@ -226,7 +229,7 @@ function AgentVoicePill({
type="button"
onClick={(e) => {
e.stopPropagation();
handlePreview(provider.providerId, voice.id, group.modelId);
handlePreview(provider.providerId, voice.id, group.modelId, voice.language);
}}
className={cn(
'shrink-0 size-6 flex items-center justify-center rounded-sm transition-colors',
@ -299,7 +302,12 @@ function TeacherVoicePill({
}, []);
const handlePreview = useCallback(
async (providerId: TTSProviderId, voiceId: string, modelId?: string) => {
async (
providerId: TTSProviderId,
voiceId: string,
modelId?: string,
voiceLanguage?: string,
) => {
const key = `${providerId}::${voiceId}`;
if (previewingId === key) {
stopPreview();
@ -308,10 +316,8 @@ function TeacherVoicePill({
stopPreview();
setPreviewingId(key);
const courseLanguage =
(typeof localStorage !== 'undefined' && localStorage.getItem('generationLanguage')) ||
'zh-CN';
const previewText = courseLanguage === 'en-US' ? 'Welcome to AI Classroom' : '欢迎来到AI课堂';
const isEnglish = voiceLanguage?.startsWith('en') ?? false;
const previewText = isEnglish ? 'Welcome to AI Classroom' : '欢迎来到AI课堂';
if (providerId === 'browser-native-tts') {
const { promise, cancel } = playBrowserTTSPreview({ text: previewText, voice: voiceId });
@ -450,7 +456,7 @@ function TeacherVoicePill({
type="button"
onClick={(e) => {
e.stopPropagation();
handlePreview(provider.providerId, voice.id, group.modelId);
handlePreview(provider.providerId, voice.id, group.modelId, voice.language);
}}
className={cn(
'shrink-0 size-6 flex items-center justify-center rounded-sm transition-colors',

View File

@ -1,7 +1,7 @@
'use client';
import { useState, useRef, useMemo } from 'react';
import { Bot, Check, ChevronLeft, Globe, Paperclip, FileText, X, Globe2 } from 'lucide-react';
import { Bot, Check, ChevronLeft, Paperclip, FileText, X, Globe2 } from 'lucide-react';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
import {
Select,
@ -29,8 +29,6 @@ const MAX_PDF_SIZE_BYTES = MAX_PDF_SIZE_MB * 1024 * 1024;
// ─── Types ───────────────────────────────────────────────────
export interface GenerationToolbarProps {
language: 'zh-CN' | 'en-US';
onLanguageChange: (lang: 'zh-CN' | 'en-US') => void;
webSearch: boolean;
onWebSearchChange: (v: boolean) => void;
onSettingsOpen: (section?: SettingsSection) => void;
@ -42,8 +40,6 @@ export interface GenerationToolbarProps {
// ─── Component ───────────────────────────────────────────────
export function GenerationToolbar({
language,
onLanguageChange,
webSearch,
onWebSearchChange,
onSettingsOpen,
@ -358,20 +354,6 @@ export function GenerationToolbar({
</Tooltip>
)}
{/* ── Language pill ── */}
<Tooltip>
<TooltipTrigger asChild>
<button
onClick={() => onLanguageChange(language === 'zh-CN' ? 'en-US' : 'zh-CN')}
className={pillMuted}
>
<Globe className="size-3.5" />
<span>{language === 'zh-CN' ? '中文' : 'EN'}</span>
</button>
</TooltipTrigger>
<TooltipContent>{t('toolbar.languageHint')}</TooltipContent>
</Tooltip>
{/* ── Separator ── */}
<div className="w-px h-4 bg-border/60 mx-1" />

View File

@ -44,16 +44,12 @@ export function PBLRenderer({ content, mode: _mode, sceneId }: PBLRendererProps)
// Add Question Agent welcome message if chat is empty and active issue has questions
const activeIssue = newConfig.issueboard.issues.find((i) => i.is_active);
if (activeIssue?.generated_questions && newConfig.chat.messages.length === 0) {
const welcomeMsg = t('pbl.chat.welcomeMessage', {
title: activeIssue.title,
questions: activeIssue.generated_questions,
});
newConfig.chat = {
messages: [
{
id: `msg_welcome_${Date.now()}`,
agent_name: activeIssue.question_agent_name,
message: welcomeMsg,
message: activeIssue.generated_questions,
timestamp: Date.now(),
read_by: [],
},

View File

@ -221,14 +221,11 @@ async function handleIssueComplete(
if (data.success && data.message) {
nextIssue.generated_questions = data.message;
// Add Question Agent welcome message
// Use LLM-generated content directly (already in the correct language)
config.chat.messages.push({
id: `msg_${Date.now()}_welcome`,
agent_name: nextIssue.question_agent_name,
message: t('pbl.chat.welcomeMessage', {
title: nextIssue.title,
questions: data.message,
}),
message: data.message,
timestamp: Date.now(),
read_by: [],
});
@ -237,14 +234,11 @@ async function handleIssueComplete(
log.error('[usePBLChat] Failed to generate questions for next issue:', error);
}
} else if (questionAgent && nextIssue.generated_questions) {
// Questions already exist, just add welcome message
// Questions already exist, use directly
config.chat.messages.push({
id: `msg_${Date.now()}_welcome`,
agent_name: nextIssue.question_agent_name,
message: t('pbl.chat.welcomeMessage', {
title: nextIssue.title,
questions: nextIssue.generated_questions,
}),
message: nextIssue.generated_questions,
timestamp: Date.now(),
read_by: [],
});

View File

@ -80,13 +80,13 @@ export function getServerVoiceList(
export interface ModelVoiceGroup {
modelId: string;
modelName: string;
voices: Array<{ id: string; name: string }>;
voices: Array<{ id: string; name: string; language?: string }>;
}
export interface ProviderWithVoices {
providerId: TTSProviderId;
providerName: string;
voices: Array<{ id: string; name: string }>; // keep for backward compat
voices: Array<{ id: string; name: string; language?: string }>;
modelGroups: ModelVoiceGroup[]; // voices grouped by model
}
@ -122,7 +122,11 @@ export function getAvailableProvidersWithVoices(
const isServerConfigured = providerConfig?.isServerConfigured === true;
if (hasApiKey || isServerConfigured) {
const allVoices = config.voices.map((v) => ({ id: v.id, name: v.name }));
const allVoices = config.voices.map((v) => ({
id: v.id,
name: v.name,
language: v.language,
}));
// Build model groups
const modelGroups: ModelVoiceGroup[] = [];
@ -130,7 +134,7 @@ export function getAvailableProvidersWithVoices(
for (const model of config.models) {
const compatibleVoices = config.voices
.filter((v) => !v.compatibleModels || v.compatibleModels.includes(model.id))
.map((v) => ({ id: v.id, name: v.name }));
.map((v) => ({ id: v.id, name: v.name, language: v.language }));
modelGroups.push({
modelId: model.id,
modelName: model.name,

View File

@ -23,6 +23,7 @@ export {
formatImageDescription,
formatImagePlaceholder,
buildVisionUserContent,
buildLanguageText,
} from './prompt-formatters';
// JSON repair
@ -38,6 +39,7 @@ export {
generateSceneActions,
createSceneWithActions,
} from './scene-generator';
export type { SceneContentOptions, SceneActionsOptions } from './scene-generator';
// Scene builder (standalone)
export {

View File

@ -37,10 +37,9 @@ export async function generateSceneOutlinesFromRequirements(
researchContext?: string;
teacherContext?: string;
},
): Promise<GenerationResult<SceneOutline[]>> {
): Promise<GenerationResult<{ languageDirective: string; outlines: SceneOutline[] }>> {
// Build available images description for the prompt
let availableImagesText =
requirements.language === 'zh-CN' ? '无可用图片' : 'No images available';
let availableImagesText = 'No images available';
let visionImages: Array<{ id: string; src: string }> | undefined;
if (pdfImages && pdfImages.length > 0) {
@ -51,11 +50,9 @@ export async function generateSceneOutlinesFromRequirements(
const textOnlySlice = allWithSrc.slice(MAX_VISION_IMAGES);
const noSrcImages = pdfImages.filter((img) => !options.imageMapping![img.id]);
const visionDescriptions = visionSlice.map((img) =>
formatImagePlaceholder(img, requirements.language),
);
const visionDescriptions = visionSlice.map((img) => formatImagePlaceholder(img));
const textDescriptions = [...textOnlySlice, ...noSrcImages].map((img) =>
formatImageDescription(img, requirements.language),
formatImageDescription(img),
);
availableImagesText = [...visionDescriptions, ...textDescriptions].join('\n');
@ -67,9 +64,7 @@ export async function generateSceneOutlinesFromRequirements(
}));
} else {
// Text-only mode: full descriptions
availableImagesText = pdfImages
.map((img) => formatImageDescription(img, requirements.language))
.join('\n');
availableImagesText = pdfImages.map((img) => formatImageDescription(img)).join('\n');
}
}
@ -98,17 +93,11 @@ export async function generateSceneOutlinesFromRequirements(
const prompts = buildPrompt(PROMPT_IDS.REQUIREMENTS_TO_OUTLINES, {
// New simplified variables
requirement: requirements.requirement,
language: requirements.language,
pdfContent: pdfText
? pdfText.substring(0, MAX_PDF_CONTENT_CHARS)
: requirements.language === 'zh-CN'
? '无'
: 'None',
pdfContent: pdfText ? pdfText.substring(0, MAX_PDF_CONTENT_CHARS) : 'None',
availableImages: availableImagesText,
userProfile: userProfileText,
mediaGenerationPolicy,
researchContext:
options?.researchContext || (requirements.language === 'zh-CN' ? '无' : 'None'),
researchContext: options?.researchContext || 'None',
// Server-side generation populates this via options; client-side populates via formatTeacherPersonaForPrompt
teacherContext: options?.teacherContext || '',
});
@ -128,20 +117,34 @@ export async function generateSceneOutlinesFromRequirements(
});
const response = await aiCall(prompts.system, prompts.user, visionImages);
const outlines = parseJsonResponse<SceneOutline[]>(response);
const parsed = parseJsonResponse<
{ languageDirective: string; outlines: SceneOutline[] } | SceneOutline[]
>(response);
if (!outlines || !Array.isArray(outlines)) {
return {
success: false,
error: 'Failed to parse scene outlines response',
};
let languageDirective: string;
let rawOutlines: SceneOutline[];
if (Array.isArray(parsed)) {
// Fallback: LLM returned old flat array format
languageDirective = 'Teach in the language that matches the user requirement.';
rawOutlines = parsed;
} else if (parsed && parsed.outlines) {
languageDirective =
parsed.languageDirective || 'Teach in the language that matches the user requirement.';
rawOutlines = parsed.outlines;
} else {
return { success: false, error: 'Failed to parse scene outlines response' };
}
// Ensure IDs, order, and language
const enriched = outlines.map((outline, index) => ({
if (!Array.isArray(rawOutlines)) {
return { success: false, error: 'Failed to parse scene outlines response' };
}
// Ensure IDs and order
const enriched = rawOutlines.map((outline, index) => ({
...outline,
id: outline.id || nanoid(),
order: index + 1,
language: requirements.language,
}));
// Replace sequential gen_img_N/gen_vid_N with globally unique IDs
@ -156,7 +159,7 @@ export async function generateSceneOutlinesFromRequirements(
totalScenes: result.length,
});
return { success: true, data: result };
return { success: true, data: { languageDirective, outlines: result } };
} catch (error) {
return { success: false, error: String(error) };
}

View File

@ -39,7 +39,7 @@ export async function runGenerationPipeline(
...session.progress,
currentStage: 1,
overallProgress: 5,
statusMessage: '正在分析需求,生成场景大纲...',
statusMessage: 'Analyzing requirements, generating outlines...',
});
const outlinesResult = await generateSceneOutlinesFromRequirements(
@ -52,19 +52,26 @@ export async function runGenerationPipeline(
if (!outlinesResult.success || !outlinesResult.data) {
throw new Error(outlinesResult.error || 'Failed to generate scene outlines');
}
session.sceneOutlines = outlinesResult.data;
callbacks?.onStageComplete?.(1, session.sceneOutlines);
const { outlines, languageDirective } = outlinesResult.data;
session.sceneOutlines = outlines;
callbacks?.onStageComplete?.(1, outlines);
// Stage 2: Generate Full Scenes
callbacks?.onProgress?.({
...session.progress,
currentStage: 2,
overallProgress: 50,
statusMessage: '正在生成场景内容...',
totalScenes: session.sceneOutlines.length,
statusMessage: 'Generating scene content...',
totalScenes: outlines.length,
});
const scenesResult = await generateFullScenes(session.sceneOutlines, store, aiCall, callbacks);
const scenesResult = await generateFullScenes(
outlines,
store,
aiCall,
callbacks,
languageDirective,
);
if (!scenesResult.success) {
throw new Error(scenesResult.error || 'Failed to generate scenes');
}
@ -76,9 +83,9 @@ export async function runGenerationPipeline(
currentStage: 2,
overallProgress: 100,
stageProgress: 100,
statusMessage: '生成完成!',
statusMessage: 'Generation complete!',
scenesGenerated: scenesResult.data?.length || 0,
totalScenes: session.sceneOutlines.length,
totalScenes: session.sceneOutlines?.length || 0,
};
return { success: true, data: session };

View File

@ -75,31 +75,27 @@ export function formatTeacherPersonaForPrompt(agents?: AgentInfo[]): string {
* Format a single PdfImage description for prompt inclusion.
* Includes dimension/aspect-ratio info when available.
*/
export function formatImageDescription(img: PdfImage, language: string): string {
export function formatImageDescription(img: PdfImage): string {
let dimInfo = '';
if (img.width && img.height) {
const ratio = (img.width / img.height).toFixed(2);
dimInfo = ` | 尺寸: ${img.width}×${img.height} (宽高比${ratio})`;
dimInfo = ` | size: ${img.width}×${img.height} (aspect ratio ${ratio})`;
}
const desc = img.description ? ` | ${img.description}` : '';
return language === 'zh-CN'
? `- **${img.id}**: 来自PDF第${img.pageNumber}页${dimInfo}${desc}`
: `- **${img.id}**: from PDF page ${img.pageNumber}${dimInfo}${desc}`;
return `- **${img.id}**: from PDF page ${img.pageNumber}${dimInfo}${desc}`;
}
/**
* Format a short image placeholder for vision mode.
* Only ID + page + dimensions + aspect ratio (no description), since the model can see the actual image.
*/
export function formatImagePlaceholder(img: PdfImage, language: string): string {
export function formatImagePlaceholder(img: PdfImage): string {
let dimInfo = '';
if (img.width && img.height) {
const ratio = (img.width / img.height).toFixed(2);
dimInfo = ` | 尺寸: ${img.width}×${img.height} (宽高比${ratio})`;
dimInfo = ` | size: ${img.width}×${img.height} (aspect ratio ${ratio})`;
}
return language === 'zh-CN'
? `- **${img.id}**: PDF第${img.pageNumber}页的图片${dimInfo} [参见附图]`
: `- **${img.id}**: image from PDF page ${img.pageNumber}${dimInfo} [see attached]`;
return `- **${img.id}**: image from PDF page ${img.pageNumber}${dimInfo} [see attached]`;
}
/**
@ -121,7 +117,7 @@ export function buildVisionUserContent(
let dimInfo = '';
if (img.width && img.height) {
const ratio = (img.width / img.height).toFixed(2);
dimInfo = ` (${img.width}×${img.height}, 宽高比${ratio})`;
dimInfo = ` (${img.width}×${img.height}, aspect ratio ${ratio})`;
}
parts.push({ type: 'text', text: `\n**${img.id}**${dimInfo}:` });
// Strip data URI prefix — AI SDK only accepts http(s) URLs or raw base64
@ -139,3 +135,16 @@ export function buildVisionUserContent(
}
return parts;
}
/**
* Build language instruction text from course-level directive and optional per-scene note.
* Used by scene content and action generators to inject into prompt templates.
*/
export function buildLanguageText(directive?: string, sceneNote?: string): string {
if (!directive && !sceneNote) return '';
let text = directive || '';
if (sceneNote) {
text += (text ? '\n\n' : '') + `Additional language note for this scene: ${sceneNote}`;
}
return text;
}

View File

@ -6,7 +6,7 @@ Key Points: {{keyPoints}}
{{courseContext}}
{{agents}}
**Language Requirement**: Generated speech content must be in the same language as the key points above.
**Language Directive**: {{languageDirective}}
Output as a JSON array directly (no explanation, no code fences, 3-6 speech segments):
[{"type":"text","content":"Opening speech content"}]

View File

@ -25,11 +25,11 @@ The following constraints must be strictly obeyed in all JavaScript logic and vi
---
## Language
## Language Directive
**Page language**: {{language}}
{{languageDirective}}
(All UI text, labels, instructions, and descriptions must be in this language)
(All UI text, labels, instructions, and descriptions must follow the language directive above)
---

View File

@ -8,6 +8,8 @@
{{courseContext}}
{{agents}}
**Language Directive**: {{languageDirective}}
Please generate the speech content for this PBL scene.
Output as a JSON array directly (no explanation, no code fences):

View File

@ -5,7 +5,7 @@ Description: {{description}}
{{courseContext}}
{{agents}}
**Language Requirement**: Generated speech content must be in the same language as the key points above.
**Language Directive**: {{languageDirective}}
Output as a JSON array directly (no explanation, no code fences, 3-6 segments):
[{"type":"text","content":"Let's test your understanding"}]

View File

@ -3,7 +3,8 @@ Description: {{description}}
Test Points: {{keyPoints}}
Question Count: {{questionCount}}, Difficulty: {{difficulty}}, Question Types: {{questionTypes}}
**Language Requirement**: Questions and options must be in the same language as the title and description above.
## Language Directive
{{languageDirective}}
Output JSON array directly (no explanation, no code blocks, no LaTeX):
[{"id":"q1","type":"single","question":"Question text","options":["Option A","Option B","Option C","Option D"],"correctAnswer":"Option A"}]

View File

@ -14,6 +14,39 @@ Based on the user's free-form requirement text, automatically infer course detai
---
## Language Inference
Infer the course language from all available signals and produce:
1. **`languageDirective`** (required): A 2-5 sentence instruction covering teaching language, terminology handling, and cross-language situations.
2. **`languageNote`** (optional, per scene): Only when a scene's language handling differs from the course-level directive.
### Decision rules (apply in order)
1. **Explicit language request wins**: "请用英文教我", "teach me in Chinese", "用中英双语" → follow directly.
2. **Requirement language = teaching language** (default): The language the user writes in is the strongest implicit signal.
3. **Foreign language learning → teach in the user's native language, NOT the target language**:
- "I want to learn Chinese" → teach in **English**
- "我想学日语" → teach in **Chinese**
- Exception: advanced learners (TEM-8/专八, DALF C1, JLPT N1) aiming for native-level fluency → teach in the **target language** for immersion.
4. **Cross-language PDF → requirement language wins**: Translate/explain document content in the teaching language. Never let the PDF language override the requirement language.
5. **Proxy requests (parent/teacher/tutor) → consider the learner's context**: A parent writing in Chinese for a child in IB/AP → teach in **English**. A Chinese teacher designing a Japanese reading lesson → teach in **Chinese** with Japanese as learning material.
6. **Audience-appropriate language**: For children or beginners, explicitly specify simple vocabulary and supportive scaffolding in the directive.
### Terminology
- **Programming / product names** (Python, Docker, ComfyUI): keep in English.
- **Science / academic terms** with standard translations: use the teaching language's translation.
- **Emerging tech terms** (AI/ML): show bilingually.
- **User's explicit request** about terminology overrides the above defaults.
---
## Design Principles
### MAIC Platform Technical Constraints
@ -171,7 +204,7 @@ Use `pbl` type when the course involves complex, multi-step project work that be
**Constraints**:
- Limit to **at most 1 PBL scene per course** (they are comprehensive and long)
- PBL scenes **require** a `pblConfig` object with: projectTopic, projectDescription, targetSkills, issueCount, language
- PBL scenes **require** a `pblConfig` object with: projectTopic, projectDescription, targetSkills, issueCount
- PBL is for substantial project work - do NOT use for simple exercises or single-step tasks
- The `pblConfig.targetSkills` should list 2-5 specific skills students will develop
- The `pblConfig.issueCount` should typically be 2-5 issues
@ -180,57 +213,60 @@ Use `pbl` type when the course involves complex, multi-step project work that be
## Output Format
You must output a JSON array where each element is a scene outline object:
Output a JSON **object** (not a bare array) with this structure:
```json
[
{
"id": "scene_1",
"type": "slide",
"title": "Scene Title",
"description": "1-2 sentences describing the teaching purpose",
"keyPoints": ["Key point 1", "Key point 2", "Key point 3"],
"teachingObjective": "Corresponding learning objective",
"estimatedDuration": 120,
"order": 1,
"suggestedImageIds": ["img_1"],
"mediaGenerations": [
{
"type": "image",
"prompt": "A diagram showing the key concept",
"elementId": "gen_img_1",
"aspectRatio": "16:9"
{
"languageDirective": "2-5 sentence instruction describing the course language behavior",
"outlines": [
{
"id": "scene_1",
"type": "slide",
"title": "Scene Title",
"description": "1-2 sentences describing the teaching purpose",
"keyPoints": ["Key point 1", "Key point 2", "Key point 3"],
"teachingObjective": "Corresponding learning objective",
"estimatedDuration": 120,
"order": 1,
"suggestedImageIds": ["img_1"],
"mediaGenerations": [
{
"type": "image",
"prompt": "A diagram showing the key concept",
"elementId": "gen_img_1",
"aspectRatio": "16:9"
}
]
},
{
"id": "scene_2",
"type": "interactive",
"title": "Interactive Exploration",
"description": "Students explore the concept through hands-on interactive visualization",
"keyPoints": ["Interactive element 1", "Observable phenomenon"],
"order": 2,
"interactiveConfig": {
"conceptName": "Concept Name",
"conceptOverview": "Brief description of what this interactive demonstrates",
"designIdea": "Describe the interactive elements: sliders, drag handles, animations, etc.",
"subject": "Physics"
}
},
{
"id": "scene_3",
"type": "quiz",
"title": "Knowledge Check",
"description": "Test student understanding of XX concept",
"keyPoints": ["Test point 1", "Test point 2"],
"order": 3,
"quizConfig": {
"questionCount": 2,
"difficulty": "medium",
"questionTypes": ["single", "multiple", "short_answer"]
}
]
},
{
"id": "scene_2",
"type": "interactive",
"title": "Interactive Exploration",
"description": "Students explore the concept through hands-on interactive visualization",
"keyPoints": ["Interactive element 1", "Observable phenomenon"],
"order": 2,
"interactiveConfig": {
"conceptName": "Concept Name",
"conceptOverview": "Brief description of what this interactive demonstrates",
"designIdea": "Describe the interactive elements: sliders, drag handles, animations, etc.",
"subject": "Physics"
}
},
{
"id": "scene_3",
"type": "quiz",
"title": "Knowledge Check",
"description": "Test student understanding of XX concept",
"keyPoints": ["Test point 1", "Test point 2"],
"order": 3,
"quizConfig": {
"questionCount": 2,
"difficulty": "medium",
"questionTypes": ["single", "multiple", "short_answer"]
}
}
]
]
}
```
### Field Descriptions
@ -249,7 +285,8 @@ You must output a JSON array where each element is a scene outline object:
| mediaGenerations | MediaGenerationRequest[] | ❌ | AI image/video generation requests when PDF images insufficient |
| quizConfig | object | ❌ | Required for quiz type, contains questionCount/difficulty/questionTypes |
| interactiveConfig | object | ❌ | Required for interactive type, contains conceptName/conceptOverview/designIdea/subject |
| pblConfig | object | ❌ | Required for pbl type, contains projectTopic/projectDescription/targetSkills/issueCount/language |
| languageNote | string | ❌ | Optional. Only include when this scene has language considerations not covered by the course-level languageDirective. |
| pblConfig | object | ❌ | Required for pbl type, contains projectTopic/projectDescription/targetSkills/issueCount |
### quizConfig Structure
@ -279,8 +316,7 @@ You must output a JSON array where each element is a scene outline object:
"projectTopic": "Main topic of the project",
"projectDescription": "Brief description of what students will build/accomplish",
"targetSkills": ["Skill 1", "Skill 2", "Skill 3"],
"issueCount": 3,
"language": "zh-CN"
"issueCount": 3
}
```
@ -288,14 +324,14 @@ You must output a JSON array where each element is a scene outline object:
## Important Reminders
1. **Must output valid JSON array format**
1. **Must output valid JSON object with `languageDirective` and `outlines` fields**
2. **type can be `"slide"`, `"quiz"`, `"interactive"`, or `"pbl"`**
3. **quiz type must include quizConfig**
4. **interactive type must include interactiveConfig** - with conceptName, conceptOverview, designIdea, and subject
5b. **pbl type must include pblConfig** - with projectTopic, projectDescription, targetSkills, issueCount, and language
5b. **pbl type must include pblConfig** - with projectTopic, projectDescription, targetSkills, and issueCount
5. Arrange appropriate number of scenes based on inferred duration (typically 1-2 scenes per minute)
6. Insert quizzes at appropriate points for knowledge checks
7. Use interactive scenes sparingly (max 1-2 per course) and only when the concept truly benefits from hands-on interaction
8. **Language Requirement**: Strictly output all content in the language specified by the user
8. **Language**: Infer from the user's requirement text and context. Output all scene content in the inferred language.
9. Regardless of information completeness, always output conforming JSON - do not ask questions or request more information
10. **No teacher identity on slides**: Scene titles and keyPoints must be neutral and topic-focused. Never include the teacher's name or role (e.g., avoid "Teacher Wang's Tips", "Teacher's Wishes"). Use generic labels like "Tips", "Summary", "Key Takeaways" instead.

View File

@ -10,11 +10,12 @@ Please generate scene outlines based on the following course requirements.
{{userProfile}}
## Course Language
## Language Context
**Required language**: {{language}}
(If language is zh-CN, all content must be in Chinese; if en-US, all content must be in English)
Infer the course language directive by applying the decision rules from the system prompt. Key reminders:
- Requirement language = teaching language (unless overridden by explicit request or learner context)
- Foreign language learning → teach in user's native language, not the target language
- PDF language does NOT override teaching language — translate/explain document content instead
---
@ -46,16 +47,21 @@ Please automatically infer the following from user requirements:
- Teaching style (formal/casual/interactive/academic)
- Visual style (minimal/colorful/professional/playful)
Then output a JSON array containing all scene outlines. Each scene must include:
Then output a JSON object with `languageDirective` and `outlines`. Each scene in the `outlines` array must include:
```json
{
"id": "scene_1",
"type": "slide" or "quiz" or "interactive",
"title": "Scene Title",
"description": "Teaching purpose description",
"keyPoints": ["Point 1", "Point 2", "Point 3"],
"order": 1
"languageDirective": "2-5 sentence instruction describing the course language behavior",
"outlines": [
{
"id": "scene_1",
"type": "slide" or "quiz" or "interactive",
"title": "Scene Title",
"description": "Teaching purpose description",
"keyPoints": ["Point 1", "Point 2", "Point 3"],
"order": 1
}
]
}
```
@ -73,10 +79,10 @@ Then output a JSON array containing all scene outlines. Each scene must include:
3. **Interactive scenes**: If a concept benefits from hands-on simulation/visualization, use `"type": "interactive"` with an `interactiveConfig` object containing `conceptName`, `conceptOverview`, `designIdea`, and `subject`. Limit to 1-2 per course.
4. **Scene count**: Based on inferred duration, typically 1-2 scenes per minute
5. **Quiz placement**: Recommend inserting a quiz every 3-5 slides for assessment
6. **Language**: Strictly output all content in the specified course language
6. **Language**: Infer from the user's requirement text and context, then output all content in the inferred language
7. **If no suitable PDF images exist** for a slide scene that would benefit from visuals, add `mediaGenerations` array with image generation prompts. Write prompts in English. Use `elementId` format like "gen_img_1", "gen_img_2" — IDs must be **globally unique across all scenes** (do NOT restart numbering per scene). To reuse a generated image in a different scene, reference the same elementId without re-declaring it in mediaGenerations. Each generated image should be visually distinct — avoid near-identical media across slides.
8. **If web search results are provided**, reference specific findings and sources in scene descriptions and keyPoints. The search results provide up-to-date information — incorporate it to make the course content current and accurate.
{{mediaGenerationPolicy}}
Please output JSON array directly without additional explanatory text.
Please output a JSON object with `languageDirective` (string) and `outlines` (array) directly without additional explanatory text.

View File

@ -6,7 +6,7 @@ Description: {{description}}
{{agents}}
{{userProfile}}
**Language Requirement**: Generated speech content must be in the same language as the key points above.
**Language Directive**: {{languageDirective}}
Output as a JSON array directly (no explanation, no code fences, 5-10 segments):
[{"type":"action","name":"spotlight","params":{"elementId":"text_xxx"}},{"type":"text","content":"Opening speech content"}]

View File

@ -18,7 +18,8 @@
Based on the scene information above, generate a complete Canvas/PPT component for one page.
**Language Requirement**: All generated text content must be in the same language as the title and description above.
## Language Directive
{{languageDirective}}
**Must Follow**:

View File

@ -20,6 +20,7 @@ import type { Action } from '@/lib/types/action';
import { applyOutlineFallbacks } from './outline-generator';
import { generateSceneContent, generateSceneActions } from './scene-generator';
import type { AgentInfo, SceneGenerationContext, AICallFn } from './pipeline-types';
import { buildLanguageText } from './prompt-formatters';
import { createLogger } from '@/lib/logger';
const log = createLogger('Generation');
@ -76,10 +77,13 @@ export async function buildSceneFromOutline(
agents?: AgentInfo[],
onPhaseChange?: (phase: 'content' | 'actions') => void,
userProfile?: string,
languageDirective?: string,
): Promise<Scene | null> {
// Apply type fallbacks
outline = applyOutlineFallbacks(outline, !!languageModel);
const langText = buildLanguageText(languageDirective, outline.languageNote);
// Step 1: Generate content (with images if available)
onPhaseChange?.('content');
log.debug(`Step 1: Generating content for: ${outline.title}`);
@ -91,16 +95,14 @@ export async function buildSceneFromOutline(
log.debug(
`imageMapping available: ${imageMapping ? Object.keys(imageMapping).length + ' keys' : 'undefined'}`,
);
const content = await generateSceneContent(
outline,
aiCall,
const content = await generateSceneContent(outline, aiCall, {
assignedImages,
imageMapping,
languageModel,
visionEnabled,
undefined,
agents,
);
languageDirective: langText,
});
if (!content) {
log.error(`Failed to generate content for: ${outline.title}`);
return null;
@ -109,7 +111,12 @@ export async function buildSceneFromOutline(
// Step 2: Generate Actions
onPhaseChange?.('actions');
log.debug(`Step 2: Generating actions for: ${outline.title}`);
const actions = await generateSceneActions(outline, content, aiCall, ctx, agents, userProfile);
const actions = await generateSceneActions(outline, content, aiCall, {
ctx,
agents,
userProfile,
languageDirective: langText,
});
log.debug(`Generated ${actions.length} actions for: ${outline.title}`);
// Build complete Scene object

View File

@ -28,6 +28,7 @@ import { parseActionsFromStructuredOutput } from './action-parser';
import { parseJsonResponse } from './json-repair';
import {
buildCourseContext,
buildLanguageText,
formatAgentsForPrompt,
formatTeacherPersonaForPrompt,
formatImageDescription,
@ -47,6 +48,25 @@ import type {
import { createLogger } from '@/lib/logger';
const log = createLogger('Generation');
// ── Options interfaces for scene generation functions ──
export interface SceneContentOptions {
assignedImages?: PdfImage[];
imageMapping?: ImageMapping;
languageModel?: LanguageModel;
visionEnabled?: boolean;
generatedMediaMapping?: ImageMapping;
agents?: AgentInfo[];
languageDirective?: string;
}
export interface SceneActionsOptions {
ctx?: SceneGenerationContext;
agents?: AgentInfo[];
userProfile?: string;
languageDirective?: string;
}
// ==================== Stage 2: Full Scenes (Two-Step) ====================
/**
@ -63,6 +83,7 @@ export async function generateFullScenes(
store: StageStore,
aiCall: AICallFn,
callbacks?: GenerationCallbacks,
languageDirective?: string,
): Promise<GenerationResult<string[]>> {
const api = createStageAPI(store);
const totalScenes = sceneOutlines.length;
@ -81,7 +102,7 @@ export async function generateFullScenes(
const results = await Promise.all(
sceneOutlines.map(async (outline, index) => {
try {
const sceneId = await generateSingleScene(outline, api, aiCall);
const sceneId = await generateSingleScene(outline, api, aiCall, languageDirective);
// Update progress (not atomic, but sufficient for UI display)
completedCount++;
@ -125,10 +146,11 @@ async function generateSingleScene(
outline: SceneOutline,
api: ReturnType<typeof createStageAPI>,
aiCall: AICallFn,
languageDirective?: string,
): Promise<string | null> {
// Step 3.1: Generate content
log.info(`Step 3.1: Generating content for: ${outline.title}`);
const content = await generateSceneContent(outline, aiCall);
const content = await generateSceneContent(outline, aiCall, { languageDirective });
if (!content) {
log.error(`Failed to generate content for: ${outline.title}`);
return null;
@ -136,7 +158,7 @@ async function generateSingleScene(
// Step 3.2: Generate Actions
log.info(`Step 3.2: Generating actions for: ${outline.title}`);
const actions = await generateSceneActions(outline, content, aiCall);
const actions = await generateSceneActions(outline, content, aiCall, { languageDirective });
log.info(`Generated ${actions.length} actions for: ${outline.title}`);
// Create complete Scene
@ -149,12 +171,7 @@ async function generateSingleScene(
export async function generateSceneContent(
outline: SceneOutline,
aiCall: AICallFn,
assignedImages?: PdfImage[],
imageMapping?: ImageMapping,
languageModel?: LanguageModel,
visionEnabled?: boolean,
generatedMediaMapping?: ImageMapping,
agents?: AgentInfo[],
options: SceneContentOptions = {},
): Promise<
| GeneratedSlideContent
| GeneratedQuizContent
@ -162,6 +179,15 @@ export async function generateSceneContent(
| GeneratedPBLContent
| null
> {
const {
assignedImages,
imageMapping,
languageModel,
visionEnabled,
generatedMediaMapping,
agents,
languageDirective,
} = options;
// If outline is interactive but missing interactiveConfig, fall back to slide
if (outline.type === 'interactive' && !outline.interactiveConfig) {
log.warn(
@ -176,6 +202,7 @@ export async function generateSceneContent(
visionEnabled,
generatedMediaMapping,
agents,
languageDirective,
);
}
@ -189,13 +216,14 @@ export async function generateSceneContent(
visionEnabled,
generatedMediaMapping,
agents,
languageDirective,
);
case 'quiz':
return generateQuizContent(outline, aiCall);
return generateQuizContent(outline, aiCall, languageDirective);
case 'interactive':
return generateInteractiveContent(outline, aiCall, outline.language);
return generateInteractiveContent(outline, aiCall, languageDirective);
case 'pbl':
return generatePBLSceneContent(outline, languageModel);
return generatePBLSceneContent(outline, languageModel, languageDirective);
default:
return null;
}
@ -466,11 +494,10 @@ async function generateSlideContent(
visionEnabled?: boolean,
generatedMediaMapping?: ImageMapping,
agents?: AgentInfo[],
languageDirective?: string,
): Promise<GeneratedSlideContent | null> {
const lang = outline.language || 'zh-CN';
// Build assigned images description for the prompt
let assignedImagesText = '无可用图片,禁止插入任何 image 元素';
let assignedImagesText = 'No images available. Do NOT insert any image elements.';
let visionImages: Array<{ id: string; src: string }> | undefined;
if (assignedImages && assignedImages.length > 0) {
@ -481,9 +508,9 @@ async function generateSlideContent(
const textOnlySlice = withSrc.slice(MAX_VISION_IMAGES);
const noSrcImages = assignedImages.filter((img) => !imageMapping[img.id]);
const visionDescriptions = visionSlice.map((img) => formatImagePlaceholder(img, lang));
const visionDescriptions = visionSlice.map((img) => formatImagePlaceholder(img));
const textDescriptions = [...textOnlySlice, ...noSrcImages].map((img) =>
formatImageDescription(img, lang),
formatImageDescription(img),
);
assignedImagesText = [...visionDescriptions, ...textDescriptions].join('\n');
@ -494,9 +521,7 @@ async function generateSlideContent(
height: img.height,
}));
} else {
assignedImagesText = assignedImages
.map((img) => formatImageDescription(img, lang))
.join('\n');
assignedImagesText = assignedImages.map((img) => formatImageDescription(img)).join('\n');
}
}
@ -521,7 +546,7 @@ async function generateSlideContent(
if (mediaParts.length > 0) {
const mediaText = mediaParts.join('\n\n');
if (assignedImagesText.includes('禁止插入') || assignedImagesText.includes('No images')) {
if (assignedImagesText.includes('No images')) {
assignedImagesText = mediaText;
} else {
assignedImagesText += `\n\n${mediaText}`;
@ -544,6 +569,7 @@ async function generateSlideContent(
canvas_width: canvasWidth,
canvas_height: canvasHeight,
teacherContext,
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!prompts) {
@ -632,6 +658,7 @@ async function generateSlideContent(
async function generateQuizContent(
outline: SceneOutline,
aiCall: AICallFn,
languageDirective?: string,
): Promise<GeneratedQuizContent | null> {
const quizConfig = outline.quizConfig || {
questionCount: 3,
@ -646,6 +673,7 @@ async function generateQuizContent(
questionCount: quizConfig.questionCount,
difficulty: quizConfig.difficulty,
questionTypes: quizConfig.questionTypes.join(', '),
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!prompts) {
@ -735,7 +763,7 @@ function normalizeQuizAnswer(question: Record<string, unknown>): string[] | unde
async function generateInteractiveContent(
outline: SceneOutline,
aiCall: AICallFn,
language: 'zh-CN' | 'en-US' = 'zh-CN',
languageDirective?: string,
): Promise<GeneratedInteractiveContent | null> {
const config = outline.interactiveConfig!;
@ -792,7 +820,7 @@ async function generateInteractiveContent(
keyPoints: (outline.keyPoints || []).map((p, i) => `${i + 1}. ${p}`).join('\n'),
scientificConstraints,
designIdea: config.designIdea,
language,
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!htmlPrompts) {
@ -826,6 +854,7 @@ async function generateInteractiveContent(
async function generatePBLSceneContent(
outline: SceneOutline,
languageModel?: LanguageModel,
languageDirective?: string,
): Promise<GeneratedPBLContent | null> {
if (!languageModel) {
log.error('LanguageModel required for PBL generation');
@ -847,7 +876,8 @@ async function generatePBLSceneContent(
projectDescription: pblConfig.projectDescription,
targetSkills: pblConfig.targetSkills,
issueCount: pblConfig.issueCount,
language: pblConfig.language,
languageDirective:
languageDirective || 'Teach in the language that matches the user requirement.',
},
languageModel,
{
@ -913,10 +943,9 @@ export async function generateSceneActions(
| GeneratedInteractiveContent
| GeneratedPBLContent,
aiCall: AICallFn,
ctx?: SceneGenerationContext,
agents?: AgentInfo[],
userProfile?: string,
options: SceneActionsOptions = {},
): Promise<Action[]> {
const { ctx, agents, userProfile, languageDirective } = options;
const agentsText = formatAgentsForPrompt(agents);
if (outline.type === 'slide' && 'elements' in content) {
@ -931,6 +960,7 @@ export async function generateSceneActions(
courseContext: buildCourseContext(ctx),
agents: agentsText,
userProfile: userProfile || '',
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!prompts) {
@ -959,6 +989,7 @@ export async function generateSceneActions(
questions: questionsText,
courseContext: buildCourseContext(ctx),
agents: agentsText,
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!prompts) {
@ -986,6 +1017,7 @@ export async function generateSceneActions(
designIdea: config?.designIdea || '',
courseContext: buildCourseContext(ctx),
agents: agentsText,
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!prompts) {
@ -1013,6 +1045,7 @@ export async function generateSceneActions(
projectDescription: pblConfig?.projectDescription || outline.description,
courseContext: buildCourseContext(ctx),
agents: agentsText,
languageDirective: buildLanguageText(languageDirective, outline.languageNote),
});
if (!prompts) {

View File

@ -73,6 +73,7 @@ async function fetchSceneContent(
style?: string;
};
agents?: AgentInfo[];
languageDirective?: string;
},
signal?: AbortSignal,
): Promise<SceneContentResult> {
@ -101,6 +102,7 @@ async function fetchSceneActions(
agents?: AgentInfo[];
previousSpeeches?: string[];
userProfile?: string;
languageDirective?: string;
},
signal?: AbortSignal,
): Promise<SceneActionsResult> {
@ -228,6 +230,7 @@ export interface GenerationParams {
};
agents?: AgentInfo[];
userProfile?: string;
languageDirective?: string;
}
export function useSceneGenerator(options: UseSceneGeneratorOptions = {}) {
@ -321,6 +324,7 @@ export function useSceneGenerator(options: UseSceneGeneratorOptions = {}) {
imageMapping: params.imageMapping,
stageInfo: params.stageInfo,
agents: params.agents,
languageDirective: params.languageDirective,
},
signal,
);
@ -354,6 +358,7 @@ export function useSceneGenerator(options: UseSceneGeneratorOptions = {}) {
agents: params.agents,
previousSpeeches,
userProfile: params.userProfile,
languageDirective: params.languageDirective,
},
signal,
);
@ -470,6 +475,7 @@ export function useSceneGenerator(options: UseSceneGeneratorOptions = {}) {
imageMapping: params.imageMapping,
stageInfo: params.stageInfo,
agents: params.agents,
languageDirective: params.languageDirective,
},
signal,
);
@ -497,6 +503,7 @@ export function useSceneGenerator(options: UseSceneGeneratorOptions = {}) {
agents: params.agents,
previousSpeeches,
userProfile: params.userProfile,
languageDirective: params.languageDirective,
},
signal,
);

View File

@ -26,7 +26,6 @@ Not every key needs explanation, but the following have non-obvious UX context t
| `home.greetingWithName` | Top-left of homepage, clickable pill that opens nickname editor | This is a **call-to-action** — the greeting doubles as an entry point for users to set their nickname. The translation must include `{{name}}` and read naturally with the default nickname (see `profile.defaultNickname`). Avoid generic greetings that hide the name (e.g. don't translate as just "Welcome"). |
| `profile.defaultNickname` | Pre-filled in the greeting and the nickname input field | Shown before the user sets a real name. Pick a warm, gender-neutral word that: (1) feels natural in the greeting, (2) clearly signals "this is a placeholder you should replace". Avoid cold terms like "User" or formal terms like "Student". Examples: EN "Learner", ZH "同学". |
| `profile.bioPlaceholder` | Textarea placeholder in the profile editor | The bio is fed to the AI teacher to personalize lessons. The placeholder should hint at this — tell users *why* filling it in helps. |
| `pbl.chat.welcomeMessage` | First message from the Question Agent in PBL mode | Contains `{{title}}` (issue title) and `{{questions}}` (multi-line generated text). The `{{questions}}` block can be long — make sure surrounding text doesn't make the message feel cluttered. Keep the framing concise. |
| `pbl.chat.issueCompleteMessage` | System message when a PBL issue is completed | Contains `{{completed}}` and `{{next}}`. Should feel like a natural progression, not a mechanical status update. |
| `generation.textTruncated` / `generation.imageTruncated` | Toast warnings during PDF-based course generation | These are technical warnings shown briefly. Keep them short and factual. `textTruncated` has `{{n}}` (character count), `imageTruncated` has `{{total}}` and `{{max}}`. |
| `agentBar.readyToLearn` | Classroom page, above the agent role list | Conversational prompt to set the mood before class starts. Should feel inviting, not instructional. |

View File

@ -10,7 +10,6 @@
"greetingWithName": "Hi, {{name}}"
},
"toolbar": {
"languageHint": "Course will be generated in this language",
"pdfParser": "Parser",
"pdfUpload": "Upload PDF",
"removePdf": "Remove file",
@ -212,7 +211,6 @@
"mentionHint": "Use @question to ask, @judge to submit for review",
"placeholder": "Type a message...",
"send": "Send",
"welcomeMessage": "Hello! I'm your Question Agent for this issue: \"{{title}}\"\n\nTo help guide your work, I've prepared some questions for you:\n\n{{questions}}\n\nFeel free to @question me anytime if you need help or clarification!",
"issueCompleteMessage": "Issue \"{{completed}}\" completed! Moving to next issue: \"{{next}}\"",
"allCompleteMessage": "🎉 All issues completed! Great work on the project!"
},

View File

@ -10,7 +10,6 @@
"greetingWithName": "こんにちは、{{name}}さん"
},
"toolbar": {
"languageHint": "コースはこの言語で生成されます",
"pdfParser": "パーサー",
"pdfUpload": "PDFをアップロード",
"removePdf": "ファイルを削除",
@ -212,7 +211,6 @@
"mentionHint": "@question で質問、@judge で提出・レビュー依頼",
"placeholder": "メッセージを入力...",
"send": "送信",
"welcomeMessage": "こんにちは!この課題の質問エージェントです:「{{title}}」\n\n学習の手助けとして、いくつかの質問を用意しました:\n\n{{questions}}\n\nいつでも @question で気軽に質問してくださいね!",
"issueCompleteMessage": "課題「{{completed}}」が完了しました!次の課題に進みます:「{{next}}」",
"allCompleteMessage": "🎉 すべての課題が完了しました!プロジェクトお疲れさまでした!"
},

View File

@ -10,7 +10,6 @@
"greetingWithName": "Привет, {{name}}"
},
"toolbar": {
"languageHint": "Курс будет сгенерирован на этом языке",
"pdfParser": "Парсер",
"pdfUpload": "Загрузить PDF",
"removePdf": "Удалить файл",
@ -212,7 +211,6 @@
"mentionHint": "Используйте @question для вопроса, @judge для проверки",
"placeholder": "Введите сообщение...",
"send": "Отправить",
"welcomeMessage": "Привет! Я ваш помощник по задаче: \"{{title}}\"\n\nЧтобы помочь вам, я подготовил несколько вопросов:\n\n{{questions}}\n\nОбращайтесь через @question, если нужна помощь!",
"issueCompleteMessage": "Задача \"{{completed}}\" выполнена! Переход к следующей: \"{{next}}\"",
"allCompleteMessage": "🎉 Все задачи выполнены! Отличная работа над проектом!"
},

View File

@ -10,7 +10,6 @@
"greetingWithName": "嗨,{{name}}"
},
"toolbar": {
"languageHint": "课程将以此语言生成",
"pdfParser": "解析器",
"pdfUpload": "上传 PDF",
"removePdf": "移除文件",
@ -212,7 +211,6 @@
"mentionHint": "使用 @question 提问,@judge 提交评审",
"placeholder": "输入消息...",
"send": "发送",
"welcomeMessage": "你好!我是本任务的提问助手,当前任务:「{{title}}」\n\n为了帮助你开展工作,我准备了一些引导问题:\n\n{{questions}}\n\n随时可以 @question 向我提问!",
"issueCompleteMessage": "任务「{{completed}}」已完成!进入下一个任务:「{{next}}」",
"allCompleteMessage": "🎉 所有任务都已完成!项目做得很棒!"
},

View File

@ -163,11 +163,9 @@ Personalize your teaching based on their background when relevant. Address them
const roleGuideline = ROLE_GUIDELINES[agentConfig.role] || ROLE_GUIDELINES.student;
// Build language constraint from stage language
const courseLanguage = storeState.stage?.language;
const languageConstraint = courseLanguage
? `\n# Language (CRITICAL)\nYou MUST speak in ${courseLanguage === 'zh-CN' ? 'Chinese (Simplified)' : courseLanguage === 'en-US' ? 'English' : courseLanguage}. ALL text content in your response MUST be in this language.\n`
: '';
// Build language constraint from stage language directive
const langDirective = storeState.stage?.languageDirective;
const languageConstraint = langDirective ? `\n# Language (CRITICAL)\n${langDirective}\n` : '';
return `# Role
You are ${agentConfig.name}.

View File

@ -25,7 +25,7 @@ export interface GeneratePBLConfig {
projectDescription: string;
targetSkills: string[];
issueCount?: number;
language: string;
languageDirective: string;
}
export interface GeneratePBLCallbacks {
@ -44,7 +44,7 @@ export async function generatePBLContent(
model: LanguageModel,
callbacks?: GeneratePBLCallbacks,
): Promise<PBLProjectConfig> {
const { language } = config;
const { languageDirective } = config;
// Initialize shared state
const projectConfig: PBLProjectConfig = {
@ -61,7 +61,7 @@ export async function generatePBLContent(
);
const projectMCP = new ProjectMCP(projectConfig);
const agentMCP = new AgentMCP(projectConfig);
const issueboardMCP = new IssueboardMCP(projectConfig, agentMCP, language);
const issueboardMCP = new IssueboardMCP(projectConfig, agentMCP, languageDirective);
callbacks?.onProgress?.('Starting PBL project generation...');
@ -287,10 +287,7 @@ export async function generatePBLContent(
{
model,
system: systemPrompt,
prompt:
language === 'zh-CN'
? `请设计一个PBL项目。现在从 project_info 模式开始,先设置项目标题和描述。`
: `Design a PBL project. Start in project_info mode by setting the project title and description.`,
prompt: `Design a PBL project. Start in project_info mode by setting the project title and description.`,
tools: pblTools,
stopWhen: stepCountIs(30),
onStepFinish: ({ toolCalls, text }) => {
@ -317,7 +314,7 @@ export async function generatePBLContent(
callbacks?.onProgress?.('PBL structure generated. Running post-processing...');
// Post-processing: activate first issue and generate initial questions
await postProcessPBL(projectConfig, model, language, callbacks);
await postProcessPBL(projectConfig, model, languageDirective, callbacks);
callbacks?.onProgress?.('PBL project generation complete!');
@ -333,7 +330,7 @@ export async function generatePBLContent(
async function postProcessPBL(
config: PBLProjectConfig,
model: LanguageModel,
language: string,
languageDirective: string,
callbacks?: GeneratePBLCallbacks,
): Promise<void> {
const { issueboard, agents } = config;
@ -360,26 +357,7 @@ async function postProcessPBL(
try {
callbacks?.onProgress?.('Generating initial questions for first issue...');
const context =
language === 'zh-CN'
? `## 任务信息
**标题**: ${firstIssue.title}
**描述**: ${firstIssue.description}
**负责人**: ${firstIssue.person_in_charge}
${firstIssue.participants.length > 0 ? `**参与者**: ${firstIssue.participants.join('、')}` : ''}
${firstIssue.notes ? `**备注**: ${firstIssue.notes}` : ''}
## 你的任务
根据以上任务信息,生成1-3个具体、可操作的引导问题,帮助学生理解和完成这个任务。每个问题应:
- 引导学生达成关键学习目标
- 具体且可操作
- 帮助分解问题
- 鼓励批判性思考
请以编号列表格式回答。`
: `## Issue Information
const context = `## Issue Information
**Title**: ${firstIssue.title}
**Description**: ${firstIssue.description}
@ -389,13 +367,16 @@ ${firstIssue.notes ? `**Notes**: ${firstIssue.notes}` : ''}
## Your Task
Based on the issue information above, generate 1-3 specific, actionable questions that will help students understand and complete this issue. Each question should:
- Guide students toward key learning objectives
- Be specific and actionable
- Help break down the problem
- Encourage critical thinking
Generate a welcome message for the student working on this issue. The message should:
1. Start with a friendly greeting introducing yourself as the guiding assistant for this issue (use a natural, localized title — do NOT use the English term "Question Agent" directly in non-English contexts)
2. Present 1-3 specific, actionable guiding questions based on the issue information above, each question should:
- Guide students toward key learning objectives
- Be specific and actionable
- Help break down the problem
- Encourage critical thinking
3. End by encouraging the student to type \`@question\` anytime for help (keep the literal \`@question\` as-is since it triggers the agent system)
Format your response as a numbered list.`;
Format the questions as a numbered list.`;
const questionResult = await callLLM(
{
@ -409,16 +390,10 @@ Format your response as a numbered list.`;
const generatedQuestions = questionResult.text;
firstIssue.generated_questions = generatedQuestions;
// Add welcome message to chat
const welcomeMessage =
language === 'zh-CN'
? `你好!我是这个任务的提问助手:"${firstIssue.title}"\n\n为了引导你的学习,我准备了一些问题:\n\n${generatedQuestions}\n\n随时 @question 我来获取帮助或澄清!`
: `Hello! I'm your Question Agent for this issue: "${firstIssue.title}"\n\nTo help guide your work, I've prepared some questions for you:\n\n${generatedQuestions}\n\nFeel free to @question me anytime if you need help or clarification!`;
config.chat.messages.push({
id: `msg_welcome_${Date.now()}`,
agent_name: firstIssue.question_agent_name,
message: welcomeMessage,
message: generatedQuestions,
timestamp: Date.now(),
read_by: [],
});

View File

@ -1,24 +1,14 @@
/**
* Agent template prompts for PBL Question and Judge agents.
*
* Migrated from PBL-Nano with multi-language support.
* Uses languageDirective for multi-language support.
*/
export function getQuestionAgentPrompt(language: string = 'en-US'): string {
if (language === 'zh-CN') {
return QUESTION_AGENT_TEMPLATE_PROMPT_ZH;
}
return QUESTION_AGENT_TEMPLATE_PROMPT;
}
export function getJudgeAgentPrompt(language: string = 'en-US'): string {
if (language === 'zh-CN') {
return JUDGE_AGENT_TEMPLATE_PROMPT_ZH;
}
return JUDGE_AGENT_TEMPLATE_PROMPT;
}
export const QUESTION_AGENT_TEMPLATE_PROMPT = `You are a Question Agent in a Project-Based Learning platform. Your role is to help students understand and complete their assigned issue.
export function getQuestionAgentPrompt(languageDirective: string): string {
const languageSection = languageDirective
? `\n## Language\n\n${languageDirective}\n\nAll responses must follow this language directive.`
: '';
return `You are a Question Agent in a Project-Based Learning platform. Your role is to help students understand and complete their assigned issue.
## Your Responsibilities:
@ -34,9 +24,14 @@ export const QUESTION_AGENT_TEMPLATE_PROMPT = `You are a Question Agent in a Pro
- Be encouraging and supportive
- Focus on learning process, not just answers
- Help students break down complex problems
- Guide them to relevant resources or thinking approaches`;
- Guide them to relevant resources or thinking approaches${languageSection}`;
}
export const JUDGE_AGENT_TEMPLATE_PROMPT = `You are a Judge Agent in a Project-Based Learning platform. Your role is to evaluate whether students have completed their assigned issue successfully.
export function getJudgeAgentPrompt(languageDirective: string): string {
const languageSection = languageDirective
? `\n## Language\n\n${languageDirective}\n\nAll responses must follow this language directive.`
: '';
return `You are a Judge Agent in a Project-Based Learning platform. Your role is to evaluate whether students have completed their assigned issue successfully.
## Your Responsibilities:
@ -56,44 +51,5 @@ export const JUDGE_AGENT_TEMPLATE_PROMPT = `You are a Judge Agent in a Project-B
- Be fair but encouraging
- Provide specific, actionable feedback
- Focus on learning outcomes, not perfection
- Celebrate successes while identifying growth areas`;
const QUESTION_AGENT_TEMPLATE_PROMPT_ZH = `你是项目式学习平台中的提问助手(Question Agent)。你的职责是帮助学生理解并完成分配给他们的任务。
## 你的职责:
1. **生成初始问题**:当任务被激活时,根据任务标题和描述生成1-3个具体、可操作的引导问题。
2. **回答学生疑问**:当学生 @mention 你时:
- 提供有用的提示和引导
- 通过反问帮助他们批判性思考
- 不直接给出答案——帮助他们自己发现解决方案
- 围绕生成的引导问题保持方向
## 准则:
- 鼓励和支持学生
- 关注学习过程,而非仅关注答案
- 帮助学生分解复杂问题
- 引导他们找到相关资源或思路`;
const JUDGE_AGENT_TEMPLATE_PROMPT_ZH = `你是项目式学习平台中的评判助手(Judge Agent)。你的职责是评估学生是否成功完成了分配的任务。
## 你的职责:
1. **评估完成度**:当学生 @mention 你时:
- 请他们解释完成了什么
- 对照任务描述和引导问题审核他们的工作
- 提供建设性反馈
- 判断任务是否完成或需要改进
2. **反馈格式**:
- 突出做得好的地方
- 指出不足和改进空间
- 如果未完成,给出明确的下一步指导
- 最终判定:"COMPLETE" 或 "NEEDS_REVISION"
## 准则:
- 公正但鼓励
- 提供具体、可操作的反馈
- 关注学习成果,而非完美
- 在肯定成就的同时指出成长空间`;
- Celebrate successes while identifying growth areas${languageSection}`;
}

View File

@ -14,13 +14,13 @@ import { getQuestionAgentPrompt, getJudgeAgentPrompt } from './agent-templates';
export class IssueboardMCP {
private config: PBLProjectConfig;
private agentMCP: AgentMCP;
private language: string;
private languageDirective: string;
private nextIssueId: number;
constructor(config: PBLProjectConfig, agentMCP: AgentMCP, language: string = 'en-US') {
constructor(config: PBLProjectConfig, agentMCP: AgentMCP, languageDirective: string = '') {
this.config = config;
this.agentMCP = agentMCP;
this.language = language;
this.languageDirective = languageDirective;
this.nextIssueId = 1;
}
@ -107,7 +107,7 @@ export class IssueboardMCP {
// Auto-create question and judge agents
this.agentMCP.createAgent({
name: questionAgentName,
system_prompt: getQuestionAgentPrompt(this.language),
system_prompt: getQuestionAgentPrompt(this.languageDirective),
default_mode: 'chat',
actor_role: 'Question Assistant for Issue',
role_division: 'development',
@ -116,7 +116,7 @@ export class IssueboardMCP {
this.agentMCP.createAgent({
name: judgeAgentName,
system_prompt: getJudgeAgentPrompt(this.language),
system_prompt: getJudgeAgentPrompt(this.languageDirective),
default_mode: 'chat',
actor_role: 'Judge for Issue Completion',
role_division: 'management',

View File

@ -2,7 +2,7 @@
* PBL Generation System Prompt
*
* Migrated from PBL-Nano's anything2pbl_nano.ts systemPrompt.
* Enhanced with multi-language support and configurable parameters.
* Uses languageDirective for multi-language support.
*/
export interface PBLSystemPromptConfig {
@ -10,15 +10,17 @@ export interface PBLSystemPromptConfig {
projectDescription: string;
targetSkills: string[];
issueCount?: number;
language: string;
languageDirective: string;
}
export function buildPBLSystemPrompt(config: PBLSystemPromptConfig): string {
const { projectTopic, projectDescription, targetSkills, issueCount = 3, language } = config;
if (language === 'zh-CN') {
return buildPBLSystemPromptZH(config);
}
const {
projectTopic,
projectDescription,
targetSkills,
issueCount = 3,
languageDirective,
} = config;
return `You are a Teaching Assistant (TA) on a Project-Based Learning platform. You are fully responsible for designing group projects for students based on the course information provided by the teacher.
@ -79,72 +81,13 @@ When you create issues:
- You do NOT need to manually create these agents
- Focus on designing meaningful issues with clear descriptions
## Language
${languageDirective}
All project content (title, description, agent names and prompts, issue titles and descriptions, questions, messages) must follow this language directive.
**IMPORTANT**: Once you have configured the project info, defined all necessary agents (roles), and created the issueboard with tasks, you MUST set your mode to **idle** to indicate completion.
Your initial mode is **project_info**.`;
}
function buildPBLSystemPromptZH(config: PBLSystemPromptConfig): string {
const { projectTopic, projectDescription, targetSkills, issueCount = 3 } = config;
return `你是项目式学习(PBL)平台的教学助手(TA)。你需要根据老师提供的课程信息,自主设计完整的学生小组项目。
## 你的职责
设计一个完整的项目:
1. 创建简洁、有吸引力的项目标题
2. 撰写简明的项目描述(2-4句话),涵盖:
- 项目内容
- 核心学习目标
- 学生将完成什么
老师提供的信息:
- **项目主题**:${projectTopic}
- **项目描述**:${projectDescription}
- **目标技能**:${targetSkills.join('、')}
- **建议任务数量**:${issueCount}
根据以上信息自主设计项目,不要请求确认或额外输入。
## 模式系统
你可以在不同模式间切换,每种模式提供不同的工具集:
- **project_info**:设置项目基本信息(标题、描述)
- **agent**:定义项目角色
- **issueboard**:配置协作工作流和任务
- **idle**:表示项目配置完成的特殊模式
你从 **project_info** 模式开始。使用 \`set_mode\` 工具在模式间切换。
## 工作流程
1. 在 **project_info** 模式中:设置项目标题和描述
2. 切换到 **agent** 模式:定义 2-4 个学生开发角色(不要创建管理角色给学生)
3. 切换到 **issueboard** 模式:创建 ${issueCount} 个顺序任务引导学生完成项目
4. 完成所有配置后,切换到 **idle** 模式
## 角色设计指南
- 创建 2-4 个**开发**角色供学生选择
- 每个角色有明确的职责和独特的系统提示
- 角色应互补(如"数据分析师"、"前端开发者"、"项目经理")
- 不要创建系统 Agent(问答/评判 Agent 会自动按任务创建)
## 任务设计指南
- 创建恰好 ${issueCount} 个任务,形成逻辑序列
- 每个任务应可由一人完成
- 任务应层层递进(前面的任务为后面的打基础)
- 每个任务需要:标题、描述、负责人(使用角色名称)和相关参与者
## 任务 Agent 自动创建
创建任务时:
- 每个任务会自动获得 Question Agent 和 Judge Agent
- 你不需要手动创建这些 Agent
- 专注于设计有意义的任务和清晰的描述
**重要**:完成项目信息、角色和任务看板配置后,你必须切换到 **idle** 模式表示完成。
你的初始模式是 **project_info**。`;
}

View File

@ -13,7 +13,6 @@ import {
} from '@/lib/generation/scene-generator';
import type { AICallFn } from '@/lib/generation/pipeline-types';
import type { AgentInfo } from '@/lib/generation/pipeline-types';
import { formatTeacherPersonaForPrompt } from '@/lib/generation/prompt-formatters';
import { getDefaultAgents } from '@/lib/orchestration/registry/store';
import { createLogger } from '@/lib/logger';
import { isProviderKeyRequired } from '@/lib/ai/providers';
@ -36,7 +35,6 @@ const log = createLogger('Classroom');
export interface GenerateClassroomInput {
requirement: string;
pdfContent?: { text: string; images: string[] };
language?: string;
enableWebSearch?: boolean;
enableImageGeneration?: boolean;
enableVideoGeneration?: boolean;
@ -98,10 +96,6 @@ function createInMemoryStore(stage: Stage): StageStore {
};
}
function normalizeLanguage(language?: string): 'zh-CN' | 'en-US' {
return language === 'en-US' ? 'en-US' : 'zh-CN';
}
function stripCodeFences(text: string): string {
let cleaned = text.trim();
if (cleaned.startsWith('```')) {
@ -112,7 +106,7 @@ function stripCodeFences(text: string): string {
async function generateAgentProfiles(
requirement: string,
language: string,
languageDirective: string,
aiCall: AICallFn,
): Promise<AgentInfo[]> {
const systemPrompt =
@ -125,7 +119,8 @@ Requirements:
- Decide the appropriate number of agents based on the course content (typically 3-5)
- Exactly 1 agent must have role "teacher", the rest can be "assistant" or "student"
- Each agent needs: name, role, persona (2-3 sentences describing personality and teaching/learning style)
- Names and personas must be in language: ${language}
- Language directive for this course: ${languageDirective}
Agent names and personas must follow this language directive.
Return a JSON object with this exact structure:
{
@ -224,31 +219,11 @@ export async function generateClassroom(
return result.text;
};
const lang = normalizeLanguage(input.language);
const requirements: UserRequirements = {
requirement,
language: lang,
};
const pdfText = pdfContent?.text || undefined;
// Resolve agents based on agentMode
let agents: AgentInfo[];
let agentMode = input.agentMode || 'default';
if (agentMode === 'generate') {
log.info('Generating custom agent profiles via LLM...');
try {
agents = await generateAgentProfiles(requirement, lang, aiCall);
log.info(`Generated ${agents.length} agent profiles`);
} catch (e) {
log.warn('Agent profile generation failed, falling back to defaults:', e);
agents = getDefaultAgents();
agentMode = 'default';
}
} else {
agents = getDefaultAgents();
}
const teacherContext = formatTeacherPersonaForPrompt(agents);
await options.onProgress?.({
step: 'researching',
progress: 10,
@ -304,7 +279,7 @@ export async function generateClassroom(
imageGenerationEnabled: input.enableImageGeneration,
videoGenerationEnabled: input.enableVideoGeneration,
researchContext,
teacherContext,
// NO teacherContext — agents haven't been generated yet
},
);
@ -313,8 +288,8 @@ export async function generateClassroom(
throw new Error(outlinesResult.error || 'Failed to generate scene outlines');
}
const outlines = outlinesResult.data;
log.info(`Generated ${outlines.length} scene outlines`);
const { languageDirective, outlines } = outlinesResult.data;
log.info(`Generated ${outlines.length} scene outlines (languageDirective: ${languageDirective})`);
await options.onProgress?.({
step: 'generating_outlines',
@ -324,12 +299,28 @@ export async function generateClassroom(
totalScenes: outlines.length,
});
// Resolve agents based on agentMode — now AFTER outlines so we can use languageDirective
let agents: AgentInfo[];
const agentMode = input.agentMode || 'default';
if (agentMode === 'generate') {
log.info('Generating custom agent profiles via LLM...');
try {
agents = await generateAgentProfiles(requirement, languageDirective, aiCall);
log.info(`Generated ${agents.length} agent profiles`);
} catch (e) {
log.warn('Agent profile generation failed, falling back to defaults:', e);
agents = getDefaultAgents();
}
} else {
agents = getDefaultAgents();
}
const stageId = nanoid(10);
const stage: Stage = {
id: stageId,
name: outlines[0]?.title || requirement.slice(0, 50),
description: undefined,
language: lang,
languageDirective,
style: 'interactive',
createdAt: Date.now(),
updatedAt: Date.now(),
@ -371,22 +362,16 @@ export async function generateClassroom(
totalScenes: outlines.length,
});
const content = await generateSceneContent(
safeOutline,
aiCall,
undefined,
undefined,
undefined,
undefined,
undefined,
agents,
);
const content = await generateSceneContent(safeOutline, aiCall, { agents, languageDirective });
if (!content) {
log.warn(`Skipping scene "${safeOutline.title}" — content generation failed`);
continue;
}
const actions = await generateSceneActions(safeOutline, content, aiCall, undefined, agents);
const actions = await generateSceneActions(safeOutline, content, aiCall, {
agents,
languageDirective,
});
log.info(`Scene "${safeOutline.title}": ${actions.length} actions`);
const sceneId = createSceneWithActions(safeOutline, content, actions, api);

View File

@ -26,7 +26,6 @@ export interface ClassroomGenerationJob {
completedAt?: string;
inputSummary: {
requirementPreview: string;
language: string;
hasPdf: boolean;
pdfTextLength: number;
pdfImageCount: number;
@ -49,7 +48,6 @@ function buildInputSummary(input: GenerateClassroomInput): ClassroomGenerationJo
return {
requirementPreview:
input.requirement.length > 200 ? `${input.requirement.slice(0, 197)}...` : input.requirement,
language: input.language || 'zh-CN',
hasPdf: !!input.pdfContent,
pdfTextLength: input.pdfContent?.text.length || 0,
pdfImageCount: input.pdfContent?.images.length || 0,

View File

@ -30,22 +30,6 @@ export type ImageMapping = Record<string, string>;
// ==================== Stage 1 Input ====================
export interface AudienceProfile {
gradeLevel: string; // "K-12", "University", "Professional"
ageRange?: string; // "6-12", "18-25"
prerequisites?: string[]; // Required prior knowledge
learningStyles?: ('visual' | 'auditory' | 'kinesthetic' | 'reading')[];
}
export interface StylePreferences {
tone: 'formal' | 'casual' | 'engaging' | 'academic';
visualStyle: 'minimalist' | 'colorful' | 'professional' | 'playful';
interactivityLevel: 'low' | 'medium' | 'high';
includeExamples: boolean;
includePractice: boolean;
language: string; // 'zh-CN', 'en-US'
}
export interface UploadedDocument {
id: string;
name: string; // Original filename
@ -64,27 +48,11 @@ export interface UploadedDocument {
*/
export interface UserRequirements {
requirement: string; // Single free-form text for all user input
language: 'zh-CN' | 'en-US'; // Course language - critical for generation
userNickname?: string; // Student nickname for personalization
userBio?: string; // Student background for personalization
webSearch?: boolean; // Enable web search for richer context
}
/**
* @deprecated Use UserRequirements instead
* Legacy structured requirements - kept for backward compatibility
*/
export interface LegacyUserRequirements {
topic: string;
description?: string;
learningObjectives: string[];
audience: AudienceProfile;
durationMinutes: number;
style: StylePreferences;
documents?: UploadedDocument[];
additionalNotes?: string;
}
// ==================== Stage 1 Output: Scene Outlines (Simplified) ====================
/**
@ -100,7 +68,7 @@ export interface SceneOutline {
teachingObjective?: string;
estimatedDuration?: number; // seconds
order: number;
language?: 'zh-CN' | 'en-US'; // Generation language (inherited from requirements)
languageNote?: string; // LLM-inferred language note for this scene
// Suggested image IDs (from PDF-extracted images)
suggestedImageIds?: string[]; // e.g., ["img_1", "img_3"]
// AI-generated media requests (when PDF images are insufficient)
@ -124,7 +92,6 @@ export interface SceneOutline {
projectDescription: string;
targetSkills: string[];
issueCount?: number;
language: 'zh-CN' | 'en-US';
};
}

View File

@ -19,7 +19,7 @@ export interface Stage {
createdAt: number;
updatedAt: number;
// Stage metadata
language?: string;
languageDirective?: string;
style?: string;
// Whiteboard data
whiteboard?: Whiteboard[];

View File

@ -44,7 +44,7 @@ export interface StageRecord {
description?: string;
createdAt: number; // timestamp
updatedAt: number; // timestamp
language?: string;
languageDirective?: string;
style?: string;
currentSceneId?: string;
agentIds?: string[]; // Agent IDs selected at creation time
@ -175,7 +175,7 @@ export function mediaFileKey(stageId: string, elementId: string): string {
// ==================== Database Definition ====================
const DATABASE_NAME = 'MAIC-Database';
const _DATABASE_VERSION = 8;
const _DATABASE_VERSION = 9;
/**
* MAIC Database Instance
@ -309,6 +309,40 @@ class MAICDatabase extends Dexie {
mediaFiles: 'id, stageId, [stageId+type]',
generatedAgents: 'id, stageId',
});
// Version 9: Migrate legacy `language` field to `languageDirective`
// Old stages stored a BCP-47 locale code (e.g. "zh-CN"); new code expects a
// natural-language directive. Convert known locales and drop the old field.
const LOCALE_TO_DIRECTIVE: Record<string, string> = {
'zh-CN': 'Deliver the entire course in Chinese (Simplified, zh-CN).',
'en-US': 'Deliver the entire course in English (en-US).',
'ja-JP': 'Deliver the entire course in Japanese (ja-JP).',
'ru-RU': 'Deliver the entire course in Russian (ru-RU).',
};
this.version(9)
.stores({
stages: 'id, updatedAt',
scenes: 'id, stageId, order, [stageId+order]',
audioFiles: 'id, createdAt',
imageFiles: 'id, createdAt',
snapshots: '++id',
chatSessions: 'id, stageId, [stageId+createdAt]',
playbackState: 'stageId',
stageOutlines: 'stageId',
mediaFiles: 'id, stageId, [stageId+type]',
generatedAgents: 'id, stageId',
})
.upgrade(async (tx) => {
const table = tx.table('stages');
await table.toCollection().modify((stage: Record<string, unknown>) => {
const lang = stage.language as string | undefined;
if (lang && !stage.languageDirective) {
stage.languageDirective =
LOCALE_TO_DIRECTIVE[lang] || `Deliver the entire course in ${lang}.`;
}
delete stage.language;
});
});
}
}

View File

@ -44,7 +44,7 @@ export async function saveStageData(stageId: string, data: StageStoreData): Prom
description: data.stage.description,
createdAt: data.stage.createdAt || now,
updatedAt: now,
language: data.stage.language,
languageDirective: data.stage.languageDirective,
style: data.stage.style,
currentSceneId: data.currentSceneId || undefined,
agentIds: data.stage.agentIds,

View File

@ -0,0 +1,312 @@
[
{
"case_id": "zh_pure_general",
"category": "zh_pure_humanities",
"requirement": "请讲解欧洲文艺复兴时期的音乐发展历程",
"ground_truth": "Teaching language: Chinese. Music and history terminology should use standard Chinese translations."
},
{
"case_id": "zh_pure_k12",
"category": "zh_pure_k12_education",
"requirement": "帮我制作一节小学三年级语文课",
"ground_truth": "Teaching language: Chinese. Use age-appropriate Chinese for primary school students."
},
{
"case_id": "zh_tech_pygame",
"category": "zh_with_english_tech_term",
"requirement": "用pygame做一个入门小游戏教程",
"ground_truth": "Teaching language: Chinese. Programming terms like pygame, Python should be kept in English."
},
{
"case_id": "zh_tech_comfyui",
"category": "zh_with_english_product_name",
"requirement": "ComfyUI零基础入门教程",
"ground_truth": "Teaching language: Chinese. Product names like ComfyUI should be kept in English. Technical terms kept in English with Chinese explanation."
},
{
"case_id": "zh_tech_alevel",
"category": "zh_with_english_exam_system",
"requirement": "设计一门A-Level化学课程,要求通俗易懂,适合基础薄弱的学生",
"ground_truth": "Teaching language: Chinese. \"A-Level\" should be kept in English. Chemistry terms should use standard Chinese translations with English originals where helpful."
},
{
"case_id": "en_pure_science",
"category": "en_pure_short",
"requirement": "Teach me about photosynthesis in plants",
"ground_truth": "Teaching language: English. Biology terms like photosynthesis should use standard English terminology."
},
{
"case_id": "en_pure_tech",
"category": "en_pure_tech",
"requirement": "Help me learn Grafana Alloy from scratch",
"ground_truth": "Teaching language: English. Technical terms like Grafana, Alloy should be kept as-is."
},
{
"case_id": "en_pure_academic",
"category": "en_pure_academic",
"requirement": "Cover CAIE 9701 Chemistry Chapter 1 and include past paper practice questions",
"ground_truth": "Teaching language: English. CAIE chemistry terminology in English. Past paper references in English."
},
{
"case_id": "zh_learn_en",
"category": "zh_user_learning_english",
"requirement": "帮我复习人教版初二下册英语第三单元的单词",
"ground_truth": "Teaching language: Chinese. This is a Chinese student memorizing English vocabulary. Course taught in Chinese with English words and translations progressively introduced."
},
{
"case_id": "en_learn_chinese",
"category": "en_user_learning_chinese",
"requirement": "I'd like to start learning Mandarin Chinese conversation basics",
"ground_truth": "Teaching language: English. This is an English speaker learning Mandarin Chinese. Teach in English, introduce Chinese characters/pinyin progressively."
},
{
"case_id": "en_learn_german",
"category": "en_user_learning_german",
"requirement": "Teach me beginner German at A1 level",
"ground_truth": "Teaching language: English. This is a beginner learning German. Teach in English, introduce German vocabulary and grammar progressively."
},
{
"case_id": "zh_baby_learn_en",
"category": "zh_young_child_learning_english",
"requirement": "我家孩子5岁,想教他认识简单的英语单词",
"ground_truth": "Teaching language: Chinese. This is a 5-year-old Chinese child learning English reading. Must teach in Chinese with simple English words introduced gradually."
},
{
"case_id": "zh_set_en",
"category": "zh_requirement_but_en_locale",
"requirement": "讲解电压、电流、电阻和功率之间的基本关系",
"ground_truth": "Teaching language: Chinese (requirement is in Chinese). Physics terms should use standard Chinese translations. The en-US locale setting should be ignored."
},
{
"case_id": "zh_set_en2",
"category": "zh_requirement_but_en_locale_tech",
"requirement": "如何从零训练一个小型AI模型",
"ground_truth": "Teaching language: Chinese (requirement is in Chinese). AI/ML terms can be kept in English or shown bilingually."
},
{
"case_id": "foreign_in_cn",
"category": "foreigner_learning_chinese_culture",
"requirement": "作为外国人,我想了解在中国日常购物的流程",
"ground_truth": "Teaching language: Chinese. The user is a foreigner learning Chinese shopping culture. Content should be in Chinese, potentially with simpler language or pinyin for key phrases."
},
{
"case_id": "spanish",
"category": "spanish_requirement",
"requirement": "Quiero aprender los fundamentos del ensayo de jarras, con explicaciones técnicas y didácticas, incluyendo ilustraciones del proceso",
"ground_truth": "Teaching language: Spanish. The requirement is in Spanish, so the course should be in Spanish. Technical terms related to jar testing should use Spanish translations."
},
{
"case_id": "german_kid",
"category": "german_child_requirement",
"requirement": "Ich bin 8 Jahre alt. Kannst du mir erklären, wie ein Elektromotor funktioniert?",
"ground_truth": "Teaching language: German. The user is an 8-year-old asking about electric motors. Use simple, child-friendly German."
},
{
"case_id": "arabic",
"category": "arabic_user_learning_english",
"requirement": "أريد تعلم اللغة الإنجليزية، مستواي حاليا A2 وأحتاج تحسين مهاراتي",
"ground_truth": "Teaching language: Arabic. This is an Arabic speaker at A2 level wanting to learn English. Teach primarily in Arabic, introducing English progressively."
},
{
"case_id": "zh_advanced_en_learner",
"category": "zh_advanced_english_learner",
"requirement": "我已过专八,想把英语口语提升到接近母语水平。目前的问题是表达时总用简单词汇,不够地道。",
"ground_truth": "Teaching language: English. The user is an advanced Chinese English learner (TEM-8) who can fully understand English but lacks native-level spoken fluency and complexity. Course should be in English, encouraging use of more sophisticated and precise expressions instead of defaulting to simple phrasing."
},
{
"case_id": "zh_translate_en_pdf",
"category": "zh_requirement_english_pdf",
"requirement": "请将这篇英文论文翻译为中文,并撰写一份内容摘要",
"ground_truth": "Teaching language: Chinese. The source document is an English academic paper (SPE/petroleum engineering). Teach in Chinese, with English technical terms preserved on first mention alongside Chinese translations, to help the student understand and summarize the paper.",
"pdfTextSample": "SPE-230629-MS\nPhysics-Based Interpretation of RFS-DSS for Far-Field Monitoring of\nFracture Conductivity\nQueendarlyn A. Nwabueze and Smith Leggett, Bob L. Herd Department of Petroleum Engineering, Texa"
},
{
"case_id": "zh_esl_teacher_en_article",
"category": "zh_teacher_english_article",
"requirement": "我是一名ESL教师,需要用这篇英文文章设计一节课,重点教授词汇、篇章结构和概括技巧",
"ground_truth": "Teaching language: Chinese. This is a Chinese ESL teacher preparing a lesson using an English article. Course should be taught in Chinese, with the English article content used as learning material. English vocabulary, sentence structures, and summary skills should be explicitly taught.",
"pdfTextSample": "Before You Read\nU7A-p.94\n7A\nA. Discussion. Look at the information and captions, paying attention to the \nwords in bold. Then answer the questions below.\n1. What kind of animals were dinosaurs? When d"
},
{
"case_id": "zh_cpp_chinese_pdf",
"category": "zh_requirement_chinese_pdf",
"requirement": "请根据上传的教学大纲,生成第五周的C++编程课程内容",
"ground_truth": "Teaching language: Chinese. Both the requirement and the PDF syllabus are in Chinese. C++ programming terms should be kept in English. Teach in Chinese following the uploaded syllabus.",
"pdfTextSample": "第5 周:复杂一点的判断\n学习主题: 多分支与逻辑运算符\n知识要点:\n多分支结构: else-if 语句\n逻辑运算符: 与(&&)、或(||)、非(!)\n运算符的优先级\n多区间判断问题(如成绩等级划分)\n学习意义: 掌握处理复杂、多条件组合的判断场景,让程序能够应对更丰富的现实问题。"
},
{
"case_id": "ja_learn_en",
"category": "language_learning",
"requirement": "英語のリスニング力を上げたい、TOEICのスコアも上げたい",
"ground_truth": "Teaching language: Japanese. This is a Japanese speaker wanting to improve English listening and TOEIC score. Teach in Japanese, introduce English listening materials and vocabulary progressively."
},
{
"case_id": "ko_learn_en",
"category": "language_learning",
"requirement": "영어 회화를 배우고 싶어요, 기초부터 시작하고 싶습니다",
"ground_truth": "Teaching language: Korean. This is a Korean speaker wanting to learn English conversation from basics. Teach in Korean, introduce English phrases and dialogue progressively."
},
{
"case_id": "en_learn_ja",
"category": "language_learning",
"requirement": "I want to learn basic Japanese for my trip to Tokyo next month",
"ground_truth": "Teaching language: English. This is an English speaker learning basic Japanese for travel. Teach in English, introduce hiragana, katakana, and useful travel phrases progressively."
},
{
"case_id": "ja_learn_zh",
"category": "language_learning",
"requirement": "中国語を勉強したいです、ビジネス中国語を身につけたい",
"ground_truth": "Teaching language: Japanese. This is a Japanese speaker learning business Chinese. Teach in Japanese, introduce Chinese characters, pinyin, and business expressions progressively. Non-Chinese/English language axis."
},
{
"case_id": "multi_target",
"category": "language_learning_multi",
"requirement": "I want to learn both Spanish and French at the same time, starting from scratch",
"ground_truth": "Teaching language: English. The learner wants to study two Romance languages simultaneously. Teach in English, introduce Spanish and French vocabulary/grammar in parallel, highlighting similarities and differences."
},
{
"case_id": "ja_immersive_en",
"category": "immersive_learning",
"requirement": "TOEIC 900点目指して、全部英語で英語を学びたい。日本語は使わないでください。",
"ground_truth": "Teaching language: English. This is an advanced Japanese English learner explicitly requesting full English immersion. Course should be entirely in English with no Japanese."
},
{
"case_id": "zh_immersive_fr",
"category": "immersive_learning",
"requirement": "我法语B2水平了,想用法语直接学习法国文学,不要用中文",
"ground_truth": "Teaching language: French. This is an advanced Chinese French learner at B2 level requesting immersive French instruction for French literature. Course should be entirely in French."
},
{
"case_id": "zh_explicit_en",
"category": "explicit_language_instruction",
"requirement": "请用英文给我讲解量子力学的基本原理",
"ground_truth": "Teaching language: English. The user explicitly requests English instruction despite writing in Chinese. Course should be in English covering quantum mechanics fundamentals."
},
{
"case_id": "en_explicit_zh",
"category": "explicit_language_instruction",
"requirement": "Explain machine learning concepts in Chinese please, I want to practice reading technical Chinese",
"ground_truth": "Teaching language: Chinese. The user explicitly requests Chinese instruction despite writing in English. Course should be in Chinese covering machine learning concepts."
},
{
"case_id": "bilingual_request",
"category": "bilingual_teaching",
"requirement": "用中英双语教我机器学习,中文解释概念,英文给出术语和代码",
"ground_truth": "Teaching language: Bilingual Chinese-English. The user explicitly requests bilingual instruction. Concepts explained in Chinese, technical terms and code in English."
},
{
"case_id": "code_switch_zh_en",
"category": "code_switching",
"requirement": "帮我学习how to use Docker来deploy一个web app",
"ground_truth": "Teaching language: Chinese. The requirement mixes Chinese and English (code-switching). Teach in Chinese with Docker/deployment technical terms kept in English."
},
{
"case_id": "minimal_zh",
"category": "minimal_ambiguous",
"requirement": "微积分",
"ground_truth": "Teaching language: Chinese. Extremely short requirement with only two Chinese characters. Teach calculus in Chinese."
},
{
"case_id": "pinyin_input",
"category": "romanized_input",
"requirement": "wo xiang xue python biancheng",
"ground_truth": "Teaching language: Chinese. The requirement is in pinyin (romanized Chinese), meaning 'I want to learn Python programming'. Teach in Chinese with Python terms in English."
},
{
"case_id": "teacher_fr_for_zh",
"category": "user_profile_teacher",
"requirement": "Help me prepare a beginner French lesson for my Chinese middle school students",
"ground_truth": "Teaching language: English. This is a teacher preparing a French lesson for Chinese middle school students. Course design in English, with lesson content considering Chinese students' perspective when introducing French."
},
{
"case_id": "parent_intl_school",
"category": "user_profile_parent",
"requirement": "我孩子12岁在国际学校读IB,帮他复习Biology的cell structure部分",
"ground_truth": "Teaching language: English. Parent writes in Chinese but the child studies IB Biology in English. Course content should be in English to match the child's learning environment."
},
{
"case_id": "bilingual_student",
"category": "user_profile_bilingual",
"requirement": "I'm Chinese-American, studying AP Physics C in high school, help me prepare for the exam",
"ground_truth": "Teaching language: English. Bilingual Chinese-American student in US high school AP Physics. Course should be in English matching the AP exam language."
},
{
"case_id": "zh_teacher_for_foreigners",
"category": "user_profile_teacher",
"requirement": "我是对外汉语老师,要给零基础的美国学生设计第一节中文课",
"ground_truth": "Teaching language: Chinese. This is a Chinese-as-a-foreign-language teacher designing a first lesson for American beginners. Course design in Chinese, but lesson content should consider English-speaking students' needs with pinyin and basic characters."
},
{
"case_id": "professional_business_en",
"category": "user_profile_professional",
"requirement": "下个月要去美国出差做presentation,帮我速成商务英语口语",
"ground_truth": "Teaching language: Chinese. A Chinese professional preparing for a business trip to the US. Teach business English presentation skills in Chinese, with English phrases and expressions for practice."
},
{
"case_id": "immigrant_de",
"category": "user_profile_immigrant",
"requirement": "Ich bin neu in Deutschland und muss schnell Deutsch für den Alltag lernen, mein Niveau ist A1",
"ground_truth": "Teaching language: German. This is a new immigrant in Germany needing everyday German at A1 level. Teach in simple, practical German for daily life situations."
},
{
"case_id": "heritage_zh",
"category": "user_profile_heritage",
"requirement": "I'm a Chinese-American, I can speak conversational Mandarin but can't read or write well. I want to improve my Chinese literacy.",
"ground_truth": "Teaching language: English. This is a heritage Chinese speaker who understands spoken Mandarin but lacks literacy. Teach in English, progressively introduce Chinese characters and reading skills building on their existing spoken knowledge."
},
{
"case_id": "tutor_math_bilingual",
"category": "user_profile_tutor",
"requirement": "我是数学家教,学生是ABC华裔,中文能听懂但更习惯英文思考,帮我准备高一数学内容",
"ground_truth": "Teaching language: Chinese. This is a Chinese math tutor whose student is an American-born Chinese who thinks in English. Course preparation in Chinese for the tutor, but math content should consider bilingual presentation to accommodate the student."
},
{
"case_id": "en_req_zh_pdf",
"category": "pdf_cross_language",
"requirement": "Summarize this Chinese research paper and explain the key findings",
"ground_truth": "Teaching language: English. The requirement is in English and the PDF is a Chinese NLP research paper. Teach in English, translating and explaining the Chinese paper's content.",
"pdfTextSample": "基于深度学习的自然语言处理技术研究综述\n摘要:近年来,深度学习技术在自然语言处理领域取得了显著进展。本文综述了基于Transformer架构的预训练语言模型"
},
{
"case_id": "en_req_en_pdf",
"category": "pdf_same_language",
"requirement": "Break down this paper chapter by chapter and create study notes",
"ground_truth": "Teaching language: English. Both the requirement and PDF are in English. Straightforward same-language case. Teach and summarize in English.",
"pdfTextSample": "Introduction to Machine Learning: A Comprehensive Survey\nAbstract: Machine learning has become a cornerstone of modern artificial intelligence. This survey covers supervised, unsupervised, and reinforcement learning paradigms"
},
{
"case_id": "zh_req_ja_pdf",
"category": "pdf_cross_language",
"requirement": "帮我翻译并讲解这篇日文材料的核心内容",
"ground_truth": "Teaching language: Chinese. The requirement is in Chinese and the PDF is in Japanese. Teach in Chinese, translating and explaining the Japanese content. Japanese terms shown with Chinese translation.",
"pdfTextSample": "ディープラーニングによる画像認識技術の最新動向\n概要:本稿では、畳み込みニューラルネットワーク(CNN)を中心とした画像認識技術の発展について概説する"
},
{
"case_id": "zh_req_fr_pdf",
"category": "pdf_cross_language",
"requirement": "请把这篇法语文献的要点整理成中文笔记",
"ground_truth": "Teaching language: Chinese. The requirement is in Chinese and the PDF is in French. Teach in Chinese, summarizing and translating the French paper's key points.",
"pdfTextSample": "L'intelligence artificielle dans l'éducation : perspectives et défis\nRésumé : Cet article examine l'impact croissant de l'intelligence artificielle sur les pratiques éducatives contemporaines"
},
{
"case_id": "ja_req_en_pdf",
"category": "pdf_cross_language",
"requirement": "この英語の論文を日本語で解説してください、専門用語も日本語に訳してください",
"ground_truth": "Teaching language: Japanese. The requirement is in Japanese and the PDF is in English. Teach in Japanese, translating and explaining the English paper. Technical terms translated to Japanese.",
"pdfTextSample": "Advances in Robotics and Autonomous Systems\nAbstract: This paper reviews recent developments in robotic perception, planning, and control systems with applications in manufacturing and healthcare"
},
{
"case_id": "en_req_multilingual_pdf",
"category": "pdf_multilingual",
"requirement": "Analyze this bilingual Chinese-English textbook and create a study guide",
"ground_truth": "Teaching language: English. The requirement is in English and the PDF is a bilingual Chinese-English textbook. Teach in English, leveraging both languages in the source material.",
"pdfTextSample": "Chapter 1: Introduction to Economics 经济学导论\n1.1 What is Economics? 什么是经济学?\nEconomics is the study of how societies allocate scarce resources.\n经济学是研究社会如何分配稀缺资源的学科。"
},
{
"case_id": "zh_teacher_ja_pdf",
"category": "pdf_teacher_perspective",
"requirement": "我是日语老师,用这篇日文短文给初级学生设计一节阅读课",
"ground_truth": "Teaching language: Chinese. This is a Chinese Japanese-language teacher using a Japanese article to design a reading lesson for beginners. Course design in Chinese, with Japanese text used as learning material. Vocabulary and grammar points explained in Chinese.",
"pdfTextSample": "桜の季節\n春になると、日本中で桜が咲きます。多くの人が公園でお花見をします。桜の花は美しいですが、すぐに散ってしまいます。"
}
]

View File

@ -0,0 +1,272 @@
/**
* Outline Language Inference — Real LLM Evaluation Tests
*
* Tests the languageDirective inferred during REAL outline generation.
* Calls the actual generateSceneOutlinesFromRequirements function,
* then uses an LLM-as-judge to evaluate the directive against ground truth.
*
* Calls real LLM APIs — meant to be run locally, NOT in CI/CD.
*
* Environment variables:
* EVAL_INFERENCE_MODEL Model for outline generation (default: DEFAULT_MODEL or gpt-4o-mini)
* EVAL_JUDGE_MODEL Model for LLM-as-judge (default: openai:gpt-4o-mini)
*
* Usage:
* EVAL_INFERENCE_MODEL=google:gemini-3-flash-preview \
* EVAL_JUDGE_MODEL=openai:gpt-4o-mini \
* pnpm vitest run tests/generation/outline-language.eval.test.ts
*
* Results are written to tests/generation/outline-language.eval.result.md
*/
import { describe, it, expect, afterAll, beforeAll } from 'vitest';
import { writeFileSync, readFileSync } from 'fs';
import { resolve } from 'path';
import { generateText } from 'ai';
import { generateSceneOutlinesFromRequirements } from '@/lib/generation/outline-generator';
import { resolveModel } from '@/lib/server/resolve-model';
import type { AICallFn } from '@/lib/generation/pipeline-types';
import { callLLM } from '@/lib/ai/llm';
// ---------------------------------------------------------------------------
// Test case definition
// ---------------------------------------------------------------------------
interface LanguageTestCase {
case_id: string;
category: string;
requirement: string;
ground_truth: string;
pdfTextSample?: string;
}
// ---------------------------------------------------------------------------
// Load test cases from JSON
// ---------------------------------------------------------------------------
const TEST_CASES: LanguageTestCase[] = JSON.parse(
readFileSync(resolve(__dirname, 'language-test-cases.json'), 'utf-8'),
);
// ---------------------------------------------------------------------------
// Model resolution
// ---------------------------------------------------------------------------
const JUDGE_MODEL_DEFAULT = 'openai:gpt-4o-mini';
/** Resolve the inference model (EVAL_INFERENCE_MODEL → DEFAULT_MODEL → gpt-4o-mini) */
function getInferenceModel() {
return resolveModel({
modelString: process.env.EVAL_INFERENCE_MODEL || undefined,
});
}
/** Resolve the judge model (EVAL_JUDGE_MODEL → gpt-4o-mini) */
function getJudgeModel() {
return resolveModel({
modelString: process.env.EVAL_JUDGE_MODEL || JUDGE_MODEL_DEFAULT,
});
}
// ---------------------------------------------------------------------------
// Build AICallFn from the inference model
// ---------------------------------------------------------------------------
async function buildAICallFn(): Promise<AICallFn> {
const { model: languageModel, modelInfo } = await getInferenceModel();
return async (systemPrompt, userPrompt, _images) => {
const result = await callLLM(
{
model: languageModel,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt },
],
maxOutputTokens: modelInfo?.outputWindow,
},
'eval-outline-language',
);
return result.text;
};
}
// ---------------------------------------------------------------------------
// LLM Judge
// ---------------------------------------------------------------------------
const JUDGE_SYSTEM_PROMPT = `You are evaluating whether a language directive for an AI course generation system is reasonable given the expected behavior.
You will be given:
1. The original user requirement
2. The generated language directive
3. The ground truth description of expected behavior
Evaluation criteria — the directive should:
- Use the correct primary teaching language
- Handle terminology in a reasonable way for the subject and audience
- For cross-language scenarios (foreign language learning, cross-language PDF), acknowledge both languages
Be LENIENT in your evaluation:
- The directive does NOT need to match the ground truth word-for-word
- Different but equally valid approaches should PASS
- If the teaching language is correct and the overall approach is reasonable, it should PASS
- Only FAIL if the directive is clearly WRONG (e.g., wrong teaching language, completely ignoring a cross-language situation)
Respond with ONLY a JSON object:
{"pass": true/false, "reason": "brief explanation (1-2 sentences)"}`;
interface JudgeResult {
pass: boolean;
reason: string;
}
async function judgeDirective(
requirement: string,
directive: string,
groundTruth: string,
): Promise<JudgeResult> {
const { model } = await getJudgeModel();
const result = await generateText({
model,
system: JUDGE_SYSTEM_PROMPT,
prompt: `Requirement: "${requirement}"\n\nGenerated directive: "${directive}"\n\nGround truth: "${groundTruth}"`,
temperature: 0,
});
try {
const text = result.text.replace(/```json\s*|\s*```/g, '').trim();
return JSON.parse(text) as JudgeResult;
} catch {
return { pass: false, reason: `Failed to parse judge response: ${result.text}` };
}
}
// ---------------------------------------------------------------------------
// Result collector
// ---------------------------------------------------------------------------
interface EvalResult {
case_id: string;
category: string;
requirement: string;
pdfTextSample?: string;
groundTruth: string;
directive: string;
outlinesCount: number;
judgePassed: boolean;
judgeReason: string;
}
const results: EvalResult[] = [];
// ---------------------------------------------------------------------------
// Test runner
// ---------------------------------------------------------------------------
describe('Outline Language Inference Evaluation', () => {
let aiCall: AICallFn;
beforeAll(async () => {
aiCall = await buildAICallFn();
});
for (const tc of TEST_CASES) {
it.concurrent(
`${tc.case_id}: ${tc.requirement.slice(0, 50)}`,
{ timeout: 120_000 },
async () => {
// Call the REAL outline generation function
const result = await generateSceneOutlinesFromRequirements(
{ requirement: tc.requirement },
tc.pdfTextSample || undefined, // pass PDF text if available
undefined, // no PDF images
aiCall,
undefined, // no callbacks
);
expect(result.success, `Outline generation failed: ${result.error}`).toBe(true);
expect(result.data).toBeDefined();
const { languageDirective, outlines } = result.data!;
expect(languageDirective, 'languageDirective should not be empty').toBeTruthy();
expect(outlines.length, 'should generate at least 1 outline').toBeGreaterThan(0);
// LLM-as-judge
const judge = await judgeDirective(tc.requirement, languageDirective, tc.ground_truth);
results.push({
case_id: tc.case_id,
category: tc.category,
requirement: tc.requirement,
pdfTextSample: tc.pdfTextSample,
groundTruth: tc.ground_truth,
directive: languageDirective,
outlinesCount: outlines.length,
judgePassed: judge.pass,
judgeReason: judge.reason,
});
expect(judge.pass, `Judge failed: ${judge.reason}`).toBe(true);
},
);
}
// Write results file after all tests
afterAll(() => {
if (results.length === 0) return;
const passed = results.filter((r) => r.judgePassed).length;
const total = results.length;
const inferenceModelStr =
process.env.EVAL_INFERENCE_MODEL || process.env.DEFAULT_MODEL || '(default: gpt-4o-mini)';
const judgeModelStr = process.env.EVAL_JUDGE_MODEL || JUDGE_MODEL_DEFAULT;
const lines: string[] = [
`# Outline Language Inference Eval Results`,
``,
`- **Date**: ${new Date().toISOString()}`,
`- **Passed**: ${passed}/${total} (${((passed / total) * 100).toFixed(0)}%)`,
`- **Inference model**: ${inferenceModelStr}`,
`- **Judge model**: ${judgeModelStr}`,
`- **Method**: real outline generation (generateSceneOutlinesFromRequirements) + LLM-as-judge`,
`- **Test cases**: curated from production data`,
``,
`## Detail`,
``,
];
for (const r of results) {
const icon = r.judgePassed ? 'PASS' : '**FAIL**';
lines.push(`### ${icon} ${r.case_id}`);
lines.push(``);
lines.push(`- **Category**: ${r.category}`);
lines.push(`- **Input**: \`${r.requirement}\``);
if (r.pdfTextSample) {
lines.push(`- **PDF sample**: \`${r.pdfTextSample.slice(0, 80)}...\``);
}
lines.push(`- **Ground truth**: ${r.groundTruth}`);
lines.push(`- **Directive**: ${r.directive}`);
lines.push(`- **Outlines generated**: ${r.outlinesCount}`);
lines.push(`- **Judge**: ${r.judgePassed ? 'PASS' : 'FAIL'} — ${r.judgeReason}`);
lines.push(``);
}
lines.push(`## Summary`);
lines.push(``);
lines.push(`| # | Case | Category | Outlines | Result | Judge reason |`);
lines.push(`|---|------|----------|----------|--------|--------------|`);
for (let i = 0; i < results.length; i++) {
const r = results[i];
const icon = r.judgePassed ? 'PASS' : 'FAIL';
lines.push(
`| ${i + 1} | ${r.case_id} | ${r.category} | ${r.outlinesCount} | ${icon} | ${r.judgeReason} |`,
);
}
lines.push(``);
const outPath = resolve(__dirname, 'outline-language.eval.result.md');
writeFileSync(outPath, lines.join('\n'), 'utf-8');
console.log(`\nResults written to: ${outPath}`);
});
});

23
tests/setup-env.ts Normal file
View File

@ -0,0 +1,23 @@
/**
* Load .env.local before tests so API keys are available.
*/
import { readFileSync } from 'fs';
import { resolve } from 'path';
const envPath = resolve(__dirname, '..', '.env.local');
try {
const content = readFileSync(envPath, 'utf-8');
for (const line of content.split('\n')) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith('#')) continue;
const eqIdx = trimmed.indexOf('=');
if (eqIdx < 0) continue;
const key = trimmed.slice(0, eqIdx).trim();
const value = trimmed.slice(eqIdx + 1).trim();
if (!process.env[key]) {
process.env[key] = value;
}
}
} catch {
// .env.local not found, skip
}

View File

@ -9,5 +9,7 @@ export default defineConfig({
},
test: {
include: ['tests/**/*.test.ts'],
exclude: ['tests/**/*.eval.test.ts'],
setupFiles: ['tests/setup-env.ts'],
},
});

14
vitest.eval.config.ts Normal file
View File

@ -0,0 +1,14 @@
import { resolve } from 'path';
import { defineConfig } from 'vitest/config';
export default defineConfig({
resolve: {
alias: {
'@': resolve(__dirname, '.'),
},
},
test: {
include: ['tests/**/*.eval.test.ts'],
setupFiles: ['tests/setup-env.ts'],
},
});