feat(choreography): shared orchestration spec in lib/choreography (#863) (#890)

* feat(choreography): shared orchestration spec in lib/choreography (#863)

Introduce lib/choreography/ as the single source of truth for the
orchestration semantics a faithful classroom-video exporter needs from
playback, so the app runtime and the exporter interpret one spec instead
of each re-implementing (and silently drifting from) the other.

Kept in lib/ rather than a package: these semantics co-evolve with the
playback engine, and the exporter will also live in the app, so both
consumers share them via ordinary imports. Purity is machine-enforced by
an eslint boundary on lib/choreography/** (blocks @/ host-app paths +
react/react-dom/gsap/framer-motion/motion), so the exporter can interpret
the spec in a pure Node environment.

- timing.ts   — timing constants + the deterministic no-audio speech
                estimate, moved verbatim from the engines.
- cursor.ts   — resolvePlaybackCursor + EMPTY_SCENE_DWELL, moved from
                lib/playback/engine-cursor.ts (typed on dsl SceneCore).
- timeline.ts — new pure resolveActionTimeline: index-domain -> time-domain
                expansion (blocking cursor-advance vs fire-and-forget visual
                duration), keyed off the DSL fire-and-forget partition.
- descriptors/— versioned, zod-schema-validated animation descriptors
                spotlight.v1 + laser.v1 (declarative: property/from/to/
                duration/easing; no implementation), pinned to the current
                overlay components.

Behavior-neutral engine refactor: lib/action/engine.ts and
lib/playback/engine.ts import from lib/choreography and the local literals
are deleted, so the timing dimension now has exactly one copy.

The spotlight/laser overlay components still hardcode their animation
values (they do not yet READ the descriptors) — tracked in #889.

Closes #863

* fix(choreography): address cross-review findings on resolveActionTimeline + descriptors

- P1: model implicit whiteboard auto-open — a wb_* mutation on a closed board
  now prepends a synthetic IMPLICIT_WB_OPEN (WB_OPEN_MS) beat, mirroring the
  engine's ensureWhiteboardOpen; open state carries across scenes and toggles
  on wb_open/wb_close (new `whiteboardOpen` option to seed it).
- P2: scale real speech audio duration by playbackSpeed too (live path sets
  AudioPlayer.setPlaybackRate), keeping it in lockstep with the estimate path.
- P2: express the spotlight mask relationship in the descriptor model —
  LayerSchema gains `role` ('content'|'mask') + `maskedBy` (subtract|intersect);
  spotlight.v1's cutout is now a mask layer the dim layer subtracts, so a
  non-React consumer reconstructs the cutout instead of a black rect.
- P3: wb_clear on an empty board is 0ms (engine early-returns), not wbClearMs(0).

Tests: tests/lib/choreography 45 pass (+6); engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): address final-audit findings on effect lifetime + spotlight descriptor

Second cross-review round (fresh-session codex final audit) surfaced three
deeper mismatches with live playback:

- P1: fire-and-forget effect lifetime is not a flat EFFECT_AUTO_CLEAR_MS. The
  engine's processNext clears effects at every scene boundary and on completion,
  and scheduleEffectClear uses one shared timer each new effect resets. Added
  clampFireAndForgetLifetimes: an effect's visual durationMs is now min(next
  scene boundary / completion, shared-timer deadline chained through later
  effects in the same scene). advancesCursorMs (0) is untouched.
- P2: spotlight dimness default is 0.5 (executeSpotlight: dimOpacity ?? 0.5; DSL
  documents 0.5), not the component's unreachable ?? 0.7 fallback. Fixed the
  descriptor param + test.
- P2: model the spotlight wrapper's enter/exit opacity fade (motion.div, no
  explicit duration → engine default). TrackSchema.durationMs is now optional to
  express "use the consumer's engine default"; dim layer carries the fade tracks.

Tests: tests/lib/choreography 50 pass (+5, incl. boundary-cut / completion-cut /
full-lifetime / shared-timer-extension / wrapper-fade); engine regression 28 pass;
tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): pin spotlight fade duration + laser dot geometry in descriptors

Third cross-review round (codex) flagged two descriptor-completeness gaps that
would make a non-Motion consumer (the exporter) diverge from the app:

- Spotlight wrapper fade: the enter/exit opacity tracks left durationMs implicit
  (Motion default). A literal consumer treats a missing duration as instant, so
  the spotlight would pop on/off. Pinned to Motion's default 300ms tween.
- Laser dot geometry: the descriptor captured only tracks, not the dot group's
  center anchor (translate -50%,-50%) or the rounded-full ring/core. A literal
  renderer would draw an offset 10px square. Added the static geometry (anchor,
  borderRadius 9999, ring inset/position) so the shape/position match the app.

Also refined the effect-lifetime docstring to cite the app's per-scene engine
teardown/completion (the actual clearEffects path) rather than an intra-engine
boundary gate that is dead in the single-scene-per-engine configuration.

The empty-scene "speech dwell → blank chat bubble" observation is pre-existing
behavior: EMPTY_SCENE_DWELL is a verbatim move from lib/playback/engine-cursor.ts
(unchanged from origin/main), out of scope for this move-only PR; tracked separately.

Tests: tests/lib/choreography 54 pass; engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): zero-duration for engine-skipped/no-op actions in timeline

Fourth cross-review round (codex) flagged two remaining timeline divergences:

- Skipped discussions: the engine skips a discussion outright (no timer) when
  it's already consumed or its agent isn't selected, but the timeline always
  charged DISCUSSION_TRIGGER_DELAY_MS. Added `isDiscussionSkipped` resolver
  (runtime-state-dependent, like getVideoDurationMs) → 0ms when skipped.
- No-op whiteboard draws: executeWbDrawText (empty content) and
  executeWbDrawTable (no rows/cols) return before any delay. The timeline now
  charges 0ms for these determinable-from-the-action no-ops instead of
  WB_DRAW_MS. (KaTeX-failure / missing-edit-target no-ops depend on runtime
  state and remain out of scope, consistent with the resolver pattern.)

Tests: tests/lib/choreography 56 pass; engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): exact-5s effect chain break + spotlight dim full-screen geometry

Fifth cross-review round (codex), both P2:

- Effect chain break at an EXACT 5s boundary: the earlier effect's clear timer
  is queued before the reading timer that triggers the later effect (same 5000ms
  delay), so it fires first — the predecessor is cleared at exactly deadlineMs,
  not extended. Changed the chain guard from `> deadlineMs` to `>= deadlineMs`.
- Spotlight dim layer full-screen geometry: the descriptor recorded only fill +
  mask relation, leaving a literal consumer no way to know the dim rect spans the
  0..100 viewport. Added explicit x/y/width/height (100×100 at origin) so the
  descriptor is self-contained.

Tests: tests/lib/choreography 57 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): no-op wb_edit_code resolver in timeline

Sixth cross-review round (codex), one P2: executeWbEditCode returns before its
delay when the edit can't apply (missing/non-code target, stale line refs). The
timeline always charged WB_EDIT_MS. Added `isEditCodeNoop` resolver (runtime-
state-dependent, same pattern as getClearElementCount / isDiscussionSkipped) →
0ms when the caller flags a no-op.

Tests: tests/lib/choreography 58 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): descriptor layer inheritance for nested effect layers

Seventh cross-review round (codex), two P2 with one root cause: the flat
layers[] model couldn't express a child layer riding a parent's animation
(the source nests some layers inside an animated wrapper). Added an
`inheritsFrom: {parentId, props}` relation to LayerSchema:

- Laser ring + core inheritsFrom the animated `dot` (left/top/opacity), so a
  literal consumer flies them in/out with the dot instead of leaving them at a
  static origin while only the dot moves.
- Spotlight border inheritsFrom `dim` (opacity), so the outline fades out with
  the wrapper instead of lingering after the dimming layer disappears.

Tests: tests/lib/choreography 60 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): address review — discussion auto-skip, explicit video policy, import allowlist

Human review (wyuc) raised two blocking timing issues + one non-blocking guard:

- Discussion dwell (blocking): a non-skipped discussion in unattended
  playback/export blocks for the trigger delay AND the ProactiveCard's own
  auto-skip countdown, not just DISCUSSION_TRIGGER_DELAY_MS. Added
  DISCUSSION_AUTO_SKIP_MS (5000) to the timing spec and charge
  DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS. ProactiveCard now reads
  the same constant (was a hardcoded 5000), so card countdown and timeline can't
  drift. A `spotlight -> discussion -> speech` timeline now extends the spotlight
  across the full discussion interval.
- play_video (blocking): an unresolved duration no longer silently becomes a
  0ms segment (which shifted every later action early). New
  `onUnresolvedVideoDuration` policy defaults to 'throw' (fail loudly); 'cap'
  assumes MAX_VIDEO_WAIT_MS, 'zero' opts back into no-dwell explicitly.
- Purity guard (non-blocking): turned the lib/choreography boundary into a true
  import allowlist. Beyond the existing @/… + render-package blocks, it now
  rejects parent-escape (../…) imports/re-exports, any bare package other than
  @openmaic/dsl / zod, and dynamic import()/require(). Negative-tested: ../store,
  a stray bare package, export * from ../playback, and import('react') all fail.

Tests: tests/lib/choreography 60 pass; engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
This commit is contained in:
杨慎 2026-07-11 13:57:06 +08:00 committed by GitHub
parent b4c1c580f8
commit 7fffec78f6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
17 changed files with 1844 additions and 43 deletions

5
.codegraph/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
# CodeGraph data files — local to each machine, not for committing.
# Ignore everything in .codegraph/ except this file itself, so transient
# files (the database, daemon.pid, sockets, logs) never show up in git.
*
!.gitignore

View File

@ -5,6 +5,7 @@ import { createPortal } from 'react-dom';
import { motion } from 'motion/react';
import { Play, Pause, X } from 'lucide-react';
import { useI18n } from '@/lib/hooks/use-i18n';
import { DISCUSSION_AUTO_SKIP_MS } from '@/lib/choreography';
import type { DiscussionAction } from '@/lib/types/action';
interface ProactiveCardProps {
@ -92,7 +93,7 @@ export const ProactiveCard = ({
useEffect(() => {
if (mode !== 'playback') return;
const duration = 5000;
const duration = DISCUSSION_AUTO_SKIP_MS;
const interval = 50;
const step = (interval / duration) * 100;
@ -201,7 +202,7 @@ export const ProactiveCard = ({
isPaused ? 'text-gray-300 dark:text-gray-600' : 'text-gray-400 dark:text-gray-500'
}`}
>
{Math.max(0, Math.ceil((progress / 100) * 5))}s
{Math.max(0, Math.ceil((progress / 100) * (DISCUSSION_AUTO_SKIP_MS / 1000)))}s
</span>
</div>

View File

@ -111,6 +111,93 @@ const eslintConfig = defineConfig([
],
},
},
// Module boundary (machine-enforced): lib/choreography is the shared
// orchestration spec (timing + action timeline). It lives in the app (not a
// package) because its semantics co-evolve with the playback engine, but it
// must stay pure so the classroom-video exporter can interpret it in a pure
// Node environment. Two guards, mirroring the package boundaries above:
// 1. NO `@/…` host-app path-alias string — it authors none; it depends only
// on @openmaic/dsl (types + the fire-and-forget partition) and relative
// siblings. The app and exporter import it, never the reverse.
// 2. NO React / DOM / render-backend runtime import (react, react-dom, gsap,
// framer-motion, motion) — these are bare specifiers the `@/` rule can't
// see. A descriptor describes animation; it never renders it.
{
files: ['lib/choreography/**/*.{ts,tsx,js,jsx,mjs,cjs}'],
rules: {
'no-restricted-syntax': [
'error',
{
selector: 'Literal[value=/^@\\//]',
message:
'lib/choreography must not reference a host-app path (@/…). It authors no `@/…` strings — depend only on @openmaic/dsl and relative siblings, so the exporter can interpret it in pure Node. The app and exporter import it, not the reverse.',
},
{
selector: 'TemplateElement[value.cooked=/^@\\//]',
message:
'lib/choreography must not reference a host-app path (@/…) in a template literal. Depend only on @openmaic/dsl and relative siblings.',
},
// Import allowlist (static imports/re-exports): the ONLY permitted
// sources are `@openmaic/dsl`(/subpaths), `zod`, and in-folder relatives
// (`./…`). Anything else — a parent-escape `../…` reaching back into the
// app, or any other bare package — fails. Enforced on Import/Export
// source string nodes via a negative-lookahead so the guard is a true
// allowlist, not a blocklist of known-bad names.
{
selector:
'ImportDeclaration > Literal.source[value=/^(?!@openmaic\\/dsl(\\/|$)|zod(\\/|$)|\\.\\/).+/]',
message:
'lib/choreography may import only from @openmaic/dsl, zod, or in-folder relatives (./…). No parent-escape (../…) into the app and no other packages — keep it pure so the exporter runs in plain Node.',
},
{
selector:
'ExportNamedDeclaration > Literal.source[value=/^(?!@openmaic\\/dsl(\\/|$)|zod(\\/|$)|\\.\\/).+/]',
message:
'lib/choreography may re-export only from @openmaic/dsl, zod, or in-folder relatives (./…).',
},
{
selector:
'ExportAllDeclaration > Literal.source[value=/^(?!@openmaic\\/dsl(\\/|$)|zod(\\/|$)|\\.\\/).+/]',
message:
'lib/choreography may re-export only from @openmaic/dsl, zod, or in-folder relatives (./…).',
},
// No dynamic import() or require() — they bypass the static allowlist and
// can pull in a render backend at runtime.
{
selector: 'ImportExpression',
message:
'lib/choreography must not use dynamic import() — it bypasses the static import allowlist. Use a top-level import from @openmaic/dsl, zod, or a relative sibling.',
},
{
selector: "CallExpression[callee.name='require']",
message:
'lib/choreography must not use require() — it bypasses the static import allowlist.',
},
],
'no-restricted-imports': [
'error',
{
patterns: [
{
group: [
'react',
'react-dom',
'react/*',
'react-dom/*',
'gsap',
'gsap/*',
'framer-motion',
'motion',
'motion/*',
],
message:
'lib/choreography must stay render-backend-agnostic (pure Node): no React / DOM / GSAP / framer-motion. It describes timing and animation as data; the app effect components and the exporter render it.',
},
],
},
],
},
},
]);
export default eslintConfig;

View File

@ -36,6 +36,18 @@ import type {
WidgetRevealAction,
} from '@/lib/types/action';
import type { CodeLine } from '@openmaic/dsl';
import {
EFFECT_AUTO_CLEAR_MS,
MAX_VIDEO_WAIT_MS,
WB_OPEN_MS,
WB_DRAW_MS,
WB_EDIT_MS,
WB_DELETE_MS,
WB_CLOSE_MS,
WIDGET_MS,
wbDrawCodeMs,
wbClearMs,
} from '@/lib/choreography';
import katex from 'katex';
import { createLogger } from '@/lib/logger';
@ -71,9 +83,6 @@ function generateLineIds(count: number): string[] {
// ==================== ActionEngine ====================
/** Default duration (ms) before fire-and-forget effects auto-clear */
const EFFECT_AUTO_CLEAR_MS = 5000;
/** Callback for sending messages to widget iframe */
export type WidgetMessageCallback = (type: string, payload: Record<string, unknown>) => void;
@ -293,7 +302,6 @@ export class ActionEngine {
// the playback engine from hanging indefinitely if the video element is
// invalid or the state change is missed.
return new Promise<void>((resolve) => {
const MAX_VIDEO_WAIT_MS = 5 * 60 * 1000; // 5 minutes
const timeout = setTimeout(() => {
unsubscribe();
log.warn(`[playVideo] Timeout waiting for video ${action.elementId} to finish`);
@ -365,7 +373,7 @@ export class ActionEngine {
useCanvasStore.getState().setWhiteboardOpen(true);
if (options.silent) return;
// Wait for open animation to complete (slow spring: stiffness 120, damping 18, mass 1.2)
await delay(2000);
await delay(WB_OPEN_MS);
}
private async executeWbDrawText(
@ -401,7 +409,7 @@ export class ActionEngine {
if (!options.silent) {
// Wait for element fade-in animation
await delay(800);
await delay(WB_DRAW_MS);
}
}
@ -432,7 +440,7 @@ export class ActionEngine {
if (!options.silent) {
// Wait for element fade-in animation
await delay(800);
await delay(WB_DRAW_MS);
}
}
@ -460,7 +468,7 @@ export class ActionEngine {
wb.data.id,
);
if (!options.silent) await delay(800);
if (!options.silent) await delay(WB_DRAW_MS);
}
private async executeWbDrawLatex(
@ -499,7 +507,7 @@ export class ActionEngine {
return;
}
if (!options.silent) await delay(800);
if (!options.silent) await delay(WB_DRAW_MS);
}
private async executeWbDrawTable(
@ -558,7 +566,7 @@ export class ActionEngine {
wb.data.id,
);
if (!options.silent) await delay(800);
if (!options.silent) await delay(WB_DRAW_MS);
}
private async executeWbDrawLine(
@ -595,7 +603,7 @@ export class ActionEngine {
if (!options.silent) {
// Wait for element fade-in animation
await delay(800);
await delay(WB_DRAW_MS);
}
}
@ -628,8 +636,8 @@ export class ActionEngine {
);
if (!options.silent) {
// Wait for typing animation: base 800ms + 50ms per line, capped at 3s
const animMs = Math.min(800 + lines.length * 50, 3000);
// Wait for typing animation (base 800ms + 50ms/line, capped at 3s)
const animMs = wbDrawCodeMs(lines.length);
await delay(animMs);
}
}
@ -697,7 +705,7 @@ export class ActionEngine {
if (!options.silent) {
// Wait for edit animation
await delay(600);
await delay(WB_EDIT_MS);
}
}
@ -709,7 +717,7 @@ export class ActionEngine {
if (!wb.success || !wb.data) return;
this.stageAPI.whiteboard.deleteElement(action.elementId, wb.data.id);
if (!options.silent) await delay(300);
if (!options.silent) await delay(WB_DELETE_MS);
}
private async executeWbClear(options: ActionExecutionOptions = {}): Promise<void> {
@ -731,8 +739,8 @@ export class ActionEngine {
// Trigger cascade exit animation
useCanvasStore.getState().setWhiteboardClearing(true);
// Wait for cascade: base 380ms + 55ms per element, capped at 1400ms
const animMs = Math.min(380 + elementCount * 55, 1400);
// Wait for cascade (base 380ms + 55ms/element, capped at 1400ms)
const animMs = wbClearMs(elementCount);
await delay(animMs);
// Actually remove elements
@ -744,7 +752,7 @@ export class ActionEngine {
useCanvasStore.getState().setWhiteboardOpen(false);
if (options.silent) return;
// Wait for close animation (500ms ease-out tween)
await delay(700);
await delay(WB_CLOSE_MS);
}
// ==================== Widget Actions ====================
@ -765,14 +773,14 @@ export class ActionEngine {
content: action.content,
});
// Quick delay for visual effect
await delay(300);
await delay(WIDGET_MS);
}
/** Execute widget setState action */
private async executeWidgetSetState(action: WidgetSetStateAction): Promise<void> {
this.sendWidgetMessage('SET_WIDGET_STATE', { state: action.state, content: action.content });
// Quick delay for state change to propagate
await delay(300);
await delay(WIDGET_MS);
}
/** Execute widget annotation action */
@ -781,12 +789,12 @@ export class ActionEngine {
target: action.target,
content: action.content,
});
await delay(300);
await delay(WIDGET_MS);
}
/** Execute widget reveal action */
private async executeWidgetReveal(action: WidgetRevealAction): Promise<void> {
this.sendWidgetMessage('REVEAL_ELEMENT', { target: action.target, content: action.content });
await delay(300);
await delay(WIDGET_MS);
}
}

View File

@ -1,5 +1,12 @@
import type { Action } from '@/lib/types/action';
import type { Scene } from '@/lib/types/stage';
/**
* Playback cursor — resolve the current action from a `(sceneIndex, actionIndex)`
* cursor over a scene list.
*
* Moved verbatim from the app's `lib/playback/engine-cursor.ts` so the app
* runtime and the video exporter walk scenes identically. Pure — types come
* from `@openmaic/dsl`, no runtime dependencies.
*/
import type { Action, SceneCore } from '@openmaic/dsl';
/**
* Synthetic dwell beat yielded for a scene that carries no actions. It is an
@ -30,10 +37,12 @@ export interface CursorResult {
* yields one {@link EMPTY_SCENE_DWELL} beat (when its action cursor is still 0)
* rather than being skipped. Returns `null` once every scene is consumed.
*
* Pure: it does not mutate inputs; the caller adopts the returned cursor.
* Pure: it does not mutate inputs; the caller adopts the returned cursor. Typed
* against {@link SceneCore} (only `id` + `actions` are read), so an app-widened
* `Scene` (extra content kinds) is accepted without casting.
*/
export function resolvePlaybackCursor(
scenes: Scene[],
scenes: SceneCore[],
sceneIndex: number,
actionIndex: number,
): CursorResult | null {

View File

@ -0,0 +1,22 @@
/**
* Animation descriptor registry — the versioned, declarative animations both
* the app effect components and the video exporter interpret.
*/
import type { AnimationDescriptor } from './types';
import { spotlightV1 } from './spotlight';
import { laserV1 } from './laser';
export * from './types';
export { spotlightV1 } from './spotlight';
export { laserV1 } from './laser';
/** All shipped descriptors, keyed by their versioned id. */
export const DESCRIPTORS: Record<string, AnimationDescriptor> = {
'spotlight.v1': spotlightV1,
'laser.v1': laserV1,
};
/** Look up a descriptor by its versioned id (e.g. 'spotlight.v1'). */
export function getDescriptor(id: string): AnimationDescriptor | undefined {
return DESCRIPTORS[id];
}

View File

@ -0,0 +1,138 @@
import type { AnimationDescriptor } from './types';
/**
* laser.v1 — a laser dot flies in from the nearest off-screen corner to the
* element center, with a pulsing ring and a glowing core.
*
* Values captured verbatim from the `LaserOverlay` effect component
* (`motion/react`):
* - dot fly-in: left/top 500ms expo-out `[0.22,1,0.36,1]`; opacity 150ms;
* start corner is `center > 50 ? 105 : -5` (percent) per axis.
* - exit: 250ms ease-in `[0.4,0,1,1]` back to the start corner (position +
* opacity).
* - ring: infinite pulse, scale 1→2.8, opacity 0.6→0, 1500ms easeOut, 300ms
* repeat delay.
* - core: 10px square (`w-2.5 h-2.5`), glow `0 0 8px 2px {color}60`.
*
* Static geometry captured too, so a literal (non-React) renderer matches the
* app rather than drawing an offset square: the dot group is centered on the
* target via `translate(-50%, -50%)` (`-translate-x/y-1/2`), and both the ring
* and core are circles (`rounded-full` → `borderRadius: 9999px`); the ring is
* `position: absolute; inset: 0` inside the group.
*
* Color default `#ff0000` — the app store's laserOptions default
* (`lib/store/canvas.ts`), which the caller passes in; the component's own
* fallback `#ff3b30` is only used when no options exist, which does not happen
* at playback time.
*/
export const laserV1: AnimationDescriptor = {
id: 'laser.v1',
version: 1,
effect: 'laser',
params: { color: '#ff0000' },
zIndex: 101,
layers: [
{
id: 'dot',
// The dot group is anchored at its own center on the target coordinate.
staticProps: { transform: 'translate(-50%, -50%)' },
tracks: [
{
property: 'left',
from: { axis: 'centerX', threshold: 50, whenAbove: 105, whenBelow: -5 },
to: { ref: 'centerX' },
durationMs: 500,
easing: { type: 'cubicBezier', points: [0.22, 1, 0.36, 1] },
phase: 'enter',
},
{
property: 'top',
from: { axis: 'centerY', threshold: 50, whenAbove: 105, whenBelow: -5 },
to: { ref: 'centerY' },
durationMs: 500,
easing: { type: 'cubicBezier', points: [0.22, 1, 0.36, 1] },
phase: 'enter',
},
{
property: 'opacity',
from: 0,
to: 1,
durationMs: 150,
phase: 'enter',
},
{
property: 'left',
from: { ref: 'centerX' },
to: { axis: 'centerX', threshold: 50, whenAbove: 105, whenBelow: -5 },
durationMs: 250,
easing: { type: 'cubicBezier', points: [0.4, 0, 1, 1] },
phase: 'exit',
},
{
property: 'top',
from: { ref: 'centerY' },
to: { axis: 'centerY', threshold: 50, whenAbove: 105, whenBelow: -5 },
durationMs: 250,
easing: { type: 'cubicBezier', points: [0.4, 0, 1, 1] },
phase: 'exit',
},
{
property: 'opacity',
from: 1,
to: 0,
durationMs: 250,
easing: { type: 'cubicBezier', points: [0.4, 0, 1, 1] },
phase: 'exit',
},
],
},
{
id: 'ring',
// Nested inside the animated dot wrapper in the source, so it rides the
// dot's fly-in/exit position + fade, then adds its own scale/opacity pulse.
inheritsFrom: { parentId: 'dot', props: ['left', 'top', 'opacity'] },
// Circular pulse ring, absolutely positioned to fill the dot group.
staticProps: {
position: 'absolute',
inset: 0,
borderRadius: 9999,
border: '1.5px solid {color}',
},
tracks: [
{
property: 'scale',
from: 1,
to: 2.8,
durationMs: 1500,
easing: { type: 'named', name: 'easeOut' },
repeat: 'infinite',
repeatDelayMs: 300,
},
{
property: 'opacity',
from: 0.6,
to: 0,
durationMs: 1500,
easing: { type: 'named', name: 'easeOut' },
repeat: 'infinite',
repeatDelayMs: 300,
},
],
},
{
id: 'core',
// Nested inside the animated dot wrapper too — rides the dot's fly-in/exit
// position + fade. It has no animation of its own.
inheritsFrom: { parentId: 'dot', props: ['left', 'top', 'opacity'] },
// Circular light core (rounded-full).
staticProps: {
width: 10,
height: 10,
borderRadius: 9999,
backgroundColor: '{color}',
boxShadow: '0 0 8px 2px {color}60',
},
tracks: [],
},
],
};

View File

@ -0,0 +1,160 @@
import type { AnimationDescriptor } from './types';
/**
* spotlight.v1 — focus a single element, dimming the rest.
*
* An SVG mask (0-100 viewBox) punches a rounded cutout over a dimmed full-screen
* rect; a white border traces the cutout. Values captured verbatim from the
* `SpotlightOverlay` effect component (`motion/react`):
* - cutout: 600ms expo-out, insets from ±8/rx4 to the tight ±~0.5/rx1 frame.
* Modeled as a `role: 'mask'` layer — it is not painted itself; the `dim`
* layer subtracts it (`maskedBy`), so a non-React consumer reconstructs the
* "dim everywhere except the cutout" compositing rather than "draw a black
* rect".
* - border: 500ms expo-out, delayed 50ms, fading in as it settles.
* - dim: static `rgba(0,0,0,{dimness})`, dimness default 0.5, with the cutout
* subtracted.
*
* The `dimness` default is 0.5 — the value the runtime actually renders: a
* spotlight action with no `dimOpacity` is stored as `action.dimOpacity ?? 0.5`
* (`ActionEngine.executeSpotlight`; DSL documents `dimOpacity` default 0.5), so
* the component's own `?? 0.7` fallback is unreachable at playback. The exporter
* must use 0.5 to match.
*
* Shared easing `[0.16, 1, 0.3, 1]` (the spotlight expo-out).
*/
export const spotlightV1: AnimationDescriptor = {
id: 'spotlight.v1',
version: 1,
effect: 'spotlight',
params: { dimness: 0.5 },
zIndex: 100,
layers: [
{
id: 'cutout',
// Geometry only — subtracted from `dim` (see its maskedBy), not painted.
role: 'mask',
staticProps: { fill: '#000000' },
tracks: [
{
property: 'x',
from: { ref: 'x', offset: -8 },
to: { ref: 'x', offset: -0.4 },
durationMs: 600,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'y',
from: { ref: 'y', offset: -8 },
to: { ref: 'y', offset: -0.6 },
durationMs: 600,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'width',
from: { ref: 'w', offset: 16 },
to: { ref: 'w', offset: 0.8 },
durationMs: 600,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'height',
from: { ref: 'h', offset: 16 },
to: { ref: 'h', offset: 1.2 },
durationMs: 600,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'rx',
from: 4,
to: 1,
durationMs: 600,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
],
},
{
id: 'border',
// The white outline sits inside the same wrapper motion.div as the dim
// rect, so it fades in/out with the whole effect. `dim` carries the
// canonical wrapper opacity fade; the border rides it (on top of its own
// geometry + border-opacity tracks below).
inheritsFrom: { parentId: 'dim', props: ['opacity'] },
staticProps: {
stroke: 'rgba(255,255,255,0.7)',
strokeWidth: 1.2,
fill: 'none',
vectorEffect: 'non-scaling-stroke',
},
tracks: [
{
property: 'x',
from: { ref: 'x', offset: -4 },
to: { ref: 'x', offset: -0.4 },
durationMs: 500,
delayMs: 50,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'y',
from: { ref: 'y', offset: -4 },
to: { ref: 'y', offset: -0.6 },
durationMs: 500,
delayMs: 50,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'width',
from: { ref: 'w', offset: 8 },
to: { ref: 'w', offset: 0.8 },
durationMs: 500,
delayMs: 50,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'height',
from: { ref: 'h', offset: 8 },
to: { ref: 'h', offset: 1.2 },
durationMs: 500,
delayMs: 50,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'opacity',
from: 0,
to: 1,
durationMs: 500,
delayMs: 50,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
{
property: 'rx',
from: 2,
to: 1,
durationMs: 500,
delayMs: 50,
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
},
],
},
{
id: 'dim',
// Full-screen dim behind the cutout, with the cutout subtracted (SVG
// <mask>: white full-cover minus the black cutout rect). Spans the whole
// 0..100 viewport (`<rect width="100" height="100">` at the origin) — made
// explicit so a literal consumer dims the entire frame rather than nothing.
maskedBy: { layerId: 'cutout', mode: 'subtract' },
staticProps: { x: 0, y: 0, width: 100, height: 100, fill: 'rgba(0,0,0,{dimness})' },
// The whole effect is wrapped in a motion.div that fades opacity 0→1 on
// enter and →0 on exit. The source sets no explicit transition, so Motion
// applies its default tween (300ms) — pinned here explicitly so a
// non-Motion consumer (the exporter) reproduces the fade instead of
// popping the spotlight on/off. This layer carries the canonical wrapper
// fade; the border layer inherits its opacity so both fade out together.
tracks: [
{ property: 'opacity', from: 0, to: 1, durationMs: 300, phase: 'enter' },
{ property: 'opacity', from: 1, to: 0, durationMs: 300, phase: 'exit' },
],
},
],
};

View File

@ -0,0 +1,190 @@
/**
* Animation descriptor model — a declarative, render-backend-agnostic
* description of an effect animation: *what property, from what value to what
* value, over how long, with what easing*. No implementation, no `motion`, no
* DOM. The app's effect components and the video exporter both interpret these,
* so the animation values live in exactly one place and cannot drift.
*
* Descriptors are versioned (`spotlight.v1`) and schema-validated: the schema
* is authored here with zod, the TS types are inferred from it (single source),
* and every shipped descriptor is checked against it in tests. The exporter can
* reuse {@link AnimationDescriptorSchema} to validate anything it interprets.
*
* Pure — depends only on zod, no React / DOM / render backend.
*/
import { z } from 'zod';
/** A field of the target element's percentage geometry (0-100 space). */
export const GeometryRefSchema = z.enum(['x', 'y', 'w', 'h', 'centerX', 'centerY']);
/**
* A value derived linearly from the target element's geometry:
* `value = geometry[ref] * scale + offset`. Used for effect positions that
* track the highlighted element (e.g. a spotlight cutout inset by a few units).
*/
export const GeometryValueSchema = z.object({
ref: GeometryRefSchema,
/** Multiplier on the geometry field. Default 1. */
scale: z.number().optional(),
/** Added after scaling. Default 0. */
offset: z.number().optional(),
});
/**
* A corner/edge fly-in start value: pick one of two off-screen positions based
* on which half of the viewport the element center sits in. Models the laser's
* `center > 50 ? 105 : -5` start rule.
*/
export const CornerValueSchema = z.object({
/** Which center axis to test. */
axis: z.enum(['centerX', 'centerY']),
/** Comparison threshold (percent). */
threshold: z.number(),
/** Value used when the center is strictly above the threshold. */
whenAbove: z.number(),
/** Value used otherwise. */
whenBelow: z.number(),
});
/**
* An animatable endpoint: a literal number, a literal string (colors; may carry
* a `{param}` placeholder), or a geometry-/corner-derived value.
*/
export const AnimatableValueSchema = z.union([
z.number(),
z.string(),
GeometryValueSchema,
CornerValueSchema,
]);
/** Easing curve. Omit on a track to use the consumer's engine default. */
export const EasingSchema = z.discriminatedUnion('type', [
z.object({
type: z.literal('cubicBezier'),
points: z.tuple([z.number(), z.number(), z.number(), z.number()]),
}),
z.object({ type: z.literal('named'), name: z.string() }),
z.object({
type: z.literal('spring'),
stiffness: z.number(),
damping: z.number(),
mass: z.number().optional(),
}),
]);
/** Which phase of the effect lifecycle a track belongs to. Default 'enter'. */
export const TrackPhaseSchema = z.enum(['enter', 'exit']);
/** A single animated property from `from` to `to` over `durationMs`. */
export const TrackSchema = z.object({
/** The property name (e.g. 'x', 'width', 'opacity', 'scale', 'left', 'top'). */
property: z.string(),
from: AnimatableValueSchema,
to: AnimatableValueSchema,
/** Omitted when the source specifies no explicit duration — use the consumer's engine default. */
durationMs: z.number().optional(),
delayMs: z.number().optional(),
/** Omitted when the source specifies no explicit easing. */
easing: EasingSchema.optional(),
phase: TrackPhaseSchema.optional(),
/** Number of repeats, or 'infinite'. Omit for no repeat. */
repeat: z.union([z.number(), z.literal('infinite')]).optional(),
repeatDelayMs: z.number().optional(),
});
/** Non-animated static value on a layer; strings may carry `{param}` placeholders. */
const StaticPropsSchema = z.record(z.string(), z.union([z.number(), z.string()]));
/** Whether a layer is painted directly, or only defines geometry used as a mask. */
export const LayerRoleSchema = z.enum(['content', 'mask']);
/**
* How a content layer is clipped by a `mask`-role layer:
* - `subtract` — the mask region is removed (made transparent). Models an SVG
* `<mask>` of a white full-cover minus a black shape (the spotlight: the dim
* rect shows everywhere *except* the cutout).
* - `intersect` — only the mask region is kept.
*/
export const MaskModeSchema = z.enum(['subtract', 'intersect']);
/** A content layer's clip relationship to a `mask`-role layer in the same descriptor. */
export const MaskRefSchema = z.object({
/** `id` of the sibling layer (role `mask`) whose animated geometry clips this one. */
layerId: z.string(),
mode: MaskModeSchema,
});
/** Which of a parent layer's animated properties a child inherits. */
export const InheritablePropSchema = z.enum(['left', 'top', 'x', 'y', 'opacity', 'scale']);
/**
* A layer's inheritance from a parent layer in the same descriptor. In the
* source, the effect nests some layers inside an animated wrapper so they ride
* its motion (the laser ring/core sit inside the animated dot; the spotlight
* border sits inside the fading SVG wrapper). A flat layer list can't express
* that, so a child names its `parentId` and the `props` it rides — a literal
* consumer composes the parent's animation for those props with the child's own
* (e.g. the ring inherits the dot's left/top/opacity and adds its own scale
* pulse). `props` defaults to all of the parent's animated properties.
*/
export const InheritRefSchema = z.object({
/** `id` of the parent layer whose animation this layer rides. */
parentId: z.string(),
/** Properties inherited from the parent. Omit to inherit all of them. */
props: z.array(InheritablePropSchema).optional(),
});
/**
* A visual layer of the effect (e.g. the spotlight cutout, its border, the
* laser ring). Groups animated `tracks` with non-animated `staticProps`.
*
* A layer with `role: 'mask'` is not painted on its own — its animated geometry
* is referenced by another layer's `maskedBy` to clip it. This lets a non-React
* consumer reconstruct compositing relationships (e.g. the spotlight dim rect
* with the cutout punched out) that independent layers alone cannot express.
*
* A layer with `inheritsFrom` rides a parent layer's animation for the named
* properties (the source nests it inside that animated wrapper), on top of its
* own `tracks`.
*/
export const LayerSchema = z.object({
id: z.string(),
/** Default `content` (painted). `mask` layers only supply geometry for a `maskedBy` ref. */
role: LayerRoleSchema.optional(),
/** This (content) layer is clipped by the referenced `mask`-role layer. */
maskedBy: MaskRefSchema.optional(),
/** This layer rides a parent layer's animation (it is nested inside it in the source). */
inheritsFrom: InheritRefSchema.optional(),
tracks: z.array(TrackSchema),
staticProps: StaticPropsSchema.optional(),
});
/** A versioned, declarative animation for one effect. */
export const AnimationDescriptorSchema = z.object({
/** Stable id including version, e.g. 'spotlight.v1'. */
id: z.string(),
/** Numeric version, bumped on any behavioral change. */
version: z.number(),
effect: z.enum(['spotlight', 'laser']),
/** Default parameter values; consumers may override (e.g. dimness, color). */
params: StaticPropsSchema.optional(),
/** Stacking order the effect renders at. */
zIndex: z.number(),
layers: z.array(LayerSchema),
});
// Types are inferred from the schemas so the schema stays the single source.
export type GeometryRef = z.infer<typeof GeometryRefSchema>;
export type GeometryValue = z.infer<typeof GeometryValueSchema>;
export type CornerValue = z.infer<typeof CornerValueSchema>;
export type AnimatableValue = z.infer<typeof AnimatableValueSchema>;
export type Easing = z.infer<typeof EasingSchema>;
export type TrackPhase = z.infer<typeof TrackPhaseSchema>;
export type Track = z.infer<typeof TrackSchema>;
export type LayerRole = z.infer<typeof LayerRoleSchema>;
export type MaskMode = z.infer<typeof MaskModeSchema>;
export type MaskRef = z.infer<typeof MaskRefSchema>;
export type InheritableProp = z.infer<typeof InheritablePropSchema>;
export type InheritRef = z.infer<typeof InheritRefSchema>;
export type Layer = z.infer<typeof LayerSchema>;
export type AnimationDescriptor = z.infer<typeof AnimationDescriptorSchema>;

25
lib/choreography/index.ts Normal file
View File

@ -0,0 +1,25 @@
/**
* `lib/choreography` — the shared orchestration/spec modules.
*
* The single source of truth for the semantics a faithful classroom-video
* exporter needs from playback, so neither the app runtime nor the exporter
* re-implements (and silently drifts from) the other:
*
* - **Timing** — playback timing constants + the deterministic no-audio speech
* estimate ({@link estimateSpeechDurationMs}).
* - **Cursor** — {@link resolvePlaybackCursor}, the scene/action walk.
* - **Timeline** — {@link resolveActionTimeline}, the index→time expansion.
* - **Descriptors** — versioned, declarative animation descriptors
* ({@link DESCRIPTORS}, e.g. `spotlight.v1`) + their zod schema.
*
* Kept in `lib/` (not a package) because these semantics co-evolve with the
* playback engine. Purity is machine-enforced: these modules import only
* `@openmaic/dsl` (types + the fire-and-forget partition), `zod` (descriptor
* schema), and pure helpers — no React / DOM / GSAP / framer-motion / render
* backend, so the exporter can interpret them in a pure Node environment. The
* eslint `@/`-boundary block on `lib/choreography/**` keeps host-app paths out.
*/
export * from './timing';
export * from './cursor';
export * from './timeline';
export * from './descriptors/index';

View File

@ -0,0 +1,403 @@
/**
* `resolveActionTimeline` — the index-domain → time-domain expansion.
*
* Playback drives actions by a `(sceneIndex, actionIndex)` cursor; a faithful
* video exporter needs the same sequence laid out on a wall-clock. This pure
* function formalizes the semantics the app's `PlaybackEngine.processNext`
* switch expresses as control flow:
*
* - **Blocking** actions (speech, whiteboard, widget, video, discussion) hold
* the cursor until they complete — the next action starts after them.
* - **Fire-and-forget** actions (spotlight, laser) do not block: playback
* continues immediately, and the effect persists visually for
* {@link EFFECT_AUTO_CLEAR_MS} before auto-clearing.
*
* The blocking/non-blocking partition is read from the DSL's
* {@link FIRE_AND_FORGET_ACTIONS} rather than hardcoded here, so the two stay
* in lockstep. Durations come from the shared {@link timing} spec.
*
* Pure, no runtime dependencies beyond `@openmaic/dsl`.
*/
import type {
Action,
SceneCore,
SpeechAction,
PlayVideoAction,
WbDrawCodeAction,
WbDrawTextAction,
WbDrawTableAction,
WbEditCodeAction,
DiscussionAction,
WbClearAction,
} from '@openmaic/dsl';
import { FIRE_AND_FORGET_ACTIONS } from '@openmaic/dsl';
import { EMPTY_SCENE_DWELL } from './cursor';
import {
EFFECT_AUTO_CLEAR_MS,
DISCUSSION_TRIGGER_DELAY_MS,
DISCUSSION_AUTO_SKIP_MS,
MAX_VIDEO_WAIT_MS,
WB_OPEN_MS,
WB_DRAW_MS,
WB_EDIT_MS,
WB_DELETE_MS,
WB_CLOSE_MS,
WIDGET_MS,
wbDrawCodeMs,
wbClearMs,
estimateSpeechDurationMs,
} from './timing';
const FIRE_AND_FORGET = new Set<string>(FIRE_AND_FORGET_ACTIONS);
/**
* Synthetic segment emitted when a whiteboard mutation runs while the board is
* closed. The app's `ActionEngine.execute()` awaits `ensureWhiteboardOpen()`
* (a `WB_OPEN_MS` open animation) before any `wb_*` action other than
* `wb_open`/`wb_close`, so the exporter must lay down that same beat or every
* later segment starts `WB_OPEN_MS` too early. Distinct id so consumers can
* tell it apart from an authored `wb_open`.
*/
export const IMPLICIT_WB_OPEN: Action = {
id: '__implicit_wb_open__',
type: 'wb_open',
} as Action;
/** Whiteboard mutations that trigger an implicit auto-open when the board is closed. */
function isImplicitOpenTrigger(type: string): boolean {
return type.startsWith('wb_') && type !== 'wb_open' && type !== 'wb_close';
}
export interface ResolveTimelineOptions {
/** Playback speed multiplier applied to speech dwell (estimate and real audio alike). Default 1. */
playbackSpeed?: number;
/**
* Real narration duration (ms) for a speech action when pre-generated audio
* exists. Return `null`/`undefined` to fall back to the deterministic
* {@link estimateSpeechDurationMs}. The exporter, which knows each clip's
* stored audio duration (issue #861), supplies this. The returned length is
* the clip's natural (1×) duration; the timeline divides it by `playbackSpeed`
* to match the live `AudioPlayer.setPlaybackRate` path.
*/
getAudioDurationMs?: (action: SpeechAction) => number | null | undefined;
/**
* Real video duration (ms) for a play_video action. Return `null`/`undefined`
* when unknown — the {@link ResolveTimelineOptions.onUnresolvedVideoDuration}
* policy then decides. A resolved value is capped at {@link MAX_VIDEO_WAIT_MS}.
*/
getVideoDurationMs?: (action: PlayVideoAction) => number | null | undefined;
/**
* What to do when a `play_video` duration is unresolved (no `getVideoDurationMs`,
* or it returned nullish). `play_video` blocks live playback until the video
* ends, so a silent 0 would shift every later action early — hence the default
* is `'throw'` (fail loudly). `'cap'` assumes the {@link MAX_VIDEO_WAIT_MS}
* safety cap; `'zero'` opts back into no-dwell explicitly.
*/
onUnresolvedVideoDuration?: 'throw' | 'cap' | 'zero';
/**
* Live whiteboard element count when a wb_clear runs (the clear animation
* scales with it). Defaults to 0 when not supplied, which yields a 0ms dwell
* (an empty clear is a no-op in the engine); the exporter, which replays
* whiteboard state, can provide the true count.
*/
getClearElementCount?: (action: WbClearAction) => number;
/**
* Whether a discussion action is skipped outright by the engine (already
* consumed, or its `agentId` isn't in the selected set) — in which case it
* contributes no dwell. Depends on runtime state the pure timeline can't see,
* so the caller supplies it; defaults to "not skipped"
* ({@link DISCUSSION_TRIGGER_DELAY_MS}).
*/
isDiscussionSkipped?: (action: DiscussionAction) => boolean;
/**
* Whether a `wb_edit_code` action is a no-op the engine skips without delay
* (target block missing / not a code element / stale line refs). Depends on
* live whiteboard state, so the caller supplies it; defaults to a normal edit
* ({@link WB_EDIT_MS}).
*/
isEditCodeNoop?: (action: WbEditCodeAction) => boolean;
/**
* Whether the whiteboard is already open when the timeline starts. Defaults to
* `false`, matching the engine's post-`resetPlaybackVisualState()` state, so
* the first whiteboard mutation triggers an implicit {@link IMPLICIT_WB_OPEN}
* beat.
*/
whiteboardOpen?: boolean;
}
export interface TimelineSegment {
action: Action;
sceneId: string;
sceneIndex: number;
actionIndex: number;
/** Wall-clock start (ms) relative to the start of playback. */
startMs: number;
/** How long the action is visually present (ms). */
durationMs: number;
/**
* How much the playback cursor advances (ms) before the next action starts.
* Equal to `durationMs` for blocking actions, `0` for fire-and-forget.
*/
advancesCursorMs: number;
/** Whether the action blocks the cursor (false only for fire-and-forget). */
blocking: boolean;
}
/** Line count of a code block, matching the app's `code.split('\n')` typing anim. */
function codeLineCount(code: string): number {
return code.split('\n').length;
}
/**
* Resolve a `play_video` action's blocking duration. Live playback treats
* `play_video` as blocking — it waits until the video ends or {@link MAX_VIDEO_WAIT_MS}
* fires — so an unknown duration must NOT silently become a zero-length segment
* (that would shift every later action early). When `getVideoDurationMs` returns
* a value it's capped at the wait limit; when it's missing/unresolved the
* {@link ResolveTimelineOptions.onUnresolvedVideoDuration} policy decides:
* `'throw'` (default — fail loudly), `'cap'` (assume the max wait), or `'zero'`
* (opt in to the old no-dwell behavior explicitly).
*/
function resolveVideoDurationMs(action: PlayVideoAction, opts: ResolveTimelineOptions): number {
const resolved = opts.getVideoDurationMs?.(action);
if (resolved != null) return Math.min(resolved, MAX_VIDEO_WAIT_MS);
switch (opts.onUnresolvedVideoDuration ?? 'throw') {
case 'zero':
return 0;
case 'cap':
return MAX_VIDEO_WAIT_MS;
case 'throw':
default:
throw new Error(
`resolveActionTimeline: play_video "${action.elementId}" has no resolved duration. ` +
`play_video is blocking, so a missing duration would silently shift later actions ` +
`early. Supply getVideoDurationMs, or set onUnresolvedVideoDuration to 'cap' or 'zero'.`,
);
}
}
/**
* The visual duration (ms) of a single action — how long it is present on
* screen. For blocking actions this is also how long the cursor waits.
*/
function actionDurationMs(action: Action, opts: ResolveTimelineOptions): number {
switch (action.type) {
case 'speech': {
const speed = opts.playbackSpeed ?? 1;
// The live path plays pre-generated audio at `AudioPlayer.setPlaybackRate(speed)`,
// so a stored clip's wall-clock dwell is its length divided by speed — the
// same scaling the no-audio estimate applies. Keeping the two paths in
// lockstep is what stops non-1× exports from drifting.
const audio = opts.getAudioDurationMs?.(action);
if (audio != null) return audio / speed;
return estimateSpeechDurationMs(action.text, { speed });
}
case 'spotlight':
case 'laser':
return EFFECT_AUTO_CLEAR_MS;
case 'discussion': {
// A discussion the engine skips outright — already consumed, or its agent
// isn't selected — contributes no dwell (`processNext` recurses with no
// timer). That skip depends on runtime state (consumed set / selected
// agents), so the caller signals it via `isDiscussionSkipped`.
if (opts.isDiscussionSkipped?.(action as DiscussionAction)) return 0;
// Otherwise: the trigger delay before the ProactiveCard shows, then — in
// unattended playback/export — the card's own auto-skip countdown before
// playback continues. (An attended viewer joining/skipping the card early
// is interactive and out of scope; this models the deterministic
// no-interaction dwell.)
return DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS;
}
case 'play_video':
return resolveVideoDurationMs(action as PlayVideoAction, opts);
case 'wb_open':
return WB_OPEN_MS;
case 'wb_draw_text': {
// The engine no-ops (no delay) when there's nothing to draw:
// `executeWbDrawText` returns early on empty content.
const content = (action as WbDrawTextAction).content ?? '';
return content ? WB_DRAW_MS : 0;
}
case 'wb_draw_table': {
// `executeWbDrawTable` returns early (no delay) when the table has no rows
// or no columns.
const data = (action as WbDrawTableAction).data;
const rows = data?.length ?? 0;
const cols = rows > 0 ? (data[0]?.length ?? 0) : 0;
return rows === 0 || cols === 0 ? 0 : WB_DRAW_MS;
}
case 'wb_draw_shape':
case 'wb_draw_chart':
case 'wb_draw_latex':
case 'wb_draw_line':
return WB_DRAW_MS;
case 'wb_draw_code':
return wbDrawCodeMs(codeLineCount((action as WbDrawCodeAction).code));
case 'wb_edit_code':
// `executeWbEditCode` returns before its delay when the edit can't apply
// (missing/non-code target, or stale lineId/lineIds). That depends on live
// whiteboard state the pure timeline can't see, so the caller signals a
// no-op via `isEditCodeNoop`; otherwise it's the normal edit animation.
return opts.isEditCodeNoop?.(action as WbEditCodeAction) ? 0 : WB_EDIT_MS;
case 'wb_clear': {
// The engine early-returns with no delay when the board is already empty
// (`executeWbClear`: elementCount === 0 → return), so an empty clear has a
// 0ms dwell, not the `wbClearMs(0)` animation floor.
const count = opts.getClearElementCount?.(action as WbClearAction) ?? 0;
return count === 0 ? 0 : wbClearMs(count);
}
case 'wb_delete':
return WB_DELETE_MS;
case 'wb_close':
return WB_CLOSE_MS;
case 'widget_highlight':
case 'widget_setState':
case 'widget_annotation':
case 'widget_reveal':
return WIDGET_MS;
default:
return 0;
}
}
/**
* Expand a scene list into an ordered wall-clock timeline. Scenes and their
* actions are visited in order; a scene with no actions yields one
* {@link EMPTY_SCENE_DWELL} beat (a blank speech clip's dwell) so it still
* shows, mirroring {@link resolvePlaybackCursor}.
*
* Whiteboard auto-open is modeled: a `wb_*` mutation (draw/edit/clear/delete)
* that runs while the board is closed is preceded by a synthetic
* {@link IMPLICIT_WB_OPEN} beat ({@link WB_OPEN_MS}), exactly as the engine's
* `ensureWhiteboardOpen` does. The open state carries across scenes and is
* toggled by `wb_open` / `wb_close`; seed it via {@link ResolveTimelineOptions.whiteboardOpen}.
*
* @returns segments in play order, each stamped with `startMs`, its visual
* `durationMs`, and how far it `advancesCursorMs`.
*
* Typed against {@link SceneCore} (only `id` + `actions` are read), so an
* app-widened `Scene` (extra content kinds) is accepted without casting.
*/
export function resolveActionTimeline(
scenes: SceneCore[],
opts: ResolveTimelineOptions = {},
): TimelineSegment[] {
const segments: TimelineSegment[] = [];
let clockMs = 0;
// Mirrors the engine's whiteboard-open flag: false after
// resetPlaybackVisualState(), flipped by wb_open / wb_close, and read to
// decide whether a wb_* mutation must first pay the implicit open animation.
let whiteboardOpen = opts.whiteboardOpen ?? false;
const push = (action: Action, sceneId: string, sceneIndex: number, actionIndex: number) => {
const durationMs = actionDurationMs(action, opts);
const blocking = !FIRE_AND_FORGET.has(action.type);
const advancesCursorMs = blocking ? durationMs : 0;
segments.push({
action,
sceneId,
sceneIndex,
actionIndex,
startMs: clockMs,
durationMs,
advancesCursorMs,
blocking,
});
clockMs += advancesCursorMs;
};
const pushWithWhiteboard = (
action: Action,
sceneId: string,
sceneIndex: number,
actionIndex: number,
) => {
// A wb_* mutation on a closed board is auto-preceded by an open animation
// (engine: `execute` awaits `ensureWhiteboardOpen`). Emit that beat first so
// later segments don't start WB_OPEN_MS early.
if (!whiteboardOpen && isImplicitOpenTrigger(action.type)) {
push(IMPLICIT_WB_OPEN, sceneId, sceneIndex, actionIndex);
whiteboardOpen = true;
}
push(action, sceneId, sceneIndex, actionIndex);
if (action.type === 'wb_open') whiteboardOpen = true;
else if (action.type === 'wb_close') whiteboardOpen = false;
};
scenes.forEach((scene, sceneIndex) => {
const actions = scene.actions ?? [];
if (actions.length === 0) {
// Empty scene → one synthetic dwell beat, exactly as the cursor yields.
push(EMPTY_SCENE_DWELL, scene.id, sceneIndex, 0);
return;
}
actions.forEach((action, actionIndex) => {
pushWithWhiteboard(action, scene.id, sceneIndex, actionIndex);
});
});
clampFireAndForgetLifetimes(segments, clockMs);
return segments;
}
/**
* Correct the visual `durationMs` of fire-and-forget effects (spotlight/laser)
* in place. Their nominal lifetime is {@link EFFECT_AUTO_CLEAR_MS}, but the
* engine cuts it short — and occasionally extends it:
*
* - **Scene boundary / completion.** The app plays one `PlaybackEngine` per
* scene: switching scenes tears the engine down (`stop()` → `clearEffects()`)
* and each scene ends via the completion path (`getCurrentAction()` returns
* null → `clearEffects()`). Either way effects never outlive their scene, so
* in this continuous clock a spotlight late in a scene (or the final action of
* the lecture) is cleared at the next scene's start / at completion, not a
* flat 5s later.
* - **Shared auto-clear timer.** `ActionEngine.scheduleEffectClear` uses one
* timer that each new effect *resets*, and `clearAllEffects` drops every active
* effect together. So back-to-back effects (within `EFFECT_AUTO_CLEAR_MS` of
* each other) all live until the last one's fire + `EFFECT_AUTO_CLEAR_MS` —
* the earlier effect is *extended*, not cleared on its own schedule.
*
* `advancesCursorMs` (0 for these) is untouched — only the visual hint changes.
*
* @param completionMs the final cursor clock (when the last scene's actions are
* exhausted and playback completes → `clearEffects`).
*/
function clampFireAndForgetLifetimes(segments: TimelineSegment[], completionMs: number): void {
// First segment startMs per scene index → the wall-clock of that scene's
// boundary clearEffects. Every scene yields ≥1 segment, so all indices exist.
const sceneStartMs = new Map<number, number>();
for (const seg of segments) {
if (!sceneStartMs.has(seg.sceneIndex)) sceneStartMs.set(seg.sceneIndex, seg.startMs);
}
segments.forEach((seg, i) => {
if (seg.blocking) return; // only fire-and-forget effects have a clearable lifetime
// The next clearEffects boundary after this effect: the start of the next
// scene, or completion if this is the last scene.
const nextSceneStart = sceneStartMs.get(seg.sceneIndex + 1);
const boundaryMs = nextSceneStart ?? completionMs;
// Shared auto-clear deadline, chained through any later effects in THIS scene
// that fire before it elapses (each resets the shared timer). Segments are in
// non-decreasing startMs order, so a forward walk suffices.
let deadlineMs = seg.startMs + EFFECT_AUTO_CLEAR_MS;
for (let j = i + 1; j < segments.length; j++) {
const other = segments[j];
if (other.sceneIndex !== seg.sceneIndex) break; // cleared at the boundary anyway
if (other.blocking) continue; // blocking actions don't touch the effect timer
// Chain breaks at exact equality too: the earlier effect's clear timer was
// registered first (same 5000ms delay as the reading timer that triggers
// the later effect), so it fires before the later effect resets it. Its
// predecessor is therefore cleared at exactly `deadlineMs`, not extended.
if (other.startMs >= deadlineMs) break; // timer already fired; chain broken
deadlineMs = Math.max(deadlineMs, other.startMs + EFFECT_AUTO_CLEAR_MS);
}
const clearMs = Math.min(boundaryMs, deadlineMs);
seg.durationMs = Math.max(0, clearMs - seg.startMs);
});
}

121
lib/choreography/timing.ts Normal file
View File

@ -0,0 +1,121 @@
/**
* Timing spec — the single source of truth for playback timing.
*
* These constants and the no-audio narration estimate were previously inlined
* in the app engines (`lib/action/engine.ts`, `lib/playback/engine.ts`). They
* are moved here verbatim so the app runtime and the video exporter interpret
* the same numbers: if either side re-implemented them, the exported video
* would silently drift whenever the app is tuned. The literals below are the
* behavior-defining values — change them here and both consumers follow.
*
* Pure: depends on nothing (no React / DOM / render backend), so it can be
* interpreted in a pure Node environment by the exporter.
*/
// ==================== Effect / scene timing ====================
/** Duration (ms) a fire-and-forget effect (spotlight/laser) stays before auto-clearing. */
export const EFFECT_AUTO_CLEAR_MS = 5000;
/** Delay (ms) before a discussion trigger shows its ProactiveCard (lets prior speech finish). */
export const DISCUSSION_TRIGGER_DELAY_MS = 3000;
/**
* Duration (ms) the ProactiveCard counts down in playback mode before it
* auto-skips and playback continues. In unattended playback / export a
* non-skipped discussion therefore blocks for
* `DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS`. The `ProactiveCard`
* component reads this same constant, so the card countdown and the timeline
* can't drift.
*/
export const DISCUSSION_AUTO_SKIP_MS = 5000;
/** Safety cap (ms) on how long playback waits for a video to finish. */
export const MAX_VIDEO_WAIT_MS = 5 * 60 * 1000;
// ==================== Whiteboard / widget action durations ====================
//
// The blocking "visual duration" of each synchronous action — how long the
// engine awaits before advancing to the next action. Lifted verbatim from the
// `delay(...)` call sites in `lib/action/engine.ts`.
/** wb_open — open animation (slow spring). */
export const WB_OPEN_MS = 2000;
/** wb_draw_* (text/shape/chart/latex/table/line) — element fade-in. */
export const WB_DRAW_MS = 800;
/** wb_edit_code — line-level edit animation. */
export const WB_EDIT_MS = 600;
/** wb_delete — element removal. */
export const WB_DELETE_MS = 300;
/** wb_close — close animation (ease-out tween). */
export const WB_CLOSE_MS = 700;
/** widget_* interactions (highlight/setState/annotation/reveal). */
export const WIDGET_MS = 300;
/**
* wb_draw_code — typing animation: base 800ms + 50ms/line, capped at 3000ms.
* Verbatim from `lib/action/engine.ts` (`Math.min(800 + lines.length * 50, 3000)`).
*/
export function wbDrawCodeMs(lineCount: number): number {
return Math.min(800 + lineCount * 50, 3000);
}
/**
* wb_clear — clear animation: base 380ms + 55ms/element, capped at 1400ms.
* Verbatim from `lib/action/engine.ts` (`Math.min(380 + elementCount * 55, 1400)`).
*/
export function wbClearMs(elementCount: number): number {
return Math.min(380 + elementCount * 55, 1400);
}
// ==================== No-audio narration estimate ====================
/**
* Characters counted as CJK for the reading-timer heuristic: CJK Unified
* Ideographs (+ Ext-A), Hiragana, Katakana, Hangul Syllables. Verbatim from the
* regex in `lib/playback/engine.ts`'s `scheduleReadingTimer`.
*/
const CJK_REGEX = /[一-鿿㐀-䶿぀-ゟ゠-ヿ가-힯]/g;
/** Text is treated as CJK when this fraction of its characters are CJK. */
const CJK_RATIO_THRESHOLD = 0.3;
/** Minimum estimated reading time (ms), regardless of length. */
const MIN_READING_MS = 2000;
/** CJK narration pace: ms per character (one char ≈ one word). */
const CJK_MS_PER_CHAR = 150;
/** Non-CJK narration pace: ms per word (≈250 WPM). */
const NON_CJK_MS_PER_WORD = 240;
export interface SpeechEstimateOptions {
/** Playback speed multiplier; the estimate is divided by it. Default 1. */
speed?: number;
}
/**
* Estimate how long narration takes to "speak" when no pre-generated audio is
* available (TTS disabled). Deterministic — the exporter and the app's
* reading-timer must agree on the dwell of an audio-less speech clip.
*
* - CJK text (>30% CJK chars): ~150ms/char.
* - Non-CJK text: ~240ms/word (≈250 WPM).
* - Floored at 2000ms, then divided by playback speed.
*
* Moved verbatim from `lib/playback/engine.ts`'s `scheduleReadingTimer`.
*/
export function estimateSpeechDurationMs(text: string, opts?: SpeechEstimateOptions): number {
const speed = opts?.speed ?? 1;
const cjkCount = (text.match(CJK_REGEX) || []).length;
const isCJK = cjkCount > text.length * CJK_RATIO_THRESHOLD;
const rawMs = isCJK
? Math.max(MIN_READING_MS, text.length * CJK_MS_PER_CHAR)
: Math.max(MIN_READING_MS, text.split(/\s+/).filter(Boolean).length * NON_CJK_MS_PER_WORD);
return rawMs / speed;
}

View File

@ -35,7 +35,11 @@ import type {
} from './types';
import type { AudioPlayer } from '@/lib/utils/audio-player';
import { ActionEngine } from '@/lib/action/engine';
import { resolvePlaybackCursor } from './engine-cursor';
import {
resolvePlaybackCursor,
estimateSpeechDurationMs,
DISCUSSION_TRIGGER_DELAY_MS,
} from '@/lib/choreography';
import {
canJumpWithinReconstructablePrefix,
isWhiteboardPlaybackAction,
@ -564,21 +568,13 @@ export class PlaybackEngine {
});
// Estimated reading time when no pre-generated audio (TTS disabled).
// CJK text: ~150ms/char (one char ≈ one word).
// Non-CJK text: ~240ms/word (≈250 WPM).
// Min 2s. Cancelled on pause; resume() calls processNext directly.
// The estimate (CJK vs word-based pace, 2s floor, speed-adjusted) lives
// in @/lib/choreography so the video exporter dwells identically.
// Cancelled on pause; resume() calls processNext directly.
const scheduleReadingTimer = () => {
if (!this.isCurrentGeneration(generation)) return;
const text = speechAction.text;
const cjkCount = (
text.match(/[\u4e00-\u9fff\u3400-\u4dbf\u3040-\u309f\u30a0-\u30ff\uac00-\ud7af]/g) || []
).length;
const isCJK = cjkCount > text.length * 0.3;
const speed = this.callbacks.getPlaybackSpeed?.() ?? 1;
const rawMs = isCJK
? Math.max(2000, text.length * 150)
: Math.max(2000, text.split(/\s+/).filter(Boolean).length * 240);
const readingMs = rawMs / speed;
const readingMs = estimateSpeechDurationMs(speechAction.text, { speed });
this.speechTimerStart = Date.now();
this.speechTimerRemaining = readingMs;
this.speechTimer = setTimeout(() => {
@ -685,7 +681,7 @@ export class PlaybackEngine {
this.currentTrigger = trigger;
this.callbacks.onProactiveShow?.(trigger);
// Engine pauses here — user calls confirmDiscussion() or skipDiscussion()
}, 3000);
}, DISCUSSION_TRIGGER_DELAY_MS);
break;
}

View File

@ -1,5 +1,5 @@
import { describe, expect, test } from 'vitest';
import { resolvePlaybackCursor, EMPTY_SCENE_DWELL } from '@/lib/playback/engine-cursor';
import { resolvePlaybackCursor, EMPTY_SCENE_DWELL } from '@/lib/choreography';
import type { Action } from '@/lib/types/action';
import type { Scene } from '@/lib/types/stage';

View File

@ -0,0 +1,163 @@
import { describe, expect, it } from 'vitest';
import {
DESCRIPTORS,
spotlightV1,
laserV1,
getDescriptor,
AnimationDescriptorSchema,
} from '@/lib/choreography';
describe('animation descriptor registry', () => {
it('registers spotlight.v1 and laser.v1 under their versioned ids', () => {
expect(Object.keys(DESCRIPTORS).sort()).toEqual(['laser.v1', 'spotlight.v1']);
expect(getDescriptor('spotlight.v1')).toBe(spotlightV1);
expect(getDescriptor('laser.v1')).toBe(laserV1);
expect(getDescriptor('nope.v1')).toBeUndefined();
});
it('ids and versions are consistent', () => {
for (const [key, d] of Object.entries(DESCRIPTORS)) {
expect(d.id).toBe(key);
expect(d.version).toBe(1);
expect(key.endsWith(`.v${d.version}`)).toBe(true);
}
});
});
describe('every shipped descriptor conforms to the zod schema', () => {
for (const [key, d] of Object.entries(DESCRIPTORS)) {
it(`${key} validates`, () => {
expect(() => AnimationDescriptorSchema.parse(d)).not.toThrow();
});
}
it('rejects a descriptor missing required fields', () => {
expect(AnimationDescriptorSchema.safeParse({ id: 'x.v1', version: 1 }).success).toBe(false);
});
it('rejects an unknown easing type', () => {
const bad = {
...spotlightV1,
layers: [
{
id: 'l',
tracks: [
{
property: 'x',
from: 0,
to: 1,
durationMs: 100,
easing: { type: 'bogus', points: [0, 0, 0, 0] },
},
],
},
],
};
expect(AnimationDescriptorSchema.safeParse(bad).success).toBe(false);
});
});
describe('spotlight.v1 pins the source animation values', () => {
it('has the dim/cutout/border layers and z-index 100', () => {
expect(spotlightV1.zIndex).toBe(100);
// 0.5 is the runtime default (executeSpotlight: action.dimOpacity ?? 0.5),
// not the component's unreachable ?? 0.7 fallback.
expect(spotlightV1.params).toMatchObject({ dimness: 0.5 });
expect(spotlightV1.layers.map((l) => l.id).sort()).toEqual(['border', 'cutout', 'dim']);
});
it('cutout uses the 600ms expo-out curve', () => {
const cutout = spotlightV1.layers.find((l) => l.id === 'cutout')!;
for (const t of cutout.tracks) {
expect(t.durationMs).toBe(600);
expect(t.easing).toEqual({ type: 'cubicBezier', points: [0.16, 1, 0.3, 1] });
}
});
it('border is 500ms, delayed 50ms', () => {
const border = spotlightV1.layers.find((l) => l.id === 'border')!;
for (const t of border.tracks) {
expect(t.durationMs).toBe(500);
expect(t.delayMs).toBe(50);
}
});
it('models the dim layer as the cutout subtracted (SVG mask), not an opaque black rect', () => {
const cutout = spotlightV1.layers.find((l) => l.id === 'cutout')!;
const dim = spotlightV1.layers.find((l) => l.id === 'dim')!;
// The cutout supplies geometry only — it is not painted on its own.
expect(cutout.role).toBe('mask');
// The dim rect is clipped by subtracting the cutout, so a non-React consumer
// reconstructs "dim everywhere except the cutout" rather than a black box.
expect(dim.maskedBy).toEqual({ layerId: 'cutout', mode: 'subtract' });
// It spans the whole 0..100 viewport — explicit so a literal consumer dims
// the full frame, not a zero-size rect.
expect(dim.staticProps).toMatchObject({ x: 0, y: 0, width: 100, height: 100 });
});
it('models the wrapper enter/exit opacity fade (Motion default 300ms)', () => {
const dim = spotlightV1.layers.find((l) => l.id === 'dim')!;
const enter = dim.tracks.find((t) => t.property === 'opacity' && t.phase === 'enter')!;
const exit = dim.tracks.find((t) => t.property === 'opacity' && t.phase === 'exit')!;
// Pinned to Motion's default tween so a non-Motion consumer reproduces the
// fade instead of popping the effect on/off.
expect(enter).toMatchObject({ from: 0, to: 1, durationMs: 300 });
expect(exit).toMatchObject({ from: 1, to: 0, durationMs: 300 });
});
it('the border rides the wrapper fade (inherits the dim layer opacity)', () => {
const border = spotlightV1.layers.find((l) => l.id === 'border')!;
// The outline sits in the same fading wrapper as the dim rect, so it fades
// out with the whole effect rather than lingering after the dimming clears.
expect(border.inheritsFrom).toEqual({ parentId: 'dim', props: ['opacity'] });
});
});
describe('laser.v1 pins the source animation values', () => {
it('has the dot/ring/core layers, z-index 101, red default', () => {
expect(laserV1.zIndex).toBe(101);
expect(laserV1.params).toMatchObject({ color: '#ff0000' });
expect(laserV1.layers.map((l) => l.id).sort()).toEqual(['core', 'dot', 'ring']);
});
it('the ring pulses infinitely, scale 1→2.8, 1500ms, 300ms repeat delay', () => {
const ring = laserV1.layers.find((l) => l.id === 'ring')!;
const scale = ring.tracks.find((t) => t.property === 'scale')!;
expect(scale).toMatchObject({
from: 1,
to: 2.8,
durationMs: 1500,
repeat: 'infinite',
repeatDelayMs: 300,
});
});
it('the dot fly-in is 500ms enter and 250ms exit', () => {
const dot = laserV1.layers.find((l) => l.id === 'dot')!;
const enterLeft = dot.tracks.find((t) => t.property === 'left' && t.phase === 'enter')!;
const exitLeft = dot.tracks.find((t) => t.property === 'left' && t.phase === 'exit')!;
expect(enterLeft.durationMs).toBe(500);
expect(exitLeft.durationMs).toBe(250);
});
it('captures the dot center-anchor and circular ring/core geometry', () => {
const dot = laserV1.layers.find((l) => l.id === 'dot')!;
const ring = laserV1.layers.find((l) => l.id === 'ring')!;
const core = laserV1.layers.find((l) => l.id === 'core')!;
// The dot group is centered on the target (translate -50%,-50%), so a
// literal renderer doesn't offset it to a top-left anchor.
expect(dot.staticProps).toMatchObject({ transform: 'translate(-50%, -50%)' });
// Ring and core are circles (rounded-full → borderRadius 9999), not squares.
expect(ring.staticProps).toMatchObject({ borderRadius: 9999, position: 'absolute', inset: 0 });
expect(core.staticProps).toMatchObject({ borderRadius: 9999, width: 10, height: 10 });
});
it('the ring and core ride the dot fly-in/exit (inherit its left/top/opacity)', () => {
const ring = laserV1.layers.find((l) => l.id === 'ring')!;
const core = laserV1.layers.find((l) => l.id === 'core')!;
// Both are nested inside the animated dot wrapper in the source, so they
// follow its motion instead of sitting at a static default origin.
expect(ring.inheritsFrom).toEqual({ parentId: 'dot', props: ['left', 'top', 'opacity'] });
expect(core.inheritsFrom).toEqual({ parentId: 'dot', props: ['left', 'top', 'opacity'] });
});
});

View File

@ -0,0 +1,383 @@
import { describe, expect, it } from 'vitest';
import {
resolveActionTimeline,
IMPLICIT_WB_OPEN,
EFFECT_AUTO_CLEAR_MS,
DISCUSSION_TRIGGER_DELAY_MS,
DISCUSSION_AUTO_SKIP_MS,
WB_OPEN_MS,
WB_DRAW_MS,
WB_EDIT_MS,
wbDrawCodeMs,
wbClearMs,
estimateSpeechDurationMs,
} from '@/lib/choreography';
import type { Action } from '@/lib/types/action';
import type { Scene } from '@/lib/types/stage';
const act = (a: Partial<Action> & { type: string }): Action => a as unknown as Action;
const speech = (id: string, text: string): Action => act({ id, type: 'speech', text });
const sc = (id: string, actions: Action[]): Scene =>
({
id,
stageId: 's',
type: 'slide',
title: id,
order: 1,
content: { type: 'slide', canvas: {} },
actions,
}) as unknown as Scene;
describe('resolveActionTimeline — blocking actions advance the cursor', () => {
it('speech falls back to the deterministic estimate and accumulates startMs', () => {
const scenes = [sc('S0', [speech('a', '中'.repeat(20)), speech('b', 'hello world')])];
const tl = resolveActionTimeline(scenes);
const d0 = estimateSpeechDurationMs('中'.repeat(20)); // 3000
const d1 = estimateSpeechDurationMs('hello world'); // 2000 floor
expect(tl).toHaveLength(2);
expect(tl[0]).toMatchObject({
sceneId: 'S0',
sceneIndex: 0,
actionIndex: 0,
startMs: 0,
durationMs: d0,
advancesCursorMs: d0,
blocking: true,
});
expect(tl[1]).toMatchObject({ startMs: d0, durationMs: d1, advancesCursorMs: d1 });
});
it('a supplied audio-duration resolver overrides the estimate', () => {
const scenes = [sc('S0', [speech('a', 'anything')])];
const tl = resolveActionTimeline(scenes, { getAudioDurationMs: () => 7777 });
expect(tl[0]).toMatchObject({ durationMs: 7777, advancesCursorMs: 7777 });
});
it('playback speed scales both the estimate and supplied audio', () => {
const scenes = [sc('S0', [speech('a', '中'.repeat(20))])];
const tl = resolveActionTimeline(scenes, { playbackSpeed: 2 });
expect(tl[0].durationMs).toBe(estimateSpeechDurationMs('中'.repeat(20)) / 2);
// Real audio is scaled the same way (the live path sets AudioPlayer playbackRate).
const withAudio = resolveActionTimeline(scenes, {
playbackSpeed: 2,
getAudioDurationMs: () => 10_000,
});
expect(withAudio[0]).toMatchObject({ durationMs: 5000, advancesCursorMs: 5000 });
});
});
describe('resolveActionTimeline — fire-and-forget effects do not advance the cursor', () => {
it('effects start at the cursor but advance it by 0 (blocking:false)', () => {
const scenes = [
sc('S0', [
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
act({ id: 'la', type: 'laser', elementId: 'e2' }),
speech('s', 'hi there'),
]),
];
const tl = resolveActionTimeline(scenes);
// Both effects and the speech start at 0 — the effects didn't move the clock.
expect(tl[0]).toMatchObject({ startMs: 0, advancesCursorMs: 0, blocking: false });
expect(tl[1]).toMatchObject({ startMs: 0, advancesCursorMs: 0, blocking: false });
expect(tl[2]).toMatchObject({ startMs: 0, blocking: true });
});
it('an effect is cleared at playback completion, not a flat 5s later', () => {
// spotlight fires at 0; the only speech ends at 2000, then processNext hits
// completion → clearEffects. So the effect lives 2000ms, not EFFECT_AUTO_CLEAR_MS.
const scenes = [
sc('S0', [act({ id: 'sp', type: 'spotlight', elementId: 'e1' }), speech('s', 'hi there')]),
];
const tl = resolveActionTimeline(scenes);
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: 2000, blocking: false });
});
it('an effect is cleared at the next scene boundary', () => {
const scenes = [
sc('S0', [act({ id: 'sp', type: 'spotlight', elementId: 'e1' }), speech('a', 'hi there')]), // 0..2000
sc('S1', [speech('b', 'more')]),
];
// clearEffects fires at the start of S1 (2000), cutting the spotlight there.
expect(resolveActionTimeline(scenes)[0].durationMs).toBe(2000);
});
it('lives its full EFFECT_AUTO_CLEAR_MS when the scene runs long enough', () => {
// speech dwell 6000 > 5000, so the auto-clear timer fires before the boundary.
const scenes = [
sc('S0', [
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
speech('s', '中'.repeat(40)),
]),
];
expect(resolveActionTimeline(scenes)[0].durationMs).toBe(EFFECT_AUTO_CLEAR_MS);
});
it('a later effect resets the shared clear timer, extending earlier effects', () => {
// spotlight@0, then speech(3000), then laser@3000 (resets the shared timer),
// then speech(6000) → completion 9000. The shared timer last fires at
// 3000+5000=8000, so BOTH effects clear together at 8000: spotlight lives
// 8000ms (extended past its own 5000), laser lives 5000ms.
const scenes = [
sc('S0', [
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
speech('a', '中'.repeat(20)), // 3000
act({ id: 'la', type: 'laser', elementId: 'e2' }),
speech('b', '中'.repeat(40)), // 6000
]),
];
const tl = resolveActionTimeline(scenes);
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: 8000, blocking: false }); // spotlight
expect(tl[2]).toMatchObject({
startMs: 3000,
durationMs: EFFECT_AUTO_CLEAR_MS,
blocking: false,
}); // laser
});
it('breaks the chain when the gap between effects exceeds EFFECT_AUTO_CLEAR_MS', () => {
// spotlight@0; speech(6000) > 5000, so the shared timer for the spotlight
// already fired at 5000 before the laser@6000 arrives. The laser starts a
// fresh 5000 window. Long trailing speech keeps both within their own scene.
const scenes = [
sc('S0', [
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
speech('a', '中'.repeat(40)), // 6000
act({ id: 'la', type: 'laser', elementId: 'e2' }),
speech('b', '中'.repeat(40)), // 6000 → completion 12000
]),
];
const tl = resolveActionTimeline(scenes);
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: EFFECT_AUTO_CLEAR_MS }); // spotlight: own 5000
expect(tl[2]).toMatchObject({ startMs: 6000, durationMs: EFFECT_AUTO_CLEAR_MS }); // laser: own 5000
});
it('breaks the chain at an EXACT 5s boundary (earlier clear timer fires first)', () => {
// spotlight@0; speech of exactly 5000ms; laser@5000. The spotlight's clear
// timer was queued first (same 5000ms delay), so it fires before the laser
// resets it — the spotlight lives exactly 5000, not 10000.
const scenes = [
sc('S0', [
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
speech('a', 'anything'), // 5000 via audio resolver
act({ id: 'la', type: 'laser', elementId: 'e2' }),
speech('b', 'anything'), // 5000 → completion 10000
]),
];
const tl = resolveActionTimeline(scenes, { getAudioDurationMs: () => 5000 });
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: EFFECT_AUTO_CLEAR_MS }); // spotlight: exactly 5000
expect(tl[2]).toMatchObject({ startMs: 5000, durationMs: EFFECT_AUTO_CLEAR_MS }); // laser: own 5000
});
it('a later scene never extends an earlier scene effect (boundary clears first)', () => {
const scenes = [
sc('S0', [act({ id: 'sp', type: 'spotlight', elementId: 'e1' }), speech('a', 'hi there')]), // 0..2000
sc('S1', [act({ id: 'la', type: 'laser', elementId: 'e2' }), speech('b', 'more')]),
];
// S0's spotlight is cut at S1's start (2000), NOT extended by S1's laser@2000.
expect(resolveActionTimeline(scenes)[0].durationMs).toBe(2000);
});
it('an effect as the last action of a scene has 0ms visual duration', () => {
// The engine fires the effect, queueMicrotask(processNext) hits completion,
// and clearEffects runs before real time elapses → 0ms.
const scenes = [
sc('S0', [speech('a', 'hi there'), act({ id: 'sp', type: 'spotlight', elementId: 'e1' })]),
];
const tl = resolveActionTimeline(scenes);
expect(tl[1]).toMatchObject({ startMs: 2000, durationMs: 0, blocking: false });
});
});
describe('resolveActionTimeline — per-action durations', () => {
it('wb_draw_code uses the line-count formula', () => {
const code = 'a\nb\nc\nd'; // 4 lines
const scenes = [sc('S0', [act({ id: 'c', type: 'wb_draw_code', code })])];
const tl = resolveActionTimeline(scenes, { whiteboardOpen: true });
expect(tl[0].durationMs).toBe(wbDrawCodeMs(4));
});
it('wb_clear is 0ms when empty and scales with the supplied live element count', () => {
const scenes = [
sc('S0', [act({ id: 'o', type: 'wb_open' }), act({ id: 'cl', type: 'wb_clear' })]),
];
// Empty board → engine early-returns with no delay, so 0ms (not wbClearMs(0)).
expect(resolveActionTimeline(scenes)[1].durationMs).toBe(0);
expect(resolveActionTimeline(scenes, { getClearElementCount: () => 10 })[1].durationMs).toBe(
wbClearMs(10),
);
});
it('wb_draw_* share WB_DRAW_MS', () => {
const scenes = [sc('S0', [act({ id: 't', type: 'wb_draw_text', content: 'hi' })])];
expect(resolveActionTimeline(scenes, { whiteboardOpen: true })[0].durationMs).toBe(WB_DRAW_MS);
});
it('no-op whiteboard draws cost 0ms (engine early-returns)', () => {
// wb_draw_text with empty content → executeWbDrawText returns before delay.
const emptyText = [sc('S0', [act({ id: 't', type: 'wb_draw_text', content: '' })])];
expect(resolveActionTimeline(emptyText, { whiteboardOpen: true })[0].durationMs).toBe(0);
// wb_draw_table with no rows → executeWbDrawTable returns early.
const emptyTable = [sc('S0', [act({ id: 'tb', type: 'wb_draw_table', data: [] })])];
expect(resolveActionTimeline(emptyTable, { whiteboardOpen: true })[0].durationMs).toBe(0);
// A populated table still costs WB_DRAW_MS.
const table = [sc('S0', [act({ id: 'tb2', type: 'wb_draw_table', data: [['a', 'b']] })])];
expect(resolveActionTimeline(table, { whiteboardOpen: true })[0].durationMs).toBe(WB_DRAW_MS);
});
it('discussion dwells for trigger delay + card auto-skip (unattended playback)', () => {
const scenes = [sc('S0', [act({ id: 'd', type: 'discussion', topic: 't' })])];
// Not skipped: the 3s trigger delay, then the ProactiveCard's 5s auto-skip
// countdown before playback continues.
expect(resolveActionTimeline(scenes)[0]).toMatchObject({
durationMs: DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS,
blocking: true,
});
});
it('a skipped discussion (consumed / agent not selected) contributes no dwell', () => {
const scenes = [sc('S0', [act({ id: 'd', type: 'discussion', topic: 't' })])];
// Engine skips it (processNext recurses with no timer) → 0ms.
expect(resolveActionTimeline(scenes, { isDiscussionSkipped: () => true })[0].durationMs).toBe(
0,
);
// Not skipped → trigger delay + card auto-skip, same as the default.
expect(resolveActionTimeline(scenes, { isDiscussionSkipped: () => false })[0].durationMs).toBe(
DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS,
);
});
it('wb_edit_code is WB_EDIT_MS by default and 0ms when the caller flags a no-op', () => {
const scenes = [
sc('S0', [
act({ id: 'e', type: 'wb_edit_code', elementId: 'c1', operation: 'delete_lines' }),
]),
];
expect(resolveActionTimeline(scenes, { whiteboardOpen: true })[0].durationMs).toBe(WB_EDIT_MS);
// Stale target / non-code element → executeWbEditCode returns before its delay.
expect(
resolveActionTimeline(scenes, { whiteboardOpen: true, isEditCodeNoop: () => true })[0]
.durationMs,
).toBe(0);
});
it('play_video uses the supplied duration (capped); unresolved is explicit, not silent 0', () => {
const scenes = [sc('S0', [act({ id: 'v', type: 'play_video', elementId: 'v1' })])];
// Supplied duration is used, capped at MAX_VIDEO_WAIT_MS (5min).
expect(resolveActionTimeline(scenes, { getVideoDurationMs: () => 12_345 })[0].durationMs).toBe(
12_345,
);
expect(
resolveActionTimeline(scenes, { getVideoDurationMs: () => 60 * 60 * 1000 })[0].durationMs,
).toBe(5 * 60 * 1000);
// Unresolved: default policy THROWS rather than silently shifting later actions early.
expect(() => resolveActionTimeline(scenes)).toThrow(/play_video/);
// Opt-in policies: 'cap' → max wait, 'zero' → explicit no-dwell.
expect(resolveActionTimeline(scenes, { onUnresolvedVideoDuration: 'cap' })[0].durationMs).toBe(
5 * 60 * 1000,
);
expect(resolveActionTimeline(scenes, { onUnresolvedVideoDuration: 'zero' })[0].durationMs).toBe(
0,
);
});
});
describe('resolveActionTimeline — scene boundaries', () => {
it('an empty scene yields one EMPTY_SCENE_DWELL beat', () => {
const scenes = [sc('S0', [])];
const tl = resolveActionTimeline(scenes);
expect(tl).toHaveLength(1);
expect(tl[0]).toMatchObject({
sceneId: 'S0',
actionIndex: 0,
// empty-text speech → 2000ms floor
durationMs: estimateSpeechDurationMs(''),
blocking: true,
});
expect(tl[0].action).toMatchObject({ type: 'speech', text: '' });
});
it('startMs accumulates across scenes in play order', () => {
const scenes = [
sc('S0', [speech('a', 'hello world')]), // 2000
sc('S1', []), // empty dwell 2000
sc('S2', [speech('c', '中'.repeat(20))]), // 3000
];
const tl = resolveActionTimeline(scenes);
expect(tl.map((s) => [s.sceneId, s.startMs])).toEqual([
['S0', 0],
['S1', 2000],
['S2', 4000],
]);
});
});
describe('resolveActionTimeline — implicit whiteboard auto-open', () => {
it('prepends a WB_OPEN_MS beat before the first wb_* mutation on a closed board', () => {
const scenes = [sc('S0', [act({ id: 'd', type: 'wb_draw_text', content: 'hi' })])];
const tl = resolveActionTimeline(scenes); // whiteboardOpen defaults to false
expect(tl).toHaveLength(2);
expect(tl[0]).toMatchObject({
action: IMPLICIT_WB_OPEN,
startMs: 0,
durationMs: WB_OPEN_MS,
advancesCursorMs: WB_OPEN_MS,
blocking: true,
});
// The real draw starts after the open animation.
expect(tl[1]).toMatchObject({ startMs: WB_OPEN_MS, durationMs: WB_DRAW_MS });
});
it('does not prepend when the board is already open (seeded or via wb_open)', () => {
const drawOnly = [sc('S0', [act({ id: 'd', type: 'wb_draw_text' })])];
expect(resolveActionTimeline(drawOnly, { whiteboardOpen: true })).toHaveLength(1);
const explicitOpen = [
sc('S0', [act({ id: 'o', type: 'wb_open' }), act({ id: 'd', type: 'wb_draw_text' })]),
];
const tl = resolveActionTimeline(explicitOpen);
// Only the authored wb_open + the draw — no synthetic beat.
expect(tl.map((s) => s.action.type)).toEqual(['wb_open', 'wb_draw_text']);
expect(tl[0].action).not.toBe(IMPLICIT_WB_OPEN);
});
it('re-opens after a wb_close (auto-open state carries across the sequence)', () => {
const scenes = [
sc('S0', [
act({ id: 'd1', type: 'wb_draw_text' }), // implicit open here
act({ id: 'x', type: 'wb_close' }),
act({ id: 'd2', type: 'wb_draw_shape' }), // implicit open again
]),
];
const tl = resolveActionTimeline(scenes);
expect(tl.map((s) => s.action.type)).toEqual([
'wb_open', // implicit (before d1)
'wb_draw_text',
'wb_close',
'wb_open', // implicit (before d2, board was closed again)
'wb_draw_shape',
]);
expect(tl.filter((s) => s.action === IMPLICIT_WB_OPEN)).toHaveLength(2);
});
it('carries open state across scene boundaries', () => {
const scenes = [
sc('S0', [act({ id: 'd1', type: 'wb_draw_text' })]), // implicit open
sc('S1', [act({ id: 'd2', type: 'wb_draw_shape' })]), // still open — no new open
];
const tl = resolveActionTimeline(scenes);
expect(tl.map((s) => s.action.type)).toEqual(['wb_open', 'wb_draw_text', 'wb_draw_shape']);
});
it('wb_clear/wb_delete on a closed board also trigger the implicit open', () => {
const scenes = [sc('S0', [act({ id: 'cl', type: 'wb_clear' })])];
const tl = resolveActionTimeline(scenes);
expect(tl[0].action).toBe(IMPLICIT_WB_OPEN);
// wb_clear on an (implicitly) empty board is still 0ms.
expect(tl[1]).toMatchObject({ startMs: WB_OPEN_MS, durationMs: 0 });
});
});

View File

@ -0,0 +1,90 @@
import { describe, expect, it } from 'vitest';
import {
EFFECT_AUTO_CLEAR_MS,
DISCUSSION_TRIGGER_DELAY_MS,
DISCUSSION_AUTO_SKIP_MS,
MAX_VIDEO_WAIT_MS,
WB_OPEN_MS,
WB_DRAW_MS,
WB_EDIT_MS,
WB_DELETE_MS,
WB_CLOSE_MS,
WIDGET_MS,
wbDrawCodeMs,
wbClearMs,
estimateSpeechDurationMs,
} from '@/lib/choreography';
describe('timing constants pin the values moved from the app engines', () => {
it('effect / scene timing', () => {
expect(EFFECT_AUTO_CLEAR_MS).toBe(5000);
expect(DISCUSSION_TRIGGER_DELAY_MS).toBe(3000);
expect(DISCUSSION_AUTO_SKIP_MS).toBe(5000);
expect(MAX_VIDEO_WAIT_MS).toBe(5 * 60 * 1000);
});
it('whiteboard / widget action durations', () => {
expect(WB_OPEN_MS).toBe(2000);
expect(WB_DRAW_MS).toBe(800);
expect(WB_EDIT_MS).toBe(600);
expect(WB_DELETE_MS).toBe(300);
expect(WB_CLOSE_MS).toBe(700);
expect(WIDGET_MS).toBe(300);
});
});
describe('wbDrawCodeMs — base 800ms + 50ms/line, capped at 3000ms', () => {
it('matches Math.min(800 + lines * 50, 3000)', () => {
expect(wbDrawCodeMs(0)).toBe(800);
expect(wbDrawCodeMs(1)).toBe(850);
expect(wbDrawCodeMs(10)).toBe(1300);
// cap kicks in at 44 lines (800 + 2200 = 3000)
expect(wbDrawCodeMs(44)).toBe(3000);
expect(wbDrawCodeMs(1000)).toBe(3000);
});
});
describe('wbClearMs — base 380ms + 55ms/element, capped at 1400ms', () => {
it('matches Math.min(380 + count * 55, 1400)', () => {
expect(wbClearMs(0)).toBe(380);
expect(wbClearMs(1)).toBe(435);
expect(wbClearMs(10)).toBe(930);
// cap kicks in at ~18.5 elements → 19 elements exceeds 1400
expect(wbClearMs(19)).toBe(1400);
expect(wbClearMs(500)).toBe(1400);
});
});
describe('estimateSpeechDurationMs — deterministic no-audio dwell', () => {
it('CJK text: ~150ms/char, floored at 2000ms', () => {
// 20 CJK chars → 20 * 150 = 3000ms
expect(estimateSpeechDurationMs('中'.repeat(20))).toBe(3000);
// short CJK below the floor
expect(estimateSpeechDurationMs('中文')).toBe(2000);
});
it('non-CJK text: ~240ms/word (≈250 WPM), floored at 2000ms', () => {
// 20 words → 20 * 240 = 4800ms
const twentyWords = Array.from({ length: 20 }, (_, i) => `w${i}`).join(' ');
expect(estimateSpeechDurationMs(twentyWords)).toBe(4800);
// few words below the floor
expect(estimateSpeechDurationMs('hello world')).toBe(2000);
});
it('CJK threshold is >30% CJK characters', () => {
// Mostly ASCII with a couple CJK chars → treated as non-CJK (word-based).
const text = 'this is a long english sentence with 中文 words in it';
const words = text.split(/\s+/).filter(Boolean).length;
expect(estimateSpeechDurationMs(text)).toBe(Math.max(2000, words * 240));
});
it('divides by playback speed', () => {
const base = estimateSpeechDurationMs('中'.repeat(20));
expect(estimateSpeechDurationMs('中'.repeat(20), { speed: 2 })).toBe(base / 2);
expect(estimateSpeechDurationMs('中'.repeat(20), { speed: 0.5 })).toBe(base / 0.5);
});
it('empty text falls to the 2000ms floor', () => {
expect(estimateSpeechDurationMs('')).toBe(2000);
});
});