* feat(choreography): shared orchestration spec in lib/choreography (#863) Introduce lib/choreography/ as the single source of truth for the orchestration semantics a faithful classroom-video exporter needs from playback, so the app runtime and the exporter interpret one spec instead of each re-implementing (and silently drifting from) the other. Kept in lib/ rather than a package: these semantics co-evolve with the playback engine, and the exporter will also live in the app, so both consumers share them via ordinary imports. Purity is machine-enforced by an eslint boundary on lib/choreography/** (blocks @/ host-app paths + react/react-dom/gsap/framer-motion/motion), so the exporter can interpret the spec in a pure Node environment. - timing.ts — timing constants + the deterministic no-audio speech estimate, moved verbatim from the engines. - cursor.ts — resolvePlaybackCursor + EMPTY_SCENE_DWELL, moved from lib/playback/engine-cursor.ts (typed on dsl SceneCore). - timeline.ts — new pure resolveActionTimeline: index-domain -> time-domain expansion (blocking cursor-advance vs fire-and-forget visual duration), keyed off the DSL fire-and-forget partition. - descriptors/— versioned, zod-schema-validated animation descriptors spotlight.v1 + laser.v1 (declarative: property/from/to/ duration/easing; no implementation), pinned to the current overlay components. Behavior-neutral engine refactor: lib/action/engine.ts and lib/playback/engine.ts import from lib/choreography and the local literals are deleted, so the timing dimension now has exactly one copy. The spotlight/laser overlay components still hardcode their animation values (they do not yet READ the descriptors) — tracked in #889. Closes #863 * fix(choreography): address cross-review findings on resolveActionTimeline + descriptors - P1: model implicit whiteboard auto-open — a wb_* mutation on a closed board now prepends a synthetic IMPLICIT_WB_OPEN (WB_OPEN_MS) beat, mirroring the engine's ensureWhiteboardOpen; open state carries across scenes and toggles on wb_open/wb_close (new `whiteboardOpen` option to seed it). - P2: scale real speech audio duration by playbackSpeed too (live path sets AudioPlayer.setPlaybackRate), keeping it in lockstep with the estimate path. - P2: express the spotlight mask relationship in the descriptor model — LayerSchema gains `role` ('content'|'mask') + `maskedBy` (subtract|intersect); spotlight.v1's cutout is now a mask layer the dim layer subtracts, so a non-React consumer reconstructs the cutout instead of a black rect. - P3: wb_clear on an empty board is 0ms (engine early-returns), not wbClearMs(0). Tests: tests/lib/choreography 45 pass (+6); engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): address final-audit findings on effect lifetime + spotlight descriptor Second cross-review round (fresh-session codex final audit) surfaced three deeper mismatches with live playback: - P1: fire-and-forget effect lifetime is not a flat EFFECT_AUTO_CLEAR_MS. The engine's processNext clears effects at every scene boundary and on completion, and scheduleEffectClear uses one shared timer each new effect resets. Added clampFireAndForgetLifetimes: an effect's visual durationMs is now min(next scene boundary / completion, shared-timer deadline chained through later effects in the same scene). advancesCursorMs (0) is untouched. - P2: spotlight dimness default is 0.5 (executeSpotlight: dimOpacity ?? 0.5; DSL documents 0.5), not the component's unreachable ?? 0.7 fallback. Fixed the descriptor param + test. - P2: model the spotlight wrapper's enter/exit opacity fade (motion.div, no explicit duration → engine default). TrackSchema.durationMs is now optional to express "use the consumer's engine default"; dim layer carries the fade tracks. Tests: tests/lib/choreography 50 pass (+5, incl. boundary-cut / completion-cut / full-lifetime / shared-timer-extension / wrapper-fade); engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): pin spotlight fade duration + laser dot geometry in descriptors Third cross-review round (codex) flagged two descriptor-completeness gaps that would make a non-Motion consumer (the exporter) diverge from the app: - Spotlight wrapper fade: the enter/exit opacity tracks left durationMs implicit (Motion default). A literal consumer treats a missing duration as instant, so the spotlight would pop on/off. Pinned to Motion's default 300ms tween. - Laser dot geometry: the descriptor captured only tracks, not the dot group's center anchor (translate -50%,-50%) or the rounded-full ring/core. A literal renderer would draw an offset 10px square. Added the static geometry (anchor, borderRadius 9999, ring inset/position) so the shape/position match the app. Also refined the effect-lifetime docstring to cite the app's per-scene engine teardown/completion (the actual clearEffects path) rather than an intra-engine boundary gate that is dead in the single-scene-per-engine configuration. The empty-scene "speech dwell → blank chat bubble" observation is pre-existing behavior: EMPTY_SCENE_DWELL is a verbatim move from lib/playback/engine-cursor.ts (unchanged from origin/main), out of scope for this move-only PR; tracked separately. Tests: tests/lib/choreography 54 pass; engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): zero-duration for engine-skipped/no-op actions in timeline Fourth cross-review round (codex) flagged two remaining timeline divergences: - Skipped discussions: the engine skips a discussion outright (no timer) when it's already consumed or its agent isn't selected, but the timeline always charged DISCUSSION_TRIGGER_DELAY_MS. Added `isDiscussionSkipped` resolver (runtime-state-dependent, like getVideoDurationMs) → 0ms when skipped. - No-op whiteboard draws: executeWbDrawText (empty content) and executeWbDrawTable (no rows/cols) return before any delay. The timeline now charges 0ms for these determinable-from-the-action no-ops instead of WB_DRAW_MS. (KaTeX-failure / missing-edit-target no-ops depend on runtime state and remain out of scope, consistent with the resolver pattern.) Tests: tests/lib/choreography 56 pass; engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): exact-5s effect chain break + spotlight dim full-screen geometry Fifth cross-review round (codex), both P2: - Effect chain break at an EXACT 5s boundary: the earlier effect's clear timer is queued before the reading timer that triggers the later effect (same 5000ms delay), so it fires first — the predecessor is cleared at exactly deadlineMs, not extended. Changed the chain guard from `> deadlineMs` to `>= deadlineMs`. - Spotlight dim layer full-screen geometry: the descriptor recorded only fill + mask relation, leaving a literal consumer no way to know the dim rect spans the 0..100 viewport. Added explicit x/y/width/height (100×100 at origin) so the descriptor is self-contained. Tests: tests/lib/choreography 57 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): no-op wb_edit_code resolver in timeline Sixth cross-review round (codex), one P2: executeWbEditCode returns before its delay when the edit can't apply (missing/non-code target, stale line refs). The timeline always charged WB_EDIT_MS. Added `isEditCodeNoop` resolver (runtime- state-dependent, same pattern as getClearElementCount / isDiscussionSkipped) → 0ms when the caller flags a no-op. Tests: tests/lib/choreography 58 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): descriptor layer inheritance for nested effect layers Seventh cross-review round (codex), two P2 with one root cause: the flat layers[] model couldn't express a child layer riding a parent's animation (the source nests some layers inside an animated wrapper). Added an `inheritsFrom: {parentId, props}` relation to LayerSchema: - Laser ring + core inheritsFrom the animated `dot` (left/top/opacity), so a literal consumer flies them in/out with the dot instead of leaving them at a static origin while only the dot moves. - Spotlight border inheritsFrom `dim` (opacity), so the outline fades out with the wrapper instead of lingering after the dimming layer disappears. Tests: tests/lib/choreography 60 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): address review — discussion auto-skip, explicit video policy, import allowlist Human review (wyuc) raised two blocking timing issues + one non-blocking guard: - Discussion dwell (blocking): a non-skipped discussion in unattended playback/export blocks for the trigger delay AND the ProactiveCard's own auto-skip countdown, not just DISCUSSION_TRIGGER_DELAY_MS. Added DISCUSSION_AUTO_SKIP_MS (5000) to the timing spec and charge DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS. ProactiveCard now reads the same constant (was a hardcoded 5000), so card countdown and timeline can't drift. A `spotlight -> discussion -> speech` timeline now extends the spotlight across the full discussion interval. - play_video (blocking): an unresolved duration no longer silently becomes a 0ms segment (which shifted every later action early). New `onUnresolvedVideoDuration` policy defaults to 'throw' (fail loudly); 'cap' assumes MAX_VIDEO_WAIT_MS, 'zero' opts back into no-dwell explicitly. - Purity guard (non-blocking): turned the lib/choreography boundary into a true import allowlist. Beyond the existing @/… + render-package blocks, it now rejects parent-escape (../…) imports/re-exports, any bare package other than @openmaic/dsl / zod, and dynamic import()/require(). Negative-tested: ../store, a stray bare package, export * from ../playback, and import('react') all fail. Tests: tests/lib/choreography 60 pass; engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
This commit is contained in:
parent
b4c1c580f8
commit
7fffec78f6
5
.codegraph/.gitignore
vendored
Normal file
5
.codegraph/.gitignore
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
# CodeGraph data files — local to each machine, not for committing.
|
||||
# Ignore everything in .codegraph/ except this file itself, so transient
|
||||
# files (the database, daemon.pid, sockets, logs) never show up in git.
|
||||
*
|
||||
!.gitignore
|
||||
@ -5,6 +5,7 @@ import { createPortal } from 'react-dom';
|
||||
import { motion } from 'motion/react';
|
||||
import { Play, Pause, X } from 'lucide-react';
|
||||
import { useI18n } from '@/lib/hooks/use-i18n';
|
||||
import { DISCUSSION_AUTO_SKIP_MS } from '@/lib/choreography';
|
||||
import type { DiscussionAction } from '@/lib/types/action';
|
||||
|
||||
interface ProactiveCardProps {
|
||||
@ -92,7 +93,7 @@ export const ProactiveCard = ({
|
||||
useEffect(() => {
|
||||
if (mode !== 'playback') return;
|
||||
|
||||
const duration = 5000;
|
||||
const duration = DISCUSSION_AUTO_SKIP_MS;
|
||||
const interval = 50;
|
||||
const step = (interval / duration) * 100;
|
||||
|
||||
@ -201,7 +202,7 @@ export const ProactiveCard = ({
|
||||
isPaused ? 'text-gray-300 dark:text-gray-600' : 'text-gray-400 dark:text-gray-500'
|
||||
}`}
|
||||
>
|
||||
{Math.max(0, Math.ceil((progress / 100) * 5))}s
|
||||
{Math.max(0, Math.ceil((progress / 100) * (DISCUSSION_AUTO_SKIP_MS / 1000)))}s
|
||||
</span>
|
||||
</div>
|
||||
|
||||
|
||||
@ -111,6 +111,93 @@ const eslintConfig = defineConfig([
|
||||
],
|
||||
},
|
||||
},
|
||||
// Module boundary (machine-enforced): lib/choreography is the shared
|
||||
// orchestration spec (timing + action timeline). It lives in the app (not a
|
||||
// package) because its semantics co-evolve with the playback engine, but it
|
||||
// must stay pure so the classroom-video exporter can interpret it in a pure
|
||||
// Node environment. Two guards, mirroring the package boundaries above:
|
||||
// 1. NO `@/…` host-app path-alias string — it authors none; it depends only
|
||||
// on @openmaic/dsl (types + the fire-and-forget partition) and relative
|
||||
// siblings. The app and exporter import it, never the reverse.
|
||||
// 2. NO React / DOM / render-backend runtime import (react, react-dom, gsap,
|
||||
// framer-motion, motion) — these are bare specifiers the `@/` rule can't
|
||||
// see. A descriptor describes animation; it never renders it.
|
||||
{
|
||||
files: ['lib/choreography/**/*.{ts,tsx,js,jsx,mjs,cjs}'],
|
||||
rules: {
|
||||
'no-restricted-syntax': [
|
||||
'error',
|
||||
{
|
||||
selector: 'Literal[value=/^@\\//]',
|
||||
message:
|
||||
'lib/choreography must not reference a host-app path (@/…). It authors no `@/…` strings — depend only on @openmaic/dsl and relative siblings, so the exporter can interpret it in pure Node. The app and exporter import it, not the reverse.',
|
||||
},
|
||||
{
|
||||
selector: 'TemplateElement[value.cooked=/^@\\//]',
|
||||
message:
|
||||
'lib/choreography must not reference a host-app path (@/…) in a template literal. Depend only on @openmaic/dsl and relative siblings.',
|
||||
},
|
||||
// Import allowlist (static imports/re-exports): the ONLY permitted
|
||||
// sources are `@openmaic/dsl`(/subpaths), `zod`, and in-folder relatives
|
||||
// (`./…`). Anything else — a parent-escape `../…` reaching back into the
|
||||
// app, or any other bare package — fails. Enforced on Import/Export
|
||||
// source string nodes via a negative-lookahead so the guard is a true
|
||||
// allowlist, not a blocklist of known-bad names.
|
||||
{
|
||||
selector:
|
||||
'ImportDeclaration > Literal.source[value=/^(?!@openmaic\\/dsl(\\/|$)|zod(\\/|$)|\\.\\/).+/]',
|
||||
message:
|
||||
'lib/choreography may import only from @openmaic/dsl, zod, or in-folder relatives (./…). No parent-escape (../…) into the app and no other packages — keep it pure so the exporter runs in plain Node.',
|
||||
},
|
||||
{
|
||||
selector:
|
||||
'ExportNamedDeclaration > Literal.source[value=/^(?!@openmaic\\/dsl(\\/|$)|zod(\\/|$)|\\.\\/).+/]',
|
||||
message:
|
||||
'lib/choreography may re-export only from @openmaic/dsl, zod, or in-folder relatives (./…).',
|
||||
},
|
||||
{
|
||||
selector:
|
||||
'ExportAllDeclaration > Literal.source[value=/^(?!@openmaic\\/dsl(\\/|$)|zod(\\/|$)|\\.\\/).+/]',
|
||||
message:
|
||||
'lib/choreography may re-export only from @openmaic/dsl, zod, or in-folder relatives (./…).',
|
||||
},
|
||||
// No dynamic import() or require() — they bypass the static allowlist and
|
||||
// can pull in a render backend at runtime.
|
||||
{
|
||||
selector: 'ImportExpression',
|
||||
message:
|
||||
'lib/choreography must not use dynamic import() — it bypasses the static import allowlist. Use a top-level import from @openmaic/dsl, zod, or a relative sibling.',
|
||||
},
|
||||
{
|
||||
selector: "CallExpression[callee.name='require']",
|
||||
message:
|
||||
'lib/choreography must not use require() — it bypasses the static import allowlist.',
|
||||
},
|
||||
],
|
||||
'no-restricted-imports': [
|
||||
'error',
|
||||
{
|
||||
patterns: [
|
||||
{
|
||||
group: [
|
||||
'react',
|
||||
'react-dom',
|
||||
'react/*',
|
||||
'react-dom/*',
|
||||
'gsap',
|
||||
'gsap/*',
|
||||
'framer-motion',
|
||||
'motion',
|
||||
'motion/*',
|
||||
],
|
||||
message:
|
||||
'lib/choreography must stay render-backend-agnostic (pure Node): no React / DOM / GSAP / framer-motion. It describes timing and animation as data; the app effect components and the exporter render it.',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
export default eslintConfig;
|
||||
|
||||
@ -36,6 +36,18 @@ import type {
|
||||
WidgetRevealAction,
|
||||
} from '@/lib/types/action';
|
||||
import type { CodeLine } from '@openmaic/dsl';
|
||||
import {
|
||||
EFFECT_AUTO_CLEAR_MS,
|
||||
MAX_VIDEO_WAIT_MS,
|
||||
WB_OPEN_MS,
|
||||
WB_DRAW_MS,
|
||||
WB_EDIT_MS,
|
||||
WB_DELETE_MS,
|
||||
WB_CLOSE_MS,
|
||||
WIDGET_MS,
|
||||
wbDrawCodeMs,
|
||||
wbClearMs,
|
||||
} from '@/lib/choreography';
|
||||
import katex from 'katex';
|
||||
import { createLogger } from '@/lib/logger';
|
||||
|
||||
@ -71,9 +83,6 @@ function generateLineIds(count: number): string[] {
|
||||
|
||||
// ==================== ActionEngine ====================
|
||||
|
||||
/** Default duration (ms) before fire-and-forget effects auto-clear */
|
||||
const EFFECT_AUTO_CLEAR_MS = 5000;
|
||||
|
||||
/** Callback for sending messages to widget iframe */
|
||||
export type WidgetMessageCallback = (type: string, payload: Record<string, unknown>) => void;
|
||||
|
||||
@ -293,7 +302,6 @@ export class ActionEngine {
|
||||
// the playback engine from hanging indefinitely if the video element is
|
||||
// invalid or the state change is missed.
|
||||
return new Promise<void>((resolve) => {
|
||||
const MAX_VIDEO_WAIT_MS = 5 * 60 * 1000; // 5 minutes
|
||||
const timeout = setTimeout(() => {
|
||||
unsubscribe();
|
||||
log.warn(`[playVideo] Timeout waiting for video ${action.elementId} to finish`);
|
||||
@ -365,7 +373,7 @@ export class ActionEngine {
|
||||
useCanvasStore.getState().setWhiteboardOpen(true);
|
||||
if (options.silent) return;
|
||||
// Wait for open animation to complete (slow spring: stiffness 120, damping 18, mass 1.2)
|
||||
await delay(2000);
|
||||
await delay(WB_OPEN_MS);
|
||||
}
|
||||
|
||||
private async executeWbDrawText(
|
||||
@ -401,7 +409,7 @@ export class ActionEngine {
|
||||
|
||||
if (!options.silent) {
|
||||
// Wait for element fade-in animation
|
||||
await delay(800);
|
||||
await delay(WB_DRAW_MS);
|
||||
}
|
||||
}
|
||||
|
||||
@ -432,7 +440,7 @@ export class ActionEngine {
|
||||
|
||||
if (!options.silent) {
|
||||
// Wait for element fade-in animation
|
||||
await delay(800);
|
||||
await delay(WB_DRAW_MS);
|
||||
}
|
||||
}
|
||||
|
||||
@ -460,7 +468,7 @@ export class ActionEngine {
|
||||
wb.data.id,
|
||||
);
|
||||
|
||||
if (!options.silent) await delay(800);
|
||||
if (!options.silent) await delay(WB_DRAW_MS);
|
||||
}
|
||||
|
||||
private async executeWbDrawLatex(
|
||||
@ -499,7 +507,7 @@ export class ActionEngine {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!options.silent) await delay(800);
|
||||
if (!options.silent) await delay(WB_DRAW_MS);
|
||||
}
|
||||
|
||||
private async executeWbDrawTable(
|
||||
@ -558,7 +566,7 @@ export class ActionEngine {
|
||||
wb.data.id,
|
||||
);
|
||||
|
||||
if (!options.silent) await delay(800);
|
||||
if (!options.silent) await delay(WB_DRAW_MS);
|
||||
}
|
||||
|
||||
private async executeWbDrawLine(
|
||||
@ -595,7 +603,7 @@ export class ActionEngine {
|
||||
|
||||
if (!options.silent) {
|
||||
// Wait for element fade-in animation
|
||||
await delay(800);
|
||||
await delay(WB_DRAW_MS);
|
||||
}
|
||||
}
|
||||
|
||||
@ -628,8 +636,8 @@ export class ActionEngine {
|
||||
);
|
||||
|
||||
if (!options.silent) {
|
||||
// Wait for typing animation: base 800ms + 50ms per line, capped at 3s
|
||||
const animMs = Math.min(800 + lines.length * 50, 3000);
|
||||
// Wait for typing animation (base 800ms + 50ms/line, capped at 3s)
|
||||
const animMs = wbDrawCodeMs(lines.length);
|
||||
await delay(animMs);
|
||||
}
|
||||
}
|
||||
@ -697,7 +705,7 @@ export class ActionEngine {
|
||||
|
||||
if (!options.silent) {
|
||||
// Wait for edit animation
|
||||
await delay(600);
|
||||
await delay(WB_EDIT_MS);
|
||||
}
|
||||
}
|
||||
|
||||
@ -709,7 +717,7 @@ export class ActionEngine {
|
||||
if (!wb.success || !wb.data) return;
|
||||
|
||||
this.stageAPI.whiteboard.deleteElement(action.elementId, wb.data.id);
|
||||
if (!options.silent) await delay(300);
|
||||
if (!options.silent) await delay(WB_DELETE_MS);
|
||||
}
|
||||
|
||||
private async executeWbClear(options: ActionExecutionOptions = {}): Promise<void> {
|
||||
@ -731,8 +739,8 @@ export class ActionEngine {
|
||||
// Trigger cascade exit animation
|
||||
useCanvasStore.getState().setWhiteboardClearing(true);
|
||||
|
||||
// Wait for cascade: base 380ms + 55ms per element, capped at 1400ms
|
||||
const animMs = Math.min(380 + elementCount * 55, 1400);
|
||||
// Wait for cascade (base 380ms + 55ms/element, capped at 1400ms)
|
||||
const animMs = wbClearMs(elementCount);
|
||||
await delay(animMs);
|
||||
|
||||
// Actually remove elements
|
||||
@ -744,7 +752,7 @@ export class ActionEngine {
|
||||
useCanvasStore.getState().setWhiteboardOpen(false);
|
||||
if (options.silent) return;
|
||||
// Wait for close animation (500ms ease-out tween)
|
||||
await delay(700);
|
||||
await delay(WB_CLOSE_MS);
|
||||
}
|
||||
|
||||
// ==================== Widget Actions ====================
|
||||
@ -765,14 +773,14 @@ export class ActionEngine {
|
||||
content: action.content,
|
||||
});
|
||||
// Quick delay for visual effect
|
||||
await delay(300);
|
||||
await delay(WIDGET_MS);
|
||||
}
|
||||
|
||||
/** Execute widget setState action */
|
||||
private async executeWidgetSetState(action: WidgetSetStateAction): Promise<void> {
|
||||
this.sendWidgetMessage('SET_WIDGET_STATE', { state: action.state, content: action.content });
|
||||
// Quick delay for state change to propagate
|
||||
await delay(300);
|
||||
await delay(WIDGET_MS);
|
||||
}
|
||||
|
||||
/** Execute widget annotation action */
|
||||
@ -781,12 +789,12 @@ export class ActionEngine {
|
||||
target: action.target,
|
||||
content: action.content,
|
||||
});
|
||||
await delay(300);
|
||||
await delay(WIDGET_MS);
|
||||
}
|
||||
|
||||
/** Execute widget reveal action */
|
||||
private async executeWidgetReveal(action: WidgetRevealAction): Promise<void> {
|
||||
this.sendWidgetMessage('REVEAL_ELEMENT', { target: action.target, content: action.content });
|
||||
await delay(300);
|
||||
await delay(WIDGET_MS);
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,5 +1,12 @@
|
||||
import type { Action } from '@/lib/types/action';
|
||||
import type { Scene } from '@/lib/types/stage';
|
||||
/**
|
||||
* Playback cursor — resolve the current action from a `(sceneIndex, actionIndex)`
|
||||
* cursor over a scene list.
|
||||
*
|
||||
* Moved verbatim from the app's `lib/playback/engine-cursor.ts` so the app
|
||||
* runtime and the video exporter walk scenes identically. Pure — types come
|
||||
* from `@openmaic/dsl`, no runtime dependencies.
|
||||
*/
|
||||
import type { Action, SceneCore } from '@openmaic/dsl';
|
||||
|
||||
/**
|
||||
* Synthetic dwell beat yielded for a scene that carries no actions. It is an
|
||||
@ -30,10 +37,12 @@ export interface CursorResult {
|
||||
* yields one {@link EMPTY_SCENE_DWELL} beat (when its action cursor is still 0)
|
||||
* rather than being skipped. Returns `null` once every scene is consumed.
|
||||
*
|
||||
* Pure: it does not mutate inputs; the caller adopts the returned cursor.
|
||||
* Pure: it does not mutate inputs; the caller adopts the returned cursor. Typed
|
||||
* against {@link SceneCore} (only `id` + `actions` are read), so an app-widened
|
||||
* `Scene` (extra content kinds) is accepted without casting.
|
||||
*/
|
||||
export function resolvePlaybackCursor(
|
||||
scenes: Scene[],
|
||||
scenes: SceneCore[],
|
||||
sceneIndex: number,
|
||||
actionIndex: number,
|
||||
): CursorResult | null {
|
||||
22
lib/choreography/descriptors/index.ts
Normal file
22
lib/choreography/descriptors/index.ts
Normal file
@ -0,0 +1,22 @@
|
||||
/**
|
||||
* Animation descriptor registry — the versioned, declarative animations both
|
||||
* the app effect components and the video exporter interpret.
|
||||
*/
|
||||
import type { AnimationDescriptor } from './types';
|
||||
import { spotlightV1 } from './spotlight';
|
||||
import { laserV1 } from './laser';
|
||||
|
||||
export * from './types';
|
||||
export { spotlightV1 } from './spotlight';
|
||||
export { laserV1 } from './laser';
|
||||
|
||||
/** All shipped descriptors, keyed by their versioned id. */
|
||||
export const DESCRIPTORS: Record<string, AnimationDescriptor> = {
|
||||
'spotlight.v1': spotlightV1,
|
||||
'laser.v1': laserV1,
|
||||
};
|
||||
|
||||
/** Look up a descriptor by its versioned id (e.g. 'spotlight.v1'). */
|
||||
export function getDescriptor(id: string): AnimationDescriptor | undefined {
|
||||
return DESCRIPTORS[id];
|
||||
}
|
||||
138
lib/choreography/descriptors/laser.ts
Normal file
138
lib/choreography/descriptors/laser.ts
Normal file
@ -0,0 +1,138 @@
|
||||
import type { AnimationDescriptor } from './types';
|
||||
|
||||
/**
|
||||
* laser.v1 — a laser dot flies in from the nearest off-screen corner to the
|
||||
* element center, with a pulsing ring and a glowing core.
|
||||
*
|
||||
* Values captured verbatim from the `LaserOverlay` effect component
|
||||
* (`motion/react`):
|
||||
* - dot fly-in: left/top 500ms expo-out `[0.22,1,0.36,1]`; opacity 150ms;
|
||||
* start corner is `center > 50 ? 105 : -5` (percent) per axis.
|
||||
* - exit: 250ms ease-in `[0.4,0,1,1]` back to the start corner (position +
|
||||
* opacity).
|
||||
* - ring: infinite pulse, scale 1→2.8, opacity 0.6→0, 1500ms easeOut, 300ms
|
||||
* repeat delay.
|
||||
* - core: 10px square (`w-2.5 h-2.5`), glow `0 0 8px 2px {color}60`.
|
||||
*
|
||||
* Static geometry captured too, so a literal (non-React) renderer matches the
|
||||
* app rather than drawing an offset square: the dot group is centered on the
|
||||
* target via `translate(-50%, -50%)` (`-translate-x/y-1/2`), and both the ring
|
||||
* and core are circles (`rounded-full` → `borderRadius: 9999px`); the ring is
|
||||
* `position: absolute; inset: 0` inside the group.
|
||||
*
|
||||
* Color default `#ff0000` — the app store's laserOptions default
|
||||
* (`lib/store/canvas.ts`), which the caller passes in; the component's own
|
||||
* fallback `#ff3b30` is only used when no options exist, which does not happen
|
||||
* at playback time.
|
||||
*/
|
||||
export const laserV1: AnimationDescriptor = {
|
||||
id: 'laser.v1',
|
||||
version: 1,
|
||||
effect: 'laser',
|
||||
params: { color: '#ff0000' },
|
||||
zIndex: 101,
|
||||
layers: [
|
||||
{
|
||||
id: 'dot',
|
||||
// The dot group is anchored at its own center on the target coordinate.
|
||||
staticProps: { transform: 'translate(-50%, -50%)' },
|
||||
tracks: [
|
||||
{
|
||||
property: 'left',
|
||||
from: { axis: 'centerX', threshold: 50, whenAbove: 105, whenBelow: -5 },
|
||||
to: { ref: 'centerX' },
|
||||
durationMs: 500,
|
||||
easing: { type: 'cubicBezier', points: [0.22, 1, 0.36, 1] },
|
||||
phase: 'enter',
|
||||
},
|
||||
{
|
||||
property: 'top',
|
||||
from: { axis: 'centerY', threshold: 50, whenAbove: 105, whenBelow: -5 },
|
||||
to: { ref: 'centerY' },
|
||||
durationMs: 500,
|
||||
easing: { type: 'cubicBezier', points: [0.22, 1, 0.36, 1] },
|
||||
phase: 'enter',
|
||||
},
|
||||
{
|
||||
property: 'opacity',
|
||||
from: 0,
|
||||
to: 1,
|
||||
durationMs: 150,
|
||||
phase: 'enter',
|
||||
},
|
||||
{
|
||||
property: 'left',
|
||||
from: { ref: 'centerX' },
|
||||
to: { axis: 'centerX', threshold: 50, whenAbove: 105, whenBelow: -5 },
|
||||
durationMs: 250,
|
||||
easing: { type: 'cubicBezier', points: [0.4, 0, 1, 1] },
|
||||
phase: 'exit',
|
||||
},
|
||||
{
|
||||
property: 'top',
|
||||
from: { ref: 'centerY' },
|
||||
to: { axis: 'centerY', threshold: 50, whenAbove: 105, whenBelow: -5 },
|
||||
durationMs: 250,
|
||||
easing: { type: 'cubicBezier', points: [0.4, 0, 1, 1] },
|
||||
phase: 'exit',
|
||||
},
|
||||
{
|
||||
property: 'opacity',
|
||||
from: 1,
|
||||
to: 0,
|
||||
durationMs: 250,
|
||||
easing: { type: 'cubicBezier', points: [0.4, 0, 1, 1] },
|
||||
phase: 'exit',
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'ring',
|
||||
// Nested inside the animated dot wrapper in the source, so it rides the
|
||||
// dot's fly-in/exit position + fade, then adds its own scale/opacity pulse.
|
||||
inheritsFrom: { parentId: 'dot', props: ['left', 'top', 'opacity'] },
|
||||
// Circular pulse ring, absolutely positioned to fill the dot group.
|
||||
staticProps: {
|
||||
position: 'absolute',
|
||||
inset: 0,
|
||||
borderRadius: 9999,
|
||||
border: '1.5px solid {color}',
|
||||
},
|
||||
tracks: [
|
||||
{
|
||||
property: 'scale',
|
||||
from: 1,
|
||||
to: 2.8,
|
||||
durationMs: 1500,
|
||||
easing: { type: 'named', name: 'easeOut' },
|
||||
repeat: 'infinite',
|
||||
repeatDelayMs: 300,
|
||||
},
|
||||
{
|
||||
property: 'opacity',
|
||||
from: 0.6,
|
||||
to: 0,
|
||||
durationMs: 1500,
|
||||
easing: { type: 'named', name: 'easeOut' },
|
||||
repeat: 'infinite',
|
||||
repeatDelayMs: 300,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'core',
|
||||
// Nested inside the animated dot wrapper too — rides the dot's fly-in/exit
|
||||
// position + fade. It has no animation of its own.
|
||||
inheritsFrom: { parentId: 'dot', props: ['left', 'top', 'opacity'] },
|
||||
// Circular light core (rounded-full).
|
||||
staticProps: {
|
||||
width: 10,
|
||||
height: 10,
|
||||
borderRadius: 9999,
|
||||
backgroundColor: '{color}',
|
||||
boxShadow: '0 0 8px 2px {color}60',
|
||||
},
|
||||
tracks: [],
|
||||
},
|
||||
],
|
||||
};
|
||||
160
lib/choreography/descriptors/spotlight.ts
Normal file
160
lib/choreography/descriptors/spotlight.ts
Normal file
@ -0,0 +1,160 @@
|
||||
import type { AnimationDescriptor } from './types';
|
||||
|
||||
/**
|
||||
* spotlight.v1 — focus a single element, dimming the rest.
|
||||
*
|
||||
* An SVG mask (0-100 viewBox) punches a rounded cutout over a dimmed full-screen
|
||||
* rect; a white border traces the cutout. Values captured verbatim from the
|
||||
* `SpotlightOverlay` effect component (`motion/react`):
|
||||
* - cutout: 600ms expo-out, insets from ±8/rx4 to the tight ±~0.5/rx1 frame.
|
||||
* Modeled as a `role: 'mask'` layer — it is not painted itself; the `dim`
|
||||
* layer subtracts it (`maskedBy`), so a non-React consumer reconstructs the
|
||||
* "dim everywhere except the cutout" compositing rather than "draw a black
|
||||
* rect".
|
||||
* - border: 500ms expo-out, delayed 50ms, fading in as it settles.
|
||||
* - dim: static `rgba(0,0,0,{dimness})`, dimness default 0.5, with the cutout
|
||||
* subtracted.
|
||||
*
|
||||
* The `dimness` default is 0.5 — the value the runtime actually renders: a
|
||||
* spotlight action with no `dimOpacity` is stored as `action.dimOpacity ?? 0.5`
|
||||
* (`ActionEngine.executeSpotlight`; DSL documents `dimOpacity` default 0.5), so
|
||||
* the component's own `?? 0.7` fallback is unreachable at playback. The exporter
|
||||
* must use 0.5 to match.
|
||||
*
|
||||
* Shared easing `[0.16, 1, 0.3, 1]` (the spotlight expo-out).
|
||||
*/
|
||||
export const spotlightV1: AnimationDescriptor = {
|
||||
id: 'spotlight.v1',
|
||||
version: 1,
|
||||
effect: 'spotlight',
|
||||
params: { dimness: 0.5 },
|
||||
zIndex: 100,
|
||||
layers: [
|
||||
{
|
||||
id: 'cutout',
|
||||
// Geometry only — subtracted from `dim` (see its maskedBy), not painted.
|
||||
role: 'mask',
|
||||
staticProps: { fill: '#000000' },
|
||||
tracks: [
|
||||
{
|
||||
property: 'x',
|
||||
from: { ref: 'x', offset: -8 },
|
||||
to: { ref: 'x', offset: -0.4 },
|
||||
durationMs: 600,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'y',
|
||||
from: { ref: 'y', offset: -8 },
|
||||
to: { ref: 'y', offset: -0.6 },
|
||||
durationMs: 600,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'width',
|
||||
from: { ref: 'w', offset: 16 },
|
||||
to: { ref: 'w', offset: 0.8 },
|
||||
durationMs: 600,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'height',
|
||||
from: { ref: 'h', offset: 16 },
|
||||
to: { ref: 'h', offset: 1.2 },
|
||||
durationMs: 600,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'rx',
|
||||
from: 4,
|
||||
to: 1,
|
||||
durationMs: 600,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'border',
|
||||
// The white outline sits inside the same wrapper motion.div as the dim
|
||||
// rect, so it fades in/out with the whole effect. `dim` carries the
|
||||
// canonical wrapper opacity fade; the border rides it (on top of its own
|
||||
// geometry + border-opacity tracks below).
|
||||
inheritsFrom: { parentId: 'dim', props: ['opacity'] },
|
||||
staticProps: {
|
||||
stroke: 'rgba(255,255,255,0.7)',
|
||||
strokeWidth: 1.2,
|
||||
fill: 'none',
|
||||
vectorEffect: 'non-scaling-stroke',
|
||||
},
|
||||
tracks: [
|
||||
{
|
||||
property: 'x',
|
||||
from: { ref: 'x', offset: -4 },
|
||||
to: { ref: 'x', offset: -0.4 },
|
||||
durationMs: 500,
|
||||
delayMs: 50,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'y',
|
||||
from: { ref: 'y', offset: -4 },
|
||||
to: { ref: 'y', offset: -0.6 },
|
||||
durationMs: 500,
|
||||
delayMs: 50,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'width',
|
||||
from: { ref: 'w', offset: 8 },
|
||||
to: { ref: 'w', offset: 0.8 },
|
||||
durationMs: 500,
|
||||
delayMs: 50,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'height',
|
||||
from: { ref: 'h', offset: 8 },
|
||||
to: { ref: 'h', offset: 1.2 },
|
||||
durationMs: 500,
|
||||
delayMs: 50,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'opacity',
|
||||
from: 0,
|
||||
to: 1,
|
||||
durationMs: 500,
|
||||
delayMs: 50,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
{
|
||||
property: 'rx',
|
||||
from: 2,
|
||||
to: 1,
|
||||
durationMs: 500,
|
||||
delayMs: 50,
|
||||
easing: { type: 'cubicBezier', points: [0.16, 1, 0.3, 1] },
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'dim',
|
||||
// Full-screen dim behind the cutout, with the cutout subtracted (SVG
|
||||
// <mask>: white full-cover minus the black cutout rect). Spans the whole
|
||||
// 0..100 viewport (`<rect width="100" height="100">` at the origin) — made
|
||||
// explicit so a literal consumer dims the entire frame rather than nothing.
|
||||
maskedBy: { layerId: 'cutout', mode: 'subtract' },
|
||||
staticProps: { x: 0, y: 0, width: 100, height: 100, fill: 'rgba(0,0,0,{dimness})' },
|
||||
// The whole effect is wrapped in a motion.div that fades opacity 0→1 on
|
||||
// enter and →0 on exit. The source sets no explicit transition, so Motion
|
||||
// applies its default tween (300ms) — pinned here explicitly so a
|
||||
// non-Motion consumer (the exporter) reproduces the fade instead of
|
||||
// popping the spotlight on/off. This layer carries the canonical wrapper
|
||||
// fade; the border layer inherits its opacity so both fade out together.
|
||||
tracks: [
|
||||
{ property: 'opacity', from: 0, to: 1, durationMs: 300, phase: 'enter' },
|
||||
{ property: 'opacity', from: 1, to: 0, durationMs: 300, phase: 'exit' },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
190
lib/choreography/descriptors/types.ts
Normal file
190
lib/choreography/descriptors/types.ts
Normal file
@ -0,0 +1,190 @@
|
||||
/**
|
||||
* Animation descriptor model — a declarative, render-backend-agnostic
|
||||
* description of an effect animation: *what property, from what value to what
|
||||
* value, over how long, with what easing*. No implementation, no `motion`, no
|
||||
* DOM. The app's effect components and the video exporter both interpret these,
|
||||
* so the animation values live in exactly one place and cannot drift.
|
||||
*
|
||||
* Descriptors are versioned (`spotlight.v1`) and schema-validated: the schema
|
||||
* is authored here with zod, the TS types are inferred from it (single source),
|
||||
* and every shipped descriptor is checked against it in tests. The exporter can
|
||||
* reuse {@link AnimationDescriptorSchema} to validate anything it interprets.
|
||||
*
|
||||
* Pure — depends only on zod, no React / DOM / render backend.
|
||||
*/
|
||||
import { z } from 'zod';
|
||||
|
||||
/** A field of the target element's percentage geometry (0-100 space). */
|
||||
export const GeometryRefSchema = z.enum(['x', 'y', 'w', 'h', 'centerX', 'centerY']);
|
||||
|
||||
/**
|
||||
* A value derived linearly from the target element's geometry:
|
||||
* `value = geometry[ref] * scale + offset`. Used for effect positions that
|
||||
* track the highlighted element (e.g. a spotlight cutout inset by a few units).
|
||||
*/
|
||||
export const GeometryValueSchema = z.object({
|
||||
ref: GeometryRefSchema,
|
||||
/** Multiplier on the geometry field. Default 1. */
|
||||
scale: z.number().optional(),
|
||||
/** Added after scaling. Default 0. */
|
||||
offset: z.number().optional(),
|
||||
});
|
||||
|
||||
/**
|
||||
* A corner/edge fly-in start value: pick one of two off-screen positions based
|
||||
* on which half of the viewport the element center sits in. Models the laser's
|
||||
* `center > 50 ? 105 : -5` start rule.
|
||||
*/
|
||||
export const CornerValueSchema = z.object({
|
||||
/** Which center axis to test. */
|
||||
axis: z.enum(['centerX', 'centerY']),
|
||||
/** Comparison threshold (percent). */
|
||||
threshold: z.number(),
|
||||
/** Value used when the center is strictly above the threshold. */
|
||||
whenAbove: z.number(),
|
||||
/** Value used otherwise. */
|
||||
whenBelow: z.number(),
|
||||
});
|
||||
|
||||
/**
|
||||
* An animatable endpoint: a literal number, a literal string (colors; may carry
|
||||
* a `{param}` placeholder), or a geometry-/corner-derived value.
|
||||
*/
|
||||
export const AnimatableValueSchema = z.union([
|
||||
z.number(),
|
||||
z.string(),
|
||||
GeometryValueSchema,
|
||||
CornerValueSchema,
|
||||
]);
|
||||
|
||||
/** Easing curve. Omit on a track to use the consumer's engine default. */
|
||||
export const EasingSchema = z.discriminatedUnion('type', [
|
||||
z.object({
|
||||
type: z.literal('cubicBezier'),
|
||||
points: z.tuple([z.number(), z.number(), z.number(), z.number()]),
|
||||
}),
|
||||
z.object({ type: z.literal('named'), name: z.string() }),
|
||||
z.object({
|
||||
type: z.literal('spring'),
|
||||
stiffness: z.number(),
|
||||
damping: z.number(),
|
||||
mass: z.number().optional(),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Which phase of the effect lifecycle a track belongs to. Default 'enter'. */
|
||||
export const TrackPhaseSchema = z.enum(['enter', 'exit']);
|
||||
|
||||
/** A single animated property from `from` to `to` over `durationMs`. */
|
||||
export const TrackSchema = z.object({
|
||||
/** The property name (e.g. 'x', 'width', 'opacity', 'scale', 'left', 'top'). */
|
||||
property: z.string(),
|
||||
from: AnimatableValueSchema,
|
||||
to: AnimatableValueSchema,
|
||||
/** Omitted when the source specifies no explicit duration — use the consumer's engine default. */
|
||||
durationMs: z.number().optional(),
|
||||
delayMs: z.number().optional(),
|
||||
/** Omitted when the source specifies no explicit easing. */
|
||||
easing: EasingSchema.optional(),
|
||||
phase: TrackPhaseSchema.optional(),
|
||||
/** Number of repeats, or 'infinite'. Omit for no repeat. */
|
||||
repeat: z.union([z.number(), z.literal('infinite')]).optional(),
|
||||
repeatDelayMs: z.number().optional(),
|
||||
});
|
||||
|
||||
/** Non-animated static value on a layer; strings may carry `{param}` placeholders. */
|
||||
const StaticPropsSchema = z.record(z.string(), z.union([z.number(), z.string()]));
|
||||
|
||||
/** Whether a layer is painted directly, or only defines geometry used as a mask. */
|
||||
export const LayerRoleSchema = z.enum(['content', 'mask']);
|
||||
|
||||
/**
|
||||
* How a content layer is clipped by a `mask`-role layer:
|
||||
* - `subtract` — the mask region is removed (made transparent). Models an SVG
|
||||
* `<mask>` of a white full-cover minus a black shape (the spotlight: the dim
|
||||
* rect shows everywhere *except* the cutout).
|
||||
* - `intersect` — only the mask region is kept.
|
||||
*/
|
||||
export const MaskModeSchema = z.enum(['subtract', 'intersect']);
|
||||
|
||||
/** A content layer's clip relationship to a `mask`-role layer in the same descriptor. */
|
||||
export const MaskRefSchema = z.object({
|
||||
/** `id` of the sibling layer (role `mask`) whose animated geometry clips this one. */
|
||||
layerId: z.string(),
|
||||
mode: MaskModeSchema,
|
||||
});
|
||||
|
||||
/** Which of a parent layer's animated properties a child inherits. */
|
||||
export const InheritablePropSchema = z.enum(['left', 'top', 'x', 'y', 'opacity', 'scale']);
|
||||
|
||||
/**
|
||||
* A layer's inheritance from a parent layer in the same descriptor. In the
|
||||
* source, the effect nests some layers inside an animated wrapper so they ride
|
||||
* its motion (the laser ring/core sit inside the animated dot; the spotlight
|
||||
* border sits inside the fading SVG wrapper). A flat layer list can't express
|
||||
* that, so a child names its `parentId` and the `props` it rides — a literal
|
||||
* consumer composes the parent's animation for those props with the child's own
|
||||
* (e.g. the ring inherits the dot's left/top/opacity and adds its own scale
|
||||
* pulse). `props` defaults to all of the parent's animated properties.
|
||||
*/
|
||||
export const InheritRefSchema = z.object({
|
||||
/** `id` of the parent layer whose animation this layer rides. */
|
||||
parentId: z.string(),
|
||||
/** Properties inherited from the parent. Omit to inherit all of them. */
|
||||
props: z.array(InheritablePropSchema).optional(),
|
||||
});
|
||||
|
||||
/**
|
||||
* A visual layer of the effect (e.g. the spotlight cutout, its border, the
|
||||
* laser ring). Groups animated `tracks` with non-animated `staticProps`.
|
||||
*
|
||||
* A layer with `role: 'mask'` is not painted on its own — its animated geometry
|
||||
* is referenced by another layer's `maskedBy` to clip it. This lets a non-React
|
||||
* consumer reconstruct compositing relationships (e.g. the spotlight dim rect
|
||||
* with the cutout punched out) that independent layers alone cannot express.
|
||||
*
|
||||
* A layer with `inheritsFrom` rides a parent layer's animation for the named
|
||||
* properties (the source nests it inside that animated wrapper), on top of its
|
||||
* own `tracks`.
|
||||
*/
|
||||
export const LayerSchema = z.object({
|
||||
id: z.string(),
|
||||
/** Default `content` (painted). `mask` layers only supply geometry for a `maskedBy` ref. */
|
||||
role: LayerRoleSchema.optional(),
|
||||
/** This (content) layer is clipped by the referenced `mask`-role layer. */
|
||||
maskedBy: MaskRefSchema.optional(),
|
||||
/** This layer rides a parent layer's animation (it is nested inside it in the source). */
|
||||
inheritsFrom: InheritRefSchema.optional(),
|
||||
tracks: z.array(TrackSchema),
|
||||
staticProps: StaticPropsSchema.optional(),
|
||||
});
|
||||
|
||||
/** A versioned, declarative animation for one effect. */
|
||||
export const AnimationDescriptorSchema = z.object({
|
||||
/** Stable id including version, e.g. 'spotlight.v1'. */
|
||||
id: z.string(),
|
||||
/** Numeric version, bumped on any behavioral change. */
|
||||
version: z.number(),
|
||||
effect: z.enum(['spotlight', 'laser']),
|
||||
/** Default parameter values; consumers may override (e.g. dimness, color). */
|
||||
params: StaticPropsSchema.optional(),
|
||||
/** Stacking order the effect renders at. */
|
||||
zIndex: z.number(),
|
||||
layers: z.array(LayerSchema),
|
||||
});
|
||||
|
||||
// Types are inferred from the schemas so the schema stays the single source.
|
||||
export type GeometryRef = z.infer<typeof GeometryRefSchema>;
|
||||
export type GeometryValue = z.infer<typeof GeometryValueSchema>;
|
||||
export type CornerValue = z.infer<typeof CornerValueSchema>;
|
||||
export type AnimatableValue = z.infer<typeof AnimatableValueSchema>;
|
||||
export type Easing = z.infer<typeof EasingSchema>;
|
||||
export type TrackPhase = z.infer<typeof TrackPhaseSchema>;
|
||||
export type Track = z.infer<typeof TrackSchema>;
|
||||
export type LayerRole = z.infer<typeof LayerRoleSchema>;
|
||||
export type MaskMode = z.infer<typeof MaskModeSchema>;
|
||||
export type MaskRef = z.infer<typeof MaskRefSchema>;
|
||||
export type InheritableProp = z.infer<typeof InheritablePropSchema>;
|
||||
export type InheritRef = z.infer<typeof InheritRefSchema>;
|
||||
export type Layer = z.infer<typeof LayerSchema>;
|
||||
export type AnimationDescriptor = z.infer<typeof AnimationDescriptorSchema>;
|
||||
25
lib/choreography/index.ts
Normal file
25
lib/choreography/index.ts
Normal file
@ -0,0 +1,25 @@
|
||||
/**
|
||||
* `lib/choreography` — the shared orchestration/spec modules.
|
||||
*
|
||||
* The single source of truth for the semantics a faithful classroom-video
|
||||
* exporter needs from playback, so neither the app runtime nor the exporter
|
||||
* re-implements (and silently drifts from) the other:
|
||||
*
|
||||
* - **Timing** — playback timing constants + the deterministic no-audio speech
|
||||
* estimate ({@link estimateSpeechDurationMs}).
|
||||
* - **Cursor** — {@link resolvePlaybackCursor}, the scene/action walk.
|
||||
* - **Timeline** — {@link resolveActionTimeline}, the index→time expansion.
|
||||
* - **Descriptors** — versioned, declarative animation descriptors
|
||||
* ({@link DESCRIPTORS}, e.g. `spotlight.v1`) + their zod schema.
|
||||
*
|
||||
* Kept in `lib/` (not a package) because these semantics co-evolve with the
|
||||
* playback engine. Purity is machine-enforced: these modules import only
|
||||
* `@openmaic/dsl` (types + the fire-and-forget partition), `zod` (descriptor
|
||||
* schema), and pure helpers — no React / DOM / GSAP / framer-motion / render
|
||||
* backend, so the exporter can interpret them in a pure Node environment. The
|
||||
* eslint `@/`-boundary block on `lib/choreography/**` keeps host-app paths out.
|
||||
*/
|
||||
export * from './timing';
|
||||
export * from './cursor';
|
||||
export * from './timeline';
|
||||
export * from './descriptors/index';
|
||||
403
lib/choreography/timeline.ts
Normal file
403
lib/choreography/timeline.ts
Normal file
@ -0,0 +1,403 @@
|
||||
/**
|
||||
* `resolveActionTimeline` — the index-domain → time-domain expansion.
|
||||
*
|
||||
* Playback drives actions by a `(sceneIndex, actionIndex)` cursor; a faithful
|
||||
* video exporter needs the same sequence laid out on a wall-clock. This pure
|
||||
* function formalizes the semantics the app's `PlaybackEngine.processNext`
|
||||
* switch expresses as control flow:
|
||||
*
|
||||
* - **Blocking** actions (speech, whiteboard, widget, video, discussion) hold
|
||||
* the cursor until they complete — the next action starts after them.
|
||||
* - **Fire-and-forget** actions (spotlight, laser) do not block: playback
|
||||
* continues immediately, and the effect persists visually for
|
||||
* {@link EFFECT_AUTO_CLEAR_MS} before auto-clearing.
|
||||
*
|
||||
* The blocking/non-blocking partition is read from the DSL's
|
||||
* {@link FIRE_AND_FORGET_ACTIONS} rather than hardcoded here, so the two stay
|
||||
* in lockstep. Durations come from the shared {@link timing} spec.
|
||||
*
|
||||
* Pure, no runtime dependencies beyond `@openmaic/dsl`.
|
||||
*/
|
||||
import type {
|
||||
Action,
|
||||
SceneCore,
|
||||
SpeechAction,
|
||||
PlayVideoAction,
|
||||
WbDrawCodeAction,
|
||||
WbDrawTextAction,
|
||||
WbDrawTableAction,
|
||||
WbEditCodeAction,
|
||||
DiscussionAction,
|
||||
WbClearAction,
|
||||
} from '@openmaic/dsl';
|
||||
import { FIRE_AND_FORGET_ACTIONS } from '@openmaic/dsl';
|
||||
import { EMPTY_SCENE_DWELL } from './cursor';
|
||||
import {
|
||||
EFFECT_AUTO_CLEAR_MS,
|
||||
DISCUSSION_TRIGGER_DELAY_MS,
|
||||
DISCUSSION_AUTO_SKIP_MS,
|
||||
MAX_VIDEO_WAIT_MS,
|
||||
WB_OPEN_MS,
|
||||
WB_DRAW_MS,
|
||||
WB_EDIT_MS,
|
||||
WB_DELETE_MS,
|
||||
WB_CLOSE_MS,
|
||||
WIDGET_MS,
|
||||
wbDrawCodeMs,
|
||||
wbClearMs,
|
||||
estimateSpeechDurationMs,
|
||||
} from './timing';
|
||||
|
||||
const FIRE_AND_FORGET = new Set<string>(FIRE_AND_FORGET_ACTIONS);
|
||||
|
||||
/**
|
||||
* Synthetic segment emitted when a whiteboard mutation runs while the board is
|
||||
* closed. The app's `ActionEngine.execute()` awaits `ensureWhiteboardOpen()`
|
||||
* (a `WB_OPEN_MS` open animation) before any `wb_*` action other than
|
||||
* `wb_open`/`wb_close`, so the exporter must lay down that same beat or every
|
||||
* later segment starts `WB_OPEN_MS` too early. Distinct id so consumers can
|
||||
* tell it apart from an authored `wb_open`.
|
||||
*/
|
||||
export const IMPLICIT_WB_OPEN: Action = {
|
||||
id: '__implicit_wb_open__',
|
||||
type: 'wb_open',
|
||||
} as Action;
|
||||
|
||||
/** Whiteboard mutations that trigger an implicit auto-open when the board is closed. */
|
||||
function isImplicitOpenTrigger(type: string): boolean {
|
||||
return type.startsWith('wb_') && type !== 'wb_open' && type !== 'wb_close';
|
||||
}
|
||||
|
||||
export interface ResolveTimelineOptions {
|
||||
/** Playback speed multiplier applied to speech dwell (estimate and real audio alike). Default 1. */
|
||||
playbackSpeed?: number;
|
||||
/**
|
||||
* Real narration duration (ms) for a speech action when pre-generated audio
|
||||
* exists. Return `null`/`undefined` to fall back to the deterministic
|
||||
* {@link estimateSpeechDurationMs}. The exporter, which knows each clip's
|
||||
* stored audio duration (issue #861), supplies this. The returned length is
|
||||
* the clip's natural (1×) duration; the timeline divides it by `playbackSpeed`
|
||||
* to match the live `AudioPlayer.setPlaybackRate` path.
|
||||
*/
|
||||
getAudioDurationMs?: (action: SpeechAction) => number | null | undefined;
|
||||
/**
|
||||
* Real video duration (ms) for a play_video action. Return `null`/`undefined`
|
||||
* when unknown — the {@link ResolveTimelineOptions.onUnresolvedVideoDuration}
|
||||
* policy then decides. A resolved value is capped at {@link MAX_VIDEO_WAIT_MS}.
|
||||
*/
|
||||
getVideoDurationMs?: (action: PlayVideoAction) => number | null | undefined;
|
||||
/**
|
||||
* What to do when a `play_video` duration is unresolved (no `getVideoDurationMs`,
|
||||
* or it returned nullish). `play_video` blocks live playback until the video
|
||||
* ends, so a silent 0 would shift every later action early — hence the default
|
||||
* is `'throw'` (fail loudly). `'cap'` assumes the {@link MAX_VIDEO_WAIT_MS}
|
||||
* safety cap; `'zero'` opts back into no-dwell explicitly.
|
||||
*/
|
||||
onUnresolvedVideoDuration?: 'throw' | 'cap' | 'zero';
|
||||
/**
|
||||
* Live whiteboard element count when a wb_clear runs (the clear animation
|
||||
* scales with it). Defaults to 0 when not supplied, which yields a 0ms dwell
|
||||
* (an empty clear is a no-op in the engine); the exporter, which replays
|
||||
* whiteboard state, can provide the true count.
|
||||
*/
|
||||
getClearElementCount?: (action: WbClearAction) => number;
|
||||
/**
|
||||
* Whether a discussion action is skipped outright by the engine (already
|
||||
* consumed, or its `agentId` isn't in the selected set) — in which case it
|
||||
* contributes no dwell. Depends on runtime state the pure timeline can't see,
|
||||
* so the caller supplies it; defaults to "not skipped"
|
||||
* ({@link DISCUSSION_TRIGGER_DELAY_MS}).
|
||||
*/
|
||||
isDiscussionSkipped?: (action: DiscussionAction) => boolean;
|
||||
/**
|
||||
* Whether a `wb_edit_code` action is a no-op the engine skips without delay
|
||||
* (target block missing / not a code element / stale line refs). Depends on
|
||||
* live whiteboard state, so the caller supplies it; defaults to a normal edit
|
||||
* ({@link WB_EDIT_MS}).
|
||||
*/
|
||||
isEditCodeNoop?: (action: WbEditCodeAction) => boolean;
|
||||
/**
|
||||
* Whether the whiteboard is already open when the timeline starts. Defaults to
|
||||
* `false`, matching the engine's post-`resetPlaybackVisualState()` state, so
|
||||
* the first whiteboard mutation triggers an implicit {@link IMPLICIT_WB_OPEN}
|
||||
* beat.
|
||||
*/
|
||||
whiteboardOpen?: boolean;
|
||||
}
|
||||
|
||||
export interface TimelineSegment {
|
||||
action: Action;
|
||||
sceneId: string;
|
||||
sceneIndex: number;
|
||||
actionIndex: number;
|
||||
/** Wall-clock start (ms) relative to the start of playback. */
|
||||
startMs: number;
|
||||
/** How long the action is visually present (ms). */
|
||||
durationMs: number;
|
||||
/**
|
||||
* How much the playback cursor advances (ms) before the next action starts.
|
||||
* Equal to `durationMs` for blocking actions, `0` for fire-and-forget.
|
||||
*/
|
||||
advancesCursorMs: number;
|
||||
/** Whether the action blocks the cursor (false only for fire-and-forget). */
|
||||
blocking: boolean;
|
||||
}
|
||||
|
||||
/** Line count of a code block, matching the app's `code.split('\n')` typing anim. */
|
||||
function codeLineCount(code: string): number {
|
||||
return code.split('\n').length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a `play_video` action's blocking duration. Live playback treats
|
||||
* `play_video` as blocking — it waits until the video ends or {@link MAX_VIDEO_WAIT_MS}
|
||||
* fires — so an unknown duration must NOT silently become a zero-length segment
|
||||
* (that would shift every later action early). When `getVideoDurationMs` returns
|
||||
* a value it's capped at the wait limit; when it's missing/unresolved the
|
||||
* {@link ResolveTimelineOptions.onUnresolvedVideoDuration} policy decides:
|
||||
* `'throw'` (default — fail loudly), `'cap'` (assume the max wait), or `'zero'`
|
||||
* (opt in to the old no-dwell behavior explicitly).
|
||||
*/
|
||||
function resolveVideoDurationMs(action: PlayVideoAction, opts: ResolveTimelineOptions): number {
|
||||
const resolved = opts.getVideoDurationMs?.(action);
|
||||
if (resolved != null) return Math.min(resolved, MAX_VIDEO_WAIT_MS);
|
||||
|
||||
switch (opts.onUnresolvedVideoDuration ?? 'throw') {
|
||||
case 'zero':
|
||||
return 0;
|
||||
case 'cap':
|
||||
return MAX_VIDEO_WAIT_MS;
|
||||
case 'throw':
|
||||
default:
|
||||
throw new Error(
|
||||
`resolveActionTimeline: play_video "${action.elementId}" has no resolved duration. ` +
|
||||
`play_video is blocking, so a missing duration would silently shift later actions ` +
|
||||
`early. Supply getVideoDurationMs, or set onUnresolvedVideoDuration to 'cap' or 'zero'.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The visual duration (ms) of a single action — how long it is present on
|
||||
* screen. For blocking actions this is also how long the cursor waits.
|
||||
*/
|
||||
function actionDurationMs(action: Action, opts: ResolveTimelineOptions): number {
|
||||
switch (action.type) {
|
||||
case 'speech': {
|
||||
const speed = opts.playbackSpeed ?? 1;
|
||||
// The live path plays pre-generated audio at `AudioPlayer.setPlaybackRate(speed)`,
|
||||
// so a stored clip's wall-clock dwell is its length divided by speed — the
|
||||
// same scaling the no-audio estimate applies. Keeping the two paths in
|
||||
// lockstep is what stops non-1× exports from drifting.
|
||||
const audio = opts.getAudioDurationMs?.(action);
|
||||
if (audio != null) return audio / speed;
|
||||
return estimateSpeechDurationMs(action.text, { speed });
|
||||
}
|
||||
case 'spotlight':
|
||||
case 'laser':
|
||||
return EFFECT_AUTO_CLEAR_MS;
|
||||
case 'discussion': {
|
||||
// A discussion the engine skips outright — already consumed, or its agent
|
||||
// isn't selected — contributes no dwell (`processNext` recurses with no
|
||||
// timer). That skip depends on runtime state (consumed set / selected
|
||||
// agents), so the caller signals it via `isDiscussionSkipped`.
|
||||
if (opts.isDiscussionSkipped?.(action as DiscussionAction)) return 0;
|
||||
// Otherwise: the trigger delay before the ProactiveCard shows, then — in
|
||||
// unattended playback/export — the card's own auto-skip countdown before
|
||||
// playback continues. (An attended viewer joining/skipping the card early
|
||||
// is interactive and out of scope; this models the deterministic
|
||||
// no-interaction dwell.)
|
||||
return DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS;
|
||||
}
|
||||
case 'play_video':
|
||||
return resolveVideoDurationMs(action as PlayVideoAction, opts);
|
||||
case 'wb_open':
|
||||
return WB_OPEN_MS;
|
||||
case 'wb_draw_text': {
|
||||
// The engine no-ops (no delay) when there's nothing to draw:
|
||||
// `executeWbDrawText` returns early on empty content.
|
||||
const content = (action as WbDrawTextAction).content ?? '';
|
||||
return content ? WB_DRAW_MS : 0;
|
||||
}
|
||||
case 'wb_draw_table': {
|
||||
// `executeWbDrawTable` returns early (no delay) when the table has no rows
|
||||
// or no columns.
|
||||
const data = (action as WbDrawTableAction).data;
|
||||
const rows = data?.length ?? 0;
|
||||
const cols = rows > 0 ? (data[0]?.length ?? 0) : 0;
|
||||
return rows === 0 || cols === 0 ? 0 : WB_DRAW_MS;
|
||||
}
|
||||
case 'wb_draw_shape':
|
||||
case 'wb_draw_chart':
|
||||
case 'wb_draw_latex':
|
||||
case 'wb_draw_line':
|
||||
return WB_DRAW_MS;
|
||||
case 'wb_draw_code':
|
||||
return wbDrawCodeMs(codeLineCount((action as WbDrawCodeAction).code));
|
||||
case 'wb_edit_code':
|
||||
// `executeWbEditCode` returns before its delay when the edit can't apply
|
||||
// (missing/non-code target, or stale lineId/lineIds). That depends on live
|
||||
// whiteboard state the pure timeline can't see, so the caller signals a
|
||||
// no-op via `isEditCodeNoop`; otherwise it's the normal edit animation.
|
||||
return opts.isEditCodeNoop?.(action as WbEditCodeAction) ? 0 : WB_EDIT_MS;
|
||||
case 'wb_clear': {
|
||||
// The engine early-returns with no delay when the board is already empty
|
||||
// (`executeWbClear`: elementCount === 0 → return), so an empty clear has a
|
||||
// 0ms dwell, not the `wbClearMs(0)` animation floor.
|
||||
const count = opts.getClearElementCount?.(action as WbClearAction) ?? 0;
|
||||
return count === 0 ? 0 : wbClearMs(count);
|
||||
}
|
||||
case 'wb_delete':
|
||||
return WB_DELETE_MS;
|
||||
case 'wb_close':
|
||||
return WB_CLOSE_MS;
|
||||
case 'widget_highlight':
|
||||
case 'widget_setState':
|
||||
case 'widget_annotation':
|
||||
case 'widget_reveal':
|
||||
return WIDGET_MS;
|
||||
default:
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Expand a scene list into an ordered wall-clock timeline. Scenes and their
|
||||
* actions are visited in order; a scene with no actions yields one
|
||||
* {@link EMPTY_SCENE_DWELL} beat (a blank speech clip's dwell) so it still
|
||||
* shows, mirroring {@link resolvePlaybackCursor}.
|
||||
*
|
||||
* Whiteboard auto-open is modeled: a `wb_*` mutation (draw/edit/clear/delete)
|
||||
* that runs while the board is closed is preceded by a synthetic
|
||||
* {@link IMPLICIT_WB_OPEN} beat ({@link WB_OPEN_MS}), exactly as the engine's
|
||||
* `ensureWhiteboardOpen` does. The open state carries across scenes and is
|
||||
* toggled by `wb_open` / `wb_close`; seed it via {@link ResolveTimelineOptions.whiteboardOpen}.
|
||||
*
|
||||
* @returns segments in play order, each stamped with `startMs`, its visual
|
||||
* `durationMs`, and how far it `advancesCursorMs`.
|
||||
*
|
||||
* Typed against {@link SceneCore} (only `id` + `actions` are read), so an
|
||||
* app-widened `Scene` (extra content kinds) is accepted without casting.
|
||||
*/
|
||||
export function resolveActionTimeline(
|
||||
scenes: SceneCore[],
|
||||
opts: ResolveTimelineOptions = {},
|
||||
): TimelineSegment[] {
|
||||
const segments: TimelineSegment[] = [];
|
||||
let clockMs = 0;
|
||||
// Mirrors the engine's whiteboard-open flag: false after
|
||||
// resetPlaybackVisualState(), flipped by wb_open / wb_close, and read to
|
||||
// decide whether a wb_* mutation must first pay the implicit open animation.
|
||||
let whiteboardOpen = opts.whiteboardOpen ?? false;
|
||||
|
||||
const push = (action: Action, sceneId: string, sceneIndex: number, actionIndex: number) => {
|
||||
const durationMs = actionDurationMs(action, opts);
|
||||
const blocking = !FIRE_AND_FORGET.has(action.type);
|
||||
const advancesCursorMs = blocking ? durationMs : 0;
|
||||
segments.push({
|
||||
action,
|
||||
sceneId,
|
||||
sceneIndex,
|
||||
actionIndex,
|
||||
startMs: clockMs,
|
||||
durationMs,
|
||||
advancesCursorMs,
|
||||
blocking,
|
||||
});
|
||||
clockMs += advancesCursorMs;
|
||||
};
|
||||
|
||||
const pushWithWhiteboard = (
|
||||
action: Action,
|
||||
sceneId: string,
|
||||
sceneIndex: number,
|
||||
actionIndex: number,
|
||||
) => {
|
||||
// A wb_* mutation on a closed board is auto-preceded by an open animation
|
||||
// (engine: `execute` awaits `ensureWhiteboardOpen`). Emit that beat first so
|
||||
// later segments don't start WB_OPEN_MS early.
|
||||
if (!whiteboardOpen && isImplicitOpenTrigger(action.type)) {
|
||||
push(IMPLICIT_WB_OPEN, sceneId, sceneIndex, actionIndex);
|
||||
whiteboardOpen = true;
|
||||
}
|
||||
push(action, sceneId, sceneIndex, actionIndex);
|
||||
if (action.type === 'wb_open') whiteboardOpen = true;
|
||||
else if (action.type === 'wb_close') whiteboardOpen = false;
|
||||
};
|
||||
|
||||
scenes.forEach((scene, sceneIndex) => {
|
||||
const actions = scene.actions ?? [];
|
||||
if (actions.length === 0) {
|
||||
// Empty scene → one synthetic dwell beat, exactly as the cursor yields.
|
||||
push(EMPTY_SCENE_DWELL, scene.id, sceneIndex, 0);
|
||||
return;
|
||||
}
|
||||
actions.forEach((action, actionIndex) => {
|
||||
pushWithWhiteboard(action, scene.id, sceneIndex, actionIndex);
|
||||
});
|
||||
});
|
||||
|
||||
clampFireAndForgetLifetimes(segments, clockMs);
|
||||
|
||||
return segments;
|
||||
}
|
||||
|
||||
/**
|
||||
* Correct the visual `durationMs` of fire-and-forget effects (spotlight/laser)
|
||||
* in place. Their nominal lifetime is {@link EFFECT_AUTO_CLEAR_MS}, but the
|
||||
* engine cuts it short — and occasionally extends it:
|
||||
*
|
||||
* - **Scene boundary / completion.** The app plays one `PlaybackEngine` per
|
||||
* scene: switching scenes tears the engine down (`stop()` → `clearEffects()`)
|
||||
* and each scene ends via the completion path (`getCurrentAction()` returns
|
||||
* null → `clearEffects()`). Either way effects never outlive their scene, so
|
||||
* in this continuous clock a spotlight late in a scene (or the final action of
|
||||
* the lecture) is cleared at the next scene's start / at completion, not a
|
||||
* flat 5s later.
|
||||
* - **Shared auto-clear timer.** `ActionEngine.scheduleEffectClear` uses one
|
||||
* timer that each new effect *resets*, and `clearAllEffects` drops every active
|
||||
* effect together. So back-to-back effects (within `EFFECT_AUTO_CLEAR_MS` of
|
||||
* each other) all live until the last one's fire + `EFFECT_AUTO_CLEAR_MS` —
|
||||
* the earlier effect is *extended*, not cleared on its own schedule.
|
||||
*
|
||||
* `advancesCursorMs` (0 for these) is untouched — only the visual hint changes.
|
||||
*
|
||||
* @param completionMs the final cursor clock (when the last scene's actions are
|
||||
* exhausted and playback completes → `clearEffects`).
|
||||
*/
|
||||
function clampFireAndForgetLifetimes(segments: TimelineSegment[], completionMs: number): void {
|
||||
// First segment startMs per scene index → the wall-clock of that scene's
|
||||
// boundary clearEffects. Every scene yields ≥1 segment, so all indices exist.
|
||||
const sceneStartMs = new Map<number, number>();
|
||||
for (const seg of segments) {
|
||||
if (!sceneStartMs.has(seg.sceneIndex)) sceneStartMs.set(seg.sceneIndex, seg.startMs);
|
||||
}
|
||||
|
||||
segments.forEach((seg, i) => {
|
||||
if (seg.blocking) return; // only fire-and-forget effects have a clearable lifetime
|
||||
|
||||
// The next clearEffects boundary after this effect: the start of the next
|
||||
// scene, or completion if this is the last scene.
|
||||
const nextSceneStart = sceneStartMs.get(seg.sceneIndex + 1);
|
||||
const boundaryMs = nextSceneStart ?? completionMs;
|
||||
|
||||
// Shared auto-clear deadline, chained through any later effects in THIS scene
|
||||
// that fire before it elapses (each resets the shared timer). Segments are in
|
||||
// non-decreasing startMs order, so a forward walk suffices.
|
||||
let deadlineMs = seg.startMs + EFFECT_AUTO_CLEAR_MS;
|
||||
for (let j = i + 1; j < segments.length; j++) {
|
||||
const other = segments[j];
|
||||
if (other.sceneIndex !== seg.sceneIndex) break; // cleared at the boundary anyway
|
||||
if (other.blocking) continue; // blocking actions don't touch the effect timer
|
||||
// Chain breaks at exact equality too: the earlier effect's clear timer was
|
||||
// registered first (same 5000ms delay as the reading timer that triggers
|
||||
// the later effect), so it fires before the later effect resets it. Its
|
||||
// predecessor is therefore cleared at exactly `deadlineMs`, not extended.
|
||||
if (other.startMs >= deadlineMs) break; // timer already fired; chain broken
|
||||
deadlineMs = Math.max(deadlineMs, other.startMs + EFFECT_AUTO_CLEAR_MS);
|
||||
}
|
||||
|
||||
const clearMs = Math.min(boundaryMs, deadlineMs);
|
||||
seg.durationMs = Math.max(0, clearMs - seg.startMs);
|
||||
});
|
||||
}
|
||||
121
lib/choreography/timing.ts
Normal file
121
lib/choreography/timing.ts
Normal file
@ -0,0 +1,121 @@
|
||||
/**
|
||||
* Timing spec — the single source of truth for playback timing.
|
||||
*
|
||||
* These constants and the no-audio narration estimate were previously inlined
|
||||
* in the app engines (`lib/action/engine.ts`, `lib/playback/engine.ts`). They
|
||||
* are moved here verbatim so the app runtime and the video exporter interpret
|
||||
* the same numbers: if either side re-implemented them, the exported video
|
||||
* would silently drift whenever the app is tuned. The literals below are the
|
||||
* behavior-defining values — change them here and both consumers follow.
|
||||
*
|
||||
* Pure: depends on nothing (no React / DOM / render backend), so it can be
|
||||
* interpreted in a pure Node environment by the exporter.
|
||||
*/
|
||||
|
||||
// ==================== Effect / scene timing ====================
|
||||
|
||||
/** Duration (ms) a fire-and-forget effect (spotlight/laser) stays before auto-clearing. */
|
||||
export const EFFECT_AUTO_CLEAR_MS = 5000;
|
||||
|
||||
/** Delay (ms) before a discussion trigger shows its ProactiveCard (lets prior speech finish). */
|
||||
export const DISCUSSION_TRIGGER_DELAY_MS = 3000;
|
||||
|
||||
/**
|
||||
* Duration (ms) the ProactiveCard counts down in playback mode before it
|
||||
* auto-skips and playback continues. In unattended playback / export a
|
||||
* non-skipped discussion therefore blocks for
|
||||
* `DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS`. The `ProactiveCard`
|
||||
* component reads this same constant, so the card countdown and the timeline
|
||||
* can't drift.
|
||||
*/
|
||||
export const DISCUSSION_AUTO_SKIP_MS = 5000;
|
||||
|
||||
/** Safety cap (ms) on how long playback waits for a video to finish. */
|
||||
export const MAX_VIDEO_WAIT_MS = 5 * 60 * 1000;
|
||||
|
||||
// ==================== Whiteboard / widget action durations ====================
|
||||
//
|
||||
// The blocking "visual duration" of each synchronous action — how long the
|
||||
// engine awaits before advancing to the next action. Lifted verbatim from the
|
||||
// `delay(...)` call sites in `lib/action/engine.ts`.
|
||||
|
||||
/** wb_open — open animation (slow spring). */
|
||||
export const WB_OPEN_MS = 2000;
|
||||
|
||||
/** wb_draw_* (text/shape/chart/latex/table/line) — element fade-in. */
|
||||
export const WB_DRAW_MS = 800;
|
||||
|
||||
/** wb_edit_code — line-level edit animation. */
|
||||
export const WB_EDIT_MS = 600;
|
||||
|
||||
/** wb_delete — element removal. */
|
||||
export const WB_DELETE_MS = 300;
|
||||
|
||||
/** wb_close — close animation (ease-out tween). */
|
||||
export const WB_CLOSE_MS = 700;
|
||||
|
||||
/** widget_* interactions (highlight/setState/annotation/reveal). */
|
||||
export const WIDGET_MS = 300;
|
||||
|
||||
/**
|
||||
* wb_draw_code — typing animation: base 800ms + 50ms/line, capped at 3000ms.
|
||||
* Verbatim from `lib/action/engine.ts` (`Math.min(800 + lines.length * 50, 3000)`).
|
||||
*/
|
||||
export function wbDrawCodeMs(lineCount: number): number {
|
||||
return Math.min(800 + lineCount * 50, 3000);
|
||||
}
|
||||
|
||||
/**
|
||||
* wb_clear — clear animation: base 380ms + 55ms/element, capped at 1400ms.
|
||||
* Verbatim from `lib/action/engine.ts` (`Math.min(380 + elementCount * 55, 1400)`).
|
||||
*/
|
||||
export function wbClearMs(elementCount: number): number {
|
||||
return Math.min(380 + elementCount * 55, 1400);
|
||||
}
|
||||
|
||||
// ==================== No-audio narration estimate ====================
|
||||
|
||||
/**
|
||||
* Characters counted as CJK for the reading-timer heuristic: CJK Unified
|
||||
* Ideographs (+ Ext-A), Hiragana, Katakana, Hangul Syllables. Verbatim from the
|
||||
* regex in `lib/playback/engine.ts`'s `scheduleReadingTimer`.
|
||||
*/
|
||||
const CJK_REGEX = /[一-鿿㐀-䶿-ゟ゠-ヿ가-]/g;
|
||||
|
||||
/** Text is treated as CJK when this fraction of its characters are CJK. */
|
||||
const CJK_RATIO_THRESHOLD = 0.3;
|
||||
|
||||
/** Minimum estimated reading time (ms), regardless of length. */
|
||||
const MIN_READING_MS = 2000;
|
||||
|
||||
/** CJK narration pace: ms per character (one char ≈ one word). */
|
||||
const CJK_MS_PER_CHAR = 150;
|
||||
|
||||
/** Non-CJK narration pace: ms per word (≈250 WPM). */
|
||||
const NON_CJK_MS_PER_WORD = 240;
|
||||
|
||||
export interface SpeechEstimateOptions {
|
||||
/** Playback speed multiplier; the estimate is divided by it. Default 1. */
|
||||
speed?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Estimate how long narration takes to "speak" when no pre-generated audio is
|
||||
* available (TTS disabled). Deterministic — the exporter and the app's
|
||||
* reading-timer must agree on the dwell of an audio-less speech clip.
|
||||
*
|
||||
* - CJK text (>30% CJK chars): ~150ms/char.
|
||||
* - Non-CJK text: ~240ms/word (≈250 WPM).
|
||||
* - Floored at 2000ms, then divided by playback speed.
|
||||
*
|
||||
* Moved verbatim from `lib/playback/engine.ts`'s `scheduleReadingTimer`.
|
||||
*/
|
||||
export function estimateSpeechDurationMs(text: string, opts?: SpeechEstimateOptions): number {
|
||||
const speed = opts?.speed ?? 1;
|
||||
const cjkCount = (text.match(CJK_REGEX) || []).length;
|
||||
const isCJK = cjkCount > text.length * CJK_RATIO_THRESHOLD;
|
||||
const rawMs = isCJK
|
||||
? Math.max(MIN_READING_MS, text.length * CJK_MS_PER_CHAR)
|
||||
: Math.max(MIN_READING_MS, text.split(/\s+/).filter(Boolean).length * NON_CJK_MS_PER_WORD);
|
||||
return rawMs / speed;
|
||||
}
|
||||
@ -35,7 +35,11 @@ import type {
|
||||
} from './types';
|
||||
import type { AudioPlayer } from '@/lib/utils/audio-player';
|
||||
import { ActionEngine } from '@/lib/action/engine';
|
||||
import { resolvePlaybackCursor } from './engine-cursor';
|
||||
import {
|
||||
resolvePlaybackCursor,
|
||||
estimateSpeechDurationMs,
|
||||
DISCUSSION_TRIGGER_DELAY_MS,
|
||||
} from '@/lib/choreography';
|
||||
import {
|
||||
canJumpWithinReconstructablePrefix,
|
||||
isWhiteboardPlaybackAction,
|
||||
@ -564,21 +568,13 @@ export class PlaybackEngine {
|
||||
});
|
||||
|
||||
// Estimated reading time when no pre-generated audio (TTS disabled).
|
||||
// CJK text: ~150ms/char (one char ≈ one word).
|
||||
// Non-CJK text: ~240ms/word (≈250 WPM).
|
||||
// Min 2s. Cancelled on pause; resume() calls processNext directly.
|
||||
// The estimate (CJK vs word-based pace, 2s floor, speed-adjusted) lives
|
||||
// in @/lib/choreography so the video exporter dwells identically.
|
||||
// Cancelled on pause; resume() calls processNext directly.
|
||||
const scheduleReadingTimer = () => {
|
||||
if (!this.isCurrentGeneration(generation)) return;
|
||||
const text = speechAction.text;
|
||||
const cjkCount = (
|
||||
text.match(/[\u4e00-\u9fff\u3400-\u4dbf\u3040-\u309f\u30a0-\u30ff\uac00-\ud7af]/g) || []
|
||||
).length;
|
||||
const isCJK = cjkCount > text.length * 0.3;
|
||||
const speed = this.callbacks.getPlaybackSpeed?.() ?? 1;
|
||||
const rawMs = isCJK
|
||||
? Math.max(2000, text.length * 150)
|
||||
: Math.max(2000, text.split(/\s+/).filter(Boolean).length * 240);
|
||||
const readingMs = rawMs / speed;
|
||||
const readingMs = estimateSpeechDurationMs(speechAction.text, { speed });
|
||||
this.speechTimerStart = Date.now();
|
||||
this.speechTimerRemaining = readingMs;
|
||||
this.speechTimer = setTimeout(() => {
|
||||
@ -685,7 +681,7 @@ export class PlaybackEngine {
|
||||
this.currentTrigger = trigger;
|
||||
this.callbacks.onProactiveShow?.(trigger);
|
||||
// Engine pauses here — user calls confirmDiscussion() or skipDiscussion()
|
||||
}, 3000);
|
||||
}, DISCUSSION_TRIGGER_DELAY_MS);
|
||||
break;
|
||||
}
|
||||
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
import { describe, expect, test } from 'vitest';
|
||||
import { resolvePlaybackCursor, EMPTY_SCENE_DWELL } from '@/lib/playback/engine-cursor';
|
||||
import { resolvePlaybackCursor, EMPTY_SCENE_DWELL } from '@/lib/choreography';
|
||||
import type { Action } from '@/lib/types/action';
|
||||
import type { Scene } from '@/lib/types/stage';
|
||||
|
||||
163
tests/lib/choreography/descriptors.test.ts
Normal file
163
tests/lib/choreography/descriptors.test.ts
Normal file
@ -0,0 +1,163 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
DESCRIPTORS,
|
||||
spotlightV1,
|
||||
laserV1,
|
||||
getDescriptor,
|
||||
AnimationDescriptorSchema,
|
||||
} from '@/lib/choreography';
|
||||
|
||||
describe('animation descriptor registry', () => {
|
||||
it('registers spotlight.v1 and laser.v1 under their versioned ids', () => {
|
||||
expect(Object.keys(DESCRIPTORS).sort()).toEqual(['laser.v1', 'spotlight.v1']);
|
||||
expect(getDescriptor('spotlight.v1')).toBe(spotlightV1);
|
||||
expect(getDescriptor('laser.v1')).toBe(laserV1);
|
||||
expect(getDescriptor('nope.v1')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ids and versions are consistent', () => {
|
||||
for (const [key, d] of Object.entries(DESCRIPTORS)) {
|
||||
expect(d.id).toBe(key);
|
||||
expect(d.version).toBe(1);
|
||||
expect(key.endsWith(`.v${d.version}`)).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('every shipped descriptor conforms to the zod schema', () => {
|
||||
for (const [key, d] of Object.entries(DESCRIPTORS)) {
|
||||
it(`${key} validates`, () => {
|
||||
expect(() => AnimationDescriptorSchema.parse(d)).not.toThrow();
|
||||
});
|
||||
}
|
||||
|
||||
it('rejects a descriptor missing required fields', () => {
|
||||
expect(AnimationDescriptorSchema.safeParse({ id: 'x.v1', version: 1 }).success).toBe(false);
|
||||
});
|
||||
|
||||
it('rejects an unknown easing type', () => {
|
||||
const bad = {
|
||||
...spotlightV1,
|
||||
layers: [
|
||||
{
|
||||
id: 'l',
|
||||
tracks: [
|
||||
{
|
||||
property: 'x',
|
||||
from: 0,
|
||||
to: 1,
|
||||
durationMs: 100,
|
||||
easing: { type: 'bogus', points: [0, 0, 0, 0] },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
expect(AnimationDescriptorSchema.safeParse(bad).success).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('spotlight.v1 pins the source animation values', () => {
|
||||
it('has the dim/cutout/border layers and z-index 100', () => {
|
||||
expect(spotlightV1.zIndex).toBe(100);
|
||||
// 0.5 is the runtime default (executeSpotlight: action.dimOpacity ?? 0.5),
|
||||
// not the component's unreachable ?? 0.7 fallback.
|
||||
expect(spotlightV1.params).toMatchObject({ dimness: 0.5 });
|
||||
expect(spotlightV1.layers.map((l) => l.id).sort()).toEqual(['border', 'cutout', 'dim']);
|
||||
});
|
||||
|
||||
it('cutout uses the 600ms expo-out curve', () => {
|
||||
const cutout = spotlightV1.layers.find((l) => l.id === 'cutout')!;
|
||||
for (const t of cutout.tracks) {
|
||||
expect(t.durationMs).toBe(600);
|
||||
expect(t.easing).toEqual({ type: 'cubicBezier', points: [0.16, 1, 0.3, 1] });
|
||||
}
|
||||
});
|
||||
|
||||
it('border is 500ms, delayed 50ms', () => {
|
||||
const border = spotlightV1.layers.find((l) => l.id === 'border')!;
|
||||
for (const t of border.tracks) {
|
||||
expect(t.durationMs).toBe(500);
|
||||
expect(t.delayMs).toBe(50);
|
||||
}
|
||||
});
|
||||
|
||||
it('models the dim layer as the cutout subtracted (SVG mask), not an opaque black rect', () => {
|
||||
const cutout = spotlightV1.layers.find((l) => l.id === 'cutout')!;
|
||||
const dim = spotlightV1.layers.find((l) => l.id === 'dim')!;
|
||||
// The cutout supplies geometry only — it is not painted on its own.
|
||||
expect(cutout.role).toBe('mask');
|
||||
// The dim rect is clipped by subtracting the cutout, so a non-React consumer
|
||||
// reconstructs "dim everywhere except the cutout" rather than a black box.
|
||||
expect(dim.maskedBy).toEqual({ layerId: 'cutout', mode: 'subtract' });
|
||||
// It spans the whole 0..100 viewport — explicit so a literal consumer dims
|
||||
// the full frame, not a zero-size rect.
|
||||
expect(dim.staticProps).toMatchObject({ x: 0, y: 0, width: 100, height: 100 });
|
||||
});
|
||||
|
||||
it('models the wrapper enter/exit opacity fade (Motion default 300ms)', () => {
|
||||
const dim = spotlightV1.layers.find((l) => l.id === 'dim')!;
|
||||
const enter = dim.tracks.find((t) => t.property === 'opacity' && t.phase === 'enter')!;
|
||||
const exit = dim.tracks.find((t) => t.property === 'opacity' && t.phase === 'exit')!;
|
||||
// Pinned to Motion's default tween so a non-Motion consumer reproduces the
|
||||
// fade instead of popping the effect on/off.
|
||||
expect(enter).toMatchObject({ from: 0, to: 1, durationMs: 300 });
|
||||
expect(exit).toMatchObject({ from: 1, to: 0, durationMs: 300 });
|
||||
});
|
||||
|
||||
it('the border rides the wrapper fade (inherits the dim layer opacity)', () => {
|
||||
const border = spotlightV1.layers.find((l) => l.id === 'border')!;
|
||||
// The outline sits in the same fading wrapper as the dim rect, so it fades
|
||||
// out with the whole effect rather than lingering after the dimming clears.
|
||||
expect(border.inheritsFrom).toEqual({ parentId: 'dim', props: ['opacity'] });
|
||||
});
|
||||
});
|
||||
|
||||
describe('laser.v1 pins the source animation values', () => {
|
||||
it('has the dot/ring/core layers, z-index 101, red default', () => {
|
||||
expect(laserV1.zIndex).toBe(101);
|
||||
expect(laserV1.params).toMatchObject({ color: '#ff0000' });
|
||||
expect(laserV1.layers.map((l) => l.id).sort()).toEqual(['core', 'dot', 'ring']);
|
||||
});
|
||||
|
||||
it('the ring pulses infinitely, scale 1→2.8, 1500ms, 300ms repeat delay', () => {
|
||||
const ring = laserV1.layers.find((l) => l.id === 'ring')!;
|
||||
const scale = ring.tracks.find((t) => t.property === 'scale')!;
|
||||
expect(scale).toMatchObject({
|
||||
from: 1,
|
||||
to: 2.8,
|
||||
durationMs: 1500,
|
||||
repeat: 'infinite',
|
||||
repeatDelayMs: 300,
|
||||
});
|
||||
});
|
||||
|
||||
it('the dot fly-in is 500ms enter and 250ms exit', () => {
|
||||
const dot = laserV1.layers.find((l) => l.id === 'dot')!;
|
||||
const enterLeft = dot.tracks.find((t) => t.property === 'left' && t.phase === 'enter')!;
|
||||
const exitLeft = dot.tracks.find((t) => t.property === 'left' && t.phase === 'exit')!;
|
||||
expect(enterLeft.durationMs).toBe(500);
|
||||
expect(exitLeft.durationMs).toBe(250);
|
||||
});
|
||||
|
||||
it('captures the dot center-anchor and circular ring/core geometry', () => {
|
||||
const dot = laserV1.layers.find((l) => l.id === 'dot')!;
|
||||
const ring = laserV1.layers.find((l) => l.id === 'ring')!;
|
||||
const core = laserV1.layers.find((l) => l.id === 'core')!;
|
||||
// The dot group is centered on the target (translate -50%,-50%), so a
|
||||
// literal renderer doesn't offset it to a top-left anchor.
|
||||
expect(dot.staticProps).toMatchObject({ transform: 'translate(-50%, -50%)' });
|
||||
// Ring and core are circles (rounded-full → borderRadius 9999), not squares.
|
||||
expect(ring.staticProps).toMatchObject({ borderRadius: 9999, position: 'absolute', inset: 0 });
|
||||
expect(core.staticProps).toMatchObject({ borderRadius: 9999, width: 10, height: 10 });
|
||||
});
|
||||
|
||||
it('the ring and core ride the dot fly-in/exit (inherit its left/top/opacity)', () => {
|
||||
const ring = laserV1.layers.find((l) => l.id === 'ring')!;
|
||||
const core = laserV1.layers.find((l) => l.id === 'core')!;
|
||||
// Both are nested inside the animated dot wrapper in the source, so they
|
||||
// follow its motion instead of sitting at a static default origin.
|
||||
expect(ring.inheritsFrom).toEqual({ parentId: 'dot', props: ['left', 'top', 'opacity'] });
|
||||
expect(core.inheritsFrom).toEqual({ parentId: 'dot', props: ['left', 'top', 'opacity'] });
|
||||
});
|
||||
});
|
||||
383
tests/lib/choreography/timeline.test.ts
Normal file
383
tests/lib/choreography/timeline.test.ts
Normal file
@ -0,0 +1,383 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
resolveActionTimeline,
|
||||
IMPLICIT_WB_OPEN,
|
||||
EFFECT_AUTO_CLEAR_MS,
|
||||
DISCUSSION_TRIGGER_DELAY_MS,
|
||||
DISCUSSION_AUTO_SKIP_MS,
|
||||
WB_OPEN_MS,
|
||||
WB_DRAW_MS,
|
||||
WB_EDIT_MS,
|
||||
wbDrawCodeMs,
|
||||
wbClearMs,
|
||||
estimateSpeechDurationMs,
|
||||
} from '@/lib/choreography';
|
||||
import type { Action } from '@/lib/types/action';
|
||||
import type { Scene } from '@/lib/types/stage';
|
||||
|
||||
const act = (a: Partial<Action> & { type: string }): Action => a as unknown as Action;
|
||||
const speech = (id: string, text: string): Action => act({ id, type: 'speech', text });
|
||||
const sc = (id: string, actions: Action[]): Scene =>
|
||||
({
|
||||
id,
|
||||
stageId: 's',
|
||||
type: 'slide',
|
||||
title: id,
|
||||
order: 1,
|
||||
content: { type: 'slide', canvas: {} },
|
||||
actions,
|
||||
}) as unknown as Scene;
|
||||
|
||||
describe('resolveActionTimeline — blocking actions advance the cursor', () => {
|
||||
it('speech falls back to the deterministic estimate and accumulates startMs', () => {
|
||||
const scenes = [sc('S0', [speech('a', '中'.repeat(20)), speech('b', 'hello world')])];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
|
||||
const d0 = estimateSpeechDurationMs('中'.repeat(20)); // 3000
|
||||
const d1 = estimateSpeechDurationMs('hello world'); // 2000 floor
|
||||
|
||||
expect(tl).toHaveLength(2);
|
||||
expect(tl[0]).toMatchObject({
|
||||
sceneId: 'S0',
|
||||
sceneIndex: 0,
|
||||
actionIndex: 0,
|
||||
startMs: 0,
|
||||
durationMs: d0,
|
||||
advancesCursorMs: d0,
|
||||
blocking: true,
|
||||
});
|
||||
expect(tl[1]).toMatchObject({ startMs: d0, durationMs: d1, advancesCursorMs: d1 });
|
||||
});
|
||||
|
||||
it('a supplied audio-duration resolver overrides the estimate', () => {
|
||||
const scenes = [sc('S0', [speech('a', 'anything')])];
|
||||
const tl = resolveActionTimeline(scenes, { getAudioDurationMs: () => 7777 });
|
||||
expect(tl[0]).toMatchObject({ durationMs: 7777, advancesCursorMs: 7777 });
|
||||
});
|
||||
|
||||
it('playback speed scales both the estimate and supplied audio', () => {
|
||||
const scenes = [sc('S0', [speech('a', '中'.repeat(20))])];
|
||||
const tl = resolveActionTimeline(scenes, { playbackSpeed: 2 });
|
||||
expect(tl[0].durationMs).toBe(estimateSpeechDurationMs('中'.repeat(20)) / 2);
|
||||
|
||||
// Real audio is scaled the same way (the live path sets AudioPlayer playbackRate).
|
||||
const withAudio = resolveActionTimeline(scenes, {
|
||||
playbackSpeed: 2,
|
||||
getAudioDurationMs: () => 10_000,
|
||||
});
|
||||
expect(withAudio[0]).toMatchObject({ durationMs: 5000, advancesCursorMs: 5000 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveActionTimeline — fire-and-forget effects do not advance the cursor', () => {
|
||||
it('effects start at the cursor but advance it by 0 (blocking:false)', () => {
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
|
||||
act({ id: 'la', type: 'laser', elementId: 'e2' }),
|
||||
speech('s', 'hi there'),
|
||||
]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
|
||||
// Both effects and the speech start at 0 — the effects didn't move the clock.
|
||||
expect(tl[0]).toMatchObject({ startMs: 0, advancesCursorMs: 0, blocking: false });
|
||||
expect(tl[1]).toMatchObject({ startMs: 0, advancesCursorMs: 0, blocking: false });
|
||||
expect(tl[2]).toMatchObject({ startMs: 0, blocking: true });
|
||||
});
|
||||
|
||||
it('an effect is cleared at playback completion, not a flat 5s later', () => {
|
||||
// spotlight fires at 0; the only speech ends at 2000, then processNext hits
|
||||
// completion → clearEffects. So the effect lives 2000ms, not EFFECT_AUTO_CLEAR_MS.
|
||||
const scenes = [
|
||||
sc('S0', [act({ id: 'sp', type: 'spotlight', elementId: 'e1' }), speech('s', 'hi there')]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: 2000, blocking: false });
|
||||
});
|
||||
|
||||
it('an effect is cleared at the next scene boundary', () => {
|
||||
const scenes = [
|
||||
sc('S0', [act({ id: 'sp', type: 'spotlight', elementId: 'e1' }), speech('a', 'hi there')]), // 0..2000
|
||||
sc('S1', [speech('b', 'more')]),
|
||||
];
|
||||
// clearEffects fires at the start of S1 (2000), cutting the spotlight there.
|
||||
expect(resolveActionTimeline(scenes)[0].durationMs).toBe(2000);
|
||||
});
|
||||
|
||||
it('lives its full EFFECT_AUTO_CLEAR_MS when the scene runs long enough', () => {
|
||||
// speech dwell 6000 > 5000, so the auto-clear timer fires before the boundary.
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
|
||||
speech('s', '中'.repeat(40)),
|
||||
]),
|
||||
];
|
||||
expect(resolveActionTimeline(scenes)[0].durationMs).toBe(EFFECT_AUTO_CLEAR_MS);
|
||||
});
|
||||
|
||||
it('a later effect resets the shared clear timer, extending earlier effects', () => {
|
||||
// spotlight@0, then speech(3000), then laser@3000 (resets the shared timer),
|
||||
// then speech(6000) → completion 9000. The shared timer last fires at
|
||||
// 3000+5000=8000, so BOTH effects clear together at 8000: spotlight lives
|
||||
// 8000ms (extended past its own 5000), laser lives 5000ms.
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
|
||||
speech('a', '中'.repeat(20)), // 3000
|
||||
act({ id: 'la', type: 'laser', elementId: 'e2' }),
|
||||
speech('b', '中'.repeat(40)), // 6000
|
||||
]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: 8000, blocking: false }); // spotlight
|
||||
expect(tl[2]).toMatchObject({
|
||||
startMs: 3000,
|
||||
durationMs: EFFECT_AUTO_CLEAR_MS,
|
||||
blocking: false,
|
||||
}); // laser
|
||||
});
|
||||
|
||||
it('breaks the chain when the gap between effects exceeds EFFECT_AUTO_CLEAR_MS', () => {
|
||||
// spotlight@0; speech(6000) > 5000, so the shared timer for the spotlight
|
||||
// already fired at 5000 before the laser@6000 arrives. The laser starts a
|
||||
// fresh 5000 window. Long trailing speech keeps both within their own scene.
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
|
||||
speech('a', '中'.repeat(40)), // 6000
|
||||
act({ id: 'la', type: 'laser', elementId: 'e2' }),
|
||||
speech('b', '中'.repeat(40)), // 6000 → completion 12000
|
||||
]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: EFFECT_AUTO_CLEAR_MS }); // spotlight: own 5000
|
||||
expect(tl[2]).toMatchObject({ startMs: 6000, durationMs: EFFECT_AUTO_CLEAR_MS }); // laser: own 5000
|
||||
});
|
||||
|
||||
it('breaks the chain at an EXACT 5s boundary (earlier clear timer fires first)', () => {
|
||||
// spotlight@0; speech of exactly 5000ms; laser@5000. The spotlight's clear
|
||||
// timer was queued first (same 5000ms delay), so it fires before the laser
|
||||
// resets it — the spotlight lives exactly 5000, not 10000.
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'sp', type: 'spotlight', elementId: 'e1' }),
|
||||
speech('a', 'anything'), // 5000 via audio resolver
|
||||
act({ id: 'la', type: 'laser', elementId: 'e2' }),
|
||||
speech('b', 'anything'), // 5000 → completion 10000
|
||||
]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes, { getAudioDurationMs: () => 5000 });
|
||||
expect(tl[0]).toMatchObject({ startMs: 0, durationMs: EFFECT_AUTO_CLEAR_MS }); // spotlight: exactly 5000
|
||||
expect(tl[2]).toMatchObject({ startMs: 5000, durationMs: EFFECT_AUTO_CLEAR_MS }); // laser: own 5000
|
||||
});
|
||||
|
||||
it('a later scene never extends an earlier scene effect (boundary clears first)', () => {
|
||||
const scenes = [
|
||||
sc('S0', [act({ id: 'sp', type: 'spotlight', elementId: 'e1' }), speech('a', 'hi there')]), // 0..2000
|
||||
sc('S1', [act({ id: 'la', type: 'laser', elementId: 'e2' }), speech('b', 'more')]),
|
||||
];
|
||||
// S0's spotlight is cut at S1's start (2000), NOT extended by S1's laser@2000.
|
||||
expect(resolveActionTimeline(scenes)[0].durationMs).toBe(2000);
|
||||
});
|
||||
|
||||
it('an effect as the last action of a scene has 0ms visual duration', () => {
|
||||
// The engine fires the effect, queueMicrotask(processNext) hits completion,
|
||||
// and clearEffects runs before real time elapses → 0ms.
|
||||
const scenes = [
|
||||
sc('S0', [speech('a', 'hi there'), act({ id: 'sp', type: 'spotlight', elementId: 'e1' })]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl[1]).toMatchObject({ startMs: 2000, durationMs: 0, blocking: false });
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveActionTimeline — per-action durations', () => {
|
||||
it('wb_draw_code uses the line-count formula', () => {
|
||||
const code = 'a\nb\nc\nd'; // 4 lines
|
||||
const scenes = [sc('S0', [act({ id: 'c', type: 'wb_draw_code', code })])];
|
||||
const tl = resolveActionTimeline(scenes, { whiteboardOpen: true });
|
||||
expect(tl[0].durationMs).toBe(wbDrawCodeMs(4));
|
||||
});
|
||||
|
||||
it('wb_clear is 0ms when empty and scales with the supplied live element count', () => {
|
||||
const scenes = [
|
||||
sc('S0', [act({ id: 'o', type: 'wb_open' }), act({ id: 'cl', type: 'wb_clear' })]),
|
||||
];
|
||||
// Empty board → engine early-returns with no delay, so 0ms (not wbClearMs(0)).
|
||||
expect(resolveActionTimeline(scenes)[1].durationMs).toBe(0);
|
||||
expect(resolveActionTimeline(scenes, { getClearElementCount: () => 10 })[1].durationMs).toBe(
|
||||
wbClearMs(10),
|
||||
);
|
||||
});
|
||||
|
||||
it('wb_draw_* share WB_DRAW_MS', () => {
|
||||
const scenes = [sc('S0', [act({ id: 't', type: 'wb_draw_text', content: 'hi' })])];
|
||||
expect(resolveActionTimeline(scenes, { whiteboardOpen: true })[0].durationMs).toBe(WB_DRAW_MS);
|
||||
});
|
||||
|
||||
it('no-op whiteboard draws cost 0ms (engine early-returns)', () => {
|
||||
// wb_draw_text with empty content → executeWbDrawText returns before delay.
|
||||
const emptyText = [sc('S0', [act({ id: 't', type: 'wb_draw_text', content: '' })])];
|
||||
expect(resolveActionTimeline(emptyText, { whiteboardOpen: true })[0].durationMs).toBe(0);
|
||||
// wb_draw_table with no rows → executeWbDrawTable returns early.
|
||||
const emptyTable = [sc('S0', [act({ id: 'tb', type: 'wb_draw_table', data: [] })])];
|
||||
expect(resolveActionTimeline(emptyTable, { whiteboardOpen: true })[0].durationMs).toBe(0);
|
||||
// A populated table still costs WB_DRAW_MS.
|
||||
const table = [sc('S0', [act({ id: 'tb2', type: 'wb_draw_table', data: [['a', 'b']] })])];
|
||||
expect(resolveActionTimeline(table, { whiteboardOpen: true })[0].durationMs).toBe(WB_DRAW_MS);
|
||||
});
|
||||
|
||||
it('discussion dwells for trigger delay + card auto-skip (unattended playback)', () => {
|
||||
const scenes = [sc('S0', [act({ id: 'd', type: 'discussion', topic: 't' })])];
|
||||
// Not skipped: the 3s trigger delay, then the ProactiveCard's 5s auto-skip
|
||||
// countdown before playback continues.
|
||||
expect(resolveActionTimeline(scenes)[0]).toMatchObject({
|
||||
durationMs: DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS,
|
||||
blocking: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('a skipped discussion (consumed / agent not selected) contributes no dwell', () => {
|
||||
const scenes = [sc('S0', [act({ id: 'd', type: 'discussion', topic: 't' })])];
|
||||
// Engine skips it (processNext recurses with no timer) → 0ms.
|
||||
expect(resolveActionTimeline(scenes, { isDiscussionSkipped: () => true })[0].durationMs).toBe(
|
||||
0,
|
||||
);
|
||||
// Not skipped → trigger delay + card auto-skip, same as the default.
|
||||
expect(resolveActionTimeline(scenes, { isDiscussionSkipped: () => false })[0].durationMs).toBe(
|
||||
DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS,
|
||||
);
|
||||
});
|
||||
|
||||
it('wb_edit_code is WB_EDIT_MS by default and 0ms when the caller flags a no-op', () => {
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'e', type: 'wb_edit_code', elementId: 'c1', operation: 'delete_lines' }),
|
||||
]),
|
||||
];
|
||||
expect(resolveActionTimeline(scenes, { whiteboardOpen: true })[0].durationMs).toBe(WB_EDIT_MS);
|
||||
// Stale target / non-code element → executeWbEditCode returns before its delay.
|
||||
expect(
|
||||
resolveActionTimeline(scenes, { whiteboardOpen: true, isEditCodeNoop: () => true })[0]
|
||||
.durationMs,
|
||||
).toBe(0);
|
||||
});
|
||||
|
||||
it('play_video uses the supplied duration (capped); unresolved is explicit, not silent 0', () => {
|
||||
const scenes = [sc('S0', [act({ id: 'v', type: 'play_video', elementId: 'v1' })])];
|
||||
// Supplied duration is used, capped at MAX_VIDEO_WAIT_MS (5min).
|
||||
expect(resolveActionTimeline(scenes, { getVideoDurationMs: () => 12_345 })[0].durationMs).toBe(
|
||||
12_345,
|
||||
);
|
||||
expect(
|
||||
resolveActionTimeline(scenes, { getVideoDurationMs: () => 60 * 60 * 1000 })[0].durationMs,
|
||||
).toBe(5 * 60 * 1000);
|
||||
// Unresolved: default policy THROWS rather than silently shifting later actions early.
|
||||
expect(() => resolveActionTimeline(scenes)).toThrow(/play_video/);
|
||||
// Opt-in policies: 'cap' → max wait, 'zero' → explicit no-dwell.
|
||||
expect(resolveActionTimeline(scenes, { onUnresolvedVideoDuration: 'cap' })[0].durationMs).toBe(
|
||||
5 * 60 * 1000,
|
||||
);
|
||||
expect(resolveActionTimeline(scenes, { onUnresolvedVideoDuration: 'zero' })[0].durationMs).toBe(
|
||||
0,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveActionTimeline — scene boundaries', () => {
|
||||
it('an empty scene yields one EMPTY_SCENE_DWELL beat', () => {
|
||||
const scenes = [sc('S0', [])];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl).toHaveLength(1);
|
||||
expect(tl[0]).toMatchObject({
|
||||
sceneId: 'S0',
|
||||
actionIndex: 0,
|
||||
// empty-text speech → 2000ms floor
|
||||
durationMs: estimateSpeechDurationMs(''),
|
||||
blocking: true,
|
||||
});
|
||||
expect(tl[0].action).toMatchObject({ type: 'speech', text: '' });
|
||||
});
|
||||
|
||||
it('startMs accumulates across scenes in play order', () => {
|
||||
const scenes = [
|
||||
sc('S0', [speech('a', 'hello world')]), // 2000
|
||||
sc('S1', []), // empty dwell 2000
|
||||
sc('S2', [speech('c', '中'.repeat(20))]), // 3000
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl.map((s) => [s.sceneId, s.startMs])).toEqual([
|
||||
['S0', 0],
|
||||
['S1', 2000],
|
||||
['S2', 4000],
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveActionTimeline — implicit whiteboard auto-open', () => {
|
||||
it('prepends a WB_OPEN_MS beat before the first wb_* mutation on a closed board', () => {
|
||||
const scenes = [sc('S0', [act({ id: 'd', type: 'wb_draw_text', content: 'hi' })])];
|
||||
const tl = resolveActionTimeline(scenes); // whiteboardOpen defaults to false
|
||||
|
||||
expect(tl).toHaveLength(2);
|
||||
expect(tl[0]).toMatchObject({
|
||||
action: IMPLICIT_WB_OPEN,
|
||||
startMs: 0,
|
||||
durationMs: WB_OPEN_MS,
|
||||
advancesCursorMs: WB_OPEN_MS,
|
||||
blocking: true,
|
||||
});
|
||||
// The real draw starts after the open animation.
|
||||
expect(tl[1]).toMatchObject({ startMs: WB_OPEN_MS, durationMs: WB_DRAW_MS });
|
||||
});
|
||||
|
||||
it('does not prepend when the board is already open (seeded or via wb_open)', () => {
|
||||
const drawOnly = [sc('S0', [act({ id: 'd', type: 'wb_draw_text' })])];
|
||||
expect(resolveActionTimeline(drawOnly, { whiteboardOpen: true })).toHaveLength(1);
|
||||
|
||||
const explicitOpen = [
|
||||
sc('S0', [act({ id: 'o', type: 'wb_open' }), act({ id: 'd', type: 'wb_draw_text' })]),
|
||||
];
|
||||
const tl = resolveActionTimeline(explicitOpen);
|
||||
// Only the authored wb_open + the draw — no synthetic beat.
|
||||
expect(tl.map((s) => s.action.type)).toEqual(['wb_open', 'wb_draw_text']);
|
||||
expect(tl[0].action).not.toBe(IMPLICIT_WB_OPEN);
|
||||
});
|
||||
|
||||
it('re-opens after a wb_close (auto-open state carries across the sequence)', () => {
|
||||
const scenes = [
|
||||
sc('S0', [
|
||||
act({ id: 'd1', type: 'wb_draw_text' }), // implicit open here
|
||||
act({ id: 'x', type: 'wb_close' }),
|
||||
act({ id: 'd2', type: 'wb_draw_shape' }), // implicit open again
|
||||
]),
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl.map((s) => s.action.type)).toEqual([
|
||||
'wb_open', // implicit (before d1)
|
||||
'wb_draw_text',
|
||||
'wb_close',
|
||||
'wb_open', // implicit (before d2, board was closed again)
|
||||
'wb_draw_shape',
|
||||
]);
|
||||
expect(tl.filter((s) => s.action === IMPLICIT_WB_OPEN)).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('carries open state across scene boundaries', () => {
|
||||
const scenes = [
|
||||
sc('S0', [act({ id: 'd1', type: 'wb_draw_text' })]), // implicit open
|
||||
sc('S1', [act({ id: 'd2', type: 'wb_draw_shape' })]), // still open — no new open
|
||||
];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl.map((s) => s.action.type)).toEqual(['wb_open', 'wb_draw_text', 'wb_draw_shape']);
|
||||
});
|
||||
|
||||
it('wb_clear/wb_delete on a closed board also trigger the implicit open', () => {
|
||||
const scenes = [sc('S0', [act({ id: 'cl', type: 'wb_clear' })])];
|
||||
const tl = resolveActionTimeline(scenes);
|
||||
expect(tl[0].action).toBe(IMPLICIT_WB_OPEN);
|
||||
// wb_clear on an (implicitly) empty board is still 0ms.
|
||||
expect(tl[1]).toMatchObject({ startMs: WB_OPEN_MS, durationMs: 0 });
|
||||
});
|
||||
});
|
||||
90
tests/lib/choreography/timing.test.ts
Normal file
90
tests/lib/choreography/timing.test.ts
Normal file
@ -0,0 +1,90 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
EFFECT_AUTO_CLEAR_MS,
|
||||
DISCUSSION_TRIGGER_DELAY_MS,
|
||||
DISCUSSION_AUTO_SKIP_MS,
|
||||
MAX_VIDEO_WAIT_MS,
|
||||
WB_OPEN_MS,
|
||||
WB_DRAW_MS,
|
||||
WB_EDIT_MS,
|
||||
WB_DELETE_MS,
|
||||
WB_CLOSE_MS,
|
||||
WIDGET_MS,
|
||||
wbDrawCodeMs,
|
||||
wbClearMs,
|
||||
estimateSpeechDurationMs,
|
||||
} from '@/lib/choreography';
|
||||
|
||||
describe('timing constants pin the values moved from the app engines', () => {
|
||||
it('effect / scene timing', () => {
|
||||
expect(EFFECT_AUTO_CLEAR_MS).toBe(5000);
|
||||
expect(DISCUSSION_TRIGGER_DELAY_MS).toBe(3000);
|
||||
expect(DISCUSSION_AUTO_SKIP_MS).toBe(5000);
|
||||
expect(MAX_VIDEO_WAIT_MS).toBe(5 * 60 * 1000);
|
||||
});
|
||||
|
||||
it('whiteboard / widget action durations', () => {
|
||||
expect(WB_OPEN_MS).toBe(2000);
|
||||
expect(WB_DRAW_MS).toBe(800);
|
||||
expect(WB_EDIT_MS).toBe(600);
|
||||
expect(WB_DELETE_MS).toBe(300);
|
||||
expect(WB_CLOSE_MS).toBe(700);
|
||||
expect(WIDGET_MS).toBe(300);
|
||||
});
|
||||
});
|
||||
|
||||
describe('wbDrawCodeMs — base 800ms + 50ms/line, capped at 3000ms', () => {
|
||||
it('matches Math.min(800 + lines * 50, 3000)', () => {
|
||||
expect(wbDrawCodeMs(0)).toBe(800);
|
||||
expect(wbDrawCodeMs(1)).toBe(850);
|
||||
expect(wbDrawCodeMs(10)).toBe(1300);
|
||||
// cap kicks in at 44 lines (800 + 2200 = 3000)
|
||||
expect(wbDrawCodeMs(44)).toBe(3000);
|
||||
expect(wbDrawCodeMs(1000)).toBe(3000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('wbClearMs — base 380ms + 55ms/element, capped at 1400ms', () => {
|
||||
it('matches Math.min(380 + count * 55, 1400)', () => {
|
||||
expect(wbClearMs(0)).toBe(380);
|
||||
expect(wbClearMs(1)).toBe(435);
|
||||
expect(wbClearMs(10)).toBe(930);
|
||||
// cap kicks in at ~18.5 elements → 19 elements exceeds 1400
|
||||
expect(wbClearMs(19)).toBe(1400);
|
||||
expect(wbClearMs(500)).toBe(1400);
|
||||
});
|
||||
});
|
||||
|
||||
describe('estimateSpeechDurationMs — deterministic no-audio dwell', () => {
|
||||
it('CJK text: ~150ms/char, floored at 2000ms', () => {
|
||||
// 20 CJK chars → 20 * 150 = 3000ms
|
||||
expect(estimateSpeechDurationMs('中'.repeat(20))).toBe(3000);
|
||||
// short CJK below the floor
|
||||
expect(estimateSpeechDurationMs('中文')).toBe(2000);
|
||||
});
|
||||
|
||||
it('non-CJK text: ~240ms/word (≈250 WPM), floored at 2000ms', () => {
|
||||
// 20 words → 20 * 240 = 4800ms
|
||||
const twentyWords = Array.from({ length: 20 }, (_, i) => `w${i}`).join(' ');
|
||||
expect(estimateSpeechDurationMs(twentyWords)).toBe(4800);
|
||||
// few words below the floor
|
||||
expect(estimateSpeechDurationMs('hello world')).toBe(2000);
|
||||
});
|
||||
|
||||
it('CJK threshold is >30% CJK characters', () => {
|
||||
// Mostly ASCII with a couple CJK chars → treated as non-CJK (word-based).
|
||||
const text = 'this is a long english sentence with 中文 words in it';
|
||||
const words = text.split(/\s+/).filter(Boolean).length;
|
||||
expect(estimateSpeechDurationMs(text)).toBe(Math.max(2000, words * 240));
|
||||
});
|
||||
|
||||
it('divides by playback speed', () => {
|
||||
const base = estimateSpeechDurationMs('中'.repeat(20));
|
||||
expect(estimateSpeechDurationMs('中'.repeat(20), { speed: 2 })).toBe(base / 2);
|
||||
expect(estimateSpeechDurationMs('中'.repeat(20), { speed: 0.5 })).toBe(base / 0.5);
|
||||
});
|
||||
|
||||
it('empty text falls to the 2000ms floor', () => {
|
||||
expect(estimateSpeechDurationMs('')).toBe(2000);
|
||||
});
|
||||
});
|
||||
Loading…
x
Reference in New Issue
Block a user