forked from longtao.wu/droboticflow
🐛 Restore acceptance captions after discarded slice navigation
This commit is contained in:
parent
e44cd95952
commit
4170c74b35
@ -0,0 +1,25 @@
|
||||
# Acceptance caption recovery after discarded slices
|
||||
|
||||
Base: e44cd959, codex/review-random-claim-fetch. Freshness confirmed against origin/dev and live source. Existing claim and heartbeat fixes remain intact.
|
||||
|
||||
## Confirmed defect
|
||||
|
||||
Focused final acceptance with source-index media reuses one Movi player and URL across slices. A keep -> discard -> keep selection can measure a temporarily unavailable picture and hide the subtitle overlay. The initial retry timers were cancelled on the first successful measurement and never rearmed on a later measurement failure. Playback could resume with no caption DOM even though the caption toggle stayed on and the caption data remained available; window resize restored it.
|
||||
|
||||
The actual acceptance component reproduced this in both source-index fixtures (ordinary and fractional media offset). A real `ReviewVideoCaption` lifecycle regression additionally ran against the fixed base source and failed at the recovery visibility assertion after late shadow-video dimensions, without resize or playback. Old-source evidence: `local-data/test-results/acceptance-caption-fix/lifecycle-red.log`. Earlier full-flow symptom capture: `local-data/diagnostics/acceptance-caption-switch/discard-recovery-probe.log` (these initial diagnostic runs also reported a browser cleanup failure; they are not passing-gate evidence).
|
||||
|
||||
## Change
|
||||
|
||||
- Re-arm bounded picture measurement retries after a failed measurement or media reload.
|
||||
- Refresh on seek completion and playback readiness; time updates only recover a missing overlay and do not repeatedly measure a healthy picture.
|
||||
- Cancel retries after recovery, media error or effect cleanup. Decode errors hide stale captions; readiness events allow recovery; unmounted effects cannot revive an old overlay.
|
||||
- Preserve caption visibility/placement preferences, all caption text, review permissions, draft/decision behavior, playback controls and rates. No network/API, database, migration or resource changes.
|
||||
- Extend the existing acceptance browser test with the exact single-camera keep -> discard -> keep click sequence and playback assertions for native, Movi and fractional-offset Movi. Add isolated error/readiness/unmount lifecycle coverage.
|
||||
|
||||
## Verification
|
||||
|
||||
`npm run verify:task -- --base e44cd959` passed 8/8, `inputsChangedDuringRun: false`, no unmapped/deferred checks. Final report: `local-data/test-results/verify-1794205856525025000/report.json`. Coverage includes type/architecture checks, changed-file format, verification-tooling tests, caption/preferences/navigation units and both browser fixtures. Screenshot checked: `local-data/test-results/acceptance-playback-rate/movi-caption-after-discard.png`.
|
||||
|
||||
Initial full browser runs passed functional assertions but failed browser-process cleanup. A scoped probe isolated the failure to killing a process that had exercised all three decoder modes with zero remaining contexts; running one mode completed cleanup normally. The acceptance test now uses and closes a separately owned browser process for each mode. Shared cleanup deadlines and failure reporting were not relaxed. Final task checks run serially and exit successfully; original failed reports remain unchanged.
|
||||
|
||||
No push, PR, image upload, deployment or production write performed. Online business acceptance remains pending deployment authorization.
|
||||
@ -14,3 +14,7 @@ User requested investigation of remaining full-regression failures. Scope also i
|
||||
## Follow-up: review renewal resilience
|
||||
|
||||
User asked to address a second supplier's automatic-pause report. Local red-capable browser reproduction confirms a single transient renewal failure permanently disables subsequent session heartbeats. Add bounded, same-generation renew-only retries while the previously confirmed lease has adequate lifetime; preserve terminal permission/ownership failures, explicit resume, drafts and claim behavior. Historical production 409 attribution is still unconfirmed. Evidence: [renewal-resilience.md](renewal-resilience.md). No production write, push, PR or deployment included.
|
||||
|
||||
## Follow-up: acceptance captions after discarded slices
|
||||
|
||||
User authorized the fix after confirming that entering focused acceptance, selecting a discarded slice and returning to a kept slice permanently removes captions. Base: e44cd959. Scope: shared picture-box recovery and browser coverage of the actual acceptance component; preserve subtitle preferences, playback/rate, review permissions, drafts and all earlier fixes. No API, customer data, push or deployment changes. Reuse this task worktree to avoid disturbing unrelated local work.
|
||||
|
||||
@ -117,7 +117,8 @@
|
||||
"^src/components/(ReviewCaptions|review-viewing)\\.tsx?$",
|
||||
"^src/hooks/useReviewDraft\\.ts$",
|
||||
"^src/lib/(review-caption-preferences\\.js|review-navigation\\.js|review-opinions\\.ts)$",
|
||||
"^src/styles/review-captions\\.css$"
|
||||
"^src/styles/review-captions\\.css$",
|
||||
"^tests/browser/review-caption-recovery\\.mjs$"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/review-target-interaction.test.mjs",
|
||||
@ -130,6 +131,11 @@
|
||||
"argv": ["node", "tests/browser/acceptance-playback-rate.mjs"],
|
||||
"env": {},
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": ["node", "tests/browser/review-caption-recovery.mjs"],
|
||||
"env": {},
|
||||
"tests": true
|
||||
}
|
||||
],
|
||||
"warnings": ["浏览器使用真实验收组件、三种播放器模式与隔离合成媒体/批注 API;不连接生产。"]
|
||||
|
||||
@ -20,7 +20,12 @@ export function usePictureBox(
|
||||
const video = videoRef.current;
|
||||
const surface = surfaceRef.current;
|
||||
if (!video || !surface) return;
|
||||
const hide = () => setPicture(null);
|
||||
let disposed = false;
|
||||
let resolved = false;
|
||||
const hide = () => {
|
||||
resolved = false;
|
||||
setPicture(null);
|
||||
};
|
||||
// Reports whether the picture could be resolved this pass. False means
|
||||
// "retry later" — the movi shadow <video> (source of the intrinsic size,
|
||||
// see below) is built asynchronously after the host fires its first
|
||||
@ -53,6 +58,7 @@ export function usePictureBox(
|
||||
const scale = Math.min(bounds.width / pictureWidth, bounds.height / pictureHeight);
|
||||
const width = pictureWidth * scale;
|
||||
const height = pictureHeight * scale;
|
||||
resolved = true;
|
||||
setPicture({
|
||||
left: bounds.left - parent.left + (bounds.width - width) / 2,
|
||||
top: bounds.top - parent.top + (bounds.height - height) / 2,
|
||||
@ -61,39 +67,63 @@ export function usePictureBox(
|
||||
});
|
||||
return true;
|
||||
};
|
||||
const observer = new ResizeObserver(() => measure());
|
||||
observer.observe(video);
|
||||
observer.observe(surface);
|
||||
video.addEventListener("loadeddata", measure);
|
||||
video.addEventListener("loadedmetadata", measure);
|
||||
video.addEventListener("canplay", measure);
|
||||
video.addEventListener("emptied", hide);
|
||||
video.addEventListener("loadstart", hide);
|
||||
video.addEventListener("error", hide);
|
||||
window.addEventListener("resize", measure);
|
||||
const retryTimers: number[] = [];
|
||||
const retryTimers = new Set<number>();
|
||||
const RETRY_DELAYS = [100, 300, 700, 1500, 3000, 6000];
|
||||
const stopRetries = () => {
|
||||
for (const timer of retryTimers.splice(0)) window.clearTimeout(timer);
|
||||
for (const timer of retryTimers) window.clearTimeout(timer);
|
||||
retryTimers.clear();
|
||||
};
|
||||
for (const delay of RETRY_DELAYS) {
|
||||
retryTimers.push(
|
||||
window.setTimeout(() => {
|
||||
if (measure()) stopRetries();
|
||||
}, delay),
|
||||
);
|
||||
}
|
||||
if (measure()) stopRetries();
|
||||
const startRetries = () => {
|
||||
if (disposed || retryTimers.size) return;
|
||||
for (const delay of RETRY_DELAYS) {
|
||||
const timer = window.setTimeout(() => {
|
||||
retryTimers.delete(timer);
|
||||
if (!disposed && measure()) stopRetries();
|
||||
}, delay);
|
||||
retryTimers.add(timer);
|
||||
}
|
||||
};
|
||||
const refresh = () => {
|
||||
if (disposed) return;
|
||||
if (measure()) stopRetries();
|
||||
else startRetries();
|
||||
};
|
||||
const reload = () => {
|
||||
hide();
|
||||
stopRetries();
|
||||
startRetries();
|
||||
};
|
||||
const fail = () => {
|
||||
hide();
|
||||
stopRetries();
|
||||
};
|
||||
// Source-index slices reuse the same player and URL. A resize/seek can
|
||||
// temporarily hide its picture without another metadata event afterwards.
|
||||
// Re-arm bounded recovery on failed measurements, not only on mount.
|
||||
const recover = () => {
|
||||
if (!resolved) refresh();
|
||||
};
|
||||
const observer = new ResizeObserver(refresh);
|
||||
observer.observe(video);
|
||||
observer.observe(surface);
|
||||
const readinessEvents = ["loadeddata", "loadedmetadata", "canplay", "seeked", "playing"];
|
||||
for (const event of readinessEvents) video.addEventListener(event, refresh);
|
||||
video.addEventListener("timeupdate", recover);
|
||||
video.addEventListener("emptied", reload);
|
||||
video.addEventListener("loadstart", reload);
|
||||
video.addEventListener("error", fail);
|
||||
window.addEventListener("resize", refresh);
|
||||
refresh();
|
||||
return () => {
|
||||
disposed = true;
|
||||
stopRetries();
|
||||
observer.disconnect();
|
||||
video.removeEventListener("loadeddata", measure);
|
||||
video.removeEventListener("loadedmetadata", measure);
|
||||
video.removeEventListener("canplay", measure);
|
||||
video.removeEventListener("emptied", hide);
|
||||
video.removeEventListener("loadstart", hide);
|
||||
video.removeEventListener("error", hide);
|
||||
window.removeEventListener("resize", measure);
|
||||
for (const event of readinessEvents) video.removeEventListener(event, refresh);
|
||||
video.removeEventListener("timeupdate", recover);
|
||||
video.removeEventListener("emptied", reload);
|
||||
video.removeEventListener("loadstart", reload);
|
||||
video.removeEventListener("error", fail);
|
||||
window.removeEventListener("resize", refresh);
|
||||
};
|
||||
}, [videoRef, surfaceRef, viewKey]);
|
||||
return picture;
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
import { browserExecutableOptions } from "../../scripts/lib/browser-executable.mjs";
|
||||
// Actual acceptance component and media reloads; API responses/media and review-draft writes are isolated synthetic fixtures.
|
||||
import { closeBrowserFixture } from "../../scripts/lib/browser-test-lifecycle.mjs";
|
||||
import { createServer } from "vite";
|
||||
import { chromium } from "playwright";
|
||||
import assert from "node:assert/strict";
|
||||
@ -53,14 +54,11 @@ execFileSync("ffmpeg", [
|
||||
`${out}/source.mp4`,
|
||||
]);
|
||||
const sourceMedia = await fs.readFile(`${out}/source.mp4`);
|
||||
let browser, server, diagnosticPage;
|
||||
const launchBrowser = () =>
|
||||
chromium.launch({
|
||||
...browserExecutableOptions(),
|
||||
|
||||
headless: true,
|
||||
args: ["--no-proxy-server"],
|
||||
});
|
||||
let browser, owner, server, diagnosticPage;
|
||||
const launchBrowser = async () => {
|
||||
owner = await chromium.launchServer({ ...browserExecutableOptions(), headless: true, args: ["--no-proxy-server"] });
|
||||
return chromium.connect(owner.wsEndpoint());
|
||||
};
|
||||
const cases = [];
|
||||
try {
|
||||
if (!base) {
|
||||
@ -85,8 +83,10 @@ try {
|
||||
await server.listen();
|
||||
base = server.resolvedUrls.local[0].replace(/\/$/, "");
|
||||
}
|
||||
browser = await launchBrowser();
|
||||
for (const engine of process.env.REVIEW_NOTE_BASELINE ? ["native"] : ["native", "movi", "movi-fractional"]) {
|
||||
// Decoder modes have independent browser-process lifetimes as well as
|
||||
// isolated API fixtures; close each owned process before the next mode.
|
||||
browser = await launchBrowser();
|
||||
const sourceIndex = engine !== "native";
|
||||
const fractionalOffset = engine === "movi-fractional" ? 0.023333333 : 0;
|
||||
const context = await browser.newContext({ viewport: { width: 1500, height: 1000 } });
|
||||
@ -201,6 +201,43 @@ try {
|
||||
);
|
||||
await page.goto(`${base}/__acceptance_playback_rate`);
|
||||
await page.getByLabel("播放速度").waitFor();
|
||||
// Keep the same single-camera source player across keep -> discard -> keep.
|
||||
// Caption DOM must recover without toggling captions or resizing the window.
|
||||
const sliceChoices = page.locator('[aria-label="验收切片选择"]');
|
||||
const leftCaption = page.locator('[data-camera-caption="left"]');
|
||||
await leftCaption.waitFor();
|
||||
assert.match(await leftCaption.innerText(), /第一片/);
|
||||
await sliceChoices.getByRole("button", { name: /已舍弃/ }).click();
|
||||
await page.getByTestId("acceptance-discard-evidence").waitFor();
|
||||
await page.waitForFunction(() => {
|
||||
const p = document.querySelector('[data-camera="left"]');
|
||||
return p?.readyState >= 2 && !p.seeking;
|
||||
});
|
||||
await sliceChoices.getByRole("button", { name: /第二片/ }).click();
|
||||
await page.waitForFunction(
|
||||
() => document.querySelector('[data-testid="review-current-slice"]')?.textContent === "第二片",
|
||||
);
|
||||
await page.waitForFunction(() => {
|
||||
const p = document.querySelector('[data-camera="left"]');
|
||||
return p?.readyState >= 2 && !p.seeking;
|
||||
});
|
||||
await leftCaption.waitFor({ timeout: 8000 });
|
||||
assert.match(await leftCaption.innerText(), /第二片/);
|
||||
await page.getByRole("button", { name: "播放", exact: true }).click();
|
||||
await page.waitForFunction(
|
||||
(offset) => {
|
||||
const p = document.querySelector('[data-camera="left"]');
|
||||
return p && !p.paused && p.currentTime > offset + 0.3;
|
||||
},
|
||||
sourceIndex ? 8 + fractionalOffset : 0,
|
||||
);
|
||||
assert.match(await leftCaption.innerText(), /第二片/);
|
||||
await page.getByRole("button", { name: "暂停", exact: true }).click();
|
||||
await page.screenshot({ path: `${out}/${engine}-caption-after-discard.png`, fullPage: true });
|
||||
await sliceChoices.getByRole("button", { name: /^第一片/ }).click();
|
||||
await page.waitForFunction(
|
||||
() => document.querySelector('[data-testid="review-current-slice"]')?.textContent === "第一片",
|
||||
);
|
||||
await page.getByRole("button", { name: "恢复双画面", exact: true }).click();
|
||||
async function assertRate(rate) {
|
||||
assert.equal(await page.getByLabel("播放速度").inputValue(), String(rate));
|
||||
@ -353,6 +390,9 @@ try {
|
||||
await page.screenshot({ path: `${out}/${engine}.png`, fullPage: true });
|
||||
cases.push({ engine, sliceChanges: 3 + shortcutChanges, retainedRates: [2, 1.5, 1.5], errors });
|
||||
await context.close();
|
||||
await closeBrowserFixture({ browser, owner });
|
||||
browser = undefined;
|
||||
owner = undefined;
|
||||
}
|
||||
await fs.writeFile(`${out}/report.json`, JSON.stringify({ passed: true, cases }, null, 2));
|
||||
console.log(JSON.stringify({ passed: true, cases }));
|
||||
@ -379,6 +419,5 @@ try {
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
await browser?.close();
|
||||
await server?.close();
|
||||
await closeBrowserFixture({ browser, owner, vite: server });
|
||||
}
|
||||
|
||||
87
tests/browser/review-caption-recovery.mjs
Normal file
87
tests/browser/review-caption-recovery.mjs
Normal file
@ -0,0 +1,87 @@
|
||||
import { createServer } from "vite";
|
||||
import { chromium } from "playwright";
|
||||
import { browserExecutableOptions } from "../../scripts/lib/browser-executable.mjs";
|
||||
import { closeBrowserFixture } from "../../scripts/lib/browser-test-lifecycle.mjs";
|
||||
import assert from "node:assert/strict";
|
||||
const server = await createServer({
|
||||
configFile: false,
|
||||
cacheDir: "local-data/test-results/review-caption-recovery/vite-cache",
|
||||
server: { host: "127.0.0.1", port: 0 },
|
||||
plugins: [
|
||||
{
|
||||
name: "caption-lifecycle",
|
||||
configureServer(s) {
|
||||
s.middlewares.use("/caption-lifecycle", async (req, res) => {
|
||||
res.setHeader("Content-Type", "text/html");
|
||||
res.end(
|
||||
await s.transformIndexHtml(
|
||||
"/caption-lifecycle",
|
||||
`<div id="root"></div><script type="module">
|
||||
import React,{useRef,useLayoutEffect} from 'react';import {createRoot} from 'react-dom/client';import {ReviewVideoCaption} from '/src/components/ReviewCaptions.tsx';
|
||||
window.pictureWidth=640;window.mediaError=null;
|
||||
function Picture(){const host=useRef(null),surface=useRef(null);useLayoutEffect(()=>{const h=host.current;h.attachShadow({mode:'open'});const inner=document.createElement('video');Object.defineProperties(inner,{videoWidth:{get:()=>window.pictureWidth},videoHeight:{get:()=>window.pictureWidth?480:0}});h.shadowRoot.append(inner);Object.defineProperties(h,{readyState:{get:()=>4},videoWidth:{get:()=>0},videoHeight:{get:()=>0},error:{get:()=>window.mediaError}});window.player=h;},[]);return React.createElement('div',{ref:surface,style:{position:'relative',width:640,height:480}},React.createElement('movi-player',{ref:host,style:{display:'block',width:640,height:480}}),React.createElement(ReviewVideoCaption,{videoRef:host,surfaceRef:surface,viewKey:'same-source-video',camera:'left',caption:{kind:'action',sliceText:'合成切片',text:'字幕正文仍存在'}}));}
|
||||
function App(){const [shown,setShown]=React.useState(true);window.setShown=setShown;return shown?React.createElement(Picture):null;}createRoot(document.getElementById('root')).render(React.createElement(App));</script>`,
|
||||
),
|
||||
);
|
||||
});
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
let browser, owner;
|
||||
try {
|
||||
await server.listen();
|
||||
owner = await chromium.launchServer({ ...browserExecutableOptions(), headless: true });
|
||||
browser = await chromium.connect(owner.wsEndpoint());
|
||||
const page = await browser.newPage();
|
||||
await page.goto(server.resolvedUrls.local[0] + "caption-lifecycle");
|
||||
await page.locator("[data-camera-caption]").waitFor();
|
||||
assert.equal(await page.locator("[data-camera-caption]").count(), 1);
|
||||
await page.evaluate(() => {
|
||||
window.pictureWidth = 0;
|
||||
window.player.dispatchEvent(new Event("loadstart"));
|
||||
window.player.dispatchEvent(new Event("canplay"));
|
||||
});
|
||||
|
||||
await page.waitForTimeout(100);
|
||||
await page.evaluate(() => {
|
||||
window.pictureWidth = 640;
|
||||
});
|
||||
await page.locator("[data-camera-caption]").waitFor({ timeout: 2000 });
|
||||
assert.equal(
|
||||
await page.locator("[data-camera-caption]").count(),
|
||||
1,
|
||||
"late dimensions recover without resize or playback",
|
||||
);
|
||||
await page.evaluate(() => {
|
||||
window.mediaError = { code: 3 };
|
||||
window.player.dispatchEvent(new Event("error"));
|
||||
});
|
||||
await page.waitForFunction(() => !document.querySelector("[data-camera-caption]"));
|
||||
await page.waitForTimeout(350);
|
||||
assert.equal(await page.locator("[data-camera-caption]").count(), 0, "decode failure must hide stale captions");
|
||||
await page.evaluate(() => {
|
||||
window.mediaError = null;
|
||||
window.player.dispatchEvent(new Event("canplay"));
|
||||
});
|
||||
await page.locator("[data-camera-caption]").waitFor();
|
||||
await page.evaluate(() => {
|
||||
window.pictureWidth = 0;
|
||||
window.player.dispatchEvent(new Event("emptied"));
|
||||
window.setShown(false);
|
||||
});
|
||||
await page.waitForTimeout(400);
|
||||
assert.equal(
|
||||
await page.locator("[data-camera-caption]").count(),
|
||||
0,
|
||||
"unmounted measurement cannot revive an old caption",
|
||||
);
|
||||
await page.evaluate(() => {
|
||||
window.pictureWidth = 640;
|
||||
window.setShown(true);
|
||||
});
|
||||
await page.locator("[data-camera-caption]").waitFor();
|
||||
console.log("PASS caption reload recovery, decode error hiding, readiness recovery and unmount/remount");
|
||||
} finally {
|
||||
await closeBrowserFixture({ browser, owner, vite: server });
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user