⚡ Add scoped development verification workflow

This commit is contained in:
Codex 2026-11-02 22:33:09 -08:00
parent c8aa1ae714
commit 8a82ecf174
9 changed files with 759 additions and 2 deletions

View File

@ -0,0 +1,8 @@
# Targeted development verification
Goal: implement the approved fast development / task / integration verification workflow.
Agent: Codex, current unified-worktree conversation
Base: codex/review-focus-layout-stability @ c8aa1ae7; continue the user's unified worktree without moving other work.
Scope: AGENTS.md, verification guidance, npm aliases, local selector/runner and its tests.
Out of scope: changing release policy, bypassing approvals, production actions or capacity acceptance.
Risk: false-positive verification; test unmapped changes, skips, empty tests, timeouts and affected-scope expansion.

View File

@ -80,11 +80,17 @@ These principles guide implementation choices. Specific business contracts and t
- Deploying to any company cluster requires the user's explicit approval immediately before the deployment; verification work must stop when that approval is absent.
- Cluster verification is read-only against customer data: use isolated synthetic fixtures or existing records without mutation, and never delete, overwrite, or add customer data. If a test requires a write or destructive operation, stop and obtain explicit approval for that exact operation.
## Development verification
- Before choosing checks for a change, read `docs/agents/verification.md`. Start with `npm run verify:plan -- --base <task-base>`; use quick checks while editing and affected task checks once before delivery. Expand only for changed dependencies, failures or uncovered risk.
- Reuse unchanged evidence within a task using the recorded source, dependency, configuration and environment identities. Report skipped, timed-out and unmapped coverage as pending; quick checks never replace release evidence.
- Keep local implementation and verification moving within the authorized scope. The existing push, PR, production-data and deployment approval rules still apply.
## Local Environment
- Local replica stack (Vite 5173, Go API 3001, snapshot Mongo 27018) is the default verification surface before any PR.
- Verification commands and known pitfalls (upstream test debt, session auth flag, workspace mount) are documented in `docs/HANDOFF.md`.
- Comparing test failures against the known-failure baseline ALWAYS goes through `npm run test:frontend:gate` / `npm run test:go:gate` (they diff against `tests/known-failures.json`; only NEW failures fail). Never hand-diff failure lists with stash/scratch files. Formatting gate: `npm run format:check`.
- Comparing test failures against the known-failure baseline ALWAYS goes through `npm run test:frontend:gate` / `npm run test:go:gate` (they diff against `tests/known-failures.json`; only NEW failures fail). Never hand-diff failure lists with stash/scratch files. Quick edits check changed-file formatting; the full formatting gate remains `npm run format:check` at integration/release scope.
- The company cluster and BOS bucket are read-only by default; any write requires explicit user approval.
## Agent skills

View File

@ -19,6 +19,10 @@
- 质检员 `fanyu-full-qc@droboticflow.local`、验收员 `fanyu-full-acceptance@droboticflow.local`、管理员 `fanyu-admin@droboticflow.local`
- **红线**:27018/27017 都是本地容器,随便写;公司集群(kubeconfig `local-data/fanyu.conf`)和 BOS 桶默认只读,任何写操作需用户明确确认。
## 日常验证入口
开发时先按[分层验证流程](agents/verification.md)选择检查。`verify:plan` 查看范围,`verify:quick` 用于编辑反馈,`verify:task` 用于本批业务流程验收;完整集成与发布检查独立执行。不要每次修改都重复本手册的所有验证命令。
## 1. 前端(5173)怎么改、怎么验证
**改动即生效**:Vite dev server 直连 `src/` 源码,保存文件浏览器自动热更(HMR)。没有"重新构建前端"这一步。

View File

@ -0,0 +1,45 @@
# 分层验证
适用于日常功能开发、修复及本批改动交付。发布仍使用 [release.py 流程](../release/README.md)。
## 工作步骤
1. 开工时用几句话确定业务结果、兼容范围、任务基线和风险。普通小改动不另写长计划;在已授权范围内连续完成实现、验证、修正。
2. `npm run verify:plan -- --base <任务起点提交>` 查看范围。默认 `HEAD` 只包含未提交改动;交付已提交任务时必须指定任务起点,不能用干净树的空计划证明已提交代码通过。
3. 编辑过程中运行 `npm run verify:quick -- --base <任务起点提交>`。相关单元测试被合并、去重执行;不默认启动浏览器、Mongo 或长时压测。
4. 本批完成后运行 `npm run verify:task -- --base <任务起点提交>`,集中验证受影响的真实业务流程。失败只修复并重跑有关部分;新改动影响其他范围时扩大验证。
5. 跨模块集成或公共依赖变化时运行 `verify:integration`。最终报告列出通过、失败、待验证和日志路径;部署单独执行发布门禁。
命令支持 `--plan`(不执行)及 `--budget-seconds N`。默认执行预算:quick 180 秒、task 600 秒、integration 3600 秒;预算耗尽会停止进程组并报告待验证,不自动重跑或改成通过。冷编译、安装依赖和机器负载会影响耗时,预算不是延迟承诺。
## 风险与完成标准
| 变更 | 开发检查 | 本批完成还需确认 |
| --- | --- | --- |
| 文档、纯样式 | 差异/格式检查 | 界面样式实际查看正常与错误状态;文案遵循现有最简规则 |
| 交互、状态和页面切换 | 类型、有关单元回归 | 一条覆盖进入、操作、退出的真实浏览器流程 |
| 领取、权限、保存、校验、返工 | 相关接口和规则回归 | 隔离存储、冲突、失败恢复及旧数据兼容 |
| 查询、轮询、缓存 | 结果等价回归 | 相同代表性规模的查询量、扫描量、耗时、取消及并发对照 |
| 工作流、迁移、资源或容量 | 专项验证 | 在途任务、回退和恢复;生产写入按具体授权执行 |
风险取决于行为和影响面,不按行数判断。共享工作台页面可能同时影响多个流程,脚本会扩大映射;确认只是文案/样式时可记录实际差异与人工检查,执行有关检查并说明未运行项,不把整批列为全通过。
120 人完整作业、8 小时运行、媒体容量和节点故障属于专项验收:容量承诺、相关架构或资源变更时执行,不绑定每次普通功能提交。发布基础门禁未因本规则减少。
## 选择与兜底
- 映射的唯一配置为 `scripts/verification-scopes.json`;当前覆盖审查切换、源校验/个人池、搜索分页、标注编辑、审核决定和统计面板。
- Git 检测包含起点以来提交、暂存、未暂存、未跟踪文件,重命名检查新旧路径。改动的前端测试和 Go 同名测试自动加入。CSS 不因单纯样式修改启动全套类型/业务测试。
- 未映射的前端代码扩大到前端基线门禁,未映射 Go 实现扩大到所属包;浏览器流程仍需人工核对。其他未映射行为报告待验证,补充映射或专项证据后才算交付完成。
- 退出码 0 仅表示所选检查通过;1 表示检查失败;2 表示超时、跳过、未映射、执行中源码变化或入口异常。报告中的 deferred/notCovered 始终保留,不把快速通过解释成完整通过。
- 遇到已有测试债务,使用既有 `test:frontend:gate` / `test:go:gate` 比较具体用例。不因失败而扩大 known-failures 基线,不手工忽略断言。
## 减少重复与环境准备
- 同一批同一输入只做一次成功验证;优先复用已有 npm 测试组、Go 编译/测试缓存和本地隔离 Mongo 服务,不为每次小改动重装依赖、重建镜像或重启整套副本。
- 浏览器/数据库测试串行运行,继续使用各自独立合成数据和清理机制。共享服务不等于共享业务状态;公司集群和 BOS 不用于这些写入测试。
- 日志及 JSON 报告写入忽略目录 `local-data/test-results/verify-*`,包含命令、基线、源码摘要、环境摘要、工具版本、安装锁文件摘要、耗时和未覆盖项。原始环境值不写入报告。
- 同任务内复用记录前,核对源码/测试/夹具、依赖安装、配置、工具版本、测试环境均未变化。源码摘要和环境摘要不是依赖安装或外部数据状态的证明。此入口不做跨运行的自动“免测缓存”;Go 自有缓存正常使用。浏览器、数据库和生产证据按各自环境重新核验。
- 更换依赖、修复失败、修改夹具、出现新问题或范围扩大时,重跑相关检查。测试报错、跳过和未启动永远不是可复用的成功证据。
此流程不增加日常审批。普通本地修改、测试及本地提交持续推进;推送、PR、公司部署、生产数据和资源修改仍遵循 `AGENTS.md` 的授权边界。

View File

@ -1,5 +1,8 @@
# 泛宇发布试行流程
日常开发按[分层验证流程](../agents/verification.md)执行,不在每次编辑后重复本发布流程。
本页发布门禁及 `fanyu-policy.json` 保持独立;快速检查报告不能作为缺失发布证据的替代。
## 开始或恢复工作
发布、部署故障、回退、上下文恢复时先读本页。状态的权威来源是

View File

@ -93,7 +93,12 @@
"test:review:decisions": "node --test tests/frontend/review-decision-controller.test.mjs tests/frontend/review-decision-completion.test.mjs tests/frontend/rework-resubmission.test.mjs",
"test:browser:review-decisions": "REVIEW_DECISION_BROWSER_TEST=1 go test ./internal/api -run '^TestReviewDecisionControllerBrowserReplica$' -count=1 -v && node tests/browser/review-decision-failure.mjs && node tests/browser/review-decision-start-race.mjs",
"test:review:drafts": "node --test tests/frontend/review-draft-persistence.test.mjs tests/frontend/rework-verdict-keys.test.mjs tests/frontend/review-draft-autosave.test.mjs tests/frontend/acceptance-legacy-compat.test.mjs",
"test:browser:review-drafts": "REVIEW_DRAFT_BROWSER_TEST=1 go test ./internal/api -run '^TestReviewDraftPersistenceBrowserReplica$' -count=1 -v"
"test:browser:review-drafts": "REVIEW_DRAFT_BROWSER_TEST=1 go test ./internal/api -run '^TestReviewDraftPersistenceBrowserReplica$' -count=1 -v",
"verify:plan": "python3 scripts/verify-change.py --plan",
"verify:quick": "python3 scripts/verify-change.py --mode quick",
"verify:task": "python3 scripts/verify-change.py --mode task",
"verify:integration": "python3 scripts/verify-change.py --mode integration",
"test:verification": "python3 -m unittest discover -s tests/tooling -p test_verification.py"
},
"dependencies": {
"@tailwindcss/vite": "^4.1.14",

View File

@ -0,0 +1,249 @@
{
"version": 1,
"scopes": [
{
"name": "review-navigation",
"paths": [
"^src/(pages/AnnotationOperations\\.tsx|hooks/useReviewContinuation\\.ts|services/api/review-continuation\\.ts)$",
"^internal/api/review_(continuation|trace)",
"^tests/browser/review-trace\\.mjs$"
],
"frontend": [
"tests/frontend/annotation-project-loading.test.mjs"
],
"go": {
"./internal/api": [
"^TestReviewContinuation"
]
},
"task": [
{
"argv": [
"go",
"test",
"./internal/api",
"-run",
"^TestReviewTraceBrowserReplica$",
"-count=1",
"-timeout=3m",
"-v"
],
"env": {
"REVIEW_TRACE_BROWSER_TEST": "1"
},
"tests": true
}
]
},
{
"name": "source-and-buffer",
"paths": [
"^src/(pages/AnnotationOperations\\.tsx|services/api/annotation-operations\\.ts)$",
"^internal/(api|store)/annotation_(source|work_buffer|claim_read|claim_visit|random_claim)",
"^internal/store/human_claim",
"^tests/browser/annotation-source-validation-replica\\.mjs$"
],
"frontend": [
"tests/frontend/annotation-buffer-*.test.mjs",
"tests/frontend/annotation-source-validation-ui.test.mjs"
],
"go": {
"./internal/api": [
"^TestSourceValidation",
"^TestPendingSourceValidation",
"^TestWorkBufferRetains",
"^TestRandomClaim"
],
"./internal/store": [
"^TestSourceAdmissionMemoryClaimFence$"
]
},
"task": [
{
"argv": [
"go",
"test",
"./internal/store",
"-run",
"^TestSourceAdmissionMongo(ClaimFence|BatchMatchesLegacyPipeline)$",
"-count=1",
"-timeout=3m",
"-v"
],
"env": {
"SOURCE_ADMISSION_LOCAL_MONGO": "1"
},
"tests": true
},
{
"argv": [
"go",
"test",
"./internal/api",
"-run",
"^TestSourceValidationBrowserReplica$",
"-count=1",
"-timeout=3m",
"-v"
],
"env": {
"SOURCE_VALIDATION_BROWSER_TEST": "1"
},
"tests": true
}
],
"warnings": [
"涉及准入/轮询:核对真实规模下查询次数、扫描量、超时与并发;局部回归不等于容量通过。"
]
},
{
"name": "pool-search",
"paths": [
"^src/pages/AnnotationOperations\\.tsx$",
"^src/.*(pool|Pool|human-loop)",
"^internal/(api|store)/(human_pool|admin_annotation_pool|human_loop)",
"^tests/browser/(review-pool-search|admin-annotation-pool)"
],
"frontend": [
"tests/frontend/annotation-pool-*.test.mjs",
"tests/frontend/review-pool-preview.test.mjs"
],
"go": {
"./internal/api": [
"^TestHumanPoolSearch",
"^TestHumanPoolQCProjectScope"
]
},
"task": [
{
"argv": [
"go",
"test",
"./internal/api",
"-run",
"^TestReviewPoolSearchBrowserReplica$",
"-count=1",
"-timeout=3m",
"-v"
],
"env": {
"REVIEW_POOL_BROWSER_TEST": "1"
},
"tests": true
}
]
},
{
"name": "annotation-editing",
"paths": [
"^src/(pages/AnnotationOperations\\.tsx|components/CaptureAnnotationWorkbench\\.tsx)$",
"^src/.*(slice-edit|SliceEdit|annotation-persistence)",
"^internal/api/(annotation_draft|annotation_slice|slice_recovery|slice_edit)",
"^tests/browser/(rework-boundary|slice-edit)"
],
"frontend": [
"tests/frontend/annotation-persistence.test.mjs",
"tests/frontend/slice-edit-session.test.mjs",
"tests/frontend/annotation-authoritative-mutation.test.mjs"
],
"go": {},
"task": [
{
"argv": [
"go",
"test",
"./internal/api",
"-run",
"^TestReworkBoundaryBrowserReplica$",
"-count=1",
"-timeout=3m",
"-v"
],
"env": {
"REWORK_BOUNDARY_BROWSER_TEST": "1"
},
"tests": true
}
]
},
{
"name": "review-decisions",
"paths": [
"^src/(pages/AnnotationOperations\\.tsx|components/CaptureAnnotationWorkbench\\.tsx)$",
"^src/.*([Rr]eview[Dd]raft|review-draft|[Rr]eview[Dd]ecision|review-decision)",
"^internal/api/review_(draft|decision|confirmation)",
"^tests/browser/review-(draft|decision|authoring)"
],
"frontend": [
"tests/frontend/review-decision-controller.test.mjs",
"tests/frontend/review-decision-completion.test.mjs",
"tests/frontend/review-draft-persistence.test.mjs",
"tests/frontend/review-draft-autosave.test.mjs"
],
"go": {},
"task": [
{
"argv": [
"npm",
"run",
"test:browser:review-drafts"
],
"tests": true
},
{
"argv": [
"npm",
"run",
"test:browser:review-decisions"
],
"tests": true
},
{
"argv": [
"npm",
"run",
"test:browser:review-permissions"
],
"tests": true
}
]
},
{
"name": "dashboards",
"paths": [
"^src/.*([Dd]ashboard|WorkforceStatistics|annotation-output)",
"^internal/(api|store)/(production_(dashboard|stats)|annotation_output|workforce)",
"^internal/(outputsnapshot|workforcestats)/",
"^tests/browser/.*dashboard"
],
"frontend": [
"tests/frontend/dashboard-presentation.test.mjs",
"tests/frontend/production-quality-dates.test.mjs"
],
"go": {
"./internal/api": [
"^TestProductionDashboardScopeIsolationAndFullCorpus$"
]
},
"task": [
{
"argv": [
"npm",
"run",
"test:browser:dashboard"
],
"tests": true
},
{
"argv": [
"python3",
"scripts/test_dashboard_stability.py"
]
}
],
"warnings": [
"统计/查询改动须额外记录同数据规模的性能对照;完整 120 人容量另行验收。"
]
}
]
}

290
scripts/verify-change.py Normal file
View File

@ -0,0 +1,290 @@
#!/usr/bin/env python3
"""Select local development checks; never substitutes for release.py evidence."""
import argparse
import ast
import hashlib
import json
import os
from pathlib import Path
import re
import signal
import subprocess
import sys
import time
ROOT = Path(__file__).resolve().parents[1]
POLICY = ROOT / 'scripts/verification-scopes.json'
def git(root, *args):
return subprocess.check_output(['git', *args], cwd=root, stderr=subprocess.PIPE).decode().strip('\n')
def changed_files(root, base):
revision = git(root, 'rev-parse', '--verify', base + '^{commit}')
changed = git(root, 'diff', '--name-only', '--no-renames', '-z', revision, '--').split('\0')
untracked = git(root, 'ls-files', '--others', '--exclude-standard', '-z').split('\0')
return revision, sorted(set(filter(None, changed + untracked)))
def command(argv, env=None, tests=False):
return {'argv': argv, 'env': env or {}, 'tests': tests}
def tooling_package_change(root, base):
try:
old = json.loads(git(root, 'show', base + ':package.json'))
new = json.loads((root/'package.json').read_text())
except (OSError, ValueError, subprocess.CalledProcessError):
return False
old_scripts, new_scripts = old.pop('scripts', {}), new.pop('scripts', {})
names = {name for name in set(old_scripts) | set(new_scripts) if old_scripts.get(name) != new_scripts.get(name)}
allowed = {'verify:plan', 'verify:quick', 'verify:task', 'verify:integration', 'test:verification'}
return old == new and bool(names) and names <= allowed
def plan_checks(root, files, policy, mode, base, tooling_package=False):
quick = [command(['git', 'diff', '--check', base, '--'])]
task = []
warnings = []
unresolved = []
scopes = []
frontend = set()
go_patterns = {}
matched = set()
for scope in policy['scopes']:
hits = [f for f in files if any(re.search(p, f) for p in scope['paths'])]
if not hits:
continue
scopes.append(scope['name'])
matched.update(hits)
for pattern in scope.get('frontend', []):
found = list(root.glob(pattern))
if not found:
unresolved.append('测试映射不存在: ' + pattern)
frontend.update(str(p.relative_to(root)) for p in found)
for pkg, patterns in scope.get('go', {}).items():
go_patterns.setdefault(pkg, set()).update(patterns)
task.extend(scope.get('task', []))
warnings.extend(scope.get('warnings', []))
web = any((f.startswith('src/') and not f.endswith('.css') or re.match(r'(package(-lock)?\.json|tsconfig.*|vite\.config.*)$', f)) and not (f == 'package.json' and tooling_package) for f in files)
if web:
quick.append(command(['npm', 'run', 'lint:web']))
formatting = [f for f in files if (root/f).is_file() and
(f.startswith('src/') and re.search(r'\.(ts|tsx|css|js)$', f) or
f.startswith('tests/frontend/') and f.endswith('.mjs'))]
if formatting:
quick.append(command(['node', 'node_modules/prettier/bin/prettier.cjs', '--check', *formatting]))
go_files = [f for f in files if f.endswith('.go') and (root/f).is_file()]
if go_files:
quick.append(command([sys.executable, 'scripts/verify-change.py', '--gofmt', *go_files]))
for f in files:
path = root/f
if f.startswith('tests/frontend/') and f.endswith('.test.mjs') and path.exists():
frontend.add(f)
continue
if f.endswith('.go'):
pkg = './' + str(Path(f).parent)
test_file = path if f.endswith('_test.go') else path.with_name(path.stem + '_test.go')
fast = []
if test_file.exists():
names = re.findall(r'^func (Test\w+)\(', test_file.read_text(), re.M)
fast = [name for name in names if not re.search(r'Browser|Mongo|Capacity|Soak|Load|Local|Benchmark', name)]
if fast:
go_patterns.setdefault(pkg, set()).update('^' + name + '$' for name in fast)
if len(fast) != len(names) and f not in matched:
unresolved.append(f + ': 外部环境测试需要显式任务级映射,不能将跳过当通过。')
if len(fast) != len(names):
warnings.append(f + ': 外部环境/容量测试不计入快速检查;核对任务级覆盖。')
if f not in matched and (not f.endswith('_test.go') or not fast):
# Package-wide fallback is explicit at task level, not every edit.
task.append(command(['go', 'test', pkg, '-timeout=5m', '-v'], tests=True))
continue
if f in matched or f.endswith('.md') or f.startswith('docs/') and f.endswith(('.png', '.jpg', '.svg')):
continue
if f.startswith('src/') and f.endswith('.css'):
warnings.append(f + ': 样式改动需人工查看受影响页面及异常状态。')
continue
if f.startswith('src/'):
task.append(command(['npm', 'run', 'test:frontend:gate']))
warnings.append(f + ': 未建立业务映射,任务级扩大到前端基线门禁;仍需人工确认浏览器流程。')
continue
if f.startswith('tests/browser/'):
unresolved.append(f + ': 为该浏览器夹具补充可运行的任务级映射。')
continue
if f == 'package.json' and tooling_package:
quick.append(command([sys.executable, '-m', 'unittest', 'discover', '-s', 'tests/tooling', '-p', 'test_verification.py'], tests=True))
continue
if f in ('package.json', 'package-lock.json', 'go.mod', 'go.sum') or re.match(r'(tsconfig|vite\.config)', f):
task.extend([command(['npm', 'run', 'test:frontend:gate']), command(['npm', 'run', 'test:go:gate']),
command(['npm', 'run', 'build:web']), command(['npm', 'run', 'test:production:boot'])])
warnings.append('依赖/公共配置变化:任务级扩大到集成检查,核对安装与锁文件。')
continue
if f in ('scripts/verify-change.py', 'scripts/verification-scopes.json') or f.startswith('tests/tooling/'):
quick.append(command([sys.executable, '-m', 'unittest', 'discover', '-s', 'tests/tooling', '-p', 'test_verification.py'], tests=True))
continue
if path.exists() and f.endswith('.py'):
quick.append(command([sys.executable, 'scripts/verify-change.py', '--python-syntax', f]))
elif path.exists() and f.endswith('.mjs'):
quick.append(command(['node', '--check', f]))
elif path.exists() and f.endswith('.sh'):
quick.append(command(['bash', '-n', f]))
unresolved.append(f + ': 尚无行为验证映射;补充映射/专项证据,语法通过不代表功能通过。')
if frontend:
quick.append(command(['node', '--test', *sorted(frontend)], tests=True))
for pkg, patterns in sorted(go_patterns.items()):
quick.append(command(['go', 'test', pkg, '-run', '|'.join(sorted(patterns)), '-timeout=2m', '-v'], tests=True))
integration = [command(['npm', 'run', x]) for x in
['lint:web', 'format:check', 'test:frontend:gate', 'test:go:gate', 'build:web', 'test:production:boot']]
def unique(commands):
out, seen = [], set()
for c in commands:
c = {**command(c['argv']), **c}
key = json.dumps(c, sort_keys=True)
if key not in seen:
out.append(c)
seen.add(key)
return out
quick, task = unique(quick), unique(task)
# Full suite gates subsume targeted frontend/Go tests, not tooling checks.
integration_extra = [c for c in quick if c['argv'][:2] not in (['node', '--test'], ['go', 'test'])
and c['argv'] != ['npm', 'run', 'lint:web']
and 'node_modules/prettier/bin/prettier.cjs' not in c['argv']]
selected = quick if mode == 'quick' else unique(integration_extra + task + integration) if mode == 'integration' else unique(quick + task)
return {'mode': mode, 'files': files, 'scopes': scopes, 'commands': selected,
'deferred': task if mode == 'quick' else [], 'unmapped': sorted(set(unresolved)),
'notes': sorted(set(warnings)), 'notCovered': [
'生产业务写验收、生产数据处置及正式发布门禁', '120 人完整作业、8 小时运行、节点故障及真实媒体容量',
'业务影响需人工核对;文件映射不能证明所有间接依赖均已覆盖']}
def fingerprint(root):
digest = hashlib.sha256()
paths = git(root, 'ls-files', '-z', '--cached', '--others', '--exclude-standard').split('\0')
for name in sorted(set(filter(None, paths))):
p = root/name
digest.update(name.encode())
if p.is_symlink():
digest.update(os.readlink(p).encode())
elif p.is_file():
digest.update(p.read_bytes())
else:
digest.update(b'<missing>')
return digest.hexdigest()
def run_check(check, root, logfile, timeout):
start = time.monotonic()
env = dict(os.environ)
# Avoid accidentally enabling unrelated opt-in suites in an inherited shell.
for key in list(env):
if key.startswith(('TEST_', 'RUN_')) or key in ('MONGO_URI', 'DATABASE_URL') or re.search(r'(_BROWSER_TEST|_LOCAL_MONGO|_MONGO_HTTP|_MONGO_STORE|_LOAD_TEST|_CAPACITY_TEST)$', key):
env.pop(key)
env.update(check['env'])
status, code = 'failed', None
with logfile.open('w') as stream:
try:
child = subprocess.Popen(check['argv'], cwd=root, env=env, stdout=stream,
stderr=subprocess.STDOUT, start_new_session=True)
try:
code = child.wait(timeout=timeout)
status = 'passed' if code == 0 else 'failed'
except (subprocess.TimeoutExpired, KeyboardInterrupt):
status = 'incomplete'
try:
os.killpg(child.pid, signal.SIGTERM)
except ProcessLookupError:
pass
try:
child.wait(timeout=2)
except subprocess.TimeoutExpired:
pass
# Kill lingering grandchildren even if the parent exited on TERM.
try:
os.killpg(child.pid, signal.SIGKILL)
except ProcessLookupError:
pass
child.wait()
except OSError as error:
stream.write(str(error))
output = logfile.read_text()
if status == 'passed' and check.get('tests'):
if re.search(r'--- SKIP:|\[no test files\]|no tests to run|# SKIP|# skipped [1-9]|ℹ skipped [1-9]|skipped=[1-9]', output):
status = 'incomplete'
if re.search(r'Ran 0 tests|# tests 0|ℹ tests 0', output):
status = 'incomplete'
return {**check, 'status': status, 'exitCode': code, 'seconds': round(time.monotonic()-start, 3), 'log': str(logfile)}
def runtime_identity(root):
versions = {'python': sys.version.split()[0]}
for name, argv in [('node', ['node', '--version']), ('go', ['go', 'version'])]:
try:
versions[name] = subprocess.check_output(argv, cwd=root, stderr=subprocess.PIPE, timeout=5).decode().strip()
except (OSError, subprocess.SubprocessError):
versions[name] = 'unavailable'
lock = root/'node_modules/.package-lock.json'
versions['installedNpmLockDigest'] = hashlib.sha256(lock.read_bytes()).hexdigest() if lock.is_file() else 'unavailable'
return versions
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--mode', choices=['quick', 'task', 'integration'], default='quick')
parser.add_argument('--base', default='HEAD', help='包含该提交以来的提交、暂存、未暂存及未跟踪改动;默认仅当前未提交改动')
parser.add_argument('--plan', action='store_true', help='只打印计划,不运行检查')
parser.add_argument('--budget-seconds', type=int, help='总时间预算,超时报告待验证')
parser.add_argument('--gofmt', nargs='+', help=argparse.SUPPRESS)
parser.add_argument('--python-syntax', help=argparse.SUPPRESS)
args = parser.parse_args(argv)
if args.gofmt:
result = subprocess.check_output(['gofmt', '-l', *args.gofmt], cwd=ROOT).decode()
print(result, end='')
return 1 if result.strip() else 0
if args.python_syntax:
ast.parse((ROOT/args.python_syntax).read_text())
return 0
budget = args.budget_seconds if args.budget_seconds is not None else {'quick': 180, 'task': 600, 'integration': 3600}[args.mode]
if budget <= 0:
parser.error('--budget-seconds must be positive')
base, files = changed_files(ROOT, args.base)
plan = plan_checks(ROOT, files, json.loads(POLICY.read_text()), args.mode, base, tooling_package_change(ROOT, base))
print(json.dumps(plan, ensure_ascii=False, indent=2), flush=True)
if args.plan:
return 0
folder = ROOT/'local-data/test-results'/f'verify-{time.time_ns()}'
folder.mkdir(parents=True, mode=0o700)
before = fingerprint(ROOT)
report = {**plan, 'base': base, 'head': git(ROOT, 'rev-parse', 'HEAD'), 'inputDigest': before, 'runtime': runtime_identity(ROOT),
'environmentDigest': hashlib.sha256(json.dumps(dict(os.environ), sort_keys=True).encode()).hexdigest(),
'startedAt': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'results': []}
start = time.monotonic()
for index, check in enumerate(plan['commands']):
remaining = budget - (time.monotonic()-start)
if remaining <= 0:
report['results'].append({**check, 'status': 'not-run', 'reason': 'budget exhausted'})
continue
result = run_check(check, ROOT, folder/f'{index:02}.log', remaining)
report['results'].append(result)
print(f"{result['status']}: {' '.join(check['argv'])} ({result['seconds']}s)", flush=True)
if result['status'] != 'passed':
report['results'].extend({**c, 'status': 'not-run', 'reason': 'previous check did not pass'} for c in plan['commands'][index+1:])
break
report['inputsChangedDuringRun'] = fingerprint(ROOT) != before
statuses = {r['status'] for r in report['results']}
code = 1 if 'failed' in statuses else 2 if (statuses - {'passed'} or plan['unmapped'] or report['inputsChangedDuringRun']) else 0
report['status'] = 'failed' if code == 1 else 'incomplete' if code == 2 else 'selected-checks-passed'
report['seconds'] = round(time.monotonic()-start, 3)
(folder/'report.json').write_text(json.dumps(report, ensure_ascii=False, indent=2)+'\n')
print(f"{report['status']}; 报告: {folder/'report.json'}")
if plan['deferred']:
print('任务级检查尚未执行;本结果不代表发布/容量验收通过。')
return code
if __name__ == '__main__':
try:
sys.exit(main())
except (OSError, ValueError, subprocess.CalledProcessError) as error:
print(f'检查入口失败: {type(error).__name__}', file=sys.stderr)
sys.exit(2)

View File

@ -0,0 +1,147 @@
import importlib.util
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[2]
spec = importlib.util.spec_from_file_location('verification', ROOT/'scripts/verify-change.py')
v = importlib.util.module_from_spec(spec)
spec.loader.exec_module(v)
POLICY = json.loads((ROOT/'scripts/verification-scopes.json').read_text())
class VerificationTest(unittest.TestCase):
def plan(self, files, mode='quick'):
return v.plan_checks(ROOT, files, POLICY, mode, 'HEAD')
def test_documentation_is_small(self):
plan = self.plan(['docs/example.md', 'AGENTS.md'])
self.assertEqual(len(plan['commands']), 1)
self.assertEqual(plan['deferred'], [])
self.assertEqual(plan['unmapped'], [])
def test_common_page_expands_and_deduplicates(self):
plan = self.plan(['src/pages/AnnotationOperations.tsx'])
self.assertGreaterEqual(len(plan['scopes']), 4)
self.assertTrue(plan['deferred'])
self.assertFalse(any('go' == c['argv'][0] and c['env'] for c in plan['commands']))
node_tests = [c for c in plan['commands'] if c['argv'][:2] == ['node', '--test']]
self.assertEqual(len(node_tests), 1)
self.assertEqual(len(node_tests[0]['argv'][2:]), len(set(node_tests[0]['argv'][2:])))
def test_source_task_includes_real_mongo_and_browser(self):
plan = self.plan(['internal/store/annotation_source_admission.go'], 'task')
self.assertTrue(any(c['env'].get('SOURCE_ADMISSION_LOCAL_MONGO') for c in plan['commands']))
self.assertTrue(any(c['env'].get('SOURCE_VALIDATION_BROWSER_TEST') for c in plan['commands']))
self.assertIn('source-and-buffer', plan['scopes'])
self.assertEqual(plan['unmapped'], [])
def test_script_alias_only_does_not_trigger_dependency_suite(self):
plan = v.plan_checks(ROOT, ['package.json'], POLICY, 'quick', 'HEAD', tooling_package=True)
self.assertEqual(plan['deferred'], [])
self.assertFalse(any('lint:web' in c['argv'] for c in plan['commands']))
self.assertTrue(any('unittest' in c['argv'] for c in plan['commands']))
def test_css_does_not_run_business_suites(self):
plan = self.plan(['src/example.css'])
self.assertEqual(plan['deferred'], [])
self.assertFalse(any('lint:web' in c['argv'] for c in plan['commands']))
def test_integration_does_not_repeat_targeted_frontend_tests(self):
plan = self.plan(['src/pages/AnnotationOperations.tsx'], 'integration')
self.assertFalse(any(c['argv'][:2] == ['node', '--test'] for c in plan['commands']))
self.assertEqual(sum('test:frontend:gate' in c['argv'] for c in plan['commands']), 1)
def test_integration_keeps_tooling_regressions(self):
plan = self.plan(['scripts/verify-change.py'], 'integration')
self.assertTrue(any('unittest' in c['argv'] for c in plan['commands']))
def test_new_frontend_falls_back_at_task_level(self):
plan = self.plan(['src/components/FutureShared.tsx'])
self.assertTrue(any('test:frontend:gate' in c['argv'] for c in plan['deferred']))
self.assertFalse(any('test:frontend:gate' in c['argv'] for c in plan['commands']))
def test_unknown_script_is_not_silently_covered(self):
self.assertTrue(self.plan(['scripts/new-data-migration.py'])['unmapped'])
self.assertTrue(self.plan(['tests/browser/new-flow.mjs'])['unmapped'])
def test_changed_frontend_test_is_always_included(self):
f = 'tests/frontend/annotation-buffer-summary.test.mjs'
plan = self.plan([f])
self.assertTrue(any(f in c['argv'] for c in plan['commands']))
def test_deleted_go_file_gets_package_fallback(self):
plan = self.plan(['internal/store/removed.go'])
self.assertTrue(any('./internal/store' in c['argv'] for c in plan['deferred']))
def test_scope_tests_and_browser_names_exist(self):
for scope in POLICY['scopes']:
for pattern in scope.get('frontend', []):
self.assertTrue(list(ROOT.glob(pattern)), pattern)
for c in scope.get('task', []):
if c['argv'][0] == 'go':
package = ROOT/c['argv'][2]
self.assertTrue(package.is_dir())
if c['argv'][:2] == ['npm', 'run']:
self.assertIn(c['argv'][2], json.loads((ROOT/'package.json').read_text())['scripts'])
def test_changes_include_committed_staged_unstaged_untracked_and_renamed(self):
with tempfile.TemporaryDirectory() as d:
root = Path(d)
def git(*args):
return subprocess.check_output(['git', *args], cwd=root, stderr=subprocess.DEVNULL).decode().strip()
git('init')
git('config', 'user.email', 'test@example.invalid')
git('config', 'user.name', 'Test')
(root/'a').write_text('initial')
git('add', 'a'); git('commit', '-m', 'initial')
base = git('rev-parse', 'HEAD')
git('mv', 'a', 'b'); git('commit', '-m', 'rename')
(root/'staged').write_text('staged'); git('add', 'staged')
(root/'b').write_text('modified')
(root/'new').write_text('untracked')
_, files = v.changed_files(root, base)
self.assertEqual(files, ['a', 'b', 'new', 'staged'])
before = v.fingerprint(root)
(root/'new').write_text('changed during check')
self.assertNotEqual(before, v.fingerprint(root))
with self.assertRaises(subprocess.CalledProcessError):
v.changed_files(root, 'does-not-exist')
def test_runner_failure_skip_empty_and_timeout_never_pass(self):
with tempfile.TemporaryDirectory() as d:
for code, expected in [
('print("ok")', 'passed'),
('raise SystemExit(3)', 'failed'),
('print("--- SKIP: TestOptIn")', 'incomplete'),
('print("testing: warning: no tests to run")', 'incomplete'),
('import time; time.sleep(3)', 'incomplete'),
]:
result = v.run_check(v.command([sys.executable, '-c', code], tests=True), Path(d), Path(d)/'run.log', .1)
self.assertEqual(result['status'], expected, code)
def test_runner_only_enables_declared_opt_ins(self):
from unittest.mock import patch
with tempfile.TemporaryDirectory() as d, patch.dict(os.environ, {'RUN_OUTPUT_SNAPSHOT_BENCHMARK': '1', 'TEST_WORKFORCE_MONGO_URI': 'remote', 'MONGO_URI': 'remote'}):
code = 'import os; assert "RUN_OUTPUT_SNAPSHOT_BENCHMARK" not in os.environ; assert "TEST_WORKFORCE_MONGO_URI" not in os.environ; assert "MONGO_URI" not in os.environ; assert os.environ["SOURCE_ADMISSION_LOCAL_MONGO"] == "1"'
c = v.command([sys.executable, '-c', code], env={'SOURCE_ADMISSION_LOCAL_MONGO': '1'})
self.assertEqual(v.run_check(c, Path(d), Path(d)/'run.log', 2)['status'], 'passed')
def test_timeout_terminates_descendants(self):
with tempfile.TemporaryDirectory() as d:
marker = Path(d)/'should-not-exist'
nested = f'import time; from pathlib import Path; time.sleep(1); Path({str(marker)!r}).touch()'
code = f'import subprocess,sys,time; subprocess.Popen([sys.executable,"-c",{nested!r}]); time.sleep(5)'
result = v.run_check(v.command([sys.executable, '-c', code]), Path(d), Path(d)/'run.log', .1)
self.assertEqual(result['status'], 'incomplete')
import time
time.sleep(1.1)
self.assertFalse(marker.exists())
if __name__ == '__main__':
unittest.main()