forked from longtao.wu/droboticflow
⚡ Add scoped development verification workflow
This commit is contained in:
parent
c8aa1ae714
commit
8a82ecf174
8
.scratch/verification-flow/work.md
Normal file
8
.scratch/verification-flow/work.md
Normal file
@ -0,0 +1,8 @@
|
||||
# Targeted development verification
|
||||
|
||||
Goal: implement the approved fast development / task / integration verification workflow.
|
||||
Agent: Codex, current unified-worktree conversation
|
||||
Base: codex/review-focus-layout-stability @ c8aa1ae7; continue the user's unified worktree without moving other work.
|
||||
Scope: AGENTS.md, verification guidance, npm aliases, local selector/runner and its tests.
|
||||
Out of scope: changing release policy, bypassing approvals, production actions or capacity acceptance.
|
||||
Risk: false-positive verification; test unmapped changes, skips, empty tests, timeouts and affected-scope expansion.
|
||||
@ -80,11 +80,17 @@ These principles guide implementation choices. Specific business contracts and t
|
||||
- Deploying to any company cluster requires the user's explicit approval immediately before the deployment; verification work must stop when that approval is absent.
|
||||
- Cluster verification is read-only against customer data: use isolated synthetic fixtures or existing records without mutation, and never delete, overwrite, or add customer data. If a test requires a write or destructive operation, stop and obtain explicit approval for that exact operation.
|
||||
|
||||
## Development verification
|
||||
|
||||
- Before choosing checks for a change, read `docs/agents/verification.md`. Start with `npm run verify:plan -- --base <task-base>`; use quick checks while editing and affected task checks once before delivery. Expand only for changed dependencies, failures or uncovered risk.
|
||||
- Reuse unchanged evidence within a task using the recorded source, dependency, configuration and environment identities. Report skipped, timed-out and unmapped coverage as pending; quick checks never replace release evidence.
|
||||
- Keep local implementation and verification moving within the authorized scope. The existing push, PR, production-data and deployment approval rules still apply.
|
||||
|
||||
## Local Environment
|
||||
|
||||
- Local replica stack (Vite 5173, Go API 3001, snapshot Mongo 27018) is the default verification surface before any PR.
|
||||
- Verification commands and known pitfalls (upstream test debt, session auth flag, workspace mount) are documented in `docs/HANDOFF.md`.
|
||||
- Comparing test failures against the known-failure baseline ALWAYS goes through `npm run test:frontend:gate` / `npm run test:go:gate` (they diff against `tests/known-failures.json`; only NEW failures fail). Never hand-diff failure lists with stash/scratch files. Formatting gate: `npm run format:check`.
|
||||
- Comparing test failures against the known-failure baseline ALWAYS goes through `npm run test:frontend:gate` / `npm run test:go:gate` (they diff against `tests/known-failures.json`; only NEW failures fail). Never hand-diff failure lists with stash/scratch files. Quick edits check changed-file formatting; the full formatting gate remains `npm run format:check` at integration/release scope.
|
||||
- The company cluster and BOS bucket are read-only by default; any write requires explicit user approval.
|
||||
|
||||
## Agent skills
|
||||
|
||||
@ -19,6 +19,10 @@
|
||||
- 质检员 `fanyu-full-qc@droboticflow.local`、验收员 `fanyu-full-acceptance@droboticflow.local`、管理员 `fanyu-admin@droboticflow.local`
|
||||
- **红线**:27018/27017 都是本地容器,随便写;公司集群(kubeconfig `local-data/fanyu.conf`)和 BOS 桶默认只读,任何写操作需用户明确确认。
|
||||
|
||||
## 日常验证入口
|
||||
|
||||
开发时先按[分层验证流程](agents/verification.md)选择检查。`verify:plan` 查看范围,`verify:quick` 用于编辑反馈,`verify:task` 用于本批业务流程验收;完整集成与发布检查独立执行。不要每次修改都重复本手册的所有验证命令。
|
||||
|
||||
## 1. 前端(5173)怎么改、怎么验证
|
||||
|
||||
**改动即生效**:Vite dev server 直连 `src/` 源码,保存文件浏览器自动热更(HMR)。没有"重新构建前端"这一步。
|
||||
|
||||
45
docs/agents/verification.md
Normal file
45
docs/agents/verification.md
Normal file
@ -0,0 +1,45 @@
|
||||
# 分层验证
|
||||
|
||||
适用于日常功能开发、修复及本批改动交付。发布仍使用 [release.py 流程](../release/README.md)。
|
||||
|
||||
## 工作步骤
|
||||
|
||||
1. 开工时用几句话确定业务结果、兼容范围、任务基线和风险。普通小改动不另写长计划;在已授权范围内连续完成实现、验证、修正。
|
||||
2. `npm run verify:plan -- --base <任务起点提交>` 查看范围。默认 `HEAD` 只包含未提交改动;交付已提交任务时必须指定任务起点,不能用干净树的空计划证明已提交代码通过。
|
||||
3. 编辑过程中运行 `npm run verify:quick -- --base <任务起点提交>`。相关单元测试被合并、去重执行;不默认启动浏览器、Mongo 或长时压测。
|
||||
4. 本批完成后运行 `npm run verify:task -- --base <任务起点提交>`,集中验证受影响的真实业务流程。失败只修复并重跑有关部分;新改动影响其他范围时扩大验证。
|
||||
5. 跨模块集成或公共依赖变化时运行 `verify:integration`。最终报告列出通过、失败、待验证和日志路径;部署单独执行发布门禁。
|
||||
|
||||
命令支持 `--plan`(不执行)及 `--budget-seconds N`。默认执行预算:quick 180 秒、task 600 秒、integration 3600 秒;预算耗尽会停止进程组并报告待验证,不自动重跑或改成通过。冷编译、安装依赖和机器负载会影响耗时,预算不是延迟承诺。
|
||||
|
||||
## 风险与完成标准
|
||||
|
||||
| 变更 | 开发检查 | 本批完成还需确认 |
|
||||
| --- | --- | --- |
|
||||
| 文档、纯样式 | 差异/格式检查 | 界面样式实际查看正常与错误状态;文案遵循现有最简规则 |
|
||||
| 交互、状态和页面切换 | 类型、有关单元回归 | 一条覆盖进入、操作、退出的真实浏览器流程 |
|
||||
| 领取、权限、保存、校验、返工 | 相关接口和规则回归 | 隔离存储、冲突、失败恢复及旧数据兼容 |
|
||||
| 查询、轮询、缓存 | 结果等价回归 | 相同代表性规模的查询量、扫描量、耗时、取消及并发对照 |
|
||||
| 工作流、迁移、资源或容量 | 专项验证 | 在途任务、回退和恢复;生产写入按具体授权执行 |
|
||||
|
||||
风险取决于行为和影响面,不按行数判断。共享工作台页面可能同时影响多个流程,脚本会扩大映射;确认只是文案/样式时可记录实际差异与人工检查,执行有关检查并说明未运行项,不把整批列为全通过。
|
||||
|
||||
120 人完整作业、8 小时运行、媒体容量和节点故障属于专项验收:容量承诺、相关架构或资源变更时执行,不绑定每次普通功能提交。发布基础门禁未因本规则减少。
|
||||
|
||||
## 选择与兜底
|
||||
|
||||
- 映射的唯一配置为 `scripts/verification-scopes.json`;当前覆盖审查切换、源校验/个人池、搜索分页、标注编辑、审核决定和统计面板。
|
||||
- Git 检测包含起点以来提交、暂存、未暂存、未跟踪文件,重命名检查新旧路径。改动的前端测试和 Go 同名测试自动加入。CSS 不因单纯样式修改启动全套类型/业务测试。
|
||||
- 未映射的前端代码扩大到前端基线门禁,未映射 Go 实现扩大到所属包;浏览器流程仍需人工核对。其他未映射行为报告待验证,补充映射或专项证据后才算交付完成。
|
||||
- 退出码 0 仅表示所选检查通过;1 表示检查失败;2 表示超时、跳过、未映射、执行中源码变化或入口异常。报告中的 deferred/notCovered 始终保留,不把快速通过解释成完整通过。
|
||||
- 遇到已有测试债务,使用既有 `test:frontend:gate` / `test:go:gate` 比较具体用例。不因失败而扩大 known-failures 基线,不手工忽略断言。
|
||||
|
||||
## 减少重复与环境准备
|
||||
|
||||
- 同一批同一输入只做一次成功验证;优先复用已有 npm 测试组、Go 编译/测试缓存和本地隔离 Mongo 服务,不为每次小改动重装依赖、重建镜像或重启整套副本。
|
||||
- 浏览器/数据库测试串行运行,继续使用各自独立合成数据和清理机制。共享服务不等于共享业务状态;公司集群和 BOS 不用于这些写入测试。
|
||||
- 日志及 JSON 报告写入忽略目录 `local-data/test-results/verify-*`,包含命令、基线、源码摘要、环境摘要、工具版本、安装锁文件摘要、耗时和未覆盖项。原始环境值不写入报告。
|
||||
- 同任务内复用记录前,核对源码/测试/夹具、依赖安装、配置、工具版本、测试环境均未变化。源码摘要和环境摘要不是依赖安装或外部数据状态的证明。此入口不做跨运行的自动“免测缓存”;Go 自有缓存正常使用。浏览器、数据库和生产证据按各自环境重新核验。
|
||||
- 更换依赖、修复失败、修改夹具、出现新问题或范围扩大时,重跑相关检查。测试报错、跳过和未启动永远不是可复用的成功证据。
|
||||
|
||||
此流程不增加日常审批。普通本地修改、测试及本地提交持续推进;推送、PR、公司部署、生产数据和资源修改仍遵循 `AGENTS.md` 的授权边界。
|
||||
@ -1,5 +1,8 @@
|
||||
# 泛宇发布试行流程
|
||||
|
||||
日常开发按[分层验证流程](../agents/verification.md)执行,不在每次编辑后重复本发布流程。
|
||||
本页发布门禁及 `fanyu-policy.json` 保持独立;快速检查报告不能作为缺失发布证据的替代。
|
||||
|
||||
## 开始或恢复工作
|
||||
|
||||
发布、部署故障、回退、上下文恢复时先读本页。状态的权威来源是
|
||||
|
||||
@ -93,7 +93,12 @@
|
||||
"test:review:decisions": "node --test tests/frontend/review-decision-controller.test.mjs tests/frontend/review-decision-completion.test.mjs tests/frontend/rework-resubmission.test.mjs",
|
||||
"test:browser:review-decisions": "REVIEW_DECISION_BROWSER_TEST=1 go test ./internal/api -run '^TestReviewDecisionControllerBrowserReplica$' -count=1 -v && node tests/browser/review-decision-failure.mjs && node tests/browser/review-decision-start-race.mjs",
|
||||
"test:review:drafts": "node --test tests/frontend/review-draft-persistence.test.mjs tests/frontend/rework-verdict-keys.test.mjs tests/frontend/review-draft-autosave.test.mjs tests/frontend/acceptance-legacy-compat.test.mjs",
|
||||
"test:browser:review-drafts": "REVIEW_DRAFT_BROWSER_TEST=1 go test ./internal/api -run '^TestReviewDraftPersistenceBrowserReplica$' -count=1 -v"
|
||||
"test:browser:review-drafts": "REVIEW_DRAFT_BROWSER_TEST=1 go test ./internal/api -run '^TestReviewDraftPersistenceBrowserReplica$' -count=1 -v",
|
||||
"verify:plan": "python3 scripts/verify-change.py --plan",
|
||||
"verify:quick": "python3 scripts/verify-change.py --mode quick",
|
||||
"verify:task": "python3 scripts/verify-change.py --mode task",
|
||||
"verify:integration": "python3 scripts/verify-change.py --mode integration",
|
||||
"test:verification": "python3 -m unittest discover -s tests/tooling -p test_verification.py"
|
||||
},
|
||||
"dependencies": {
|
||||
"@tailwindcss/vite": "^4.1.14",
|
||||
|
||||
249
scripts/verification-scopes.json
Normal file
249
scripts/verification-scopes.json
Normal file
@ -0,0 +1,249 @@
|
||||
{
|
||||
"version": 1,
|
||||
"scopes": [
|
||||
{
|
||||
"name": "review-navigation",
|
||||
"paths": [
|
||||
"^src/(pages/AnnotationOperations\\.tsx|hooks/useReviewContinuation\\.ts|services/api/review-continuation\\.ts)$",
|
||||
"^internal/api/review_(continuation|trace)",
|
||||
"^tests/browser/review-trace\\.mjs$"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/annotation-project-loading.test.mjs"
|
||||
],
|
||||
"go": {
|
||||
"./internal/api": [
|
||||
"^TestReviewContinuation"
|
||||
]
|
||||
},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"-run",
|
||||
"^TestReviewTraceBrowserReplica$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"REVIEW_TRACE_BROWSER_TEST": "1"
|
||||
},
|
||||
"tests": true
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "source-and-buffer",
|
||||
"paths": [
|
||||
"^src/(pages/AnnotationOperations\\.tsx|services/api/annotation-operations\\.ts)$",
|
||||
"^internal/(api|store)/annotation_(source|work_buffer|claim_read|claim_visit|random_claim)",
|
||||
"^internal/store/human_claim",
|
||||
"^tests/browser/annotation-source-validation-replica\\.mjs$"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/annotation-buffer-*.test.mjs",
|
||||
"tests/frontend/annotation-source-validation-ui.test.mjs"
|
||||
],
|
||||
"go": {
|
||||
"./internal/api": [
|
||||
"^TestSourceValidation",
|
||||
"^TestPendingSourceValidation",
|
||||
"^TestWorkBufferRetains",
|
||||
"^TestRandomClaim"
|
||||
],
|
||||
"./internal/store": [
|
||||
"^TestSourceAdmissionMemoryClaimFence$"
|
||||
]
|
||||
},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/store",
|
||||
"-run",
|
||||
"^TestSourceAdmissionMongo(ClaimFence|BatchMatchesLegacyPipeline)$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"SOURCE_ADMISSION_LOCAL_MONGO": "1"
|
||||
},
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"-run",
|
||||
"^TestSourceValidationBrowserReplica$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"SOURCE_VALIDATION_BROWSER_TEST": "1"
|
||||
},
|
||||
"tests": true
|
||||
}
|
||||
],
|
||||
"warnings": [
|
||||
"涉及准入/轮询:核对真实规模下查询次数、扫描量、超时与并发;局部回归不等于容量通过。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "pool-search",
|
||||
"paths": [
|
||||
"^src/pages/AnnotationOperations\\.tsx$",
|
||||
"^src/.*(pool|Pool|human-loop)",
|
||||
"^internal/(api|store)/(human_pool|admin_annotation_pool|human_loop)",
|
||||
"^tests/browser/(review-pool-search|admin-annotation-pool)"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/annotation-pool-*.test.mjs",
|
||||
"tests/frontend/review-pool-preview.test.mjs"
|
||||
],
|
||||
"go": {
|
||||
"./internal/api": [
|
||||
"^TestHumanPoolSearch",
|
||||
"^TestHumanPoolQCProjectScope"
|
||||
]
|
||||
},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"-run",
|
||||
"^TestReviewPoolSearchBrowserReplica$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"REVIEW_POOL_BROWSER_TEST": "1"
|
||||
},
|
||||
"tests": true
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "annotation-editing",
|
||||
"paths": [
|
||||
"^src/(pages/AnnotationOperations\\.tsx|components/CaptureAnnotationWorkbench\\.tsx)$",
|
||||
"^src/.*(slice-edit|SliceEdit|annotation-persistence)",
|
||||
"^internal/api/(annotation_draft|annotation_slice|slice_recovery|slice_edit)",
|
||||
"^tests/browser/(rework-boundary|slice-edit)"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/annotation-persistence.test.mjs",
|
||||
"tests/frontend/slice-edit-session.test.mjs",
|
||||
"tests/frontend/annotation-authoritative-mutation.test.mjs"
|
||||
],
|
||||
"go": {},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"go",
|
||||
"test",
|
||||
"./internal/api",
|
||||
"-run",
|
||||
"^TestReworkBoundaryBrowserReplica$",
|
||||
"-count=1",
|
||||
"-timeout=3m",
|
||||
"-v"
|
||||
],
|
||||
"env": {
|
||||
"REWORK_BOUNDARY_BROWSER_TEST": "1"
|
||||
},
|
||||
"tests": true
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "review-decisions",
|
||||
"paths": [
|
||||
"^src/(pages/AnnotationOperations\\.tsx|components/CaptureAnnotationWorkbench\\.tsx)$",
|
||||
"^src/.*([Rr]eview[Dd]raft|review-draft|[Rr]eview[Dd]ecision|review-decision)",
|
||||
"^internal/api/review_(draft|decision|confirmation)",
|
||||
"^tests/browser/review-(draft|decision|authoring)"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/review-decision-controller.test.mjs",
|
||||
"tests/frontend/review-decision-completion.test.mjs",
|
||||
"tests/frontend/review-draft-persistence.test.mjs",
|
||||
"tests/frontend/review-draft-autosave.test.mjs"
|
||||
],
|
||||
"go": {},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"npm",
|
||||
"run",
|
||||
"test:browser:review-drafts"
|
||||
],
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"npm",
|
||||
"run",
|
||||
"test:browser:review-decisions"
|
||||
],
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"npm",
|
||||
"run",
|
||||
"test:browser:review-permissions"
|
||||
],
|
||||
"tests": true
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "dashboards",
|
||||
"paths": [
|
||||
"^src/.*([Dd]ashboard|WorkforceStatistics|annotation-output)",
|
||||
"^internal/(api|store)/(production_(dashboard|stats)|annotation_output|workforce)",
|
||||
"^internal/(outputsnapshot|workforcestats)/",
|
||||
"^tests/browser/.*dashboard"
|
||||
],
|
||||
"frontend": [
|
||||
"tests/frontend/dashboard-presentation.test.mjs",
|
||||
"tests/frontend/production-quality-dates.test.mjs"
|
||||
],
|
||||
"go": {
|
||||
"./internal/api": [
|
||||
"^TestProductionDashboardScopeIsolationAndFullCorpus$"
|
||||
]
|
||||
},
|
||||
"task": [
|
||||
{
|
||||
"argv": [
|
||||
"npm",
|
||||
"run",
|
||||
"test:browser:dashboard"
|
||||
],
|
||||
"tests": true
|
||||
},
|
||||
{
|
||||
"argv": [
|
||||
"python3",
|
||||
"scripts/test_dashboard_stability.py"
|
||||
]
|
||||
}
|
||||
],
|
||||
"warnings": [
|
||||
"统计/查询改动须额外记录同数据规模的性能对照;完整 120 人容量另行验收。"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
290
scripts/verify-change.py
Normal file
290
scripts/verify-change.py
Normal file
@ -0,0 +1,290 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Select local development checks; never substitutes for release.py evidence."""
|
||||
import argparse
|
||||
import ast
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
POLICY = ROOT / 'scripts/verification-scopes.json'
|
||||
|
||||
|
||||
def git(root, *args):
|
||||
return subprocess.check_output(['git', *args], cwd=root, stderr=subprocess.PIPE).decode().strip('\n')
|
||||
|
||||
|
||||
def changed_files(root, base):
|
||||
revision = git(root, 'rev-parse', '--verify', base + '^{commit}')
|
||||
changed = git(root, 'diff', '--name-only', '--no-renames', '-z', revision, '--').split('\0')
|
||||
untracked = git(root, 'ls-files', '--others', '--exclude-standard', '-z').split('\0')
|
||||
return revision, sorted(set(filter(None, changed + untracked)))
|
||||
|
||||
|
||||
def command(argv, env=None, tests=False):
|
||||
return {'argv': argv, 'env': env or {}, 'tests': tests}
|
||||
|
||||
|
||||
def tooling_package_change(root, base):
|
||||
try:
|
||||
old = json.loads(git(root, 'show', base + ':package.json'))
|
||||
new = json.loads((root/'package.json').read_text())
|
||||
except (OSError, ValueError, subprocess.CalledProcessError):
|
||||
return False
|
||||
old_scripts, new_scripts = old.pop('scripts', {}), new.pop('scripts', {})
|
||||
names = {name for name in set(old_scripts) | set(new_scripts) if old_scripts.get(name) != new_scripts.get(name)}
|
||||
allowed = {'verify:plan', 'verify:quick', 'verify:task', 'verify:integration', 'test:verification'}
|
||||
return old == new and bool(names) and names <= allowed
|
||||
|
||||
|
||||
def plan_checks(root, files, policy, mode, base, tooling_package=False):
|
||||
quick = [command(['git', 'diff', '--check', base, '--'])]
|
||||
task = []
|
||||
warnings = []
|
||||
unresolved = []
|
||||
scopes = []
|
||||
frontend = set()
|
||||
go_patterns = {}
|
||||
matched = set()
|
||||
for scope in policy['scopes']:
|
||||
hits = [f for f in files if any(re.search(p, f) for p in scope['paths'])]
|
||||
if not hits:
|
||||
continue
|
||||
scopes.append(scope['name'])
|
||||
matched.update(hits)
|
||||
for pattern in scope.get('frontend', []):
|
||||
found = list(root.glob(pattern))
|
||||
if not found:
|
||||
unresolved.append('测试映射不存在: ' + pattern)
|
||||
frontend.update(str(p.relative_to(root)) for p in found)
|
||||
for pkg, patterns in scope.get('go', {}).items():
|
||||
go_patterns.setdefault(pkg, set()).update(patterns)
|
||||
task.extend(scope.get('task', []))
|
||||
warnings.extend(scope.get('warnings', []))
|
||||
web = any((f.startswith('src/') and not f.endswith('.css') or re.match(r'(package(-lock)?\.json|tsconfig.*|vite\.config.*)$', f)) and not (f == 'package.json' and tooling_package) for f in files)
|
||||
if web:
|
||||
quick.append(command(['npm', 'run', 'lint:web']))
|
||||
formatting = [f for f in files if (root/f).is_file() and
|
||||
(f.startswith('src/') and re.search(r'\.(ts|tsx|css|js)$', f) or
|
||||
f.startswith('tests/frontend/') and f.endswith('.mjs'))]
|
||||
if formatting:
|
||||
quick.append(command(['node', 'node_modules/prettier/bin/prettier.cjs', '--check', *formatting]))
|
||||
go_files = [f for f in files if f.endswith('.go') and (root/f).is_file()]
|
||||
if go_files:
|
||||
quick.append(command([sys.executable, 'scripts/verify-change.py', '--gofmt', *go_files]))
|
||||
for f in files:
|
||||
path = root/f
|
||||
if f.startswith('tests/frontend/') and f.endswith('.test.mjs') and path.exists():
|
||||
frontend.add(f)
|
||||
continue
|
||||
if f.endswith('.go'):
|
||||
pkg = './' + str(Path(f).parent)
|
||||
test_file = path if f.endswith('_test.go') else path.with_name(path.stem + '_test.go')
|
||||
fast = []
|
||||
if test_file.exists():
|
||||
names = re.findall(r'^func (Test\w+)\(', test_file.read_text(), re.M)
|
||||
fast = [name for name in names if not re.search(r'Browser|Mongo|Capacity|Soak|Load|Local|Benchmark', name)]
|
||||
if fast:
|
||||
go_patterns.setdefault(pkg, set()).update('^' + name + '$' for name in fast)
|
||||
if len(fast) != len(names) and f not in matched:
|
||||
unresolved.append(f + ': 外部环境测试需要显式任务级映射,不能将跳过当通过。')
|
||||
if len(fast) != len(names):
|
||||
warnings.append(f + ': 外部环境/容量测试不计入快速检查;核对任务级覆盖。')
|
||||
if f not in matched and (not f.endswith('_test.go') or not fast):
|
||||
# Package-wide fallback is explicit at task level, not every edit.
|
||||
task.append(command(['go', 'test', pkg, '-timeout=5m', '-v'], tests=True))
|
||||
continue
|
||||
if f in matched or f.endswith('.md') or f.startswith('docs/') and f.endswith(('.png', '.jpg', '.svg')):
|
||||
continue
|
||||
if f.startswith('src/') and f.endswith('.css'):
|
||||
warnings.append(f + ': 样式改动需人工查看受影响页面及异常状态。')
|
||||
continue
|
||||
if f.startswith('src/'):
|
||||
task.append(command(['npm', 'run', 'test:frontend:gate']))
|
||||
warnings.append(f + ': 未建立业务映射,任务级扩大到前端基线门禁;仍需人工确认浏览器流程。')
|
||||
continue
|
||||
if f.startswith('tests/browser/'):
|
||||
unresolved.append(f + ': 为该浏览器夹具补充可运行的任务级映射。')
|
||||
continue
|
||||
if f == 'package.json' and tooling_package:
|
||||
quick.append(command([sys.executable, '-m', 'unittest', 'discover', '-s', 'tests/tooling', '-p', 'test_verification.py'], tests=True))
|
||||
continue
|
||||
if f in ('package.json', 'package-lock.json', 'go.mod', 'go.sum') or re.match(r'(tsconfig|vite\.config)', f):
|
||||
task.extend([command(['npm', 'run', 'test:frontend:gate']), command(['npm', 'run', 'test:go:gate']),
|
||||
command(['npm', 'run', 'build:web']), command(['npm', 'run', 'test:production:boot'])])
|
||||
warnings.append('依赖/公共配置变化:任务级扩大到集成检查,核对安装与锁文件。')
|
||||
continue
|
||||
if f in ('scripts/verify-change.py', 'scripts/verification-scopes.json') or f.startswith('tests/tooling/'):
|
||||
quick.append(command([sys.executable, '-m', 'unittest', 'discover', '-s', 'tests/tooling', '-p', 'test_verification.py'], tests=True))
|
||||
continue
|
||||
if path.exists() and f.endswith('.py'):
|
||||
quick.append(command([sys.executable, 'scripts/verify-change.py', '--python-syntax', f]))
|
||||
elif path.exists() and f.endswith('.mjs'):
|
||||
quick.append(command(['node', '--check', f]))
|
||||
elif path.exists() and f.endswith('.sh'):
|
||||
quick.append(command(['bash', '-n', f]))
|
||||
unresolved.append(f + ': 尚无行为验证映射;补充映射/专项证据,语法通过不代表功能通过。')
|
||||
if frontend:
|
||||
quick.append(command(['node', '--test', *sorted(frontend)], tests=True))
|
||||
for pkg, patterns in sorted(go_patterns.items()):
|
||||
quick.append(command(['go', 'test', pkg, '-run', '|'.join(sorted(patterns)), '-timeout=2m', '-v'], tests=True))
|
||||
integration = [command(['npm', 'run', x]) for x in
|
||||
['lint:web', 'format:check', 'test:frontend:gate', 'test:go:gate', 'build:web', 'test:production:boot']]
|
||||
def unique(commands):
|
||||
out, seen = [], set()
|
||||
for c in commands:
|
||||
c = {**command(c['argv']), **c}
|
||||
key = json.dumps(c, sort_keys=True)
|
||||
if key not in seen:
|
||||
out.append(c)
|
||||
seen.add(key)
|
||||
return out
|
||||
quick, task = unique(quick), unique(task)
|
||||
# Full suite gates subsume targeted frontend/Go tests, not tooling checks.
|
||||
integration_extra = [c for c in quick if c['argv'][:2] not in (['node', '--test'], ['go', 'test'])
|
||||
and c['argv'] != ['npm', 'run', 'lint:web']
|
||||
and 'node_modules/prettier/bin/prettier.cjs' not in c['argv']]
|
||||
selected = quick if mode == 'quick' else unique(integration_extra + task + integration) if mode == 'integration' else unique(quick + task)
|
||||
return {'mode': mode, 'files': files, 'scopes': scopes, 'commands': selected,
|
||||
'deferred': task if mode == 'quick' else [], 'unmapped': sorted(set(unresolved)),
|
||||
'notes': sorted(set(warnings)), 'notCovered': [
|
||||
'生产业务写验收、生产数据处置及正式发布门禁', '120 人完整作业、8 小时运行、节点故障及真实媒体容量',
|
||||
'业务影响需人工核对;文件映射不能证明所有间接依赖均已覆盖']}
|
||||
|
||||
|
||||
def fingerprint(root):
|
||||
digest = hashlib.sha256()
|
||||
paths = git(root, 'ls-files', '-z', '--cached', '--others', '--exclude-standard').split('\0')
|
||||
for name in sorted(set(filter(None, paths))):
|
||||
p = root/name
|
||||
digest.update(name.encode())
|
||||
if p.is_symlink():
|
||||
digest.update(os.readlink(p).encode())
|
||||
elif p.is_file():
|
||||
digest.update(p.read_bytes())
|
||||
else:
|
||||
digest.update(b'<missing>')
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def run_check(check, root, logfile, timeout):
|
||||
start = time.monotonic()
|
||||
env = dict(os.environ)
|
||||
# Avoid accidentally enabling unrelated opt-in suites in an inherited shell.
|
||||
for key in list(env):
|
||||
if key.startswith(('TEST_', 'RUN_')) or key in ('MONGO_URI', 'DATABASE_URL') or re.search(r'(_BROWSER_TEST|_LOCAL_MONGO|_MONGO_HTTP|_MONGO_STORE|_LOAD_TEST|_CAPACITY_TEST)$', key):
|
||||
env.pop(key)
|
||||
env.update(check['env'])
|
||||
status, code = 'failed', None
|
||||
with logfile.open('w') as stream:
|
||||
try:
|
||||
child = subprocess.Popen(check['argv'], cwd=root, env=env, stdout=stream,
|
||||
stderr=subprocess.STDOUT, start_new_session=True)
|
||||
try:
|
||||
code = child.wait(timeout=timeout)
|
||||
status = 'passed' if code == 0 else 'failed'
|
||||
except (subprocess.TimeoutExpired, KeyboardInterrupt):
|
||||
status = 'incomplete'
|
||||
try:
|
||||
os.killpg(child.pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
try:
|
||||
child.wait(timeout=2)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
# Kill lingering grandchildren even if the parent exited on TERM.
|
||||
try:
|
||||
os.killpg(child.pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
child.wait()
|
||||
except OSError as error:
|
||||
stream.write(str(error))
|
||||
output = logfile.read_text()
|
||||
if status == 'passed' and check.get('tests'):
|
||||
if re.search(r'--- SKIP:|\[no test files\]|no tests to run|# SKIP|# skipped [1-9]|ℹ skipped [1-9]|skipped=[1-9]', output):
|
||||
status = 'incomplete'
|
||||
if re.search(r'Ran 0 tests|# tests 0|ℹ tests 0', output):
|
||||
status = 'incomplete'
|
||||
return {**check, 'status': status, 'exitCode': code, 'seconds': round(time.monotonic()-start, 3), 'log': str(logfile)}
|
||||
|
||||
|
||||
def runtime_identity(root):
|
||||
versions = {'python': sys.version.split()[0]}
|
||||
for name, argv in [('node', ['node', '--version']), ('go', ['go', 'version'])]:
|
||||
try:
|
||||
versions[name] = subprocess.check_output(argv, cwd=root, stderr=subprocess.PIPE, timeout=5).decode().strip()
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
versions[name] = 'unavailable'
|
||||
lock = root/'node_modules/.package-lock.json'
|
||||
versions['installedNpmLockDigest'] = hashlib.sha256(lock.read_bytes()).hexdigest() if lock.is_file() else 'unavailable'
|
||||
return versions
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--mode', choices=['quick', 'task', 'integration'], default='quick')
|
||||
parser.add_argument('--base', default='HEAD', help='包含该提交以来的提交、暂存、未暂存及未跟踪改动;默认仅当前未提交改动')
|
||||
parser.add_argument('--plan', action='store_true', help='只打印计划,不运行检查')
|
||||
parser.add_argument('--budget-seconds', type=int, help='总时间预算,超时报告待验证')
|
||||
parser.add_argument('--gofmt', nargs='+', help=argparse.SUPPRESS)
|
||||
parser.add_argument('--python-syntax', help=argparse.SUPPRESS)
|
||||
args = parser.parse_args(argv)
|
||||
if args.gofmt:
|
||||
result = subprocess.check_output(['gofmt', '-l', *args.gofmt], cwd=ROOT).decode()
|
||||
print(result, end='')
|
||||
return 1 if result.strip() else 0
|
||||
if args.python_syntax:
|
||||
ast.parse((ROOT/args.python_syntax).read_text())
|
||||
return 0
|
||||
budget = args.budget_seconds if args.budget_seconds is not None else {'quick': 180, 'task': 600, 'integration': 3600}[args.mode]
|
||||
if budget <= 0:
|
||||
parser.error('--budget-seconds must be positive')
|
||||
base, files = changed_files(ROOT, args.base)
|
||||
plan = plan_checks(ROOT, files, json.loads(POLICY.read_text()), args.mode, base, tooling_package_change(ROOT, base))
|
||||
print(json.dumps(plan, ensure_ascii=False, indent=2), flush=True)
|
||||
if args.plan:
|
||||
return 0
|
||||
folder = ROOT/'local-data/test-results'/f'verify-{time.time_ns()}'
|
||||
folder.mkdir(parents=True, mode=0o700)
|
||||
before = fingerprint(ROOT)
|
||||
report = {**plan, 'base': base, 'head': git(ROOT, 'rev-parse', 'HEAD'), 'inputDigest': before, 'runtime': runtime_identity(ROOT),
|
||||
'environmentDigest': hashlib.sha256(json.dumps(dict(os.environ), sort_keys=True).encode()).hexdigest(),
|
||||
'startedAt': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'results': []}
|
||||
start = time.monotonic()
|
||||
for index, check in enumerate(plan['commands']):
|
||||
remaining = budget - (time.monotonic()-start)
|
||||
if remaining <= 0:
|
||||
report['results'].append({**check, 'status': 'not-run', 'reason': 'budget exhausted'})
|
||||
continue
|
||||
result = run_check(check, ROOT, folder/f'{index:02}.log', remaining)
|
||||
report['results'].append(result)
|
||||
print(f"{result['status']}: {' '.join(check['argv'])} ({result['seconds']}s)", flush=True)
|
||||
if result['status'] != 'passed':
|
||||
report['results'].extend({**c, 'status': 'not-run', 'reason': 'previous check did not pass'} for c in plan['commands'][index+1:])
|
||||
break
|
||||
report['inputsChangedDuringRun'] = fingerprint(ROOT) != before
|
||||
statuses = {r['status'] for r in report['results']}
|
||||
code = 1 if 'failed' in statuses else 2 if (statuses - {'passed'} or plan['unmapped'] or report['inputsChangedDuringRun']) else 0
|
||||
report['status'] = 'failed' if code == 1 else 'incomplete' if code == 2 else 'selected-checks-passed'
|
||||
report['seconds'] = round(time.monotonic()-start, 3)
|
||||
(folder/'report.json').write_text(json.dumps(report, ensure_ascii=False, indent=2)+'\n')
|
||||
print(f"{report['status']}; 报告: {folder/'report.json'}")
|
||||
if plan['deferred']:
|
||||
print('任务级检查尚未执行;本结果不代表发布/容量验收通过。')
|
||||
return code
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
sys.exit(main())
|
||||
except (OSError, ValueError, subprocess.CalledProcessError) as error:
|
||||
print(f'检查入口失败: {type(error).__name__}', file=sys.stderr)
|
||||
sys.exit(2)
|
||||
147
tests/tooling/test_verification.py
Normal file
147
tests/tooling/test_verification.py
Normal file
@ -0,0 +1,147 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
spec = importlib.util.spec_from_file_location('verification', ROOT/'scripts/verify-change.py')
|
||||
v = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(v)
|
||||
POLICY = json.loads((ROOT/'scripts/verification-scopes.json').read_text())
|
||||
|
||||
|
||||
class VerificationTest(unittest.TestCase):
|
||||
def plan(self, files, mode='quick'):
|
||||
return v.plan_checks(ROOT, files, POLICY, mode, 'HEAD')
|
||||
|
||||
def test_documentation_is_small(self):
|
||||
plan = self.plan(['docs/example.md', 'AGENTS.md'])
|
||||
self.assertEqual(len(plan['commands']), 1)
|
||||
self.assertEqual(plan['deferred'], [])
|
||||
self.assertEqual(plan['unmapped'], [])
|
||||
|
||||
def test_common_page_expands_and_deduplicates(self):
|
||||
plan = self.plan(['src/pages/AnnotationOperations.tsx'])
|
||||
self.assertGreaterEqual(len(plan['scopes']), 4)
|
||||
self.assertTrue(plan['deferred'])
|
||||
self.assertFalse(any('go' == c['argv'][0] and c['env'] for c in plan['commands']))
|
||||
node_tests = [c for c in plan['commands'] if c['argv'][:2] == ['node', '--test']]
|
||||
self.assertEqual(len(node_tests), 1)
|
||||
self.assertEqual(len(node_tests[0]['argv'][2:]), len(set(node_tests[0]['argv'][2:])))
|
||||
|
||||
def test_source_task_includes_real_mongo_and_browser(self):
|
||||
plan = self.plan(['internal/store/annotation_source_admission.go'], 'task')
|
||||
self.assertTrue(any(c['env'].get('SOURCE_ADMISSION_LOCAL_MONGO') for c in plan['commands']))
|
||||
self.assertTrue(any(c['env'].get('SOURCE_VALIDATION_BROWSER_TEST') for c in plan['commands']))
|
||||
self.assertIn('source-and-buffer', plan['scopes'])
|
||||
self.assertEqual(plan['unmapped'], [])
|
||||
|
||||
def test_script_alias_only_does_not_trigger_dependency_suite(self):
|
||||
plan = v.plan_checks(ROOT, ['package.json'], POLICY, 'quick', 'HEAD', tooling_package=True)
|
||||
self.assertEqual(plan['deferred'], [])
|
||||
self.assertFalse(any('lint:web' in c['argv'] for c in plan['commands']))
|
||||
self.assertTrue(any('unittest' in c['argv'] for c in plan['commands']))
|
||||
|
||||
def test_css_does_not_run_business_suites(self):
|
||||
plan = self.plan(['src/example.css'])
|
||||
self.assertEqual(plan['deferred'], [])
|
||||
self.assertFalse(any('lint:web' in c['argv'] for c in plan['commands']))
|
||||
|
||||
def test_integration_does_not_repeat_targeted_frontend_tests(self):
|
||||
plan = self.plan(['src/pages/AnnotationOperations.tsx'], 'integration')
|
||||
self.assertFalse(any(c['argv'][:2] == ['node', '--test'] for c in plan['commands']))
|
||||
self.assertEqual(sum('test:frontend:gate' in c['argv'] for c in plan['commands']), 1)
|
||||
|
||||
def test_integration_keeps_tooling_regressions(self):
|
||||
plan = self.plan(['scripts/verify-change.py'], 'integration')
|
||||
self.assertTrue(any('unittest' in c['argv'] for c in plan['commands']))
|
||||
|
||||
def test_new_frontend_falls_back_at_task_level(self):
|
||||
plan = self.plan(['src/components/FutureShared.tsx'])
|
||||
self.assertTrue(any('test:frontend:gate' in c['argv'] for c in plan['deferred']))
|
||||
self.assertFalse(any('test:frontend:gate' in c['argv'] for c in plan['commands']))
|
||||
|
||||
def test_unknown_script_is_not_silently_covered(self):
|
||||
self.assertTrue(self.plan(['scripts/new-data-migration.py'])['unmapped'])
|
||||
self.assertTrue(self.plan(['tests/browser/new-flow.mjs'])['unmapped'])
|
||||
|
||||
def test_changed_frontend_test_is_always_included(self):
|
||||
f = 'tests/frontend/annotation-buffer-summary.test.mjs'
|
||||
plan = self.plan([f])
|
||||
self.assertTrue(any(f in c['argv'] for c in plan['commands']))
|
||||
|
||||
def test_deleted_go_file_gets_package_fallback(self):
|
||||
plan = self.plan(['internal/store/removed.go'])
|
||||
self.assertTrue(any('./internal/store' in c['argv'] for c in plan['deferred']))
|
||||
|
||||
def test_scope_tests_and_browser_names_exist(self):
|
||||
for scope in POLICY['scopes']:
|
||||
for pattern in scope.get('frontend', []):
|
||||
self.assertTrue(list(ROOT.glob(pattern)), pattern)
|
||||
for c in scope.get('task', []):
|
||||
if c['argv'][0] == 'go':
|
||||
package = ROOT/c['argv'][2]
|
||||
self.assertTrue(package.is_dir())
|
||||
if c['argv'][:2] == ['npm', 'run']:
|
||||
self.assertIn(c['argv'][2], json.loads((ROOT/'package.json').read_text())['scripts'])
|
||||
|
||||
def test_changes_include_committed_staged_unstaged_untracked_and_renamed(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
root = Path(d)
|
||||
def git(*args):
|
||||
return subprocess.check_output(['git', *args], cwd=root, stderr=subprocess.DEVNULL).decode().strip()
|
||||
git('init')
|
||||
git('config', 'user.email', 'test@example.invalid')
|
||||
git('config', 'user.name', 'Test')
|
||||
(root/'a').write_text('initial')
|
||||
git('add', 'a'); git('commit', '-m', 'initial')
|
||||
base = git('rev-parse', 'HEAD')
|
||||
git('mv', 'a', 'b'); git('commit', '-m', 'rename')
|
||||
(root/'staged').write_text('staged'); git('add', 'staged')
|
||||
(root/'b').write_text('modified')
|
||||
(root/'new').write_text('untracked')
|
||||
_, files = v.changed_files(root, base)
|
||||
self.assertEqual(files, ['a', 'b', 'new', 'staged'])
|
||||
before = v.fingerprint(root)
|
||||
(root/'new').write_text('changed during check')
|
||||
self.assertNotEqual(before, v.fingerprint(root))
|
||||
with self.assertRaises(subprocess.CalledProcessError):
|
||||
v.changed_files(root, 'does-not-exist')
|
||||
|
||||
def test_runner_failure_skip_empty_and_timeout_never_pass(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
for code, expected in [
|
||||
('print("ok")', 'passed'),
|
||||
('raise SystemExit(3)', 'failed'),
|
||||
('print("--- SKIP: TestOptIn")', 'incomplete'),
|
||||
('print("testing: warning: no tests to run")', 'incomplete'),
|
||||
('import time; time.sleep(3)', 'incomplete'),
|
||||
]:
|
||||
result = v.run_check(v.command([sys.executable, '-c', code], tests=True), Path(d), Path(d)/'run.log', .1)
|
||||
self.assertEqual(result['status'], expected, code)
|
||||
|
||||
def test_runner_only_enables_declared_opt_ins(self):
|
||||
from unittest.mock import patch
|
||||
with tempfile.TemporaryDirectory() as d, patch.dict(os.environ, {'RUN_OUTPUT_SNAPSHOT_BENCHMARK': '1', 'TEST_WORKFORCE_MONGO_URI': 'remote', 'MONGO_URI': 'remote'}):
|
||||
code = 'import os; assert "RUN_OUTPUT_SNAPSHOT_BENCHMARK" not in os.environ; assert "TEST_WORKFORCE_MONGO_URI" not in os.environ; assert "MONGO_URI" not in os.environ; assert os.environ["SOURCE_ADMISSION_LOCAL_MONGO"] == "1"'
|
||||
c = v.command([sys.executable, '-c', code], env={'SOURCE_ADMISSION_LOCAL_MONGO': '1'})
|
||||
self.assertEqual(v.run_check(c, Path(d), Path(d)/'run.log', 2)['status'], 'passed')
|
||||
|
||||
def test_timeout_terminates_descendants(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
marker = Path(d)/'should-not-exist'
|
||||
nested = f'import time; from pathlib import Path; time.sleep(1); Path({str(marker)!r}).touch()'
|
||||
code = f'import subprocess,sys,time; subprocess.Popen([sys.executable,"-c",{nested!r}]); time.sleep(5)'
|
||||
result = v.run_check(v.command([sys.executable, '-c', code]), Path(d), Path(d)/'run.log', .1)
|
||||
self.assertEqual(result['status'], 'incomplete')
|
||||
import time
|
||||
time.sleep(1.1)
|
||||
self.assertFalse(marker.exists())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Loading…
x
Reference in New Issue
Block a user