📝 Record the 2026-10-20 internal test lane revision and tickets 06–15
Some checks failed
web-ci / checks (push) Has been cancelled
Some checks failed
web-ci / checks (push) Has been cancelled
Recovered from uncommitted work in the dev-mainline worktree (written 2026-10-19/20, never committed). Revises ADR 0004 and the spec/map: the lane verifies annotation → QC → final acceptance only, uses node-local storage with no BOS credentials, and promotes the same image to production. Adds the two domain terms and tickets 06–15. Docs only; no code, configuration or cluster changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
b542d71119
commit
de96cfc2be
@ -1,6 +1,6 @@
|
||||
# 独立内部测试环境落地计划
|
||||
|
||||
Status: proposed
|
||||
Status: superseded — 2026-10-20 ADR-0004 修订替代了专用节点、新云盘、合成素材先行、Ingress 与 5 账号/6 样本等前提;见 [spec.md](spec.md) 当前修订
|
||||
Date: 2026-09-27
|
||||
Scope: 本轮仅制定计划,不部署、不申请付费资源、不复制客户样本、不提交 PR。
|
||||
|
||||
|
||||
@ -25,3 +25,7 @@ Blocked by: 无(触发=用户启动执行模式/部署后线上验证)
|
||||
## 验证命令
|
||||
|
||||
- 复现栈/测试通道浏览器手工证据记录到本文件 `## Comments`
|
||||
|
||||
## Comments
|
||||
|
||||
- 2026-10-20:内部测试环境修订后,第 1 项(无 movi 服务时的浏览器播放)由 [12](12-cluster-http-verifier.md) 的浏览器播放场景覆盖,结果作为 `video-playback` 测试环境部分的证据;本票其余内容不变。
|
||||
|
||||
49
.scratch/internal-test-lane/issues/06-renderer-local-node.md
Normal file
49
.scratch/internal-test-lane/issues/06-renderer-local-node.md
Normal file
@ -0,0 +1,49 @@
|
||||
# Render the environment on a pinned node with local storage and a private entry
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: none
|
||||
|
||||
Spec: Environment. Code: `scripts/internal_test_lane/environment.py`, `tests/internal_test_lane/test_environment.py`.
|
||||
|
||||
Today the renderer creates an RWX workspace PVC and an Ingress, forces the Secure cookie and does
|
||||
not pin nodes. The cluster has only `csi-bos` (no-provisioner) and no IngressClass.
|
||||
|
||||
- Replace the RWX PVC and Ingress with static local PVs for Mongo and workspace: `Retain`, RWO,
|
||||
node affinity on `kubernetes.io/hostname`, explicit claim binding, per-batch names as today.
|
||||
- Config gains `node` and `hostRoot`. Refuse an empty node, a `hostRoot` outside a dedicated
|
||||
internal-test prefix, `droboticflow-fanyu`, and any production PV/PVC name.
|
||||
- Pin API, processor, Mongo and initializer to the node.
|
||||
- Render a one-time directory-preparation pod (pattern: `k8s/qwen-node-cache.prepare-dir.template.yaml`)
|
||||
that creates only this batch's directories under `hostRoot`, owned for the Mongo user.
|
||||
- Entry mode `private-forward`: ClusterIP only, no Ingress or TLS secret, and
|
||||
`DROBOTICFLOW_SESSION_COOKIE_SECURE=false` only in this mode; other modes keep `true`.
|
||||
- Processor runs `ANNOTATION_PROCESSOR_ROLE=all`, concurrency 1.
|
||||
- Refuse any `BOS_*`, `ANNOTATION_WORKSPACE_*` or csi-bos reference.
|
||||
- `internal-test-quota` keeps its stable name and fits API, processor, Mongo, initializer and the
|
||||
preparation pod with finite limits.
|
||||
- Keep default-deny; allow only app pods → Mongo. Whether port-forward needs an extra rule under
|
||||
Cilium stays an explicit unverified item; never open namespace-external ingress to get it working.
|
||||
|
||||
Capacity (PV size and ResourceQuota do not limit what is actually written to a local disk, and
|
||||
node `.4`'s disk is shared with kubelet and images; filling it evicts pods on that node, which can
|
||||
include production operator Jobs):
|
||||
|
||||
- Config gains a capacity budget: bundle source bytes, an output factor for slice media and results
|
||||
(default 2.0 × source until a real batch measures it), Mongo, dump, and retention of 3 archived
|
||||
batches plus the current one. The renderer prints the total.
|
||||
- The directory-preparation pod reports `df` for the filesystem holding `hostRoot`. The operator
|
||||
imports that report; the tool refuses to prepare when free space minus the budget would leave less
|
||||
than 25% of the filesystem free (kubelet hard eviction defaults to 10%). The threshold can be
|
||||
raised, not set below 15%.
|
||||
- The same preflight is required before seed, baseline start, upgrade and archive. A crossed
|
||||
threshold means stop new work (operator scales API and processor to 0) and record it; nothing is
|
||||
deleted automatically.
|
||||
- Every pod gets ephemeral-storage requests and limits, so log and container-layer growth evicts the
|
||||
test pod rather than its neighbours.
|
||||
- Use a filesystem project quota on `hostRoot` when the node filesystem supports it. Otherwise the
|
||||
batch record states "disk usage not enforced by the filesystem; bounded by preflight only".
|
||||
|
||||
Done when: public-boundary tests cover the rendered shape, the budget arithmetic, the threshold
|
||||
refusal and each other refusal, existing tests are
|
||||
updated, and the deployment section of `docs/internal-test-lane.md` matches. No cluster access.
|
||||
@ -0,0 +1,24 @@
|
||||
# Derive test runtime config from a production snapshot
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 06
|
||||
|
||||
Spec: Environment (runtime configuration).
|
||||
|
||||
- Input: a read-only snapshot of the production `droboticflow-api` and
|
||||
`droboticflow-annotation-processor` Deployments plus the ConfigMaps they reference. Secrets are
|
||||
never read; only references appear.
|
||||
- Copy non-secret env from the snapshot, then apply an explicit override allowlist:
|
||||
- isolation: namespace / `K8S_NAMESPACE`, Mongo URI and database (test Secret),
|
||||
`WORKSPACE_MOUNT_PATH`, removal of `BOS_ENDPOINT` and every `ANNOTATION_WORKSPACE_*` key,
|
||||
the private-forward cookie setting;
|
||||
- capacity: replicas, `ANNOTATION_PROCESSOR_ROLE`, concurrency keys, resources.
|
||||
- `DROBOTICFLOW_DATABASE_BOOTSTRAP` follows production (currently `verify-existing`).
|
||||
- A reference to a production Secret or ConfigMap that the test namespace lacks is refused, not dropped.
|
||||
- Emit `config-diff.json` per batch (key, class, before/after; secret refs shown as references);
|
||||
`batch.json` binds its SHA256.
|
||||
- Refuse snapshots from another namespace or with a missing Deployment.
|
||||
|
||||
Done when: tests cover the allowlist, refusal of unlisted differences and stable diff output, and
|
||||
the guide explains the diff and why other differences disqualify a batch as evidence.
|
||||
28
.scratch/internal-test-lane/issues/08-bundle-seed-v2.md
Normal file
28
.scratch/internal-test-lane/issues/08-bundle-seed-v2.md
Normal file
@ -0,0 +1,28 @@
|
||||
# Bundle and seed v2: production projects, two suppliers, five business roles
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: none
|
||||
|
||||
Spec: Data and accounts.
|
||||
|
||||
Today `scripts/internal_test_lane/lifecycle.py` (around lines 155, 201, 476) requires exactly six
|
||||
samples, three each for BLF-data and SL-data, and `internal/testlane/plan.go` (around lines
|
||||
133–302) hardcodes six samples, two dictionaries, those two projects, one supplier and the roles
|
||||
admin / supplier_manager / annotator / quality_reviewer / acceptance_reviewer.
|
||||
|
||||
- New schema `droboticflow.internal-test-seed/v2`: projects come from the bundle (name,
|
||||
workflow-definition snapshot, three dictionaries each); 1–12 samples, at most 3 per project;
|
||||
kinds `scratch`, `snapshot` at `annotation`, `snapshot` at `quality_review`.
|
||||
- Suppliers `test-a` and `test-b`, both authorized on every test project; tasks start unassigned
|
||||
in the shared project pool.
|
||||
- Accounts: annotator and quality reviewer per supplier, plus a `test-a` acceptance reviewer with
|
||||
supplier final-acceptance authorization. No platform admin, no supplier manager.
|
||||
`credentials.py` emits exactly these five.
|
||||
- The seeded database must start under the production bootstrap mode (`verify-existing`).
|
||||
- v1 bundles exist only as local synthetic fixtures: refuse v1 with a clear message. Keep every
|
||||
existing refusal (non-empty target, non-test URI, plan/manifest hash mismatch).
|
||||
- `tests/internal_test_lane/make_synthetic_bundle.py` produces v2.
|
||||
|
||||
Done when: Go and Python tests are updated, a fresh local Mongo seeds from a synthetic v2 bundle,
|
||||
and the guide documents v2.
|
||||
@ -0,0 +1,42 @@
|
||||
# Export samples from a restored backup and download their media
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 08
|
||||
|
||||
Spec: Data and accounts, run-sequence steps 1–2.
|
||||
|
||||
Restoring the backup (today's `scripts/local/fetch-mongo-snapshot.sh` execs into a production API
|
||||
pod, stages the archive in chunks under that pod's `/tmp`, then runs `docker rm -f
|
||||
fanyu-mongo-restore`, recreates 27018 and restores with `--drop`; its default backup directory is
|
||||
the fixed `fanyu-v58-ca15d60d`):
|
||||
|
||||
- Leave that script unchanged for its existing 27018 use; add a separate restore path for the lane.
|
||||
- List backup directories under `/workspace/system/backups/` and show the chosen one's name, date
|
||||
and archive size. If the newest backup is older than the selection window, stop: taking a new
|
||||
production backup is a separate approved operation.
|
||||
- Read the archive by byte ranges straight from the mounted file, with no staging in the production
|
||||
pod. If staging cannot be avoided, check the archive size against the pod's free ephemeral space
|
||||
first and refuse above a threshold. Verify the full SHA256 locally.
|
||||
- Restore into a new container with an explicit unique name and an explicit unused loopback port.
|
||||
Refuse when the name exists or the port is in use, and refuse 27017–27020 (existing local
|
||||
stacks). Never `docker rm -f` an existing container and never `--drop` into one.
|
||||
|
||||
Exporting:
|
||||
|
||||
- Connect only to an explicit loopback Mongo URI; refuse non-loopback hosts and the existing
|
||||
local containers on 27017/27018.
|
||||
- `candidates`: per project active in the backup's recent window (default 14 days, flag), list
|
||||
tasks by stage with Capture name, part count, required media keys and sizes. No account names
|
||||
or phone numbers.
|
||||
- `export`: from chosen task IDs (at most 3 per project, 12 total) write a v2 bundle skeleton:
|
||||
project config, workflow-definition snapshot, dictionaries, slice/action content for snapshot
|
||||
kinds, and required media (both cameras, both PTS, IMU) as BOS keys.
|
||||
- Download only those keys, read-only, with credentials from the untracked local file (same
|
||||
approach as `scripts/local/download-capture.sh`); record size and SHA256 into bundle assets;
|
||||
resume safely; never write to BOS.
|
||||
- Print the total size before downloading; refuse above a cap (default 20 GiB).
|
||||
|
||||
Done when: tests use a synthetic Mongo fixture, a mocked BOS client and a mocked `kubectl`/`docker`
|
||||
boundary that prove the refusals (existing container, busy port, staging over threshold). This ticket does not touch
|
||||
a real backup or BOS.
|
||||
@ -0,0 +1,31 @@
|
||||
# release.py: selectable policy, validator coverage, worker guard
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: none
|
||||
|
||||
Spec: Promotion. Code: `scripts/release.py` (policy hardcoded at line 19), `docs/release/`.
|
||||
|
||||
- `--policy <path>`; without it, behaviour stays on `docs/release/fanyu-policy.json`, so existing
|
||||
records keep working. Historical records with another policy hash follow the existing read-only rules.
|
||||
- The Fanyu policy adds `droboticflow-annotation-validator`: same image patch, its own role env
|
||||
(`validation`) and resources preserved; the cluster check verifies each Deployment's role.
|
||||
- New `docs/release/internal-test-policy.json`: namespace `droboticflow-internal-test`, local-PV
|
||||
workspace, no bucket/BOS checks, API + processor (role `all`). Each policy refuses the other
|
||||
namespace by name and UID.
|
||||
- Split `annotation-delivery` in the Fanyu policy's `requiredEvidence` into `annotation-results`
|
||||
and `delivery-packaging`. `delivery-packaging` is satisfied by either:
|
||||
- a packaging test record from an environment that runs the worker and package operators (the
|
||||
test lane cannot produce it), or
|
||||
- an unchanged-delivery-inputs check that `release.py` computes itself: the diff between the
|
||||
production API build commit and the candidate over a policy-declared path list is empty. The path
|
||||
list (result-JSON writers, release partition, part-index delivery, package operators) is derived
|
||||
from the code and reviewed with the policy. A non-empty diff requires the packaging test record.
|
||||
Records under the old policy hash stay historical; an in-progress record must be prepared again.
|
||||
- Worker guard: read the production worker's build commit from the read-only snapshot; refuse
|
||||
the plan if `internal/worker` or `cmd/worker` differ between it and the candidate, pointing to
|
||||
separate verification.
|
||||
|
||||
Done when: `tests/release` (simulated Kubernetes) covers policy selection, three-Deployment patch
|
||||
and partial failure, the worker guard, cross-policy refusal, and both `delivery-packaging` paths
|
||||
(empty diff accepted, non-empty diff without a packaging record refused), and `docs/release/README.md` is updated.
|
||||
@ -0,0 +1,36 @@
|
||||
# Bind test-lane batch reports to release evidence
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 10, 13
|
||||
|
||||
Spec: Promotion.
|
||||
|
||||
- `release.py evidence <id> --check <c> --test-lane-batch <batch-id>` reads the local batch
|
||||
record and requires: state `business-verified`, `candidateImage` equal to the release image
|
||||
digest, a config diff limited to allowed classes, and unchanged report hashes. It stores the
|
||||
batch ID, digests and report hash; anything else is refused. Batches in `failed` or
|
||||
`outcome-unknown` are always refused.
|
||||
- Start point: for the upgrade-rehearsal part of `change-safety`, the batch's `baselineImage` must
|
||||
equal the `before` image the release plan recorded for each Deployment it covers. Production may run
|
||||
different starting digests (today the validator runs `7699d455…` while api and processor run
|
||||
`1a1ead7a…`); a Deployment whose `before` image differs from `baselineImage` is listed as not
|
||||
covered by the rehearsal.
|
||||
- Each check needs named scenarios in the report, all `passed`. A scenario that is missing, failed or
|
||||
unverified refuses the check.
|
||||
|
||||
| Release check | Required scenarios |
|
||||
| --- | --- |
|
||||
| `role-login` | five business-role logins; role-boundary refusals (admin logins come from production smoke) |
|
||||
| `supplier-permissions` | cross-supplier claim refused; other supplier's annotation invisible |
|
||||
| `review-acceptance` | QC pass; QC return → rework resubmit; final acceptance |
|
||||
| `slice-recovery` | processor restart with work resumed; in-flight tasks resumed after upgrade |
|
||||
| `video-playback` | browser playback for both cameras: first frame, playback advancing (test-lane part) |
|
||||
| `annotation-results` | accepted results contain normal, discard, invalid and blank cases as entered |
|
||||
| `change-safety` (rehearsal part) | start-point match; in-flight state created on baseline; resumed on candidate |
|
||||
|
||||
`delivery-packaging` cannot be satisfied by a batch (see ticket 10).
|
||||
- Production-side smoke evidence (ticket 14) is still required by `verify`.
|
||||
|
||||
Done when: tests cover digest mismatch, start-point mismatch, a partially covered Deployment, a
|
||||
missing or failed scenario, an unverified or failed batch, a tampered report and a disallowed check.
|
||||
@ -0,0 +1,29 @@
|
||||
# HTTP business verifier for cluster batches
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 08
|
||||
|
||||
Spec: Batch flow steps 2 and 4. Today `tests/internal_test_lane/verify_local.py` reads fixed
|
||||
local paths and assumes one supplier.
|
||||
|
||||
- Inputs: loopback base URL (port-forward), the private accounts file and the executed plan.
|
||||
Refuse any non-loopback host.
|
||||
- Scenarios: five logins and role-boundary refusals; `test-a` claims and `test-b` can neither
|
||||
claim nor see that annotation; slicing → normal / discard / invalid / blank → submit → QC return
|
||||
→ rework resubmit → QC pass → final acceptance; snapshot samples; frame range.
|
||||
- Browser playback with Playwright (already a dependency): for both cameras in normal and focus
|
||||
layout, the first frame renders (video ready and a non-blank sampled frame) and playback time
|
||||
advances within ±20% of wall-clock time over 5 s. Record what happens without a movi service; this
|
||||
covers item 1 of ticket 05 for the test lane. HTTP media responses alone never pass this scenario.
|
||||
- `--phase in-flight` stops after leaving claimed, slicing-pending and submission-pending tasks;
|
||||
`--phase after-upgrade` resumes exactly those tasks.
|
||||
- Processor-restart recovery: the tool prints the restart step for the operator, then waits and
|
||||
checks the work resumes; it never restarts pods itself.
|
||||
- Every scenario has a stable ID and a `passed` / `failed` / `unverified` status, matching the table
|
||||
in ticket 11.
|
||||
- One final JSON report: batch ID, observed build commit/digest (cross-checked with `batch.json`),
|
||||
sample keys, task and item IDs, per-check status. No credentials; never overwrites.
|
||||
|
||||
Done when: it runs against a fresh local stack with two suppliers, and tests cover report shape
|
||||
and refusals.
|
||||
26
.scratch/internal-test-lane/issues/13-upgrade-rehearsal.md
Normal file
26
.scratch/internal-test-lane/issues/13-upgrade-rehearsal.md
Normal file
@ -0,0 +1,26 @@
|
||||
# Batch flow: baseline-to-candidate upgrade rehearsal
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 06, 07, 10, 12
|
||||
|
||||
Spec: Batch flow.
|
||||
|
||||
- Batch config gains pinned `baselineImage` and `candidateImage`; `batch.json` binds both.
|
||||
`baselineImage` comes from a read-only snapshot of the production Deployments taken at prepare
|
||||
time, and the snapshot's per-Deployment images are recorded for the start-point check in ticket 11.
|
||||
- States: prepared → seeded → baseline-running → in-flight-created → upgraded →
|
||||
business-verified → archived. Each transition imports operator-provided evidence as today;
|
||||
the tool never probes the cluster.
|
||||
- The upgrade runs `release.py` with the internal-test policy, the same code path as production.
|
||||
- If the candidate needs a migration that production would run as a separate reviewed step, the
|
||||
rehearsal runs that same procedure in the test namespace; otherwise its change-safety part
|
||||
cannot be recorded.
|
||||
- `status` shows which digest ran in each phase.
|
||||
- Failure path: any phase may move to `failed` or `outcome-unknown`. Either one means stop new work
|
||||
(operator scales API and processor to 0), archive with that state kept (ticket 15), and never
|
||||
promote the batch to `business-verified`. The next attempt is a new batch with new identities; a
|
||||
contaminated batch is not resumed.
|
||||
|
||||
Done when: lifecycle tests cover transitions, refusals and both failure states, a local end-to-end run uses two locally
|
||||
built binaries (production commit → dev-mainline HEAD), and the guide is updated.
|
||||
31
.scratch/internal-test-lane/issues/14-production-smoke.md
Normal file
31
.scratch/internal-test-lane/issues/14-production-smoke.md
Normal file
@ -0,0 +1,31 @@
|
||||
# Post-promotion production smoke, no customer-data writes by default
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 12
|
||||
|
||||
Spec: Promotion (production smoke).
|
||||
|
||||
- Uses the untracked production verification accounts file. Before any write mode, confirm those
|
||||
accounts belong to the frozen internal organization.
|
||||
- An HTTP method does not decide whether a call writes. Task detail GET runs interrupted-slice
|
||||
recovery and can finalize a pending submission (`internal/api/annotation_operations.go`, around
|
||||
line 999); frame-range GET persists a timeline index on a cache miss
|
||||
(`internal/api/annotation_timeline_index.go`, around line 102). Keep an endpoint list in the tool,
|
||||
each entry classified by server-side effect: none / possible write / forbidden.
|
||||
- Default mode promises no customer-data writes, not zero writes: logins (admin roles when accounts
|
||||
are supplied), project and task lists, opening an existing internal-organization task, browser
|
||||
playback (reusing ticket 12's check), frame range. Endpoints marked "possible write" are called
|
||||
only on internal-organization objects, and the report lists every such call and the object it
|
||||
touched. Calls outside the list, and any call on a non-internal organization, are refused.
|
||||
- BOS path: the smoke asserts that frame range loads from the BOS source of an internal-organization
|
||||
task (this exercises the BOS read and HEAD). Cache invalidation after a source ETag change would
|
||||
need a production object to change, so it stays unverified.
|
||||
- Write mode needs `--write --confirm <token>` and explicit internal-organization task IDs:
|
||||
claim → slicing → submit → QC → acceptance. Refuse customer organization IDs, report how many
|
||||
internal tasks remain, never create tasks.
|
||||
- Report JSON bound to the release ID and the observed build commit.
|
||||
|
||||
Done when: tests against a mocked HTTP server cover unlisted-endpoint refusal, organization
|
||||
refusal, the side-effect report (including cold frame-range cache and an interrupted task) and the
|
||||
confirm token. This ticket does not access production.
|
||||
17
.scratch/internal-test-lane/issues/15-batch-archive.md
Normal file
17
.scratch/internal-test-lane/issues/15-batch-archive.md
Normal file
@ -0,0 +1,17 @@
|
||||
# Archive cluster batches to local storage
|
||||
|
||||
Type: task
|
||||
Status: ready-for-agent
|
||||
Blocked by: 06
|
||||
|
||||
Spec: Batch flow step 5.
|
||||
|
||||
- Render an archive pod in the test namespace that runs `mongodump` of the batch database into
|
||||
the batch directory on the node; render the pull commands into
|
||||
`local-data/internal-test-lane/batches/<batch-id>/` with a SHA256 manifest.
|
||||
- Dumps count toward the capacity budget in ticket 06, and archiving requires a passing preflight.
|
||||
- `archive` evidence additionally requires the pulled dump, reports and config diff with hashes.
|
||||
- Retention: latest 3 archived batches plus the current one. `cleanup-plan` also lists the
|
||||
node-side directories for a separately approved cluster cleanup; the tool never deletes remotely.
|
||||
|
||||
Done when: tests cover manifest verification and retention selection.
|
||||
@ -128,3 +128,82 @@ Q1–Q3 均已采用推荐项。
|
||||
- [01 environment](issues/01-environment.md), [02 seed](issues/02-seed.md), [03 lifecycle](issues/03-lifecycle.md) implemented and locally verified.
|
||||
- [04 integration](issues/04-integration.md): core five-role scratch/snapshot workflows verified through final acceptance; standard delivery-package worker/graph remains uncovered. See [verification and two-axis review](verification.md).
|
||||
- No customer assets or production runtime changes. Company deployment, real sample copy and production publication remain separately authorized operations.
|
||||
|
||||
## 2026-10-19 修订讨论
|
||||
|
||||
背景:客户已投产,用户不能再用客户账号测试。希望基于 `codex/dev-mainline-20261019` 在线上单开测试环境,用少量真实客户样本模拟全流程;通过后用脚本把同一版本发布到生产。本节不构成部署、复制或账号创建授权。
|
||||
|
||||
### 2026-10-19 Round 1(Q1–Q10 全部采用推荐项)
|
||||
|
||||
- Q1:修订 ADR-0004 而非另起方案。三处修订:首次部署即用真实客户样本(不再先走合成素材阶段);全流程覆盖标准交付打包;测试结果作为生产发布验收证据。其余既定决策不变。
|
||||
- Q2:“推送到生产”定义为**版本晋升**:测试环境运行 dev-mainline 已提交 commit 构建的不可变镜像 digest,通过后经 `scripts/release.py` 将同一 digest 发布到生产;测试数据、账号、配置、存储卷不进生产;测试批次验收报告挂入 release 记录作为写入类证据;每次生产部署仍需当次授权。
|
||||
- Q3:测试环境运行配置从生产只读导出非机密项,仅覆盖隔离字段(namespace、数据库、存储、入口)与容量字段(并发 1、资源上限);每批记录与生产的配置差异,差异外不一致的项不能作为晋升证据。
|
||||
- Q4:纳入标准交付打包:测试 namespace 内独立工作流 worker(仅限测试 namespace 创建 Job)、算子镜像、生产打包工作流快照;不启用 `fanyu-high-capacity` 档,打包并发 1,产物只写测试存储、不进客户交付出口。
|
||||
- Q5:新建测试专用 BOS bucket,用生产只读凭证复制选中 Capture;测试环境的 csi-bos 存储卷与 AK/SK 只对测试 bucket 有权限。不只读挂载生产 bucket,不改用 local PV。
|
||||
- Q6:测试 namespace 内独立 Mongo 实例(不在生产 Mongo 上另建库);从生产只读导出选中项目配置(项目、工作流定义快照、三类字典)及样本,以新身份生成测试任务副本;不复制客户账号、会话、领取、在途处理和正式审查记录。
|
||||
- Q7:每个在产项目 3 条(待切片、保留切片/动作快照、质检阶段快照),总数 ≤ 12;精确清单与体积只读盘点后由用户确认。
|
||||
- Q8:两个测试供应商 `test-a`、`test-b`,各含供应商管理员/标注员/质检员,另加平台管理员和最终验收员,约 8 个账号(替代原 5 账号)。
|
||||
- Q9:首版仅本机 127.0.0.1 端口转发,供用户本人使用;不建公网入口、不改生产 Ingress/DNS。
|
||||
- Q10:“多个 pod”指一套测试环境内多个组件,一次一个候选版本;不并行多套环境。
|
||||
|
||||
### 集群只读盘点(2026-10-19)
|
||||
|
||||
- 三个 Ready 节点:`10.189.16.33`(56 CPU / ~220 GiB / 2 GPU,requests CPU 15%、limits 37%);`10.189.16.4`、`10.189.16.5`(各 8 CPU / ~31 GiB,后者 CPU limits 已超售至 185%)。均无污点。采样时实际 CPU 使用接近 0。
|
||||
- 存储类仅 `csi-bos`(no-provisioner,静态 PV);无 IngressClass。生产工作区 `droboticflow-workspace-pv` 200Gi 为 csi-bosplugin。
|
||||
- `droboticflow-internal-test` namespace 于 2026-09-27 已建,仅有 `internal-test-quota`(pods 3、requests 1 CPU/2Gi、limits 3 CPU/10.5Gi、存储 20Gi、PVC 8)、`internal-test-default-deny` 网络策略及 CCE 自动镜像拉取 Secret;无工作负载。配额不足以容纳 worker 与算子 Job。
|
||||
- 生产 `droboticflow-fanyu` 应用组件:api ×4 与 annotation-processor ×2(同镜像,build commit 630a95c8 = 当前 dev-mainline HEAD,processor role=revision);annotation-validator ×2(另一镜像,963b530f,role=validation,2026-10-17 建);worker ×2(worker 镜像,963b530f);drosync-coordinator ×2;GPU 服务 headpose/wilor 无就绪副本。`release.py` 策略只覆盖 api 与 processor。
|
||||
- **已有生产内测试组织**:所有生产应用组件的 `ANNOTATION_WORKSPACE_MAPPINGS` 含 `org-internal-0bd5aa2e…`,读取生产 bucket `internal-test/blf-20260927-…/assets`,写 `system/runs/run-internal-*`,交付至 `internal-test/…/deliveries/TEST-internal-BLF__…`。来源为 2026-09-27 release 记录 `test-readonly-fix-20260927`(API-only 配置变更,7 个测试账号覆盖 5 角色)。这是生产库内隔离组织方案,与 ADR-0004 的独立环境方案并存,待决定其去留。
|
||||
|
||||
### 2026-10-19 Round 2(Q11–Q15 采用推荐项;Q16 被用户否决)
|
||||
|
||||
- Q11:保留既有生产内测试组织 `org-internal-…`,冻结并收窄为**晋升后生产冒烟**:每次晋升后用 1 条样本验证生产自身配置、凭证与存储;不再新增样本或项目。
|
||||
- Q12:候选版本为一个 dev-mainline commit 构建的镜像;测试环境各组件 1 副本;drosync-coordinator 不在测试环境和候选范围内。(组件范围已被 2026-10-20 范围收窄修订,见下文。)
|
||||
- Q13:每批先以生产当前 digest 启动并导入样本、制造在途状态,再用**同一晋升脚本**指向测试环境升级到候选版本,演练迁移与在途恢复,产出 change-safety 证据。修订 ADR-0004“一批一版本”为“一批 = 生产版本 → 候选版本”。
|
||||
- Q14:测试环境固定到 `10.189.16.4`(无生产应用 Pod),不给生产节点加污点、不改生产调度;按需上调测试配额;观察到争用再申请专用节点。
|
||||
- Q15:测试 Mongo 为测试 namespace 内 Pod,主版本与生产一致(待只读核实),数据放 `.4` 本地盘专用目录的静态卷。
|
||||
- Q16(否决):用户无法也不愿创建 BOS 子用户或测试 bucket。
|
||||
|
||||
### 2026-10-20 用户范围收窄
|
||||
|
||||
- 本轮只测试**标注 → 质检 → 最终验收**;Capture 接入及算子处理不在本问题范围。
|
||||
- 不需要平台管理员的全局管理功能。
|
||||
- 用户希望直接从 BOS 读取相应文件。代码核实:该链路并非只读——标注处理器在领取后自行生成切片媒体,送检与终审发布结果文件,均写入 `WORKSPACE_MOUNT_PATH` 下;算子 Job 与 worker 确实不参与。依据 [annotation-workspace-mappings.md](../../docs/annotation-workspace-mappings.md):“只读委托仅解决素材与帧范围加载,不支持生成切片产物”。
|
||||
- 由此 Round 1 Q4(纳入标准交付打包)失效;Q5、Q8、Q12、Q15 中依赖测试 bucket、worker 或平台管理员的部分待 Round 3 重新确认。
|
||||
- 事实:本地 Capture 样例(单 part、480p)约 235 MB;12 条源素材约数 GB。生产 worker(963b530f)与 api(630a95c8)之间 5 个提交未改 worker 代码。
|
||||
|
||||
### 2026-10-20 Round 3(Q16′–Q21 全部采用推荐项)
|
||||
|
||||
- Q16′(替代 Q16、Q5):以本机已有 BOS 凭证、按 `scripts/local/download-capture.sh` 方式一次性只读下载选中 Capture,校验 SHA256 后上传到 `.4` 本地盘测试工作区。测试环境为纯本地工作区模式:不挂载 BOS、不持有任何 BOS 凭证,切片与结果写本地盘。bosfs 读取与 BOS ETag 版本检查不在测试环境覆盖范围,由 Q11 晋升后生产冒烟覆盖。不采用只读挂载生产 bucket,不写入生产 bucket 的 `internal-test/` 前缀。
|
||||
- Q17(撤回 Round 1 Q4):不含 worker、算子 Job 与标准交付打包。测试环境组件为 API ×1、标注处理器 ×1(role `all`,并发 1)、Mongo ×1。终审通过的任务停在待交付。
|
||||
- Q18(修订 Q12):晋升只更新 API 镜像的三个组件 api、annotation-processor、annotation-validator;worker 保持现版本。候选改动 `internal/worker`、`cmd/worker` 时晋升脚本拒绝并要求另行验证。需扩展 `release.py` 策略纳入 validator。
|
||||
- Q19(修订 Q8):5 个业务账号——test-a 标注员与质检员、test-b 标注员与质检员、1 名归属 test-a 且开通供应商最终验收授权的最终验收员;无平台管理员与供应商管理员。项目、供应商、成员与任务由 seed 命令建立。管理类登录由生产冒烟只读检查。
|
||||
- Q20:样本选择、项目配置与字典导出、Mongo 版本确认均基于 `scripts/local/fetch-mongo-snapshot.sh` 拉取的最新备份,在本地独立容器完成,不查询线上库。选样清单交用户确认。
|
||||
- Q21:测试批次报告记录候选镜像 digest 与配置差异摘要;`release.py evidence` 接收测试报告时校验 digest 一致且批次为 business-verified,否则拒绝。测试环境无法产出的证据(含交付打包)保持 unverified。
|
||||
|
||||
### 2026-10-20 Round 4(Q22–Q25 全部采用推荐项)
|
||||
|
||||
- Q22:逐步单独授权,前一步通过才询问下一步:① 拉取最新 Mongo 备份到新的本地容器(不动 27018)并选样;② 确认清单后从 BOS 只读下载样本;③ 从 dev-mainline 干净 worktree 构建候选 API 镜像并推送到公司镜像仓库;④ 测试 namespace 写入(配额、`.4` 专用目录、静态卷、Mongo、上传素材、seed、删除初始化 Pod),写前复核集群与 namespace UID;⑤ 以生产当前 digest 启动、制造在途状态、用晋升脚本升级到候选并跑全流程;⑥ 按 `release.py` 流程晋升生产并做生产冒烟。
|
||||
- Q23:晋升后生产冒烟默认只读(全角色登录、列表、内部组织任务、视频播放与帧范围);仅当候选改动领取、切片、送检或审查写路径时,经当次批准在内部组织剩余任务上做写入冒烟;任务耗尽时报告,不自动补充,不使用客户任务。
|
||||
- Q24:归档时 `mongodump`、结果报告与配置差异摘要写 `.4` 本地盘并拉回 `local-data/internal-test-lane/batches/<batch-id>/`;保留最近 3 批加当前批;旧批次本地产物预览后确认删除;集群侧清理另列清单与授权。
|
||||
- Q25:复现特定客户问题时可追加该 Capture 为样本,按 ①→②→④ 流程逐次批准,活动总数 ≤ 12;新身份,不回写来源任务。
|
||||
- 前沿已清空,待用户确认达成共同理解后修订 ADR-0004 与 spec 并拆分实施票。
|
||||
|
||||
### 2026-10-20 共同理解确认
|
||||
|
||||
- 用户确认四轮决定达成共同理解。已记录:[ADR-0004 2026-10-20 修订](../../docs/adr/0004-isolated-internal-test-lane.md)、[spec.md 当前修订](spec.md)、实施票 [06](issues/06-renderer-local-node.md)–[15](issues/15-batch-archive.md)(05 为主线既有 movi 冒烟票);[部署计划](deployment-plan.md) 标记为 superseded;术语“内部测试环境”“版本晋升”写入 [CONTEXT.md](../../CONTEXT.md)。
|
||||
- 依赖:06、08、10、14 无前置;07←06;09←08;12←08;15←06;13←06/07/10/12;11←10/13。既有 05(movi 直连播放冒烟,暂缓)可在本环境部署后执行。
|
||||
- 未开始实施,未提交;集群、BOS、备份拉取与生产晋升均未执行,按 spec 的 Run sequence 逐步授权。
|
||||
|
||||
### 2026-10-20 外部审阅采纳
|
||||
|
||||
用户采纳外部审阅的 7 点及补充项,第 6 点选方案 A。均已代码核实并写入票据与 spec:
|
||||
|
||||
- 备份恢复:现有 `fetch-mongo-snapshot.sh` 会在生产 API Pod 的 `/tmp` 分块暂存归档,并删除重建 27018 容器、`--drop` 恢复;默认备份目录固定为 `fanyu-v58-ca15d60d`。改为独立容器名与端口、拒绝覆盖、按字节区间直接读取不在生产 Pod 暂存、展示备份日期,过旧则停止([09](issues/09-backup-sample-export.md))。
|
||||
- 生产冒烟:任务详情 GET 会恢复中断切片并可能完成待定送检,帧范围 GET 缓存缺失时写入索引;“只允许 GET”不等于只读。改为按服务端副作用分类端点,默认承诺不写客户数据,内部组织上的副作用写入列入报告([14](issues/14-production-smoke.md),新增依赖 12)。
|
||||
- 本地盘容量:预算 + `df` 预检 + 剩余 25% 停止阈值 + 临时存储上限;文件系统配额视节点支持,否则记录为仅预检约束([06](issues/06-renderer-local-node.md)、[15](issues/15-batch-archive.md))。
|
||||
- 升级起点:`baselineImage` 须等于发布计划记录的各 Deployment 旧镜像;当前 validator(`7699d455…`)与 api/processor(`1a1ead7a…`)起点不同,不一致者列为演练未覆盖([11](issues/11-release-evidence-binding.md)、[13](issues/13-upgrade-rehearsal.md))。
|
||||
- 检查项→必需场景映射,缺项、失败、未验证一律拒绝([11](issues/11-release-evidence-binding.md)、[12](issues/12-cluster-http-verifier.md))。
|
||||
- 发布证据:`annotation-delivery` 拆为 `annotation-results` 与 `delivery-packaging`;后者由打包测试记录,或 `release.py` 自行核对策略声明的交付输入路径未改动满足([10](issues/10-release-policy-scope.md))。ADR-0004 修订已补记。
|
||||
- 播放与 ETag:浏览器(Playwright)首帧与播放推进;ETag 变化后的缓存失效保持未验证;05 号票第 1 项由 12 覆盖。
|
||||
- 失败路径:`failed` / `outcome-unknown` 停机、保留状态归档、重开新批次([13](issues/13-upgrade-rehearsal.md))。
|
||||
- 依赖更新:14←12。
|
||||
|
||||
@ -1,4 +1,166 @@
|
||||
# Internal test lane — implementation baseline
|
||||
# Internal test lane
|
||||
|
||||
Status: ready-for-agent
|
||||
Current revision: 2026-10-20
|
||||
Code baseline: `codex/dev-mainline-20261019` @ `630a95c8`
|
||||
|
||||
Decisions: [map.md](map.md) (2026-10-19/20, Rounds 1–4) and the 2026-10-20 revision of
|
||||
[ADR-0004](../../docs/adr/0004-isolated-internal-test-lane.md). This revision authorizes local
|
||||
implementation and local verification of tickets 06–15 only. Pulling a backup, downloading
|
||||
from BOS, pushing images, writing to the cluster, creating accounts there and promoting to
|
||||
production are separately approved steps (see Run sequence).
|
||||
|
||||
## User-facing outcome
|
||||
|
||||
Before promoting a dev-mainline candidate, the developer runs one batch in the isolated test
|
||||
environment. Up to 12 real customer samples, chosen from the newest production backup and
|
||||
downloaded once read-only from BOS, are claimed, sliced, annotated, submitted, reviewed and
|
||||
accepted by five test business accounts in two test suppliers. The batch first runs the
|
||||
production image, then upgrades to the candidate with the same promotion script used for
|
||||
production. A business-verified report bound to the candidate digest becomes release evidence;
|
||||
that digest is then promoted to production's API-image components and checked by a production
|
||||
smoke that is read-only by default.
|
||||
|
||||
## Scope
|
||||
|
||||
In: annotation → quality review → final acceptance, including slice generation, source-index
|
||||
validation, save, submission, rework return, processor restart recovery, supplier claim
|
||||
exclusivity and invisibility, browser video playback (first frame, playback advancing) and frame
|
||||
range from the local workspace.
|
||||
|
||||
Out, and recorded as unverified wherever release evidence asks for it: workflow worker,
|
||||
operator Jobs, standard delivery packaging, Capture ingestion, cache invalidation after a BOS
|
||||
source ETag change, load and capacity, platform-admin and supplier-admin
|
||||
management.
|
||||
|
||||
## Environment
|
||||
|
||||
- Namespace `droboticflow-internal-test` (exists; only `internal-test-quota` and
|
||||
`internal-test-default-deny` today). Every pod is pinned to node `10.189.16.4`; production
|
||||
nodes get no taints and production scheduling is unchanged.
|
||||
- Components: API ×1; annotation processor ×1 with role `all`, concurrency 1; Mongo ×1 with
|
||||
the production major version; an initializer pod during import only; a one-time
|
||||
directory-preparation pod.
|
||||
- Storage: static local PVs on a dedicated directory of node `.4` for Mongo and workspace,
|
||||
`Retain`, node affinity, RWO. No csi-bos volume, BOS endpoint, storage connection,
|
||||
workspace mapping or BOS credential exists in the namespace.
|
||||
- Entry: ClusterIP only, reached through `kubectl port-forward` bound to `127.0.0.1`. The
|
||||
session cookie may drop `Secure` only in this explicit private-forward mode.
|
||||
- Runtime configuration is derived from a read-only snapshot of the production API and
|
||||
processor Deployments. Only isolation fields (namespace, database, workspace/storage,
|
||||
entry/cookie) and capacity fields (replicas, processor role, concurrency, resources) are
|
||||
overridden; the database bootstrap mode follows production. Each batch records the diff;
|
||||
any other difference disqualifies the batch as promotion evidence.
|
||||
- Quota fits API, processor, Mongo and initializer with finite CPU, memory and storage limits.
|
||||
- Disk: PV size and quota do not bound real writes to the node's local disk, which kubelet and
|
||||
images share. A budget (samples, slice output, Mongo, dumps, 3 retained batches plus the current
|
||||
one) is checked against a `df` preflight before seed, start, upgrade and archive; below 25% free
|
||||
after the budget, work stops. Pods carry ephemeral-storage limits; a filesystem quota is used where
|
||||
the node supports it, otherwise the batch record says the limit is preflight-only.
|
||||
|
||||
## Data and accounts
|
||||
|
||||
- Selection and export read a backup restored into a new local container with its own name and
|
||||
loopback port; the restore refuses an existing container name, a busy port and 27017–27020, and
|
||||
never drops into an existing database. The archive is read by byte ranges from the backup mount,
|
||||
without staging inside the production pod. The chosen backup's name and date are shown; if the
|
||||
newest one is older than the selection window, work stops, because a new production backup is a
|
||||
separately approved operation. The live database is never queried.
|
||||
- Per in-production project, up to 3 samples: scratch (pending annotation), annotation
|
||||
snapshot (existing slices and actions), quality-review snapshot. At most 12 in total. The
|
||||
user confirms the final list (Capture, project, template kind, size) before download.
|
||||
- Media is downloaded read-only from production BOS on the developer machine, SHA256-verified
|
||||
and bundled. Project config, the workflow-definition snapshot and the three dictionaries come
|
||||
from the backup. Customer accounts, sessions, leases, in-flight work, formal review records
|
||||
and notification targets are not copied.
|
||||
- Suppliers `test-a` and `test-b`. Accounts: annotator and quality reviewer for each supplier,
|
||||
plus one final acceptance reviewer in `test-a` with supplier final-acceptance authorization.
|
||||
All work identities are new; source IDs are kept only for traceability. The seeded database
|
||||
must start under the production bootstrap mode.
|
||||
- Reproduction samples may be added later through the same export → download → import path,
|
||||
approved each time, keeping the active total at 12 or fewer.
|
||||
|
||||
## Batch flow
|
||||
|
||||
1. Prepare a batch with `baselineImage` (production's current API digest, from a read-only
|
||||
Deployment snapshot that also records each Deployment's image) and `candidateImage`.
|
||||
2. Seed, start on `baselineImage`, and create in-flight state: claimed tasks, slicing pending,
|
||||
submission pending.
|
||||
3. Upgrade to `candidateImage` with `release.py` and the internal-test policy.
|
||||
4. Run the HTTP verifier and the browser playback check on the candidate: five roles, supplier
|
||||
isolation, normal and rework paths, processor restart recovery. Record `business-verified` with the candidate digest and
|
||||
the config diff.
|
||||
5. Archive: `mongodump`, reports and config diff pulled to
|
||||
`local-data/internal-test-lane/batches/<batch-id>/`. Keep the latest 3 batches plus the
|
||||
current one; older local outputs are removed only after preview and confirmation. Cluster-side
|
||||
cleanup is a separate approved list.
|
||||
|
||||
Any phase can end in `failed` or `outcome-unknown`: stop new work, archive with that state kept,
|
||||
never mark the batch `business-verified`, and start over with a new batch.
|
||||
|
||||
## Promotion
|
||||
|
||||
- `release.py` takes an explicit policy; without one it keeps today's Fanyu behaviour.
|
||||
- The Fanyu policy covers `droboticflow-api`, `droboticflow-annotation-processor` and
|
||||
`droboticflow-annotation-validator`. The worker is not touched; promotion is refused when the
|
||||
candidate differs from the production worker's build commit under `internal/worker` or
|
||||
`cmd/worker`. Shared-package compatibility with the old worker stays a change-safety review item.
|
||||
- The Fanyu policy's `annotation-delivery` becomes `annotation-results` (satisfiable by a batch) and
|
||||
`delivery-packaging` (a packaging test record, or `release.py`'s own check that the policy-declared
|
||||
delivery-input paths are unchanged between the production API commit and the candidate).
|
||||
- A test-lane report counts as evidence only if its candidate digest equals the release image
|
||||
digest, the batch is `business-verified`, and every scenario the requested check needs is present
|
||||
and `passed` (mapping in ticket 11). For the rehearsal part of `change-safety`, `baselineImage`
|
||||
must also equal each covered Deployment's `before` image in the release plan; Deployments with a
|
||||
different starting digest are listed as not covered. Evidence the batch cannot produce stays
|
||||
unverified.
|
||||
- After promotion, the production smoke runs in the frozen internal organization and by default
|
||||
writes no customer data. Some GETs write (task detail recovery, frame-range index on a cache miss),
|
||||
so endpoints are classified by server-side effect, possible writes are confined to
|
||||
internal-organization objects and listed in the report. A write smoke runs only when the candidate changes claim, slicing, submission or
|
||||
review write paths, approved each time, on remaining internal tasks only. Customer tasks are
|
||||
never used and internal tasks are never replenished automatically.
|
||||
|
||||
## Run sequence (each step approved separately)
|
||||
|
||||
1. Pull the newest backup (name and date shown) into a new local container, select samples, user
|
||||
confirms the list.
|
||||
2. Download the confirmed samples from BOS, read-only.
|
||||
3. Build the candidate API image from a clean dev-mainline worktree and push it to the company
|
||||
registry.
|
||||
4. Test-namespace writes: quota, node directory, PVs/PVCs, Mongo, upload, seed, delete the
|
||||
initializer. Re-verify the cluster and namespace UID first.
|
||||
5. Start on the production digest, create in-flight state, upgrade, verify, record evidence.
|
||||
6. Promote with `release.py` prepare → check → plan → approval → deploy → verify, then run the
|
||||
production smoke.
|
||||
|
||||
## Verification seams
|
||||
|
||||
- Public-boundary tests for the renderer, lifecycle CLI, seed planner and `release.py`
|
||||
(simulated Kubernetes, as today).
|
||||
- Local end-to-end on a fresh Mongo port and API port (not 27017/27018/3001): a v2 bundle from
|
||||
synthetic media, two suppliers, five roles, baseline → candidate upgrade, and a verifier report
|
||||
bound to the digest.
|
||||
- Cluster behaviour (local PV binding on `.4`, port-forward under Cilium default-deny, quota)
|
||||
stays unverified until run-sequence step 4 is approved.
|
||||
|
||||
## Tickets
|
||||
|
||||
[06](issues/06-renderer-local-node.md) ·
|
||||
[07](issues/07-runtime-config-parity.md) ·
|
||||
[08](issues/08-bundle-seed-v2.md) ·
|
||||
[09](issues/09-backup-sample-export.md) ·
|
||||
[10](issues/10-release-policy-scope.md) ·
|
||||
[11](issues/11-release-evidence-binding.md) ·
|
||||
[12](issues/12-cluster-http-verifier.md) ·
|
||||
[13](issues/13-upgrade-rehearsal.md) ·
|
||||
[14](issues/14-production-smoke.md) ·
|
||||
[15](issues/15-batch-archive.md)
|
||||
|
||||
Existing [05](issues/05-movi-smoke-and-sl3-evidence.md) (movi direct-playback smoke, deferred by the user)
|
||||
fits this environment once it is deployed: it has no movi service.
|
||||
|
||||
## 2026-09-27 baseline (implemented; superseded where it conflicts)
|
||||
|
||||
Status: implemented 2026-09-27 (local synthetic verification); 2026-10-12 drift catch-up re-verified (suites + five-role HTTP flow; movi browser smoke pending), see verification.md new section and final section
|
||||
Baseline: 17e7de7e (implementation); drift-reviewed 2026-10-12 at 5a956624
|
||||
@ -9,7 +171,7 @@ The current turn authorizes local implementation and synthetic validation only:
|
||||
no customer database, BOS assets, production deployment or live credentials are accessed.
|
||||
Actual cluster provisioning and the six real sample transfers remain separately approved operations.
|
||||
|
||||
## User-facing outcome
|
||||
### User-facing outcome
|
||||
|
||||
An internal developer can prepare a small, independently runnable environment
|
||||
using the same application/business UI, five isolated role accounts and a fixed
|
||||
@ -17,7 +179,7 @@ six-source sample pack (three BLF-data, three SL-data). They can reproduce a
|
||||
known state or start from slicing, archive results and explicitly prepare the
|
||||
next batch/version without putting customer work into a test queue.
|
||||
|
||||
## Delivery components
|
||||
### Delivery components
|
||||
|
||||
1. Offline environment renderer and configuration validation. Fixed test namespace,
|
||||
distinct host, authenticated dedicated Mongo, per-batch database and PVCs,
|
||||
@ -35,7 +197,7 @@ next batch/version without putting customer work into a test queue.
|
||||
prepared, seeded and business-verified states; no claim of deployed/accepted
|
||||
solely because manifests were rendered or records inserted.
|
||||
|
||||
## Explicit behavior
|
||||
### Explicit behavior
|
||||
|
||||
- Test supplier `test`; platform admin plus supplier manager, annotator, quality
|
||||
reviewer and acceptance reviewer. Customer identities are not cloned.
|
||||
@ -66,7 +228,7 @@ next batch/version without putting customer work into a test queue.
|
||||
- Same immutable app image is used for eventual promotion, with production runtime
|
||||
settings retained. Candidate success does not authorize production release.
|
||||
|
||||
## Confirmed verification seams
|
||||
### Confirmed verification seams
|
||||
|
||||
The user explicitly accepted these on 2026-09-27:
|
||||
|
||||
@ -79,7 +241,7 @@ typecheck/build, then full relevant suites at the end. Review against baseline
|
||||
17e7de7e along standards and specification axes. Existing full-suite failures
|
||||
must be reported separately; do not claim a wholly green repository.
|
||||
|
||||
## Current-turn acceptance
|
||||
### Current-turn acceptance
|
||||
|
||||
Local commands, manifests, initialization, lifecycle and synthetic workflow evidence
|
||||
are runnable and documented. No company access is required to demonstrate refusal
|
||||
@ -87,7 +249,7 @@ checks and business behavior. Real sample inventory, exact company allocation,
|
||||
external ingress/storage prerequisites, cluster deployment and capacity validation
|
||||
remain explicitly unverified until separately authorized; do not fabricate them.
|
||||
|
||||
## 2026-10-12 漂移复核与追进度工单(用户批准落档)
|
||||
### 2026-10-12 漂移复核与追进度工单(用户批准落档)
|
||||
|
||||
9-27 实现完整存在且未被后续演进破坏:Go 包测试 2026-10-12 重跑全绿(`go test ./internal/testlane/...`),种子命令 `cmd/internal-test-seed` 与 `scripts/internal_test_lane.py` 冒烟正常,五角色账号模型与当前 `internal/api/auth.go` 服务端校验完全一致;实现后零维护提交、零破坏。但距验证基线 `17e7de7e` 约 174 个提交已进入主线,差距如下:
|
||||
|
||||
|
||||
@ -177,6 +177,12 @@ _Avoid_:客户公共池(内部测试不参与客户任务的竞争领取)
|
||||
**测试任务副本**:以经批准的客户样本为来源、拥有独立身份和处理历史的内部测试任务;副本的修改不回写来源任务。
|
||||
_Avoid_:供应商任务副本(不代表给每家生产供应商复制客户任务)
|
||||
|
||||
**内部测试环境**:与客户生产隔离、一次运行一个候选版本的完整平台,用经批准的少量客户样本模拟客户生产处理流程;其中的账号、任务、审查和交付不计入客户生产、统计或交付出口。
|
||||
_Avoid_:测试 pod(一套环境包含多个组件)、生产内测试项目(与客户共用业务数据,不构成环境隔离)
|
||||
|
||||
**版本晋升**:把已在内部测试环境验收的同一候选版本发布到客户生产;只带走版本本身,不带走测试数据、账号或配置,每次发布仍需单独授权。测试通过不等于获准发布。
|
||||
_Avoid_:推送 pod、迁移测试环境
|
||||
|
||||
**撤销供应商项目授权**:停止供应商领取该项目的新任务,但保留其既有任务的标注、质检与返工处理资格,直至存量完成;不等于停用账号或释放已领取任务。
|
||||
|
||||
**生产统计异常**:仅指当前正式处理中的执行故障,包括运行失败、源采集处理失败、切片或送检处理作业失败、交付打包失败。供应商归属不同、部分未领取、归属待核对、审核退回、业务升级处理及正常等待不单独计为异常;这些状态保留在明细,不增加红色异常数量。
|
||||
|
||||
@ -15,3 +15,17 @@ status: accepted
|
||||
Q7–Q13 随后明确:必要素材复制到测试存储;5 个独立测试角色账号;固定 6 份样本模板,最多 6 条活动测试任务,手动重建;切片并发 1,资源不足暂停;受控命令管理,按批次切换版本,保留最近 3 批结果并在确认后清理更旧测试产物。
|
||||
|
||||
本决策不授权创建账号、复制样本、部署或修改客户数据。具体来源清单、资源额度、账号使用者和执行审批见[需求讨论](../../.scratch/internal-test-lane/map.md),完整方案确认后再推进实施准备。
|
||||
|
||||
## 2026-10-20 修订
|
||||
|
||||
客户已投产,内部不能再借客户账号验证;候选版本须先在测试环境走完人工标注链路,再把同一镜像晋升到生产。以下决定替代上文及 2026-09-27 部署计划中与之冲突的部分,逐条依据见需求讨论中 2026-10-19/20 四轮记录。
|
||||
|
||||
- **范围**:只验证标注 → 质检 → 最终验收。工作流 worker、算子 Job、标准交付打包和 Capture 接入不在测试环境中运行;对应发布证据保持未验证。
|
||||
- **存储**:不创建 BOS 子用户或测试 bucket。选中样本从生产 BOS 一次性只读下载,导入测试节点本地盘;测试环境是纯本地工作区,不挂载 BOS、不持有任何 BOS 凭证,切片与结果只写本地盘。未采用只读挂载生产 bucket:测试命名空间须持有全桶读写凭证,隔离只剩只读挂载和应用前缀检查。未采用写入生产 bucket 的 `internal-test/` 前缀:测试产物会进入客户生产存储。代价是 BOS 读取与 ETag 版本检查只能在晋升后的生产冒烟中验证。
|
||||
- **资源**:固定在当前没有生产应用 Pod 的节点 `10.189.16.4`,不新增节点、不改生产调度;Mongo 与工作区用该节点本地盘上的静态卷。生产算子 Job 仍可能调度到该节点,观察到争用再申请专用节点。
|
||||
- **样本与账号**:从最新生产备份选样,不查询线上库;每个在产项目 3 条、总数不超过 12。两个测试供应商、5 个业务角色账号,不设平台管理员和供应商管理员,用以覆盖多供应商领取与不可见。
|
||||
- **版本**:每批先以生产当前镜像启动并制造在途状态,再用与生产相同的晋升脚本升级到候选版本。一批对应“生产版本 → 候选版本”,替代上文“按批次一次运行一个版本”。
|
||||
- **晋升**:只晋升到使用同一镜像的 api、annotation-processor、annotation-validator;worker 不随之更新,候选改动 worker 代码时拒绝晋升。测试报告只有与本次发布镜像 digest 一致时才作为发布证据。原 `annotation-delivery` 证据拆为标注结果与交付打包两项:前者可由测试批次满足,后者需要打包测试,或经核对候选版本未改动交付输入。
|
||||
- **生产内测试组织**:2026-09-27 起生产库中的内部测试组织冻结,只用于晋升后的生产冒烟,默认只读。
|
||||
|
||||
每次执行仍逐步单独授权:拉取备份、下载样本、构建推送镜像、写入测试命名空间、启动与升级演练、晋升生产。
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user