✅ Trace POC acceptance run ids

This commit is contained in:
eust-w 2026-06-23 23:14:40 +08:00
parent 220aa65a2d
commit 07eaeb22b2
5 changed files with 13 additions and 2 deletions

View File

@ -221,7 +221,7 @@ DEPLOY_ENV=poc npm run acceptance:poc-strict
`acceptance:poc-strict` refuses dependency-isolation scopes by requiring `POC_ACCEPTANCE_REQUIRED_EXTERNALS` to include `bos,dcloud,mongo`. It runs `acceptance:poc-readiness`, probes the two D-Cloud FAT contracts from `droboticflow-dcloud-secret`, and only then launches `acceptance:poc-operators`. Readiness or D-Cloud probe failures emit a standard `DROBOTICFLOW_POC_OPERATOR_ACCEPTANCE=<json>` preflight-failure result and stop before the Kubernetes acceptance Job is created; the D-Cloud probe reports only status, response shape, and configured-header booleans, never secret values or business payloads. When `POC_ACCEPTANCE_IMPORT_RESULT=true` and `POC_ACCEPTANCE_IMPORT_API_BASE_URL`, `API_BASE_URL`, or `BASE_URL` is configured, those blocked preflights are imported into the `POC 算子验收` platform history exactly like Job results. Use narrower `POC_ACCEPTANCE_REQUIRED_EXTERNALS` only when intentionally running `acceptance:poc-operators` as dependency-isolation diagnostics, not as final strict evidence.
`scripts/poc-operator-full-acceptance.sh` loads the `poc` deployment environment on `dev`, reads the current `OPERATOR_RUNNER_IMAGE`, workspace PVC, Mongo secret, and BOS workspace secret references from Kubernetes, mounts `scripts/poc_operator_full_acceptance_job.py` into a temporary Job, and prints one `DROBOTICFLOW_POC_OPERATOR_ACCEPTANCE=<json>` result. The Job executes builtin modules from the deployed runner image and writes evidence under `/workspace/acceptance/<run-id>/`; if the PVC is misconfigured and the runner cannot write there, the run fails by default instead of silently accepting temporary evidence. Set `POC_ACCEPTANCE_ALLOW_TMP_FALLBACK=true` only for diagnostic runs that intentionally write to `/tmp/droboticflow-acceptance/<run-id>/` and report `workspaceFallbackReason`.
`scripts/poc-operator-full-acceptance.sh` loads the `poc` deployment environment on `dev`, reads the current `OPERATOR_RUNNER_IMAGE`, workspace PVC, Mongo secret, and BOS workspace secret references from Kubernetes, mounts `scripts/poc_operator_full_acceptance_job.py` into a temporary Job, and prints one `DROBOTICFLOW_POC_OPERATOR_ACCEPTANCE=<json>` result. The Job executes builtin modules from the deployed runner image and writes evidence under `/workspace/acceptance/<run-id>/`; `<run-id>` is set from `POC_OPERATOR_ACCEPTANCE_JOB_NAME` so the raw result, Kubernetes Job, imported platform run source, and workspace evidence directory stay traceable to the same acceptance execution. If the PVC is misconfigured and the runner cannot write there, the run fails by default instead of silently accepting temporary evidence. Set `POC_ACCEPTANCE_ALLOW_TMP_FALLBACK=true` only for diagnostic runs that intentionally write to `/tmp/droboticflow-acceptance/<run-id>/` and report `workspaceFallbackReason`.
Final POC operator acceptance requires real input fixtures that already exist inside the Job container, usually on the shared workspace PVC mounted at `${WORKSPACE_MOUNT_PATH:-/workspace}`. Prefer the default `POC_ACCEPTANCE_FIXTURE_MANIFEST_PATH=/workspace/fixtures/poc-operator-acceptance/poc-operator-acceptance.json` file so the platform run records one reproducible fixture contract:
@ -269,7 +269,7 @@ By default `POC_ACCEPTANCE_REQUIRED_EXTERNALS=bos,dcloud,mongo`, so missing `BOS
The result JSON exposes `acceptanceScope`, `fullExternalAcceptance`, `fixtureSources`, `generatedFixtureCount`, `realFixtureCount`, `allFixturesReal`, `operatorPassedCount`, `coveredOperatorCount`, `allOperatorCoverageComplete`, `blockingOperatorCoverageComplete`, `externalVerificationPassed`, `externalVerificationMissing`, `externalVerificationPassedCount`, `externalVerificationRequiredCount`, `operatorRepositoryCount`, `operatorRepositoryRequiredCount`, `operatorRepositoryCoverageComplete`, `operatorRepositoryPolicyClean`, `operatorRepositoryPolicyViolations`, `statusCounts`, and skipped-count fields. The Job sets `fullExternalAcceptance=true` only when the same strict evidence set is already present in the raw result: scope is `full`, required externals include `bos,dcloud,mongo`, every required external system is verified, every operator is covered and passed, every builtin operator has a matching local repository manifest, repository policy violations are empty, no failed/required-skipped/missing-coverage entries exist, `allFixturesReal=true`, `generatedFixtureCount=0`, and `realFixtureCount>0`. Treat `fullExternalAcceptance=true` plus `status=passed` as the strict full-acceptance signal; any `dependency-isolation` result can prove the configured dependencies but not the omitted external operators. External verification is counted by system: BOS requires both real upload and BOS incremental scan records, Mongo requires a write followed by an independent count/drop verification, and D-Cloud requires both FAT API operators to pass. The Monitoring summary card applies the same strict display rules.
When the Job emits a `DROBOTICFLOW_POC_OPERATOR_ACCEPTANCE=<json>` marker, the wrapper extracts it from the Job logs even if the Job exits failed. If `POC_ACCEPTANCE_IMPORT_API_BASE_URL`, `API_BASE_URL`, or `BASE_URL` is set, it posts that result to `/api/acceptance/poc-operator-result` and records it as a normal platform run under the `POC 算子验收` project. In session-auth POC deployments, the acceptance-import endpoint accepts either an admin/pipeline/operator session through `DROBOTICFLOW_ACCEPTANCE_EMAIL` and `DROBOTICFLOW_ACCEPTANCE_PASSWORD`, or the shared `DROBOTICFLOW_API_TOKEN` for this fixed POC-result import route only; regular business writes still require a user session. The imported run contains a `poc-operator-full-acceptance-result.json` artifact, so the Monitoring page can show the same strict-vs-dependency-isolation summary that appears in the raw Job log. For `fullExternalAcceptance=true`, the import must also carry Kubernetes source evidence (`source.namespace`, `source.jobName`, `source.image`, and `source.imageId`); for every imported result, any provided `source.imageId` must be a full `@sha256:<64 hex>` image digest, must identify the same repository as `source.image`, and any provided `source.image` must not contain whitespace. The Monitoring card displays these fields so a final pass is tied to the exact POC Job and operator-runner image digest that produced it. Set `POC_ACCEPTANCE_IMPORT_RESULT=false` for one-off diagnostics that should not create or update platform acceptance history.
When the Job emits a `DROBOTICFLOW_POC_OPERATOR_ACCEPTANCE=<json>` marker, the wrapper extracts it from the Job logs even if the Job exits failed. If `POC_ACCEPTANCE_IMPORT_API_BASE_URL`, `API_BASE_URL`, or `BASE_URL` is set, it posts that result to `/api/acceptance/poc-operator-result` and records it as a normal platform run under the `POC 算子验收` project. In session-auth POC deployments, the acceptance-import endpoint accepts either an admin/pipeline/operator session through `DROBOTICFLOW_ACCEPTANCE_EMAIL` and `DROBOTICFLOW_ACCEPTANCE_PASSWORD`, or the shared `DROBOTICFLOW_API_TOKEN` for this fixed POC-result import route only; regular business writes still require a user session. The imported run contains a `poc-operator-full-acceptance-result.json` artifact, so the Monitoring page can show the same strict-vs-dependency-isolation summary that appears in the raw Job log. For `fullExternalAcceptance=true`, the import must also carry Kubernetes source evidence (`source.namespace`, `source.jobName`, `source.image`, and `source.imageId`); for every imported result, `result.runId` should equal `source.jobName`, any provided `source.imageId` must be a full `@sha256:<64 hex>` image digest, must identify the same repository as `source.image`, and any provided `source.image` must not contain whitespace. The Monitoring card displays these fields so a final pass is tied to the exact POC Job and operator-runner image digest that produced it. Set `POC_ACCEPTANCE_IMPORT_RESULT=false` for one-off diagnostics that should not create or update platform acceptance history.
After importing a POC operator result, verify the platform record with the same Kubernetes/Mongo evidence path instead of relying on the import response alone:

View File

@ -781,6 +781,8 @@ $image_pull_secrets_yaml
imagePullPolicy: Always
command: ["python", "/acceptance/poc_operator_full_acceptance_job.py"]
env:
- name: POC_ACCEPTANCE_RUN_ID
value: $job_name_json
- name: WORKSPACE_MOUNT_PATH
value: $workspace_mount_json
- name: POC_ACCEPTANCE_REQUIRED_EXTERNALS

View File

@ -168,6 +168,7 @@ const repositoryManifestErrors = data.static && Array.isArray(data.static.reposi
: [];
const summary = {{
runId: run.id,
acceptanceRunId: data.runId,
status: run.status,
artifactName: artifact.name,
acceptanceStatus: data.status,
@ -203,6 +204,7 @@ const summary = {{
importJobName: source.jobName,
importImage: source.image,
importSourceComplete: Boolean(source.namespace && source.jobName && source.image && source.imageId),
runIdMatchesJobName: source.jobName ? data.runId === source.jobName : null,
importImageMatches: expectedImage ? source.image === expectedImage : null,
imageIdHasSha256: /{IMAGE_DIGEST_PATTERN}/.test(source.imageId || ""),
humanReview: human ? {{
@ -213,6 +215,7 @@ const summary = {{
}};
if (summary.status !== expectedRunStatus) throw new Error(JSON.stringify({{ reason: "run status mismatch", summary }}));
if (summary.acceptanceStatus !== expectedAcceptanceStatus) throw new Error(JSON.stringify({{ reason: "acceptance status mismatch", summary }}));
if (summary.runIdMatchesJobName === false) throw new Error(JSON.stringify({{ reason: "acceptance runId does not match import jobName", summary }}));
if (expectedScope && summary.acceptanceScope !== expectedScope) throw new Error(JSON.stringify({{ reason: "acceptance scope mismatch", summary }}));
if (expectedImage && !summary.importImageMatches) throw new Error(JSON.stringify({{ reason: "runner image mismatch", summary }}));
if (requireKubernetesSource && !summary.importSourceComplete) throw new Error(JSON.stringify({{ reason: "missing kubernetes import source", summary }}));

View File

@ -135,6 +135,10 @@ def test_verify_poc_acceptance_import_creates_secret_backed_mongo_probe(tmp_path
assert env_by_name["EXPECTED_EXTERNALS"]["value"] == '["mongo"]'
assert env_by_name["REQUIRE_REAL_FIXTURES"]["value"] == "true"
assert env_by_name["REQUIRE_KUBERNETES_SOURCE"]["value"] == "true"
mongo_verify_js = env_by_name["MONGO_VERIFY_JS"]["value"]
assert "acceptanceRunId: data.runId" in mongo_verify_js
assert "runIdMatchesJobName: source.jobName ? data.runId === source.jobName : null" in mongo_verify_js
assert "acceptance runId does not match import jobName" in mongo_verify_js
assert "MONGO_URI" not in json.dumps(manifest.get("metadata", {}))
assert calls_path.read_text(encoding="utf-8").splitlines() == [
"-n droboticflow delete pod droboticflow-poc-import-verify-test --ignore-not-found=true",

View File

@ -547,6 +547,8 @@ def test_poc_operator_acceptance_renders_strict_poc_job_manifest():
assert 'name: "droboticflow-operator-acceptance-test"' in result.stdout
assert 'namespace: "droboticflow"' in result.stdout
assert 'image: "registry.example.com/dataflow/droboticflow-operator-runner:test"' in result.stdout
assert "name: POC_ACCEPTANCE_RUN_ID" in result.stdout
assert 'value: "droboticflow-operator-acceptance-test"' in result.stdout
assert 'value: "bos,dcloud,mongo"' in result.stdout
assert 'value: "false"' in result.stdout
assert "name: POC_ACCEPTANCE_ALLOW_GENERATED_FIXTURES" in result.stdout