From d5da39f5c6d96d35924ae2dceb7229f2862c1792 Mon Sep 17 00:00:00 2001 From: eust-w Date: Thu, 5 Nov 2026 05:09:20 +0800 Subject: [PATCH] =?UTF-8?q?=F0=9F=94=96=20Prepare=20POC=20rendering=20buil?= =?UTF-8?q?d=20with=20API-only=20rollout?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bump the workspace build version to 1.1.10 per the release policy. Add poc.api-image to pin only the API container by digest, with server dry-run, POC/repository guards and a resource-version precondition. Preserve all other deployments, operator pins, ConfigMaps and Secrets. Co-Authored-By: Claude Sonnet 5 --- .claude/commands/droflow-deploy.md | 1 + .claude/skills/droflow-ops/references/poc.md | 13 +++ docs/deployment.md | 19 +++++ docs/ops-catalog.md | 1 + internal/opscatalog/catalog_generated.go | 41 ++++++++++ ops/catalog/poc.yaml | 21 +++++ scripts/ops-rollout-api-image.sh | 43 ++++++++++ src/lib/ops-catalog.generated.ts | 38 +++++++++ tests/backend/ops_rollout_api_image_test.py | 85 ++++++++++++++++++++ tools/drosync/Cargo.lock | 16 ++-- tools/drosync/Cargo.toml | 2 +- 11 files changed, 271 insertions(+), 9 deletions(-) create mode 100755 scripts/ops-rollout-api-image.sh create mode 100644 tests/backend/ops_rollout_api_image_test.py diff --git a/.claude/commands/droflow-deploy.md b/.claude/commands/droflow-deploy.md index 9bbb392b..062f3acc 100644 --- a/.claude/commands/droflow-deploy.md +++ b/.claude/commands/droflow-deploy.md @@ -11,6 +11,7 @@ argument-hint: "[要做什么,例如 构建推送 POC 镜像]" - `poc.images`:构建推送 POC 镜像——用集群内 Kaniko 从当前工作树构建 api/worker/runner 等镜像并推送到 dataflow-poc(变更、需确认) - `poc.deploy`:部署 POC——渲染清单并滚动更新 droboticflow 命名空间(使用最近一次构建的镜像标签)(变更、需确认) +- `poc.api-image`:更新 POC API 镜像——只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥(变更、需确认、可 dry-run) - `poc.ship`:构建并部署 POC——依次构建推送镜像、部署、查看就绪度(变更、需确认) - `poc.rollout-restart`:滚动重启——滚动重启 API 与 worker 并等待就绪(不换镜像)(变更、需确认、可 dry-run) - `poc.manifests-validate`:校验 K8s 清单——用占位镜像渲染清单并做 kubectl client dry-run 校验(只读) diff --git a/.claude/skills/droflow-ops/references/poc.md b/.claude/skills/droflow-ops/references/poc.md index 138827b6..c0f52f85 100644 --- a/.claude/skills/droflow-ops/references/poc.md +++ b/.claude/skills/droflow-ops/references/poc.md @@ -8,6 +8,7 @@ POC 环境(命名空间 droboticflow)的构建、部署、清理与数据收 |---|---|---|---|---|---| | `poc.images`
构建推送 POC 镜像 | 用集群内 Kaniko 从当前工作树构建 api/worker/runner 等镜像并推送到 dataflow-poc([runbook](../../../../docs/deployment.md#2-build-and-push-images))
不在产品内:构建上下文是本机工作树,产品侧没有你的未发布代码;产品里请用 poc.rollout-restart | 发布 | 变更、需确认 | `env`(poc,默认 poc)
`build_operators`(bool,默认 false) | cli claude | | `poc.deploy`
部署 POC | 渲染清单并滚动更新 droboticflow 命名空间(使用最近一次构建的镜像标签)([runbook](../../../../docs/deployment.md#4-deploy))
不在产品内:需要本机 registry 凭据与最近一次构建产出的镜像标签;产品里请用 poc.rollout-restart | 发布 | 变更、需确认 | `env`(poc,默认 poc) | cli claude | +| `poc.api-image`
更新 POC API 镜像 | 只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥([runbook](../../../../docs/deployment.md#api-only-poc-rollout))
不在产品内:由本机核对新构建的镜像摘要后执行 | 发布 | 变更、需确认、可 dry-run | `env`(poc,默认 poc)
`image`(string,必填) | cli claude | | `poc.ship`
构建并部署 POC | 依次构建推送镜像、部署、查看就绪度([runbook](../../../../docs/deployment.md))
不在产品内:包含 poc.images,依赖本机工作树 | 发布 | 变更、需确认 | — | cli claude | | `poc.rollout-restart`
滚动重启 | 滚动重启 API 与 worker 并等待就绪(不换镜像)([runbook](../../../../docs/operations.md#kubernetes-verification)) | 发布 | 变更、需确认、可 dry-run | `env`(poc/wjzh,默认 poc)
`target`(all/api/worker,默认 all) | cli claude product | | `poc.manifests-validate`
校验 K8s 清单 | 用占位镜像渲染清单并做 kubectl client dry-run 校验([runbook](../../../../docs/deployment.md#6-manifest-variables))
不在产品内:依赖本机 node_modules 与工作树 | 只读 | 只读 | — | cli claude | @@ -41,6 +42,18 @@ go run ./cmd/droctl run poc.deploy --surface claude Runbook:`docs/deployment.md#4-deploy` +### `poc.api-image` — 更新 POC API 镜像 + +只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥 + +```bash +go run ./cmd/droctl run poc.api-image --arg image= --surface claude +``` + +前置检查:`branch-is-poc`、`kube-reachable` + +Runbook:`docs/deployment.md#api-only-poc-rollout` + ### `poc.ship` — 构建并部署 POC 依次构建推送镜像、部署、查看就绪度 diff --git a/docs/deployment.md b/docs/deployment.md index 4a0a199d..0e7ec7cc 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -157,6 +157,25 @@ The deploy script performs these actions: The main deployment path no longer creates a platform MongoDB Pod or Service. You must provision the backing MongoDB outside this manifest and provide its URI explicitly. +### API-only POC rollout + +For a frontend/API-only update, do not run the full environment deployment: it also rewrites +Secrets, ConfigMaps and other components. Build just `api` with `poc.images` and +`POC_KANIKO_COMPONENTS=api`, then take the pushed digest from the completed Kaniko Job log. + +```bash +go run ./cmd/droctl run poc.api-image --arg image=/dataflow-poc/droboticflow-api@sha256: --dry-run +``` + +```bash +go run ./cmd/droctl run poc.api-image --arg image=/dataflow-poc/droboticflow-api@sha256: --no-dry-run --yes +``` + +This action requires branch `poc`, namespace `droboticflow` and an image digest in the current API +repository. It updates only container `api` and `DROBOTICFLOW_IMAGE`, checks the deployment +resource version for concurrent changes, and waits for the rollout. It never updates worker, +coordinator, operator pins, ConfigMaps or Secrets. Record the previous revision for rollback. + ## 5. Services Created - `droboticflow-api`: NodePort service that exposes the frontend and API diff --git a/docs/ops-catalog.md b/docs/ops-catalog.md index 03cc9bd3..5d2c973d 100644 --- a/docs/ops-catalog.md +++ b/docs/ops-catalog.md @@ -112,6 +112,7 @@ POC 环境(命名空间 droboticflow)的构建、部署、清理与数据收 |---|---|---|---|---|---| | `poc.images`
构建推送 POC 镜像 | 用集群内 Kaniko 从当前工作树构建 api/worker/runner 等镜像并推送到 dataflow-poc([runbook](../docs/deployment.md#2-build-and-push-images))
不在产品内:构建上下文是本机工作树,产品侧没有你的未发布代码;产品里请用 poc.rollout-restart | 发布 | 变更、需确认 | `env`(poc,默认 poc)
`build_operators`(bool,默认 false) | cli claude | | `poc.deploy`
部署 POC | 渲染清单并滚动更新 droboticflow 命名空间(使用最近一次构建的镜像标签)([runbook](../docs/deployment.md#4-deploy))
不在产品内:需要本机 registry 凭据与最近一次构建产出的镜像标签;产品里请用 poc.rollout-restart | 发布 | 变更、需确认 | `env`(poc,默认 poc) | cli claude | +| `poc.api-image`
更新 POC API 镜像 | 只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥([runbook](../docs/deployment.md#api-only-poc-rollout))
不在产品内:由本机核对新构建的镜像摘要后执行 | 发布 | 变更、需确认、可 dry-run | `env`(poc,默认 poc)
`image`(string,必填) | cli claude | | `poc.ship`
构建并部署 POC | 依次构建推送镜像、部署、查看就绪度([runbook](../docs/deployment.md))
不在产品内:包含 poc.images,依赖本机工作树 | 发布 | 变更、需确认 | — | cli claude | | `poc.rollout-restart`
滚动重启 | 滚动重启 API 与 worker 并等待就绪(不换镜像)([runbook](../docs/operations.md#kubernetes-verification)) | 发布 | 变更、需确认、可 dry-run | `env`(poc/wjzh,默认 poc)
`target`(all/api/worker,默认 all) | cli claude product | | `poc.manifests-validate`
校验 K8s 清单 | 用占位镜像渲染清单并做 kubectl client dry-run 校验([runbook](../docs/deployment.md#6-manifest-variables))
不在产品内:依赖本机 node_modules 与工作树 | 只读 | 只读 | — | cli claude | diff --git a/internal/opscatalog/catalog_generated.go b/internal/opscatalog/catalog_generated.go index f492e7bf..b3c3fea7 100644 --- a/internal/opscatalog/catalog_generated.go +++ b/internal/opscatalog/catalog_generated.go @@ -1087,6 +1087,47 @@ var generatedActions = []Action{ Runbook: "docs/deployment.md#4-deploy", FailureModes: []string{"docs/operations.md#image-pull-failure", "docs/operations.md#worker-cannot-launch-jobs"}, }, + { + ID: "poc.api-image", + Domain: "poc", + Title: "更新 POC API 镜像", + Summary: "只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥", + Tier: TierRelease, + Mutates: true, + Confirm: true, + Surfaces: []Surface{SurfaceCLI, SurfaceClaude}, + SurfaceNote: "由本机核对新构建的镜像摘要后执行", + Args: []Arg{ + { + Name: "env", + Type: "enum", + Summary: "部署环境", + Values: []string{"poc"}, + Default: "poc", + HasDefault: true, + MapsTo: ArgMapping{Kind: "env", Name: "DEPLOY_ENV"}, + }, + { + Name: "image", + Type: "string", + Summary: "dataflow-poc API 镜像的完整 sha256 引用", + Required: true, + MapsTo: ArgMapping{Kind: "env", Name: "API_IMAGE"}, + }, + }, + Preflight: []Preflight{ + {ID: "branch-is-poc", Kind: "git-branch", Expect: "poc"}, + {ID: "kube-reachable", Kind: "kube-reachable", EnvArg: "env"}, + }, + Execution: Execution{ + Kind: "script", + Script: "scripts/ops-rollout-api-image.sh", + DryRunEnv: "DRY_RUN", + Interpreter: []string{"bash"}, + TimeoutSeconds: 900, + }, + Runbook: "docs/deployment.md#api-only-poc-rollout", + }, { ID: "poc.ship", Domain: "poc", diff --git a/ops/catalog/poc.yaml b/ops/catalog/poc.yaml index cdce470c..0bf875fa 100644 --- a/ops/catalog/poc.yaml +++ b/ops/catalog/poc.yaml @@ -44,6 +44,27 @@ actions: runbook: docs/deployment.md#4-deploy failure_modes: [docs/operations.md#image-pull-failure, docs/operations.md#worker-cannot-launch-jobs] + - id: poc.api-image + title: 更新 POC API 镜像 + summary: 只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥 + tier: release + mutates: true + confirm: true + surfaces: [cli, claude] + surface_note: 由本机核对新构建的镜像摘要后执行 + args: + - {name: env, type: enum, values: [poc], default: poc, summary: 部署环境, maps_to: {kind: env, name: DEPLOY_ENV}} + - {name: image, type: string, required: true, summary: dataflow-poc API 镜像的完整 sha256 引用, maps_to: {kind: env, name: API_IMAGE}} + preflight: + - {id: branch-is-poc, kind: git-branch, expect: poc} + - {id: kube-reachable, kind: kube-reachable, env_arg: env} + execution: + kind: script + script: scripts/ops-rollout-api-image.sh + dry_run_env: DRY_RUN + timeout_seconds: 900 + runbook: docs/deployment.md#api-only-poc-rollout + - id: poc.ship title: 构建并部署 POC summary: 依次构建推送镜像、部署、查看就绪度 diff --git a/scripts/ops-rollout-api-image.sh b/scripts/ops-rollout-api-image.sh new file mode 100755 index 00000000..16e884e2 --- /dev/null +++ b/scripts/ops-rollout-api-image.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# Pin only the POC API image and its reported image metadata. Never re-render cluster configuration. +set -euo pipefail +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +source "$ROOT_DIR/scripts/load-deploy-env.sh" + +if [[ "${DEPLOY_ENV:-}" != "poc" || "${K8S_NAMESPACE:-}" != "droboticflow" ]]; then + printf 'ERROR: this action only targets POC namespace droboticflow.\n' >&2 + exit 2 +fi +if [[ ! "${API_IMAGE:-}" =~ ^[^/[:space:]]+/dataflow-poc/droboticflow-api@sha256:[a-f0-9]{64}$ ]]; then + printf 'ERROR: API_IMAGE must be a dataflow-poc/droboticflow-api image pinned by sha256 digest.\n' >&2 + exit 2 +fi +case "${DRY_RUN:-true}" in + true) dry_mode=server ;; + false) dry_mode=none ;; + *) printf 'ERROR: DRY_RUN must be true or false.\n' >&2; exit 2 ;; +esac +export API_IMAGE + +patch="$(kubectl -n "$K8S_NAMESPACE" get deployment droboticflow-api -o json | python3 -c ' +import json, os, sys +current = json.load(sys.stdin) +image = os.environ["API_IMAGE"] +repo = image.split("@", 1)[0] +api = next(c for c in current["spec"]["template"]["spec"]["containers"] if c["name"] == "api") +old = api["image"] +if not (old == repo or old.startswith(repo + "@") or old.startswith(repo + ":")): + raise SystemExit("refusing to switch the API to a different image repository") +print("Previous API revision=" + current["metadata"].get("annotations", {}).get("deployment.kubernetes.io/revision", "unknown") + " image=" + old, file=sys.stderr) +print(json.dumps({ + "metadata": {"resourceVersion": current["metadata"]["resourceVersion"]}, + "spec": {"template": {"spec": {"containers": [{"name": "api", "image": image, + "env": [{"name": "DROBOTICFLOW_IMAGE", "value": image, "valueFrom": None}]}]}}} +})) +')" +kubectl -n "$K8S_NAMESPACE" patch deployment droboticflow-api --type strategic --dry-run="$dry_mode" -p "$patch" +if [[ "${DRY_RUN:-true}" == "true" ]]; then + printf 'DRY-RUN: validated API-only image update; no deployment was changed.\n' + exit 0 +fi +kubectl -n "$K8S_NAMESPACE" rollout status deployment/droboticflow-api --timeout=600s diff --git a/src/lib/ops-catalog.generated.ts b/src/lib/ops-catalog.generated.ts index ac3244bf..62c384b0 100644 --- a/src/lib/ops-catalog.generated.ts +++ b/src/lib/ops-catalog.generated.ts @@ -1342,6 +1342,44 @@ export const OPS_ACTIONS: OpsAction[] = [ "supportsDryRun": false, "runbook": "docs/deployment.md#4-deploy" }, + { + "id": "poc.api-image", + "domain": "poc", + "title": "更新 POC API 镜像", + "summary": "只更新 API 镜像及镜像标记,保留 Worker、算子、配置与密钥", + "tier": "release", + "roles": [ + "admin", + "pipeline_engineer" + ], + "mutates": true, + "confirm": true, + "surfaces": [ + "cli", + "claude" + ], + "surfaceNote": "由本机核对新构建的镜像摘要后执行", + "args": [ + { + "name": "env", + "type": "enum", + "summary": "部署环境", + "values": [ + "poc" + ], + "default": "poc", + "required": false + }, + { + "name": "image", + "type": "string", + "summary": "dataflow-poc API 镜像的完整 sha256 引用", + "required": true + } + ], + "supportsDryRun": true, + "runbook": "docs/deployment.md#api-only-poc-rollout" + }, { "id": "poc.ship", "domain": "poc", diff --git a/tests/backend/ops_rollout_api_image_test.py b/tests/backend/ops_rollout_api_image_test.py new file mode 100644 index 00000000..40aa2830 --- /dev/null +++ b/tests/backend/ops_rollout_api_image_test.py @@ -0,0 +1,85 @@ +import json +import os +import pathlib +import subprocess +import sys + +import pytest + +ROOT = pathlib.Path(__file__).resolve().parents[2] +SCRIPT = ROOT / "scripts" / "ops-rollout-api-image.sh" +IMAGE = "registry.example/dataflow-poc/droboticflow-api@sha256:" + "a" * 64 + + +def run_rollout(tmp_path, **overrides): + calls = tmp_path / "calls.jsonl" + binary = tmp_path / "kubectl" + binary.write_text( + f"#!{sys.executable}\n" + "import json, os, pathlib, sys\n" + f"calls = pathlib.Path({str(calls)!r})\n" + "args = sys.argv[1:]\n" + "with calls.open('a') as f: f.write(json.dumps(args) + '\\n')\n" + "if args[2] == 'get':\n" + " print(json.dumps({'metadata': {'resourceVersion': '21', 'annotations': {'deployment.kubernetes.io/revision': '7'}}, 'spec': {'template': {'spec': {'containers': [{'name': 'api', 'image': 'registry.example/dataflow-poc/droboticflow-api:old'}]}}}}))\n" + "elif args[2] == 'patch' and os.environ.get('FAKE_PATCH_FAIL') == 'true':\n" + " sys.exit(1)\n" + ) + binary.chmod(0o755) + env = { + **os.environ, + "PATH": str(tmp_path) + os.pathsep + os.environ["PATH"], + "DROBOTICFLOW_DEPLOY_ENV_LOADED": "poc", + "DEPLOY_ENV": "poc", + "K8S_NAMESPACE": "droboticflow", + "API_IMAGE": IMAGE, + "DRY_RUN": "true", + **overrides, + } + result = subprocess.run(["bash", str(SCRIPT)], env=env, capture_output=True, text=True) + commands = [json.loads(line) for line in calls.read_text().splitlines()] if calls.exists() else [] + return result, commands + + +def test_api_only_dry_run_does_not_start_rollout(tmp_path): + result, commands = run_rollout(tmp_path) + assert result.returncode == 0, result.stderr + assert len(commands) == 2 + assert "--dry-run=server" in commands[1] + patch = json.loads(commands[1][commands[1].index("-p") + 1]) + assert patch["metadata"] == {"resourceVersion": "21"} + containers = patch["spec"]["template"]["spec"]["containers"] + assert containers == [{"name": "api", "image": IMAGE, "env": [{"name": "DROBOTICFLOW_IMAGE", "value": IMAGE, "valueFrom": None}]}] + + +def test_api_only_rollout_changes_only_api(tmp_path): + result, commands = run_rollout(tmp_path, DRY_RUN="false") + assert result.returncode == 0, result.stderr + assert len(commands) == 3 + assert "--dry-run=server" not in commands[1] + assert commands[2] == ["-n", "droboticflow", "rollout", "status", "deployment/droboticflow-api", "--timeout=600s"] + + +@pytest.mark.parametrize("overrides", [ + {"API_IMAGE": IMAGE.replace("dataflow-poc", "fanyu")}, + {"API_IMAGE": "registry.example/dataflow-poc/droboticflow-api:latest"}, + {"K8S_NAMESPACE": "other"}, + {"DEPLOY_ENV": "wjzh"}, + {"DRY_RUN": "invalid"}, +]) +def test_invalid_target_rejected_before_kubectl(tmp_path, overrides): + result, commands = run_rollout(tmp_path, **overrides) + assert result.returncode != 0 + assert commands == [] + + +def test_different_registry_is_rejected(tmp_path): + result, commands = run_rollout(tmp_path, API_IMAGE=IMAGE.replace("registry.example", "other.example")) + assert result.returncode != 0 + assert len(commands) == 1 + + +def test_patch_conflict_stops_without_waiting_or_retrying(tmp_path): + result, commands = run_rollout(tmp_path, DRY_RUN="false", FAKE_PATCH_FAIL="true") + assert result.returncode != 0 + assert len(commands) == 2 diff --git a/tools/drosync/Cargo.lock b/tools/drosync/Cargo.lock index 023f3795..89907655 100644 --- a/tools/drosync/Cargo.lock +++ b/tools/drosync/Cargo.lock @@ -683,7 +683,7 @@ dependencies = [ [[package]] name = "drosync-cli" -version = "1.1.9" +version = "1.1.10" dependencies = [ "anyhow", "base64", @@ -703,7 +703,7 @@ dependencies = [ [[package]] name = "drosync-core" -version = "1.1.9" +version = "1.1.10" dependencies = [ "base64", "cap-fs-ext", @@ -727,7 +727,7 @@ dependencies = [ [[package]] name = "drosync-daemon" -version = "1.1.9" +version = "1.1.10" dependencies = [ "anyhow", "base64", @@ -754,7 +754,7 @@ dependencies = [ [[package]] name = "drosync-platform" -version = "1.1.9" +version = "1.1.10" dependencies = [ "directories", "keyring", @@ -767,7 +767,7 @@ dependencies = [ [[package]] name = "drosync-protocol" -version = "1.1.9" +version = "1.1.10" dependencies = [ "chrono", "serde", @@ -776,7 +776,7 @@ dependencies = [ [[package]] name = "drosync-state" -version = "1.1.9" +version = "1.1.10" dependencies = [ "chrono", "drosync-core", @@ -790,7 +790,7 @@ dependencies = [ [[package]] name = "drosync-transfer" -version = "1.1.9" +version = "1.1.10" dependencies = [ "base64", "drosync-core", @@ -810,7 +810,7 @@ dependencies = [ [[package]] name = "drosync-updater" -version = "1.1.9" +version = "1.1.10" dependencies = [ "base64", "drosync-protocol", diff --git a/tools/drosync/Cargo.toml b/tools/drosync/Cargo.toml index 19a9d39c..a5ea0a00 100644 --- a/tools/drosync/Cargo.toml +++ b/tools/drosync/Cargo.toml @@ -12,7 +12,7 @@ members = [ ] [workspace.package] -version = "1.1.9" +version = "1.1.10" edition = "2024" license = "Apache-2.0" rust-version = "1.89"