🛡️ Require explicit deploy image registry

This commit is contained in:
eust-w 2026-06-17 17:36:08 +08:00
parent 881819d746
commit 28435bb746
13 changed files with 254 additions and 78 deletions

View File

@ -13,7 +13,11 @@ K8S_NAMESPACE=droboticflow
K8S_IMAGE_PULL_SECRET_NAME=droboticflow-registry
WORKSPACE_PVC_NAME=droboticflow-workspace
WORKSPACE_MOUNT_PATH=/workspace
REGISTRY_HOST=
REGISTRY_NAMESPACE=
IMAGE_TAG=
API_IMAGE=
WORKER_IMAGE=
OPERATOR_RUNNER_IMAGE=
OPERATOR_GPU_NODE_SELECTOR=
OPERATOR_GPU_TOLERATIONS_JSON=

View File

@ -94,8 +94,9 @@ Key environment variables:
- `K8S_IMAGE_PULL_SECRET_NAME`: image pull secret injected into operator-runner Jobs
- `WORKSPACE_PVC_NAME`: shared PVC used by worker, API, and operator-runner Jobs
- `WORKSPACE_MOUNT_PATH`: shared workspace mount path, default `/workspace`
- `IMAGE_TAG`: optional shared image tag for build/deploy/render scripts; scripts default to the current git short SHA
- `OPERATOR_RUNNER_IMAGE`: container image used for node execution Jobs; required for worker startup, while build/deploy/render scripts default it to the current git image tag
- `REGISTRY_HOST` / `REGISTRY_NAMESPACE`: explicit registry coordinates used by build/deploy/render scripts to derive image references when the three image variables below are not set
- `IMAGE_TAG`: optional shared image tag for build/deploy/render scripts; scripts default to the current git short SHA only after explicit registry coordinates are configured
- `API_IMAGE` / `WORKER_IMAGE` / `OPERATOR_RUNNER_IMAGE`: container images used for the API, worker, and node execution Jobs; set all three explicitly, or set `REGISTRY_HOST` and `REGISTRY_NAMESPACE` so scripts can derive them
- `OPERATOR_GPU_NODE_SELECTOR`: optional comma-separated node selector for GPU Jobs, for example `accelerator=nvidia`
- `OPERATOR_GPU_TOLERATIONS_JSON`: optional Kubernetes tolerations JSON array for GPU Jobs
- `OPERATOR_GPU_RUNTIME_CLASS`: optional runtimeClassName for GPU Jobs, for example `nvidia`
@ -116,7 +117,7 @@ Key environment variables:
- `VITE_DEV_API_BASE_URL`: optional Vite dev proxy target for `/api`, default `http://127.0.0.1:3001`
- `VITE_DEV_WORKER_BASE_URL`: optional Vite dev proxy target for `/worker-api`, default `http://127.0.0.1:3002`
Explicit startup values are validated before API/worker startup, before Kubernetes manifest rendering, and before `scripts/deploy-k8s.sh` performs cluster writes. `MONGO_URI` is required at runtime and must be a `mongodb://` or `mongodb+srv://` URI without whitespace or unreplaced placeholder markers. `PORT`/`WORKER_PORT` must be valid TCP ports, Mongo retry count and retry delay must be positive, worker poll interval must be positive, replica counts must be non-negative in manifests, NodePort must be in the Kubernetes `30000..32767` range, and `OPERATOR_JOB_TTL_SECONDS` must be non-negative (`0` disables TTL injection). Manifest and deploy wrappers also reject invalid Kubernetes namespace/Secret/PVC/Service names, non-absolute workspace mount paths, malformed worker/BOS service URLs, malformed non-empty AI service URLs, missing AI base/model when an AI key is configured, and container image references with whitespace. Multi-line ConfigMap block values such as GPU tolerations JSON and shell command allow/deny regexes are indented before manifest output so readable multi-line input stays valid YAML. Explicit boolean env values accept `true/false`, `1/0`, `yes/no`, or `on/off`; other spellings fail fast instead of silently falling back, and accepted true aliases drive deploy-time branches such as public LoadBalancer creation.
Explicit startup values are validated before API/worker startup, before Kubernetes manifest rendering, and before `scripts/deploy-k8s.sh` performs cluster writes. `MONGO_URI` is required at runtime and must be a `mongodb://` or `mongodb+srv://` URI without whitespace or unreplaced placeholder markers. `PORT`/`WORKER_PORT` must be valid TCP ports, Mongo retry count and retry delay must be positive, worker poll interval must be positive, replica counts must be non-negative in manifests, NodePort must be in the Kubernetes `30000..32767` range, and `OPERATOR_JOB_TTL_SECONDS` must be non-negative (`0` disables TTL injection). Manifest, build, and deploy wrappers no longer carry a repository-default image registry: callers must set `API_IMAGE`, `WORKER_IMAGE`, and `OPERATOR_RUNNER_IMAGE` explicitly, or set `REGISTRY_HOST` and `REGISTRY_NAMESPACE` so those image references can be derived from `IMAGE_TAG`. Manifest and deploy wrappers also reject invalid Kubernetes namespace/Secret/PVC/Service names, non-absolute workspace mount paths, malformed worker/BOS service URLs, malformed non-empty AI service URLs, missing AI base/model when an AI key is configured, and container image references with whitespace. Multi-line ConfigMap block values such as GPU tolerations JSON and shell command allow/deny regexes are indented before manifest output so readable multi-line input stays valid YAML. Explicit boolean env values accept `true/false`, `1/0`, `yes/no`, or `on/off`; other spellings fail fast instead of silently falling back, and accepted true aliases drive deploy-time branches such as public LoadBalancer creation.
Development and acceptance entrypoints resolve API endpoints through shared helpers instead of each script carrying its own localhost default. Node/Vite entrypoints use `scripts/lib/service-endpoints.mjs`, Python operational scripts use `scripts/lib/droboticflow_env.py`, and shell wrappers use `scripts/lib/service-endpoints.sh`. `BASE_URL` is accepted by browser-oriented acceptance scripts, while deployment/reconcile wrappers preserve `API_BASE_URL` as their primary environment variable. Explicit endpoint values must be absolute `http://` or `https://` URLs so malformed values fail before Vite, curl, Playwright, or urllib requests are attempted. Browser-side `VITE_API_BASE_URL` follows the same rule, with the empty value reserved for same-origin API requests.

View File

@ -8,7 +8,7 @@ This project deploys three images:
- `droboticflow-worker`
- `droboticflow-operator-runner`
Default Baidu registry naming:
Example Baidu registry naming:
- `ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-api:<tag>`
- `ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-worker:<tag>`
@ -25,7 +25,10 @@ docker login ccr-29eug8s3-pub.cnc.bj.baidubce.com --username <registry-user>
## 2. Build and Push Images
```bash
IMAGE_TAG=20260408 ./scripts/build-and-push-images.sh
REGISTRY_HOST=ccr-29eug8s3-pub.cnc.bj.baidubce.com \
REGISTRY_NAMESPACE=dataflow \
IMAGE_TAG=20260408 \
./scripts/build-and-push-images.sh
```
Image contents:
@ -45,7 +48,11 @@ OPERATOR_RUNNER_IMAGE=...
You can also load a named deployment preset:
```bash
DEPLOY_ENV=wjzh IMAGE_TAG=20260421 ./scripts/build-and-push-images.sh
DEPLOY_ENV=wjzh \
REGISTRY_HOST=ccr-29eug8s3-pub.cnc.bj.baidubce.com \
REGISTRY_NAMESPACE=dataflow \
IMAGE_TAG=20260421 \
./scripts/build-and-push-images.sh
```
On the `dev` branch, the scripts auto-resolve to `DEPLOY_ENV=poc` when you leave `DEPLOY_ENV` unset.
@ -58,6 +65,9 @@ Recommended environment variables:
```bash
export K8S_NAMESPACE=droboticflow
export REGISTRY_HOST=ccr-29eug8s3-pub.cnc.bj.baidubce.com
export REGISTRY_NAMESPACE=dataflow
export IMAGE_TAG=20260408
export API_IMAGE=ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-api:20260408
export WORKER_IMAGE=ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-worker:20260408
export OPERATOR_RUNNER_IMAGE=ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-operator-runner:20260408
@ -89,16 +99,17 @@ The loader also looks for `$HOME/.config/droboticflow/kubeconfigs/wjzh.config` a
For the `poc` environment, use [`scripts/deploy-envs/poc.env`](../scripts/deploy-envs/poc.env). It is separated from `wjzh` and defaults to:
- `MONGO_DB=poc`
- `REGISTRY_NAMESPACE=dataflow-poc`
- `API_PUBLIC_LOAD_BALANCER=true`
- the same namespace, registry host, AI defaults, and PVC settings as `wjzh`
- the same namespace, AI defaults, and PVC settings as `wjzh`
The repository does not hardcode registry coordinates for either preset. Provide either all three image references (`API_IMAGE`, `WORKER_IMAGE`, `OPERATOR_RUNNER_IMAGE`) or `REGISTRY_HOST` plus `REGISTRY_NAMESPACE` so build, render, and deploy scripts can derive the image references from `IMAGE_TAG`.
Store `poc` secrets outside git in:
- `$HOME/.config/droboticflow/deploy/poc.secrets.env`
- `.env.poc.local`
At minimum, the `poc` secrets file should provide `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `MONGO_URI`, `DROBOTICFLOW_API_TOKEN`, and for first-time workspace setup `WORKSPACE_BOS_BUCKET`, `BOS_ACCESS_KEY_ID`, and `BOS_SECRET_ACCESS_KEY`. Replace every placeholder in `MONGO_URI` before running API, worker, render, or deploy commands; runtime startup rejects missing values, whitespace, and unreplaced markers such as `<host>`.
At minimum, the `poc` secrets file should provide `REGISTRY_HOST`, `REGISTRY_NAMESPACE` or the three explicit image references, `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `MONGO_URI`, `DROBOTICFLOW_API_TOKEN`, and for first-time workspace setup `WORKSPACE_BOS_BUCKET`, `BOS_ACCESS_KEY_ID`, and `BOS_SECRET_ACCESS_KEY`. Replace every placeholder in `MONGO_URI` before running API, worker, render, or deploy commands; runtime startup rejects missing values, whitespace, and unreplaced markers such as `<host>`.
The loader also checks `$HOME/.config/droboticflow/kubeconfigs/poc.config` automatically. On the `dev` branch and `poc` branches, trying to deploy with another `DEPLOY_ENV` now fails fast to avoid cross-environment mistakes.
@ -180,6 +191,8 @@ http://<node-ip>:30080
- `AI_API_BASE_URL`
- `AI_MODEL`
- `AI_API_KEY` (validation only; the manifest reads the key from `droboticflow-ai-secret`)
- `REGISTRY_HOST`
- `REGISTRY_NAMESPACE`
- `IMAGE_TAG`
- `API_IMAGE`
- `WORKER_IMAGE`
@ -198,7 +211,7 @@ http://<node-ip>:30080
- The main manifest only deploys the application components. It does not create MongoDB resources in-cluster.
- `MONGO_URI` is required by API and worker startup and must be a concrete `mongodb://` or `mongodb+srv://` URI without whitespace or unreplaced placeholder markers. Kubernetes deployment must reference an external MongoDB instance; localhost, `mongodb:27017`, and in-cluster `.svc` addresses are rejected by `scripts/deploy-k8s.sh`.
- Startup configuration is validated before manifest output and before `scripts/deploy-k8s.sh` performs any cluster writes. `MONGO_CONNECT_RETRIES`, `MONGO_CONNECT_RETRY_DELAY_MS`, and `WORKER_POLL_INTERVAL_MS` must be positive integers; `API_REPLICAS`, `WORKER_REPLICAS`, and `OPERATOR_JOB_TTL_SECONDS` must be non-negative integers; `API_NODE_PORT` must be in `30000..32767` when `API_SERVICE_TYPE=NodePort`; boolean env values must be one of `true/false`, `1/0`, `yes/no`, or `on/off`. Kubernetes namespace, image-pull-secret, workspace PVC, and public LoadBalancer Service names must be lowercase DNS-compatible values; workspace mount paths must be absolute container paths; worker service URLs must be absolute `http://` or `https://` URLs; non-empty BOS and AI endpoint values must also be absolute `http://` or `https://` URLs; if `AI_API_KEY` is configured, both `AI_API_BASE_URL` and `AI_MODEL` are required and there is no provider/model hardcoded fallback; container image references must be non-empty and whitespace-free. Multi-line ConfigMap block values, including `OPERATOR_GPU_TOLERATIONS_JSON`, `OPERATOR_COMMAND_ALLOWLIST_REGEX`, and `OPERATOR_COMMAND_DENYLIST_REGEX`, are indented before manifest output so formatted JSON and multi-line regex lists remain valid YAML.
- Startup configuration is validated before manifest output and before `scripts/deploy-k8s.sh` performs any cluster writes. `MONGO_CONNECT_RETRIES`, `MONGO_CONNECT_RETRY_DELAY_MS`, and `WORKER_POLL_INTERVAL_MS` must be positive integers; `API_REPLICAS`, `WORKER_REPLICAS`, and `OPERATOR_JOB_TTL_SECONDS` must be non-negative integers; `API_NODE_PORT` must be in `30000..32767` when `API_SERVICE_TYPE=NodePort`; boolean env values must be one of `true/false`, `1/0`, `yes/no`, or `on/off`. Kubernetes namespace, image-pull-secret, workspace PVC, and public LoadBalancer Service names must be lowercase DNS-compatible values; workspace mount paths must be absolute container paths; worker service URLs must be absolute `http://` or `https://` URLs; non-empty BOS and AI endpoint values must also be absolute `http://` or `https://` URLs; if `AI_API_KEY` is configured, both `AI_API_BASE_URL` and `AI_MODEL` are required and there is no provider/model hardcoded fallback; container image references must be non-empty and whitespace-free. `scripts/render-manifests.sh`, `scripts/deploy-k8s.sh`, and `scripts/build-and-push-images.sh` require explicit `API_IMAGE`, `WORKER_IMAGE`, and `OPERATOR_RUNNER_IMAGE` values unless `REGISTRY_HOST` and `REGISTRY_NAMESPACE` are both set. Multi-line ConfigMap block values, including `OPERATOR_GPU_TOLERATIONS_JSON`, `OPERATOR_COMMAND_ALLOWLIST_REGEX`, and `OPERATOR_COMMAND_DENYLIST_REGEX`, are indented before manifest output so formatted JSON and multi-line regex lists remain valid YAML.
- `BOS_ENDPOINT` is rendered into the API and worker ConfigMap as an empty value when unset. Leave it unset only when every BOS scan config or storage connection provides its own endpoint; if both the workflow/connection endpoint and global `BOS_ENDPOINT` are empty, real BOS operations fail with an explicit missing-endpoint error instead of falling back to a hardcoded region.
- API and worker entrypoints are Go binaries with Mongo startup retry, so they tolerate the database becoming ready after the Deployments start.
- The API proxies `/worker-api/*` to `WORKER_API_BASE_URL`, defaulting to the in-cluster `http://droboticflow-worker:3002` Service. Override it only when the worker Service name, namespace, or exposure model differs.
@ -217,7 +230,7 @@ http://<node-ip>:30080
- `DROBOTICFLOW_SESSION_COOKIE_SECURE=true` sets the session cookie `Secure` attribute. `scripts/deploy-k8s.sh` defaults it to `true` when `API_PUBLIC_LOAD_BALANCER=true`; local development can leave it `false`.
- `DROBOTICFLOW_TRUSTED_PROXY_CIDRS` controls which reverse proxies may supply `X-Forwarded-For` / `X-Real-IP` for rate limiting. Leave it empty unless ingress or load balancer source ranges are known.
- Session-authenticated unsafe requests require `X-Droboticflow-CSRF-Token` to match the readable `droboticflow_csrf` cookie. The bundled frontend sets this header automatically.
- `IMAGE_TAG` defaults to the current git short SHA in build, deploy, and render scripts. Set it explicitly for release names or rollback-friendly tags.
- `IMAGE_TAG` defaults to the current git short SHA in build, deploy, and render scripts only when explicit registry coordinates are configured. Set it explicitly for release names or rollback-friendly tags.
- `scripts/smoke-test.sh` requires `SMOKE_SOURCE_PATH`, a real path visible to operator-runner Jobs. The `真实数据源` runner infers filesystem/PVC paths as `LOCAL` and `bos://bucket/prefix` paths as `BOS`; if no real source is configured, the source node fails instead of emitting an empty manifest. It can exercise session-authenticated deployments when `DROBOTICFLOW_SMOKE_EMAIL` and `DROBOTICFLOW_SMOKE_PASSWORD` are set; it logs in, persists cookies, and sends the CSRF header on write requests.
- `scripts/reconcile-system-data.sh` delegates to `scripts/reconcile_system_data.py` and uses the same protected-client behavior. Set `DROBOTICFLOW_API_TOKEN` for token-protected APIs when session auth is not enforced. For public session-auth deployments, set `DROBOTICFLOW_SMOKE_EMAIL` and `DROBOTICFLOW_SMOKE_PASSWORD` to an organization admin account because the reconcile flow updates settings, users, templates, and operators. Builtin operators are read from the live `/api/operators` catalog and refreshed to `OPERATOR_RUNNER_IMAGE`; the script does not carry a separate builtin catalog.
- `scripts/object-storage-acceptance.mjs` runs browser create, preview, rename, upload, delete, and optional video playback checks against a live storage connection. It never rewrites storage connection metadata; provide `OBJECT_STORAGE_CONNECTION_ID`, `OBJECT_STORAGE_CONNECTION_NAME`, or `OBJECT_STORAGE_BUCKET` explicitly. `OBJECT_STORAGE_ACCEPTANCE_PAYLOAD_FILE` is required and must point to a non-empty real UTF-8 text or JSON payload used for create/upload/preview checks; set `OBJECT_STORAGE_ACCEPTANCE_CONTENT_TYPE` when auto-detection is not enough. Set `OBJECT_STORAGE_VIDEO_KEY` only when an existing playable object should be validated. `PLAYWRIGHT_IGNORE_HTTPS_ERRORS` must be an explicit boolean spelling when set. The script supports `DROBOTICFLOW_API_TOKEN` for token-protected non-session APIs and `DROBOTICFLOW_SMOKE_EMAIL` / `DROBOTICFLOW_SMOKE_PASSWORD` for session-authenticated browser access.

View File

@ -8,7 +8,7 @@ This note tracks the production-facing safeguards that must stay true for Drobod
- The API proxies `/worker-api/*` through `WORKER_API_BASE_URL`; the default is the in-cluster worker Service.
- Worker proxy health, stats, metrics, and retry calls use short upstream timeouts. Manual `process-once` uses a long timeout because it can execute real queued work.
- Explicit shell operators are disabled by default through `OPERATOR_ALLOW_SHELL_COMMANDS=false`; allowlist and denylist regexes are platform-controlled and cannot be overridden by workflow hyperparams.
- Environment-specific values such as `K8S_ENDPOINT`, registry username, and workspace BOS bucket have no private repository defaults; provide them from local deploy secrets or CI secrets.
- Environment-specific values such as `K8S_ENDPOINT`, registry host/namespace, registry username, deploy image references, and workspace BOS bucket have no private repository defaults; provide them from local deploy secrets or CI secrets.
- Manifest rendering and deployment wrappers reject invalid Kubernetes namespace/Secret/PVC/Service names, non-absolute workspace mount paths, malformed worker/AI/BOS service URLs, and image references with whitespace before emitting manifests or calling `kubectl`. Multi-line ConfigMap block values are indented before manifest output so formatted GPU tolerations JSON and multi-line shell command policy regexes remain valid YAML.
- Local development and acceptance endpoints are resolved through shared helpers instead of per-script localhost literals. Vite dev proxy targets can be overridden with `VITE_DEV_API_BASE_URL` and `VITE_DEV_WORKER_BASE_URL`; browser-oriented acceptance scripts accept `BASE_URL` or `API_BASE_URL`, while reconcile/deploy wrappers keep `API_BASE_URL` as the primary operational variable. Explicit endpoint values must be absolute `http://` or `https://` URLs and fail before Vite, curl, Playwright, or urllib requests are attempted. Browser-side `VITE_API_BASE_URL` follows the same absolute http(s) rule, with an empty value reserved for same-origin API calls.
- Data reconciliation uses real API calls and supports protected deployments through `DROBOTICFLOW_API_TOKEN` or session login credentials plus CSRF headers. Public session-auth environments must use an organization admin account for reconcile because it updates settings and management resources.

View File

@ -33,7 +33,7 @@
"docker:build:worker": "docker build -f Dockerfile.worker -t droboticflow-worker:local .",
"docker:build:operator-runner": "docker build -f Dockerfile.operator-runner -t droboticflow-operator-runner:local .",
"k8s:render": "./scripts/render-manifests.sh",
"k8s:validate": "./scripts/render-manifests.sh | kubectl apply --dry-run=client --validate=false -f -",
"k8s:validate": "API_IMAGE=droboticflow-api:validate WORKER_IMAGE=droboticflow-worker:validate OPERATOR_RUNNER_IMAGE=droboticflow-operator-runner:validate ./scripts/render-manifests.sh | kubectl apply --dry-run=client --validate=false -f -",
"k8s:deploy": "./scripts/deploy-k8s.sh",
"k8s:cleanup-jobs": "./scripts/cleanup-operator-jobs.sh",
"gpu:smoke": "./scripts/gpu-smoke-test.sh",

View File

@ -14,15 +14,14 @@ export DEPLOY_ENV
source "$ROOT_DIR/scripts/load-deploy-env.sh"
DEPLOY_ENV="${DEPLOY_ENV:-}"
REGISTRY_HOST="${REGISTRY_HOST:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-dataflow}"
IMAGE_TAG="${IMAGE_TAG:-$(git -C "$ROOT_DIR" rev-parse --short HEAD 2>/dev/null || date +%Y%m%d%H%M%S)}"
REGISTRY_HOST="${REGISTRY_HOST:-}"
REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-}"
PLATFORM="${PLATFORM:-linux/amd64}"
PUSH="${PUSH:-true}"
API_IMAGE="${API_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-api:$IMAGE_TAG}"
WORKER_IMAGE="${WORKER_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-worker:$IMAGE_TAG}"
OPERATOR_RUNNER_IMAGE="${OPERATOR_RUNNER_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-operator-runner:$IMAGE_TAG}"
# shellcheck source=scripts/lib/image-config.sh
source "$ROOT_DIR/scripts/lib/image-config.sh"
droboticflow_resolve_image_configuration
docker buildx inspect >/dev/null 2>&1 || docker buildx create --use >/dev/null

View File

@ -4,9 +4,9 @@
export K8S_NAMESPACE="${K8S_NAMESPACE:-droboticflow}"
export REGISTRY_SERVER="${REGISTRY_SERVER:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
export REGISTRY_HOST="${REGISTRY_HOST:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
export REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-dataflow-poc}"
export REGISTRY_SERVER="${REGISTRY_SERVER:-${REGISTRY_HOST:-}}"
export REGISTRY_HOST="${REGISTRY_HOST:-}"
export REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-}"
export REGISTRY_USERNAME="${REGISTRY_USERNAME:-}"
export CREATE_REGISTRY_SECRET="${CREATE_REGISTRY_SECRET:-true}"

View File

@ -4,9 +4,9 @@
export K8S_NAMESPACE="${K8S_NAMESPACE:-droboticflow}"
export REGISTRY_SERVER="${REGISTRY_SERVER:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
export REGISTRY_HOST="${REGISTRY_HOST:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
export REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-dataflow}"
export REGISTRY_SERVER="${REGISTRY_SERVER:-${REGISTRY_HOST:-}}"
export REGISTRY_HOST="${REGISTRY_HOST:-}"
export REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-}"
export REGISTRY_USERNAME="${REGISTRY_USERNAME:-}"
export CREATE_REGISTRY_SECRET="${CREATE_REGISTRY_SECRET:-true}"

View File

@ -16,9 +16,9 @@ DEPLOY_ENV="${DEPLOY_ENV:-}"
K8S_NAMESPACE="${K8S_NAMESPACE:-droboticflow}"
REGISTRY_SERVER="${REGISTRY_SERVER:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
REGISTRY_HOST="${REGISTRY_HOST:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-dataflow}"
REGISTRY_HOST="${REGISTRY_HOST:-}"
REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-}"
REGISTRY_SERVER="${REGISTRY_SERVER:-$REGISTRY_HOST}"
REGISTRY_USERNAME="${REGISTRY_USERNAME:-}"
REGISTRY_PASSWORD="${REGISTRY_PASSWORD:-}"
CREATE_REGISTRY_SECRET="${CREATE_REGISTRY_SECRET:-true}"
@ -39,11 +39,9 @@ API_PUBLIC_LOAD_BALANCER="${API_PUBLIC_LOAD_BALANCER:-false}"
API_PUBLIC_SERVICE_NAME="${API_PUBLIC_SERVICE_NAME:-droboticflow-api-public}"
API_REPLICAS="${API_REPLICAS:-1}"
WORKER_REPLICAS="${WORKER_REPLICAS:-1}"
DEFAULT_IMAGE_TAG="$(git -C "$ROOT_DIR" rev-parse --short HEAD 2>/dev/null || date +%Y%m%d%H%M%S)"
IMAGE_TAG="${IMAGE_TAG:-$DEFAULT_IMAGE_TAG}"
API_IMAGE="${API_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-api:$IMAGE_TAG}"
WORKER_IMAGE="${WORKER_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-worker:$IMAGE_TAG}"
OPERATOR_RUNNER_IMAGE="${OPERATOR_RUNNER_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-operator-runner:$IMAGE_TAG}"
# shellcheck source=scripts/lib/image-config.sh
source "$ROOT_DIR/scripts/lib/image-config.sh"
droboticflow_resolve_image_configuration
OPERATOR_GPU_NODE_SELECTOR="${OPERATOR_GPU_NODE_SELECTOR:-}"
OPERATOR_GPU_TOLERATIONS_JSON="${OPERATOR_GPU_TOLERATIONS_JSON:-}"
OPERATOR_GPU_RUNTIME_CLASS="${OPERATOR_GPU_RUNTIME_CLASS:-}"
@ -64,6 +62,7 @@ source "$ROOT_DIR/scripts/lib/deploy-validation.sh"
validate_deploy_startup_configuration
if boolean_true CREATE_REGISTRY_SECRET "$CREATE_REGISTRY_SECRET"; then
: "${REGISTRY_SERVER:?REGISTRY_SERVER is required when CREATE_REGISTRY_SECRET=true}"
: "${REGISTRY_USERNAME:?REGISTRY_USERNAME is required when CREATE_REGISTRY_SECRET=true}"
: "${REGISTRY_PASSWORD:?REGISTRY_PASSWORD is required when CREATE_REGISTRY_SECRET=true}"
fi

View File

@ -0,0 +1,36 @@
#!/usr/bin/env bash
droboticflow_default_image_tag() {
git -C "$ROOT_DIR" rev-parse --short HEAD 2>/dev/null || date +%Y%m%d%H%M%S
}
droboticflow_resolve_image_reference() {
local variable_name="$1"
local image_name="$2"
local explicit_value="${!variable_name:-}"
if [[ -n "$explicit_value" ]]; then
printf '%s\n' "$explicit_value"
return 0
fi
if [[ -z "${REGISTRY_HOST:-}" || -z "${REGISTRY_NAMESPACE:-}" ]]; then
echo "ERROR: $variable_name is required unless REGISTRY_HOST and REGISTRY_NAMESPACE are both set." >&2
echo "Set explicit API_IMAGE, WORKER_IMAGE, and OPERATOR_RUNNER_IMAGE values, or configure REGISTRY_HOST and REGISTRY_NAMESPACE from local deploy secrets or CI." >&2
exit 1
fi
printf '%s/%s/%s:%s\n' "$REGISTRY_HOST" "$REGISTRY_NAMESPACE" "$image_name" "$IMAGE_TAG"
}
droboticflow_resolve_image_configuration() {
if [[ -z "${API_IMAGE:-}" || -z "${WORKER_IMAGE:-}" || -z "${OPERATOR_RUNNER_IMAGE:-}" ]]; then
IMAGE_TAG="${IMAGE_TAG:-$(droboticflow_default_image_tag)}"
else
IMAGE_TAG="${IMAGE_TAG:-}"
fi
API_IMAGE="$(droboticflow_resolve_image_reference API_IMAGE droboticflow-api)"
WORKER_IMAGE="$(droboticflow_resolve_image_reference WORKER_IMAGE droboticflow-worker)"
OPERATOR_RUNNER_IMAGE="$(droboticflow_resolve_image_reference OPERATOR_RUNNER_IMAGE droboticflow-operator-runner)"
export IMAGE_TAG API_IMAGE WORKER_IMAGE OPERATOR_RUNNER_IMAGE
}

View File

@ -43,14 +43,11 @@ OPERATOR_COMMAND_DENYLIST_REGEX="${OPERATOR_COMMAND_DENYLIST_REGEX:-}"
DROBOTICFLOW_REQUIRE_SESSION_AUTH="${DROBOTICFLOW_REQUIRE_SESSION_AUTH:-$API_PUBLIC_LOAD_BALANCER}"
DROBOTICFLOW_SESSION_COOKIE_SECURE="${DROBOTICFLOW_SESSION_COOKIE_SECURE:-$API_PUBLIC_LOAD_BALANCER}"
DROBOTICFLOW_TRUSTED_PROXY_CIDRS="${DROBOTICFLOW_TRUSTED_PROXY_CIDRS:-}"
REGISTRY_HOST="${REGISTRY_HOST:-ccr-29eug8s3-pub.cnc.bj.baidubce.com}"
REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-dataflow}"
DEFAULT_IMAGE_TAG="$(git -C "$ROOT_DIR" rev-parse --short HEAD 2>/dev/null || date +%Y%m%d%H%M%S)"
IMAGE_TAG="${IMAGE_TAG:-$DEFAULT_IMAGE_TAG}"
API_IMAGE="${API_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-api:$IMAGE_TAG}"
WORKER_IMAGE="${WORKER_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-worker:$IMAGE_TAG}"
OPERATOR_RUNNER_IMAGE="${OPERATOR_RUNNER_IMAGE:-$REGISTRY_HOST/$REGISTRY_NAMESPACE/droboticflow-operator-runner:$IMAGE_TAG}"
# shellcheck source=scripts/lib/image-config.sh
source "$ROOT_DIR/scripts/lib/image-config.sh"
REGISTRY_HOST="${REGISTRY_HOST:-}"
REGISTRY_NAMESPACE="${REGISTRY_NAMESPACE:-}"
droboticflow_resolve_image_configuration
# shellcheck source=scripts/lib/deploy-validation.sh
source "$ROOT_DIR/scripts/lib/deploy-validation.sh"
@ -84,41 +81,89 @@ OPERATOR_GPU_TOLERATIONS_JSON_BLOCK="$(render_block_scalar "$OPERATOR_GPU_TOLERA
OPERATOR_COMMAND_ALLOWLIST_REGEX_BLOCK="$(render_block_scalar "$OPERATOR_COMMAND_ALLOWLIST_REGEX")"
OPERATOR_COMMAND_DENYLIST_REGEX_BLOCK="$(render_block_scalar "$OPERATOR_COMMAND_DENYLIST_REGEX")"
rendered="$(cat "$TEMPLATE")"
rendered="${rendered//__NAMESPACE__/$K8S_NAMESPACE}"
rendered="${rendered//__MONGO_DB__/$MONGO_DB}"
rendered="${rendered//__MONGO_CONNECT_RETRIES__/$MONGO_CONNECT_RETRIES}"
rendered="${rendered//__MONGO_CONNECT_RETRY_DELAY_MS__/$MONGO_CONNECT_RETRY_DELAY_MS}"
rendered="${rendered//__WORKER_POLL_INTERVAL_MS__/$WORKER_POLL_INTERVAL_MS}"
rendered="${rendered//__WORKER_API_BASE_URL__/$WORKER_API_BASE_URL}"
rendered="${rendered//__BOS_ENDPOINT__/$BOS_ENDPOINT}"
rendered="${rendered//__K8S_IMAGE_PULL_SECRET_NAME__/$K8S_IMAGE_PULL_SECRET_NAME}"
rendered="${rendered//__WORKSPACE_PVC_NAME__/$WORKSPACE_PVC_NAME}"
rendered="${rendered//__WORKSPACE_MOUNT_PATH__/$WORKSPACE_MOUNT_PATH}"
rendered="${rendered//__API_SERVICE_TYPE__/$API_SERVICE_TYPE}"
rendered="${rendered//__API_NODE_PORT_BLOCK__/$API_NODE_PORT_BLOCK}"
rendered="${rendered//__API_REPLICAS__/$API_REPLICAS}"
rendered="${rendered//__WORKER_REPLICAS__/$WORKER_REPLICAS}"
rendered="${rendered//__AI_API_BASE_URL__/$AI_API_BASE_URL}"
rendered="${rendered//__AI_MODEL__/$AI_MODEL}"
rendered="${rendered//__API_IMAGE__/$API_IMAGE}"
rendered="${rendered//__WORKER_IMAGE__/$WORKER_IMAGE}"
rendered="${rendered//__OPERATOR_RUNNER_IMAGE__/$OPERATOR_RUNNER_IMAGE}"
rendered="${rendered//__OPERATOR_GPU_NODE_SELECTOR__/$OPERATOR_GPU_NODE_SELECTOR}"
rendered="${rendered//__OPERATOR_GPU_TOLERATIONS_JSON__/$OPERATOR_GPU_TOLERATIONS_JSON_BLOCK}"
rendered="${rendered//__OPERATOR_GPU_RUNTIME_CLASS__/$OPERATOR_GPU_RUNTIME_CLASS}"
rendered="${rendered//__OPERATOR_JOB_TTL_SECONDS__/$OPERATOR_JOB_TTL_SECONDS}"
rendered="${rendered//__OPERATOR_ALLOW_SHELL_COMMANDS__/$OPERATOR_ALLOW_SHELL_COMMANDS}"
rendered="${rendered//__OPERATOR_COMMAND_ALLOWLIST_REGEX__/$OPERATOR_COMMAND_ALLOWLIST_REGEX_BLOCK}"
rendered="${rendered//__OPERATOR_COMMAND_DENYLIST_REGEX__/$OPERATOR_COMMAND_DENYLIST_REGEX_BLOCK}"
rendered="${rendered//__DROBOTICFLOW_REQUIRE_SESSION_AUTH__/$DROBOTICFLOW_REQUIRE_SESSION_AUTH}"
rendered="${rendered//__DROBOTICFLOW_SESSION_COOKIE_SECURE__/$DROBOTICFLOW_SESSION_COOKIE_SECURE}"
rendered="${rendered//__DROBOTICFLOW_TRUSTED_PROXY_CIDRS__/$DROBOTICFLOW_TRUSTED_PROXY_CIDRS}"
export \
TEMPLATE \
K8S_NAMESPACE \
MONGO_DB \
MONGO_CONNECT_RETRIES \
MONGO_CONNECT_RETRY_DELAY_MS \
WORKER_POLL_INTERVAL_MS \
WORKER_API_BASE_URL \
BOS_ENDPOINT \
K8S_IMAGE_PULL_SECRET_NAME \
WORKSPACE_PVC_NAME \
WORKSPACE_MOUNT_PATH \
API_SERVICE_TYPE \
API_NODE_PORT_BLOCK \
API_REPLICAS \
WORKER_REPLICAS \
AI_API_BASE_URL \
AI_MODEL \
API_IMAGE \
WORKER_IMAGE \
OPERATOR_RUNNER_IMAGE \
OPERATOR_GPU_NODE_SELECTOR \
OPERATOR_GPU_TOLERATIONS_JSON_BLOCK \
OPERATOR_GPU_RUNTIME_CLASS \
OPERATOR_JOB_TTL_SECONDS \
OPERATOR_ALLOW_SHELL_COMMANDS \
OPERATOR_COMMAND_ALLOWLIST_REGEX_BLOCK \
OPERATOR_COMMAND_DENYLIST_REGEX_BLOCK \
DROBOTICFLOW_REQUIRE_SESSION_AUTH \
DROBOTICFLOW_SESSION_COOKIE_SECURE \
DROBOTICFLOW_TRUSTED_PROXY_CIDRS
if grep -Eq '__[A-Z0-9_]+__' <<<"$rendered"; then
echo "ERROR: rendered manifest still contains unresolved placeholders:" >&2
grep -Eo '__[A-Z0-9_]+__' <<<"$rendered" | sort -u >&2
exit 1
fi
python3 - <<'PY'
import os
import pathlib
import re
import sys
printf '%s\n' "$rendered"
template = pathlib.Path(os.environ["TEMPLATE"]).read_text(encoding="utf-8")
replacements = {
"__NAMESPACE__": os.environ["K8S_NAMESPACE"],
"__MONGO_DB__": os.environ["MONGO_DB"],
"__MONGO_CONNECT_RETRIES__": os.environ["MONGO_CONNECT_RETRIES"],
"__MONGO_CONNECT_RETRY_DELAY_MS__": os.environ["MONGO_CONNECT_RETRY_DELAY_MS"],
"__WORKER_POLL_INTERVAL_MS__": os.environ["WORKER_POLL_INTERVAL_MS"],
"__WORKER_API_BASE_URL__": os.environ["WORKER_API_BASE_URL"],
"__BOS_ENDPOINT__": os.environ["BOS_ENDPOINT"],
"__K8S_IMAGE_PULL_SECRET_NAME__": os.environ["K8S_IMAGE_PULL_SECRET_NAME"],
"__WORKSPACE_PVC_NAME__": os.environ["WORKSPACE_PVC_NAME"],
"__WORKSPACE_MOUNT_PATH__": os.environ["WORKSPACE_MOUNT_PATH"],
"__API_SERVICE_TYPE__": os.environ["API_SERVICE_TYPE"],
"__API_NODE_PORT_BLOCK__": os.environ["API_NODE_PORT_BLOCK"],
"__API_REPLICAS__": os.environ["API_REPLICAS"],
"__WORKER_REPLICAS__": os.environ["WORKER_REPLICAS"],
"__AI_API_BASE_URL__": os.environ["AI_API_BASE_URL"],
"__AI_MODEL__": os.environ["AI_MODEL"],
"__API_IMAGE__": os.environ["API_IMAGE"],
"__WORKER_IMAGE__": os.environ["WORKER_IMAGE"],
"__OPERATOR_RUNNER_IMAGE__": os.environ["OPERATOR_RUNNER_IMAGE"],
"__OPERATOR_GPU_NODE_SELECTOR__": os.environ["OPERATOR_GPU_NODE_SELECTOR"],
"__OPERATOR_GPU_TOLERATIONS_JSON__": os.environ["OPERATOR_GPU_TOLERATIONS_JSON_BLOCK"],
"__OPERATOR_GPU_RUNTIME_CLASS__": os.environ["OPERATOR_GPU_RUNTIME_CLASS"],
"__OPERATOR_JOB_TTL_SECONDS__": os.environ["OPERATOR_JOB_TTL_SECONDS"],
"__OPERATOR_ALLOW_SHELL_COMMANDS__": os.environ["OPERATOR_ALLOW_SHELL_COMMANDS"],
"__OPERATOR_COMMAND_ALLOWLIST_REGEX__": os.environ["OPERATOR_COMMAND_ALLOWLIST_REGEX_BLOCK"],
"__OPERATOR_COMMAND_DENYLIST_REGEX__": os.environ["OPERATOR_COMMAND_DENYLIST_REGEX_BLOCK"],
"__DROBOTICFLOW_REQUIRE_SESSION_AUTH__": os.environ["DROBOTICFLOW_REQUIRE_SESSION_AUTH"],
"__DROBOTICFLOW_SESSION_COOKIE_SECURE__": os.environ["DROBOTICFLOW_SESSION_COOKIE_SECURE"],
"__DROBOTICFLOW_TRUSTED_PROXY_CIDRS__": os.environ["DROBOTICFLOW_TRUSTED_PROXY_CIDRS"],
}
rendered = template
for placeholder, value in replacements.items():
rendered = rendered.replace(placeholder, value)
unresolved = sorted(set(re.findall(r"__[A-Z0-9_]+__", rendered)))
if unresolved:
print("ERROR: rendered manifest still contains unresolved placeholders:", file=sys.stderr)
for placeholder in unresolved:
print(placeholder, file=sys.stderr)
raise SystemExit(1)
sys.stdout.write(rendered)
if not rendered.endswith("\n"):
sys.stdout.write("\n")
PY

View File

@ -20,11 +20,15 @@ DEPLOY_ENV_KEYS = {
"DROBOTICFLOW_API_TOKEN",
"DROBOTICFLOW_REQUIRE_SESSION_AUTH",
"DROBOTICFLOW_SESSION_COOKIE_SECURE",
"IMAGE_TAG",
"K8S_IMAGE_PULL_SECRET_NAME",
"K8S_NAMESPACE",
"MONGO_URI",
"OPERATOR_RUNNER_IMAGE",
"REGISTRY_HOST",
"REGISTRY_PASSWORD",
"REGISTRY_SERVER",
"REGISTRY_NAMESPACE",
"REGISTRY_USERNAME",
"WORKER_API_BASE_URL",
"WORKER_IMAGE",
@ -48,6 +52,7 @@ def write_fake_kubectl(tmp_path: pathlib.Path, calls_path: pathlib.Path) -> path
"if args[:2] == ['get', 'namespace']:\n"
" raise SystemExit(0)\n"
"if args and args[0] == 'apply':\n"
" sys.stdin.read()\n"
" print('applied')\n"
" raise SystemExit(0)\n"
"if len(args) >= 2 and args[0] == '-n':\n"
@ -83,6 +88,9 @@ def run_deploy(tmp_path: pathlib.Path, extra_env: dict[str, str]) -> tuple[subpr
"DROBOTICFLOW_DEPLOY_ENV_LOADED": "test",
"K8S_NAMESPACE": "droboticflow-test",
"MONGO_URI": "mongodb://mongo.example.net:27017/droboticflow",
"API_IMAGE": "registry.example.com/dataflow/droboticflow-api:test",
"WORKER_IMAGE": "registry.example.com/dataflow/droboticflow-worker:test",
"OPERATOR_RUNNER_IMAGE": "registry.example.com/dataflow/droboticflow-operator-runner:test",
}
)
env.update(extra_env)
@ -151,6 +159,41 @@ def test_deploy_rejects_ai_key_without_explicit_runtime_before_kubectl(tmp_path)
assert not calls_path.exists(), calls_path.read_text(encoding="utf-8") if calls_path.exists() else ""
def test_deploy_requires_explicit_images_or_registry_configuration_before_kubectl(tmp_path):
result, calls_path = run_deploy(
tmp_path,
{
"API_IMAGE": "",
"WORKER_IMAGE": "",
"OPERATOR_RUNNER_IMAGE": "",
"CREATE_REGISTRY_SECRET": "off",
},
)
assert result.returncode == 1, result.stderr + result.stdout
assert "API_IMAGE is required unless REGISTRY_HOST and REGISTRY_NAMESPACE are both set" in result.stderr
assert not calls_path.exists(), calls_path.read_text(encoding="utf-8") if calls_path.exists() else ""
def test_deploy_derives_images_from_explicit_registry_configuration(tmp_path):
result, calls_path = run_deploy(
tmp_path,
{
"API_IMAGE": "",
"WORKER_IMAGE": "",
"OPERATOR_RUNNER_IMAGE": "",
"REGISTRY_HOST": "registry.example.com",
"REGISTRY_NAMESPACE": "dataflow-poc",
"IMAGE_TAG": "release-20260617",
"CREATE_REGISTRY_SECRET": "off",
},
)
assert result.returncode == 0, result.stderr + result.stdout
calls = calls_path.read_text(encoding="utf-8")
assert "apply --dry-run=client -f" in calls
def test_deploy_treats_public_load_balancer_boolean_alias_as_true(tmp_path):
result, calls_path = run_deploy(
tmp_path,

View File

@ -24,24 +24,35 @@ CONFIG_ENV_KEYS = {
"DROBOTICFLOW_REQUIRE_SESSION_AUTH",
"DROBOTICFLOW_SESSION_COOKIE_SECURE",
"DEPLOY_ENV",
"IMAGE_TAG",
"K8S_IMAGE_PULL_SECRET_NAME",
"K8S_NAMESPACE",
"OPERATOR_COMMAND_ALLOWLIST_REGEX",
"OPERATOR_COMMAND_DENYLIST_REGEX",
"OPERATOR_GPU_TOLERATIONS_JSON",
"OPERATOR_RUNNER_IMAGE",
"REGISTRY_HOST",
"REGISTRY_NAMESPACE",
"WORKER_API_BASE_URL",
"WORKER_IMAGE",
"WORKSPACE_MOUNT_PATH",
"WORKSPACE_PVC_NAME",
}
DEFAULT_IMAGE_ENV = {
"API_IMAGE": "registry.example.com/dataflow/droboticflow-api:test",
"WORKER_IMAGE": "registry.example.com/dataflow/droboticflow-worker:test",
"OPERATOR_RUNNER_IMAGE": "registry.example.com/dataflow/droboticflow-operator-runner:test",
}
def run_render_manifests(extra_env: dict[str, str]) -> subprocess.CompletedProcess:
def run_render_manifests(extra_env: dict[str, str], *, include_default_images: bool = True) -> subprocess.CompletedProcess:
env = os.environ.copy()
for key in CONFIG_ENV_KEYS:
env.pop(key, None)
env["DROBOTICFLOW_DEPLOY_ENV_LOADED"] = "test"
if include_default_images:
env.update(DEFAULT_IMAGE_ENV)
env.update(extra_env)
return subprocess.run(["bash", str(SCRIPT)], cwd=ROOT, env=env, text=True, capture_output=True, check=False)
@ -78,6 +89,31 @@ def test_render_manifests_accepts_valid_startup_configuration():
assert 'DROBOTICFLOW_REQUIRE_SESSION_AUTH: "yes"' in result.stdout
def test_render_manifests_derives_images_from_explicit_registry_configuration():
result = run_render_manifests(
{
"REGISTRY_HOST": "registry.example.com",
"REGISTRY_NAMESPACE": "dataflow-poc",
"IMAGE_TAG": "release-20260617",
},
include_default_images=False,
)
assert result.returncode == 0, result.stderr + result.stdout
assert "image: registry.example.com/dataflow-poc/droboticflow-api:release-20260617" in result.stdout
assert "image: registry.example.com/dataflow-poc/droboticflow-worker:release-20260617" in result.stdout
assert 'OPERATOR_RUNNER_IMAGE: "registry.example.com/dataflow-poc/droboticflow-operator-runner:release-20260617"' in result.stdout
def test_render_manifests_requires_explicit_images_or_registry_configuration():
result = run_render_manifests({}, include_default_images=False)
assert_render_validation_failure(
result,
"API_IMAGE is required unless REGISTRY_HOST and REGISTRY_NAMESPACE are both set",
)
def test_render_manifests_allows_empty_optional_global_runtime_config():
result = run_render_manifests({})