16 KiB
Deployment Guide
Images
This project deploys three images:
droboticflow-apidroboticflow-workerdroboticflow-operator-runner
Default Baidu registry naming:
ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-api:<tag>ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-worker:<tag>ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-operator-runner:<tag>
1. Log In to the Registry
Use the public endpoint unless you are deploying from the same VPC as the registry:
docker login ccr-29eug8s3-pub.cnc.bj.baidubce.com --username <registry-user>
2. Build and Push Images
IMAGE_TAG=20260408 ./scripts/build-and-push-images.sh
Image contents:
droboticflow-api: Vite frontend assets + Go API binarydroboticflow-worker: Go worker binarydroboticflow-operator-runner: Python runtime +backend/operator-runner/main.py
The script prints the exact image URLs. Reuse those values for deployment:
API_IMAGE=...
WORKER_IMAGE=...
OPERATOR_RUNNER_IMAGE=...
You can also load a named deployment preset:
DEPLOY_ENV=wjzh IMAGE_TAG=20260421 ./scripts/build-and-push-images.sh
On the dev branch, the scripts auto-resolve to DEPLOY_ENV=poc when you leave DEPLOY_ENV unset.
3. Prepare Kubernetes Access
Point kubectl at the target cluster before deploying. The deployment scripts do not store kubeconfig in the repository.
Recommended environment variables:
export K8S_NAMESPACE=droboticflow
export API_IMAGE=ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-api:20260408
export WORKER_IMAGE=ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-worker:20260408
export OPERATOR_RUNNER_IMAGE=ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/droboticflow-operator-runner:20260408
export OPERATOR_GPU_NODE_SELECTOR=accelerator=nvidia
export OPERATOR_GPU_TOLERATIONS_JSON='[{"key":"nvidia.com/gpu","operator":"Exists","effect":"NoSchedule"}]'
export OPERATOR_GPU_RUNTIME_CLASS=nvidia
export MONGO_URI=mongodb://<user>:<password>@<external-host>:27017/
export REGISTRY_SERVER=ccr-29eug8s3-pub.cnc.bj.baidubce.com
export REGISTRY_USERNAME=<registry-user>
export REGISTRY_PASSWORD=<registry-password>
export K8S_IMAGE_PULL_SECRET_NAME=droboticflow-registry
export DROBOTICFLOW_API_TOKEN=<optional-shared-api-token>
export AI_API_BASE_URL=https://coding.dashscope.aliyuncs.com/v1
export AI_MODEL=glm-5
export AI_API_KEY=<dashscope-compatible-server-key>
For the wjzh environment, the repo now carries non-secret defaults in scripts/deploy-envs/wjzh.env. Sensitive values stay out of git and can be stored in either:
$HOME/.config/droboticflow/deploy/wjzh.secrets.env.env.wjzh.local
The loader also looks for $HOME/.config/droboticflow/kubeconfigs/wjzh.config and exports KUBECONFIG automatically when DEPLOY_ENV=wjzh is set.
For the poc environment, use scripts/deploy-envs/poc.env. It is separated from wjzh and defaults to:
MONGO_DB=pocREGISTRY_NAMESPACE=dataflow-pocAPI_PUBLIC_LOAD_BALANCER=true- the same namespace, registry host, AI defaults, and PVC settings as
wjzh
Store poc secrets outside git in:
$HOME/.config/droboticflow/deploy/poc.secrets.env.env.poc.local
At minimum, the poc secrets file should provide REGISTRY_USERNAME, REGISTRY_PASSWORD, MONGO_URI, DROBOTICFLOW_API_TOKEN, and for first-time workspace setup WORKSPACE_BOS_BUCKET, BOS_ACCESS_KEY_ID, and BOS_SECRET_ACCESS_KEY.
The loader also checks $HOME/.config/droboticflow/kubeconfigs/poc.config automatically. On the dev branch and poc branches, trying to deploy with another DEPLOY_ENV now fails fast to avoid cross-environment mistakes.
For first-time POC cluster setup, configure the shared workspace before deploying the app:
DEPLOY_ENV=poc ./scripts/configure-poc-environment.sh
This installs the BOS CSI node plugin and creates the droboticflow-workspace PV/PVC. WORKSPACE_BOS_BUCKET has no repository default; it must come from the local poc secrets file or the current shell. Do not commit BOS keys, bucket names used for private data, or Mongo credentials.
The POC environment also creates a public droboticflow-api-public LoadBalancer service. The standard droboticflow-api service defaults to NodePort for compatibility, but can be rendered as ClusterIP with API_SERVICE_TYPE=ClusterIP when exposure is handled by the LoadBalancer or an ingress. The public access URL should use the LoadBalancer external IP. When API_PUBLIC_LOAD_BALANCER=true, scripts/deploy-k8s.sh requires DROBOTICFLOW_API_TOKEN, requires DROBOTICFLOW_REQUIRE_SESSION_AUTH=true, and defaults DROBOTICFLOW_SESSION_COOKIE_SECURE=true; if the token is omitted locally, the script can reuse an existing droboticflow-auth-secret from the target namespace.
4. Deploy
./scripts/deploy-k8s.sh
Or, with the reusable preset:
DEPLOY_ENV=wjzh ./scripts/deploy-k8s.sh
Or for the POC environment:
DEPLOY_ENV=poc ./scripts/deploy-k8s.sh
The deploy script performs these actions:
- creates the namespace if it does not exist
- creates or updates the image pull secret
- loads
MONGO_URIfrom the current cluster secret when omitted, then validates that it points to an external MongoDB instance - creates or updates the
droboticflow-mongo-secretSecret withMONGO_URI - creates or updates the
droboticflow-ai-secretSecret for server-side AI generation - renders
k8s/droboticflow.template.yaml - validates the rendered manifest with
kubectl apply --dry-run=client - applies the manifest to the cluster
- waits for API and worker rollouts
The main deployment path no longer creates a platform MongoDB Pod or Service. You must provision the backing MongoDB outside this manifest and provide its URI explicitly.
5. Services Created
droboticflow-api: NodePort service that exposes the frontend and APIdroboticflow-worker: ClusterIP service for worker health/debugging
Default API NodePort:
30080
If the cluster node IP is reachable, the UI and API are exposed at:
http://<node-ip>:30080
6. Manifest Variables
scripts/render-manifests.sh accepts:
K8S_NAMESPACEMONGO_DBMONGO_CONNECT_RETRIESMONGO_CONNECT_RETRY_DELAY_MSWORKER_POLL_INTERVAL_MSWORKER_API_BASE_URLAPI_NODE_PORTAPI_REPLICASWORKER_REPLICASK8S_IMAGE_PULL_SECRET_NAMEDROBOTICFLOW_API_TOKENDROBOTICFLOW_REQUIRE_SESSION_AUTHDROBOTICFLOW_SESSION_COOKIE_SECUREAI_API_BASE_URLAI_MODELIMAGE_TAGAPI_IMAGEWORKER_IMAGEOPERATOR_RUNNER_IMAGEOPERATOR_GPU_NODE_SELECTOROPERATOR_GPU_TOLERATIONS_JSONOPERATOR_GPU_RUNTIME_CLASSOPERATOR_JOB_TTL_SECONDSOPERATOR_ALLOW_SHELL_COMMANDSOPERATOR_COMMAND_ALLOWLIST_REGEXOPERATOR_COMMAND_DENYLIST_REGEXDROBOTICFLOW_TRUSTED_PROXY_CIDRSAPI_SERVICE_TYPE
7. Notes
- The main manifest only deploys the application components. It does not create MongoDB resources in-cluster.
MONGO_URImust reference an external MongoDB instance; localhost,mongodb:27017, and in-cluster.svcaddresses are rejected byscripts/deploy-k8s.sh.- Startup numeric and boolean configuration is validated before manifest output.
MONGO_CONNECT_RETRIES,MONGO_CONNECT_RETRY_DELAY_MS, andWORKER_POLL_INTERVAL_MSmust be positive integers;API_REPLICAS,WORKER_REPLICAS, andOPERATOR_JOB_TTL_SECONDSmust be non-negative integers;API_NODE_PORTmust be in30000..32767whenAPI_SERVICE_TYPE=NodePort; boolean env values must be one oftrue/false,1/0,yes/no, oron/off. - API and worker entrypoints are Go binaries with Mongo startup retry, so they tolerate the database becoming ready after the Deployments start.
- The API proxies
/worker-api/*toWORKER_API_BASE_URL, defaulting to the in-clusterhttp://droboticflow-worker:3002Service. Override it only when the worker Service name, namespace, or exposure model differs. - The worker runs with Kubernetes RBAC that allows creating and watching Jobs in the namespace.
- The operator-runner Job template inherits the private registry pull secret name from
K8S_IMAGE_PULL_SECRET_NAME. - Operator-runner Jobs apply CPU and memory requests/limits from system defaults and node overrides. A positive GPU value is emitted as
nvidia.com/gpuin both requests and limits; the target cluster must have GPU nodes and the NVIDIA device plugin/runtime configured. - GPU-only scheduling settings are optional:
OPERATOR_GPU_NODE_SELECTORaccepts comma-separatedkey=valuepairs,OPERATOR_GPU_TOLERATIONS_JSONaccepts a Kubernetes tolerations JSON array, andOPERATOR_GPU_RUNTIME_CLASSsetsruntimeClassName. These settings are only injected into Jobs that request a positive GPU count. Invalid selector or tolerations values fail worker startup. - Finished operator-runner Jobs receive
ttlSecondsAfterFinishedfromOPERATOR_JOB_TTL_SECONDS, default86400. Usenpm run k8s:cleanup-jobsto delete old finished Jobs that predate TTL, or setDRY_RUN=truefirst to list candidates. - Explicit
COMMAND/SCRIPTexecution is disabled by default. SetOPERATOR_ALLOW_SHELL_COMMANDS=trueonly for deployments that need shell operators, and constrain it withOPERATOR_COMMAND_ALLOWLIST_REGEX/OPERATOR_COMMAND_DENYLIST_REGEXwhere possible. These values are injected by the worker and are not overridable from workflow hyperparams. - The operator-runner is not a standalone Deployment. It is launched on demand as a Python Job container.
- Failed operator-runner Jobs report Job conditions, Pod phase, container waiting or termination reasons, Pod events, and available logs back through the run error path.
- Worker runtime metrics are available from
GET /worker-api/metricsin Prometheus text format. Through the API, stats/metrics and worker control endpoints require an admin/operator session or an API token when session auth is not enforced. The worker Service includes Prometheus scrape annotations for internal collection, and the endpoint includes duration histograms for dispatch loop processing and node execution. - AI generation runs through the API Deployment.
AI_API_KEYis mounted through thedroboticflow-ai-secretSecret and must not be baked into frontend env vars. DROBOTICFLOW_API_TOKENis mounted throughdroboticflow-auth-secret. When enabled, non-browser clients call APIs withAuthorization: Bearer <token>orX-Droboticflow-API-Token. Do not bake this shared token into frontend assets.DROBOTICFLOW_REQUIRE_SESSION_AUTH=trueenables HttpOnly cookie-session login for browser business APIs. Public LoadBalancer deploys require it so role checks are not bypassed by a shared machine token. Bootstrap remains available through/api/auth/bootstrap-super-adminbefore the first super admin exists.DROBOTICFLOW_SESSION_COOKIE_SECURE=truesets the session cookieSecureattribute.scripts/deploy-k8s.shdefaults it totruewhenAPI_PUBLIC_LOAD_BALANCER=true; local development can leave itfalse.DROBOTICFLOW_TRUSTED_PROXY_CIDRScontrols which reverse proxies may supplyX-Forwarded-For/X-Real-IPfor rate limiting. Leave it empty unless ingress or load balancer source ranges are known.- Session-authenticated unsafe requests require
X-Droboticflow-CSRF-Tokento match the readabledroboticflow_csrfcookie. The bundled frontend sets this header automatically. IMAGE_TAGdefaults to the current git short SHA in build, deploy, and render scripts. Set it explicitly for release names or rollback-friendly tags.scripts/smoke-test.shrequiresSMOKE_SOURCE_PATH, a real path visible to operator-runner Jobs. The真实数据源runner infers filesystem/PVC paths asLOCALandbos://bucket/prefixpaths asBOS; if no real source is configured, the source node fails instead of emitting an empty manifest. It can exercise session-authenticated deployments whenDROBOTICFLOW_SMOKE_EMAILandDROBOTICFLOW_SMOKE_PASSWORDare set; it logs in, persists cookies, and sends the CSRF header on write requests.scripts/reconcile-system-data.shdelegates toscripts/reconcile_system_data.pyand uses the same protected-client behavior. SetDROBOTICFLOW_API_TOKENfor token-protected APIs when session auth is not enforced. For public session-auth deployments, setDROBOTICFLOW_SMOKE_EMAILandDROBOTICFLOW_SMOKE_PASSWORDto an organization admin account because the reconcile flow updates settings, users, templates, and operators. Builtin operators are read from the live/api/operatorscatalog and refreshed toOPERATOR_RUNNER_IMAGE; the script does not carry a separate builtin catalog.scripts/object-storage-acceptance.mjsruns browser create, preview, rename, upload, delete, and optional video playback checks against a live storage connection. It never rewrites storage connection metadata; provideOBJECT_STORAGE_CONNECTION_ID,OBJECT_STORAGE_CONNECTION_NAME, orOBJECT_STORAGE_BUCKETexplicitly.OBJECT_STORAGE_ACCEPTANCE_PAYLOAD_FILEis required and must point to a non-empty real UTF-8 text or JSON payload used for create/upload/preview checks; setOBJECT_STORAGE_ACCEPTANCE_CONTENT_TYPEwhen auto-detection is not enough. SetOBJECT_STORAGE_VIDEO_KEYonly when an existing playable object should be validated. The script supportsDROBOTICFLOW_API_TOKENfor token-protected non-session APIs andDROBOTICFLOW_SMOKE_EMAIL/DROBOTICFLOW_SMOKE_PASSWORDfor session-authenticated browser access.scripts/playwright-preview-operators-check.mjsruns browser/operator preview acceptance with caller-supplied payloads and real source paths. ProvidePREVIEW_CHECK_CSV_PAYLOAD_FILE,PREVIEW_CHECK_TEXT_PAYLOAD_FILE,PREVIEW_CHECK_JSON_PAYLOAD_FILE,PREVIEW_CHECK_JSON_EXTRACT_FIELD,PREVIEW_CHECK_JSON_FILTER_FIELD,PREVIEW_CHECK_JSON_FIELD_MAP_JSON,PREVIEW_CHECK_SCAN_SOURCE_PATH,PREVIEW_CHECK_ROSBAG_SOURCE_PATH, andPREVIEW_CHECK_LEROBOT_SOURCE_PATH. The script no longer generates built-in CSV/text/JSON/ROSBag/LeRobot data.scripts/xplatform-live-acceptance.pyruns the X-platform chain only when the caller provides the real source path plus explicit acceptance criteria. ProvideXPLATFORM_RULES_JSON, allXPLATFORM_MIN_*counts, finite normalizedXPLATFORM_MIN_CLARITY,XPLATFORM_MAX_SHAKE, andXPLATFORM_MAX_HAND_OCCLUSION, finite positiveXPLATFORM_CLARITY_SCALEandXPLATFORM_FRAME_INTERVAL_SECONDS, and positive integerXPLATFORM_SEGMENT_DURATION_SECONDS; the script has no built-in sampling rule or threshold fallback.npm run gpu:smokecreates a temporary GPU Job in the target namespace and runsnvidia-smi. OverrideGPU_SMOKE_IMAGE,GPU_RESOURCE_NAME,GPU_RESOURCE_COUNT, and theOPERATOR_GPU_*scheduling variables when the cluster uses custom GPU images or labels. The script validates the Job name, resource name/count, node selector, tolerations JSON, and runtime class before anykubectlcall so malformed GPU acceptance inputs fail locally instead of mutating the cluster with a bad manifest.- GitHub Actions includes a manual
Acceptanceworkflow for deployed-environment checks. ConfigureSMOKE_SOURCE_PATHfor API smoke,KUBE_CONFIG_B64for Kubernetes access, and optionallyDROBOTICFLOW_API_TOKEN,DROBOTICFLOW_SMOKE_EMAIL,DROBOTICFLOW_SMOKE_PASSWORD,OBJECT_STORAGE_CONNECTION_ID,OBJECT_STORAGE_CONNECTION_NAME,OBJECT_STORAGE_BUCKET,OBJECT_STORAGE_ACCEPTANCE_PAYLOAD_B64,OBJECT_STORAGE_ACCEPTANCE_CONTENT_TYPE,OBJECT_STORAGE_VIDEO_KEY,PREVIEW_CHECK_CSV_PAYLOAD_B64,PREVIEW_CHECK_TEXT_PAYLOAD_B64,PREVIEW_CHECK_JSON_PAYLOAD_B64, the preview JSON field settings, the preview scan/ROSBag/LeRobot source paths, and the X-platform source path plus explicit rules/threshold/count inputs for authenticated acceptance tests. - If you use DashScope-compatible models, prefer a backend-allowed API key. Do not deploy tool-only Coding Plan keys into the cluster.
- Production hardening still depends on the target cluster policy: TLS ingress, network policies, backup strategy, and secret rotation.