7.6 KiB
DRoboticFlow
DRoboticFlow is a workflow orchestration platform for embodied AI and vision pipelines. This repository now contains:
- a React/Vite frontend
- a Go-based MongoDB-backed REST API
- a Go-based worker service that executes workflow DAGs
- a Python-based Kubernetes Job operator runner
- deployment assets for MongoDB, Kubernetes, and Baidu Cloud Container Registry
Architecture
cmd/api: Go API entrypoint and SPA hostcmd/worker: Go worker entrypoint and worker control APIinternal/api: REST handlers and request validationinternal/worker: run scheduler, DAG executor, audit pause/resume, worker control APIinternal/store: Mongo-backed repositories plus in-memory test storeinternal/k8s: Kubernetes Job launcher for operator nodesbackend/operator-runner: Python entrypoint used by Kubernetes Jobssrc: existing frontend, now wired to real APIsk8s: Kubernetes manifest templatescripts: build, render, deploy, and smoke test scriptsdocs: API, deployment, operations, and design documents
Local Development
Prerequisites
- Node.js 22+
- Go 1.24+
- Python 3.9+
- MongoDB 7+ reachable at
MONGO_URI
Setup
-
Install dependencies:
npm install -
Copy the example environment file and adjust values:
cp .env.example .env.local -
Start the Go API and frontend:
npm run dev -
Start the full stack including Go worker:
npm run dev:full
Useful Scripts
npm run lint: type-check frontend and verify Go formattingnpm test: Go unit and integration testsnpm run build: build frontend assets and Go binariesnpm run verify: lint + test + buildnpm run smoke: local smoke test against the running API and workernpm run reconcile:data -- <api-base-url>: remove legacy demo data, reset settings, and bootstrap built-in operators through the real API
Runtime Configuration
Key environment variables:
PORT: API port, default3001WORKER_PORT: worker port, default3002MONGO_URI: MongoDB connection stringMONGO_DB: Mongo database nameMONGO_CONNECT_RETRIES: startup retry count for API and worker Mongo connectionsMONGO_CONNECT_RETRY_DELAY_MS: delay between Mongo connection retriesWORKER_POLL_INTERVAL_MS: worker polling intervalK8S_ENABLED: enable Kubernetes Job execution in the workerK8S_NAMESPACE: namespace used for JobsK8S_IMAGE_PULL_SECRET_NAME: image pull secret injected into operator-runner JobsOPERATOR_RUNNER_IMAGE: container image used for node execution JobsAI_API_BASE_URL: OpenAI-compatible DashScope endpoint used by server-side AI generation, defaulthttps://coding.dashscope.aliyuncs.com/v1AI_MODEL: server-side model name, defaultglm-5AI_API_KEY: server-side DashScope-compatible API key used for workflow/operator generationVITE_API_BASE_URL: optional frontend API base URL override
The frontend never reads the AI key directly. Workflow generation and operator generation are proxied through the Go API so secrets stay on the server side.
The settings page can update K8s access details, the worker defaults, the global timezone, provider, baseURL, model, and the server-side AI key. GET /api/settings only returns whether secrets exist via k8s.hasKubeconfig / ai.hasAPIKey; it does not echo plaintext kubeconfig or API keys back to the browser. Timestamps in the UI are rendered in the configured IANA timezone with compact minute-level formatting for cards, lists, and summaries, while logs keep finer granularity when needed. OSS-related bucket and path settings belong on OSS operators in workflows, not in the global system settings page. The OSS browser APIs also now require the current node SOURCE_PATH and only list content inside that node-scoped root.
Workflow execution concurrency now has two levels:
- system default:
settings.worker.maxConcurrentRuns - workflow override:
workflow.maxConcurrentRuns
The worker enforces both. A workflow value of 0 means "inherit the global default". Dispatch only starts queued runs while the global active run count is below the system limit and the active count for that workflow is below its effective limit.
Workflow-level execution safeguards are configured alongside the other top-bar workflow settings in the editor:
workflow.nodeTimeoutSeconds0: disable workflow-level per-node timeout1+: fail an operator execution once a single node exceeds the configured seconds
workflow.runTimeoutSeconds0: disable workflow-level timeout1+: fail the current run once total execution time exceeds the configured seconds
workflow.maxRetryAttempts0: disable automatic retry1+: automatically enqueue a new retry run after a failure, up to the configured count
workflow.retryBackoffSeconds0: retry immediately1+: keep the retry run queued until its backoff window expires
Automatic retries create real follow-up runs with trigger=retry, attempt=2+, parentRunId pointing at the failed attempt, and availableAt marking the first dispatchable time after backoff. Monitoring and history stay auditable because the original run is never mutated into the retry attempt.
Workflow templates now carry the same workflow-level defaults as project workflows. When you save a workflow as a template, duplicate a template, edit a template, or instantiate a template into a project, the workflow type, schedule, and execution-safeguard fields move across with it instead of being dropped or reset.
Artifact previews are persisted with the run artifacts themselves. JSON artifacts keep their structured payload in artifact.data, while non-JSON artifacts can attach a compact artifact.data.preview block for text, CSV/TSV, JSONL, images, audio/video metadata, PDF/document hints, SQLite/rosbag summaries, and LeRobot dataset summaries. Artifacts also carry portId / portName, which lets the worker route multi-output operators by edge handle instead of mixing all outputs together.
The built-in operator catalog now includes real data-processing and robotics helpers on top of the original source/merge/upload/audit flow: 目录扫描, CSV 转 JSON, JSON 转 CSV, JSON 字段提取, JSON 过滤, JSON 映射重命名, 文本模板渲染, 条件分流, LeRobot 数据集概览, ROSBag 数据概览, and Shell 命令执行.
Deployment
The repository includes three container images:
droboticflow-apidroboticflow-workerdroboticflow-operator-runner
The API image builds the Vite frontend and a Go binary into one container. The worker image contains a Go binary. The operator-runner image is Python-only and is launched by the worker as a Kubernetes Job.
Build and push images:
./scripts/build-and-push-images.sh
If the target cluster cannot pull the upstream mongo:7 image from Docker Hub, mirror it into the same Baidu registry and set MONGO_IMAGE during deployment.
Render Kubernetes manifests:
./scripts/render-manifests.sh
Deploy to Kubernetes:
./scripts/deploy-k8s.sh
Reconcile persistent API data after deployment:
./scripts/reconcile-system-data.sh http://<node-ip>:30080
Run a smoke test locally:
npm run smoke
Run a smoke test against a deployed cluster:
./scripts/smoke-test.sh http://<node-ip>:30080
See the detailed guides: