18 KiB
Raw Blame History

API Reference

REST API

Base path: /api

Health and Dashboard

  • GET /api/health: service health check
  • GET /api/ai/status: returns whether server-side AI generation is enabled and which model is configured
  • POST /api/ai/operators/generate: generate a custom operator definition from natural language
  • POST /api/ai/workflows/generate: generate a workflow graph from natural language and the current operator catalog
  • GET /api/dashboard: aggregate counts for projects, operators, jobs, and audits

Auth and Organizations

  • GET /api/auth/bootstrap-status: whether first super-admin bootstrap is required
  • POST /api/auth/bootstrap-super-admin: create the first super admin
  • POST /api/auth/login: log in with email and password and receive an HttpOnly session cookie
  • POST /api/auth/logout: revoke the current session cookie
  • GET /api/auth/me: return the current user and organization
  • GET /api/admin/organizations
  • POST /api/admin/organizations
  • PUT /api/admin/organizations/:id
  • DELETE /api/admin/organizations/:id
  • GET /api/admin/organizations/:id/users
  • POST /api/admin/organizations/:id/users
  • PUT /api/admin/users/:id
  • DELETE /api/admin/users/:id
  • POST /api/admin/users/:id/reset-password

Session auth uses an HttpOnly droboticflow_session cookie. Existing token-based deployments remain compatible by default; set DROBOTICFLOW_REQUIRE_SESSION_AUTH=true to require login for business APIs.

POST /api/ai/operators/generate

{
  "description": "生成一个算子,用于接收视频输入,抽取关键帧并输出图片列表。"
}

典型响应:

{
  "name": "关键帧抽取",
  "category": "AI",
  "description": "接收视频输入,按固定时间间隔抽取关键帧并输出图片列表与摘要。",
  "inputs": [
    { "name": "input", "type": "video", "transmission": "bus", "description": "视频输入或视频 manifest" }
  ],
  "outputs": [
    { "name": "frames", "type": "image", "transmission": "bus", "description": "关键帧图片列表" },
    { "name": "summary", "type": "json", "transmission": "bus", "description": "抽帧摘要" }
  ],
  "hyperparams": [
    { "key": "FRAME_INTERVAL_SECONDS", "value": "5", "description": "按秒抽帧间隔", "type": "string" },
    { "key": "MAX_FRAMES_PER_VIDEO", "value": "50", "description": "单视频最多保留关键帧数", "type": "string" }
  ],
  "skillDefinition": "接收视频文件路径或包含视频路径的 JSON manifest。若路径为 bos://...,先物化到本地工作目录,再按 FRAME_INTERVAL_SECONDS 抽取关键帧,输出图片列表与摘要。"
}

如果你想把这份响应直接转成自定义算子 Markdown,可以把响应保存为本地 JSON 后运行:

python3 scripts/generate_operator_docs.py --input-json /path/to/generated-operator.json

POST /api/ai/workflows/generate

{
  "description": "从对象存储读取视频,抽帧后调用目标检测算子,再把检测结果回传到对象存储。"
}

Projects

  • GET /api/projects
  • POST /api/projects
  • GET /api/projects/:id
  • PUT /api/projects/:id
  • DELETE /api/projects/:id
  • GET /api/projects/:id/workflows
  • POST /api/projects/:id/workflows

POST /api/projects

{
  "name": "自动驾驶新项目",
  "description": "处理多摄像头感知数据"
}

Workflows

  • GET /api/workflows/:id
  • PUT /api/workflows/:id
  • DELETE /api/workflows/:id
  • POST /api/workflows/:id/run

PUT /api/workflows/:id

{
  "name": "视觉处理流程",
  "description": "包含扫描、推理、上传",
  "type": "batch",
  "maxConcurrentRuns": 2,
  "nodeTimeoutSeconds": 120,
  "runTimeoutSeconds": 300,
  "maxRetryAttempts": 1,
  "retryBackoffSeconds": 30,
  "nodes": [],
  "edges": []
}

POST /api/workflows/:id/run

{
  "trigger": "manual"
}

Scan workflows are created through the same workflow APIs. Their payload can include kind: "scan" and a workflow-level scanConfig object. For BOS scans, scanConfig now supports either inline credentials or a reusable connectionId:

{
  "name": "X-platform BOS 扫描",
  "type": "streaming",
  "kind": "scan",
  "schedule": {
    "mode": "every_days_at_time",
    "dayInterval": 2,
    "timeOfDay": "18:30",
    "timezone": "Asia/Shanghai"
  },
  "scanConfig": {
    "dataSource": "BOS",
    "connectionId": "storage-abc123",
    "bucket": "datastorage",
    "prefix": "algo/X-platform-test",
    "scanMode": "INCREMENTAL",
    "enabled": true
  },
  "nodes": [],
  "edges": []
}

When connectionId is present, the workflow stores the reference instead of duplicating AK/SK. The worker and BOS preview API resolve the referenced storage connection at execution time.

Workflow-level schedule now supports three modes:

  • interval_seconds
    • uses interval
  • every_hours
    • uses intervalHours
  • every_days_at_time
    • uses dayInterval, timeOfDay, and optional timezone

Templates

  • GET /api/templates
  • POST /api/templates
  • GET /api/templates/:id
  • PUT /api/templates/:id
  • DELETE /api/templates/:id

Templates now persist the same workflow-level defaults used by project workflows, including type, schedule, maxConcurrentRuns, nodeTimeoutSeconds, runTimeoutSeconds, maxRetryAttempts, and retryBackoffSeconds.

Operators

  • GET /api/operators
  • POST /api/operators
  • GET /api/operators/:id
  • PUT /api/operators/:id
  • DELETE /api/operators/:id

The built-in catalog includes the original workflow primitives plus:

  • BOS 增量扫描
  • X 平台数据集索引
  • 采集目录校验
  • 视频质检
  • 数据过滤
  • 视频切片
  • 视频标注
  • 成品打包
  • 抽检采样
  • 目录扫描
  • CSV 转 JSON
  • JSON 转 CSV
  • JSON 字段提取
  • JSON 过滤
  • JSON 映射重命名
  • 文本模板渲染
  • 条件分流
  • LeRobot 数据集概览
  • ROSBag 数据概览
  • Shell 命令执行

POST /api/operators

{
  "name": "自定义分割算子",
  "type": "custom",
  "category": "AI",
  "description": "执行图像分割",
  "inputs": [{ "id": "in-1", "name": "图像", "type": "image" }],
  "outputs": [{ "id": "out-1", "name": "结果", "type": "json" }],
  "image": "ccr.example.com/dataflow/custom-seg:latest"
}

Runs and Monitoring

  • GET /api/runs
  • GET /api/runs/:id
  • GET /api/workflows/:id/runs
  • POST /api/runs/:id/rerun-subtree
  • GET /api/runs/:id/logs
  • GET /api/runs/:id/resources
  • GET /api/runs/:id/artifacts

GET /api/runs/:id/artifacts

Each artifact item can now include:

  • portId / portName: the originating output port so downstream edges can route multi-output operators correctly
  • metadata.mimeType / metadata.sha256: persisted file metadata
  • data: the real JSON payload for JSON artifacts
  • data.preview: a compact preview object for non-JSON artifacts such as text, CSV/TSV, JSONL, image thumbnails, audio/video metadata, PDF hints, SQLite/rosbag summaries, and LeRobot dataset summaries

POST /api/runs/:id/rerun-subtree

  • body: { "nodeId": "<workflow-node-id>" }
  • only allowed on failed runs
  • creates a queued child run with:
    • trigger = rerun_subtree
    • parentRunId = source failed run id
    • restartFromNodeId = selected node id
  • the child run reuses completed upstream baseline executions and re-executes the selected node plus its downstream subtree

Data Platform

  • GET /api/scan-project
  • GET /api/scan-configs
  • POST /api/scan-configs
  • GET /api/scan-configs/:id
  • PUT /api/scan-configs/:id
  • DELETE /api/scan-configs/:id
  • POST /api/scan-configs/:id/run
  • GET /api/source-data/collections
  • GET /api/source-data/collections/:runId/:collectionId
  • GET /api/source-data/collections/:runId/:collectionId/entries?prefix=<dir>
  • GET /api/packages
  • GET /api/packages/:runId/:packageName
  • GET /api/sampling-tasks
  • GET /api/sampling-tasks/:id
  • POST /api/sampling-items/:id/review

POST /api/scan-configs

{
  "name": "X-platform-test 自采数据扫描",
  "description": "扫描并索引真实采集目录",
  "sourcePath": "bos://datastorage/algo/X-platform-test",
  "bucket": "datastorage",
  "prefix": "algo/X-platform-test",
  "pattern": "",
  "interval": 86400,
  "enabled": true,
  "dataSource": "BOS"
}

Scan configs are implemented as real workflows under the system project 数据源扫描, not as a second execution engine. For BOS-backed pipelines the first node now uses the built-in BOS 增量扫描 source operator, which persists checkpoint state per workflowId + nodeId + sourceFingerprint. The worker only commits that checkpoint after the whole run succeeds, so downstream failures do not silently skip data on the next retry. GET /api/scan-project resolves the system project for UI redirects, POST /api/scan-configs/:id/run enqueues a normal run, and GET /api/source-data/collections returns data from any completed workflow that produced a real dataset-index.json artifact.

For BOS-backed X-platform scans, source collections are inferred from real meta.json roots inside the scanned prefix. collectionId is a URL-safe stable slug for routing, collectionName is the human-readable leaf folder name such as BJ_001_0001_OsmoNano_2026-03-19_14-51-43, and relativePath preserves the full BOS-relative directory such as raw/BJ_001_0001_OsmoNano_2026-03-19_14-51-43. The frontend groups these source collections together with completed package outputs inside a single 数据资产 module, and drives project/workflow scoping via query filters instead of separate first-level menus.

GET /api/packages

Returns finished data packages aggregated from real packages-manifest.json run artifacts. Each item includes the source projectId, workflowId, runId, package metadata, embedded quality / annotation summaries, and a previewArtifact when a previewable file such as the clip video exists.

GET /api/sampling-tasks

Sampling tasks are synced from real audit-samples.json run artifacts. The API persists review state separately from the originating run so reviewers can submit decisions later without mutating the original workflow outputs.

Audit

  • GET /api/audit-tasks
  • POST /api/audit-tasks/:id/approve
  • POST /api/audit-tasks/:id/reject

Users and Settings

  • GET /api/users
  • PUT /api/users/:id
  • GET /api/storage/connections
  • POST /api/storage/connections
  • GET /api/storage/connections/:id
  • PUT /api/storage/connections/:id
  • DELETE /api/storage/connections/:id
  • GET /api/external/business-db/tokens
  • POST /api/external/business-db/tokens
  • PUT /api/external/business-db/tokens/:id
  • POST /api/external/business-db/tokens/:id/rotate
  • DELETE /api/external/business-db/tokens/:id
  • GET /api/settings
  • PUT /api/settings

POST /api/storage/connections

{
  "name": "BOS 主连接",
  "description": "X-platform 扫描使用",
  "provider": "BOS",
  "endpoint": "https://bj.bcebos.com",
  "accessKeyId": "ak-...",
  "secretAccessKey": "sk-..."
}

GET /api/storage/connections

The response never returns plaintext AK/SK. It only exposes hasAccessKeyId, hasSecretAccessKey, and masked previews so the UI can confirm a real secret is already stored.

GET /api/settings

{
  "id": "system",
  "k8s": {
    "enabled": true,
    "endpoint": "https://120.48.161.230:6443",
    "namespace": "droboticflow",
    "hasKubeconfig": true
  },
  "worker": {
    "pollIntervalMs": 2000,
    "maxConcurrentRuns": 2
  },
  "ai": {
    "provider": "dashscope-openai-compatible",
    "baseURL": "https://coding.dashscope.aliyuncs.com/v1",
    "model": "glm-5",
    "hasAPIKey": true
  },
  "timezone": "Asia/Shanghai"
}

PUT /api/settings

{
  "timezone": "Asia/Shanghai",
  "worker": {
    "maxConcurrentRuns": 2
  },
  "ai": {
    "provider": "dashscope-openai-compatible",
    "baseURL": "https://coding.dashscope.aliyuncs.com/v1",
    "model": "glm-5",
    "apiKey": "sk-..."
  }
}

External Business DB Write API

For a copy-paste-ready partner handoff guide, see docs/external-business-db-write-api-integration.md.

Use this surface when a third party needs to write into a preconfigured business Mongo connection without seeing the raw Mongo URI. The external caller must use a server-generated API key in header X-API-Key.

Internal token-management routes:

  • GET /api/external/business-db/tokens
  • POST /api/external/business-db/tokens
  • PUT /api/external/business-db/tokens/:id
  • POST /api/external/business-db/tokens/:id/rotate
  • DELETE /api/external/business-db/tokens/:id

POST /api/external/business-db/tokens

{
  "name": "partner-a",
  "description": "订单同步写入",
  "enabled": true,
  "scopes": [
    {
      "connectionId": "mongo_abc123",
      "collections": ["orders", "customers"]
    }
  ]
}

Successful create and rotate responses return the plaintext apiKey once:

{
  "token": {
    "id": "exttoken_abc123",
    "name": "partner-a",
    "enabled": true,
    "tokenPreview": "dbw_****7f3a",
    "scopes": [
      {
        "connectionId": "mongo_abc123",
        "collections": ["customers", "orders"]
      }
    ]
  },
  "apiKey": "dbw_9f..."
}

Third-party write routes:

  • POST /api/external/business-db/documents
  • PATCH /api/external/business-db/documents/:documentId
  • POST /api/external/business-db/upserts

All of these routes require header:

X-API-Key: dbw_...

POST /api/external/business-db/documents

{
  "connectionId": "mongo_abc123",
  "collection": "orders",
  "document": {
    "externalId": "EXT-1001",
    "status": "new",
    "amount": 20
  }
}

PATCH /api/external/business-db/documents/:documentId

{
  "connectionId": "mongo_abc123",
  "collection": "orders",
  "patch": {
    "status": "done",
    "amount": 25
  }
}

POST /api/external/business-db/upserts

{
  "connectionId": "mongo_abc123",
  "collection": "orders",
  "match": {
    "externalId": "EXT-1001"
  },
  "patch": {
    "status": "done",
    "amount": 25
  },
  "setOnInsert": {
    "source": "partner-a"
  }
}

External write responses use:

  • operation: "created" for inserts and create-path upserts
  • operation: "updated" for patch and update-path upserts

External write validation intentionally does not expose raw Mongo update operators. document, patch, match, and setOnInsert must all be plain JSON objects.

Workflow create/update payloads also accept workflow-level execution settings:

  • 0: inherit the global worker default
  • 1+: workflow-specific maximum active runs

Dispatch enforces both the global worker limit and the workflow-specific limit.

  • runTimeoutSeconds
    • 0: disable workflow-level timeout
    • 1+: fail the current run if total execution time exceeds this many seconds
  • nodeTimeoutSeconds
    • 0: disable workflow-level per-node timeout
    • 1+: fail the current node if a single operator execution exceeds this many seconds
  • maxRetryAttempts
    • 0: disable automatic retry
    • 1+: automatically enqueue up to N retry runs after a failed attempt
  • retryBackoffSeconds
    • 0: enqueue retry runs immediately
    • 1+: delay retry dispatch by the configured number of seconds

Run responses may also include:

  • attempt: current attempt number, starting from 1
  • parentRunId: the failed run that spawned this retry run
  • availableAt: when this queued run becomes eligible for dispatch; retry backoff uses this field

Storage

  • GET /api/storage/buckets?sourcePath=<node SOURCE_PATH>
  • GET /api/storage/objects?sourcePath=<node SOURCE_PATH>&bucket=<bucket>&prefix=<prefix>

Worker API

Base path: /worker-api

  • GET /worker-api/health: worker health check
  • GET /worker-api/stats: current queue/running/audit counts
  • GET /worker-api/metrics: Prometheus text metrics for queue depth, dispatch counts, node execution counts, cumulative durations, and duration histograms
  • POST /worker-api/process-once: process one queued run immediately
  • POST /worker-api/runs/:id/retry: reset a run to queued and clear prior execution state

Response Shape Notes

  • list endpoints return { "items": [...] }
  • delete endpoints return 204 No Content
  • validation failures return 400
  • missing or invalid external API key returns 401
  • authenticated external token without the requested connection or collection permission returns 403
  • missing resources return 404
  • AI service unavailable returns 503
  • upstream AI gateway failures return 502
  • unhandled server errors return 500

Implementation Notes

  • /api/* is served by the Go API binary in cmd/api.
  • /worker-api/* is served by the Go worker binary in cmd/worker.
  • Workflow execution and storage models are defined in internal/domain and internal/store.
  • AI generation is implemented by internal/ai and is called only from the Go API, never directly from the browser.
  • GET /api/settings never returns the saved AI key in plaintext. It only exposes ai.hasAPIKey so the browser can show whether a server-side key is already configured.
  • timezone uses IANA timezone identifiers such as Asia/Shanghai or America/Los_Angeles. The frontend formats timestamps with second precision in that timezone.
  • Monitoring previews are rendered from the persisted artifact payload itself, so text/table/robotics previews keep working after the Kubernetes Job has exited and its temporary filesystem is gone.
  • OSS 相关配置不通过 /api/settings 管理。系统设置页只维护 K8s、worker、AI 和默认资源;OSS bucket/prefix 等参数由具体工作流节点维护。
  • /api/storage/* 也不再读取系统级存储根目录。浏览请求必须显式传入当前 OSS 节点超参里的 SOURCE_PATH,API 只在这个节点上下文内列 bucket 和对象。
  • Kubernetes Jobs launched for operator nodes use the Python entrypoint at backend/operator-runner/main.py.