2026-06-17 13:35:21 +08:00

21 KiB
Raw Blame History

API Reference

REST API

Base path: /api

Health and Dashboard

  • GET /api/health: service health check
  • GET /api/ai/status: returns whether server-side AI generation is enabled and which model is configured
  • POST /api/ai/operators/generate: generate a custom operator definition from natural language
  • POST /api/ai/workflows/generate: generate a workflow graph from natural language and the current operator catalog
  • GET /api/dashboard: aggregate counts for projects, operators, jobs, and audits

Auth and Organizations

  • GET /api/auth/bootstrap-status: whether first super-admin bootstrap is required
  • POST /api/auth/bootstrap-super-admin: create the first super admin
  • POST /api/auth/login: log in with email and password and receive an HttpOnly session cookie
  • POST /api/auth/logout: revoke the current session cookie
  • GET /api/auth/me: return the current user and organization
  • GET /api/admin/organizations
  • POST /api/admin/organizations
  • PUT /api/admin/organizations/:id
  • DELETE /api/admin/organizations/:id
  • GET /api/admin/organizations/:id/users
  • POST /api/admin/organizations/:id/users
  • PUT /api/admin/users/:id
  • DELETE /api/admin/users/:id
  • POST /api/admin/users/:id/reset-password

Session auth uses an HttpOnly droboticflow_session cookie. Existing token-based local deployments remain compatible by default; set DROBOTICFLOW_REQUIRE_SESSION_AUTH=true to require login for business APIs. Unsafe session-authenticated requests must also send X-Droboticflow-CSRF-Token with the value from the readable droboticflow_csrf cookie. Public LoadBalancer deployments require session auth so business role checks remain active.

POST /api/ai/operators/generate

{
  "description": "生成一个算子,用于接收视频输入,抽取关键帧并输出图片列表。"
}

典型响应:

{
  "name": "关键帧抽取",
  "category": "AI",
  "description": "接收视频输入,按固定时间间隔抽取关键帧并输出图片列表与摘要。",
  "inputs": [
    { "name": "input", "type": "video", "transmission": "bus", "description": "视频输入或视频 manifest" }
  ],
  "outputs": [
    { "name": "frames", "type": "image", "transmission": "bus", "description": "关键帧图片列表" },
    { "name": "summary", "type": "json", "transmission": "bus", "description": "抽帧摘要" }
  ],
  "hyperparams": [
    { "key": "FRAME_INTERVAL_SECONDS", "value": "5", "description": "按秒抽帧间隔", "type": "string" },
    { "key": "MAX_FRAMES_PER_VIDEO", "value": "50", "description": "单视频最多保留关键帧数", "type": "string" }
  ],
  "skillDefinition": "接收视频文件路径或包含视频路径的 JSON manifest。若路径为 bos://...,先物化到本地工作目录,再按 FRAME_INTERVAL_SECONDS 抽取关键帧,输出图片列表与摘要。"
}

如果你想把这份响应直接转成自定义算子 Markdown,可以把响应保存为本地 JSON 后运行:

python3 scripts/generate_operator_docs.py --input-json /path/to/generated-operator.json

POST /api/ai/workflows/generate

{
  "description": "从对象存储读取视频,抽帧后调用目标检测算子,再把检测结果回传到对象存储。"
}

Projects

  • GET /api/projects
  • POST /api/projects
  • GET /api/projects/:id
  • PUT /api/projects/:id
  • DELETE /api/projects/:id
  • GET /api/projects/:id/workflows
  • POST /api/projects/:id/workflows

POST /api/projects

{
  "name": "自动驾驶新项目",
  "description": "处理多摄像头感知数据"
}

Workflows

  • GET /api/workflows/:id
  • PUT /api/workflows/:id
  • DELETE /api/workflows/:id
  • POST /api/workflows/:id/run

PUT /api/workflows/:id

{
  "name": "视觉处理流程",
  "description": "包含扫描、推理、上传",
  "type": "batch",
  "maxConcurrentRuns": 2,
  "nodeTimeoutSeconds": 120,
  "runTimeoutSeconds": 300,
  "maxRetryAttempts": 1,
  "retryBackoffSeconds": 30,
  "nodes": [],
  "edges": []
}

POST /api/workflows/:id/run

{
  "trigger": "manual"
}

Scan workflows are created through the same workflow APIs. Their payload can include kind: "scan" and a workflow-level scanConfig object. For BOS scans, scanConfig now supports either inline credentials or a reusable connectionId:

{
  "name": "X-platform BOS 扫描",
  "type": "streaming",
  "kind": "scan",
  "schedule": {
    "mode": "every_days_at_time",
    "dayInterval": 2,
    "timeOfDay": "18:30",
    "timezone": "Asia/Shanghai"
  },
  "scanConfig": {
    "dataSource": "BOS",
    "connectionId": "storage-abc123",
    "bucket": "datastorage",
    "prefix": "algo/X-platform-test",
    "scanMode": "INCREMENTAL",
    "enabled": true
  },
  "nodes": [],
  "edges": []
}

When connectionId is present, the workflow stores the reference instead of duplicating AK/SK. The worker and BOS preview API resolve the referenced storage connection at execution time.

Workflow-level schedule now supports three modes:

  • interval_seconds
    • uses interval
  • every_hours
    • uses intervalHours
  • every_days_at_time
    • uses dayInterval, timeOfDay, and optional timezone

Templates

  • GET /api/templates
  • POST /api/templates
  • GET /api/templates/:id
  • PUT /api/templates/:id
  • DELETE /api/templates/:id

Templates now persist the same workflow-level defaults used by project workflows, including type, schedule, maxConcurrentRuns, nodeTimeoutSeconds, runTimeoutSeconds, maxRetryAttempts, and retryBackoffSeconds.

Operators

  • GET /api/operators
  • POST /api/operators
  • GET /api/operators/:id
  • PUT /api/operators/:id
  • DELETE /api/operators/:id

The built-in catalog includes the original workflow primitives plus:

  • BOS 增量扫描
  • X 平台数据集索引
  • 采集目录校验
  • 视频质检
  • 数据过滤
  • 视频切片
  • 视频标注
  • 成品打包
  • 抽检采样
  • 目录扫描
  • CSV 转 JSON
  • JSON 转 CSV
  • JSON 字段提取
  • JSON 过滤
  • JSON 映射重命名
  • 文本模板渲染
  • 条件分流
  • LeRobot 数据集概览
  • ROSBag 数据概览
  • Shell 命令执行

POST /api/operators

{
  "name": "自定义分割算子",
  "type": "custom",
  "category": "AI",
  "description": "执行图像分割",
  "inputs": [{ "id": "in-1", "name": "图像", "type": "image" }],
  "outputs": [{ "id": "out-1", "name": "结果", "type": "json" }],
  "image": "ccr.example.com/dataflow/custom-seg:latest"
}

Runs and Monitoring

  • GET /api/runs
  • GET /api/runs/:id
  • GET /api/workflows/:id/runs
  • POST /api/runs/:id/rerun-subtree
  • GET /api/runs/:id/logs
  • GET /api/runs/:id/resources
  • GET /api/runs/:id/artifacts

GET /api/runs/:id/logs, GET /api/runs/:id/resources, and GET /api/runs/:id/artifacts accept optional limit and offset query parameters. skip is accepted as an alias for offset. Empty values use defaults, explicit limit values must be positive integers, explicit offset / skip values must be non-negative integers, and oversized limits are capped by the API. Invalid explicit pagination returns 400 instead of falling back to a different page. Logs and resource samples default to the latest 1000 items when the run contains a large history; artifacts default from the start. The response includes:

{
  "items": [],
  "total": 1200,
  "limit": 1000,
  "offset": 200,
  "truncated": true
}

Resource samples include timestamp, cpu, and memory. When NVIDIA GPU metrics are available, samples may also include gpu, gpuMemoryUsedMiB, gpuMemoryTotalMiB, gpuName, and gpuCount.

GET /api/runs/:id/artifacts

Each artifact item can now include:

  • portId / portName: the originating output port so downstream edges can route multi-output operators correctly
  • metadata.mimeType / metadata.sha256: persisted file metadata
  • data: the real JSON payload for JSON artifacts
  • data.preview: a compact preview object for non-JSON artifacts such as text, CSV/TSV, JSONL, image thumbnails, audio/video metadata, PDF hints, SQLite/rosbag summaries, and LeRobot dataset summaries

When data.dataRef.payloadPath points to a local file, data-asset endpoints treat that file as authoritative external JSON object data. Unreadable, malformed, or non-object payloads are omitted instead of falling back to stale inline data. Data-asset integer summaries such as item counts, byte sizes, and frame counts must be exact integer values; lossy numeric values are ignored rather than truncated.

POST /api/runs/:id/rerun-subtree

  • body: { "nodeId": "<workflow-node-id>" }
  • only allowed on failed runs
  • creates a queued child run with:
    • trigger = rerun_subtree
    • parentRunId = source failed run id
    • restartFromNodeId = selected node id
  • the child run reuses completed upstream baseline executions and re-executes the selected node plus its downstream subtree

Data Platform

  • GET /api/scan-project
  • GET /api/scan-configs
  • POST /api/scan-configs
  • GET /api/scan-configs/:id
  • PUT /api/scan-configs/:id
  • DELETE /api/scan-configs/:id
  • POST /api/scan-configs/:id/run
  • GET /api/source-data/collections
  • GET /api/source-data/collections/:runId/:collectionId
  • GET /api/source-data/collections/:runId/:collectionId/entries?prefix=<dir>
  • GET /api/packages
  • GET /api/packages/:runId/:packageName
  • GET /api/sampling-tasks
  • GET /api/sampling-tasks/:id
  • POST /api/sampling-items/:id/review

POST /api/scan-configs

{
  "name": "X-platform-test 自采数据扫描",
  "description": "扫描并索引真实采集目录",
  "sourcePath": "bos://<bucket>/<prefix>",
  "bucket": "<bucket>",
  "prefix": "<prefix>",
  "pattern": "",
  "interval": 86400,
  "enabled": true,
  "dataSource": "BOS"
}

Scan configs are implemented as real workflows under the system project 数据源扫描, not as a second execution engine. For BOS-backed pipelines the first node now uses the built-in BOS 增量扫描 source operator, which persists checkpoint state per workflowId + nodeId + sourceFingerprint. The worker only commits that checkpoint after the whole run succeeds, so downstream failures do not silently skip data on the next retry. GET /api/scan-project resolves the system project for UI redirects, POST /api/scan-configs/:id/run enqueues a normal run, and GET /api/source-data/collections returns data from any completed workflow that produced a real dataset-index.json artifact.

For BOS-backed X-platform scans, source collections are inferred from real meta.json roots inside the scanned prefix. collectionId is a URL-safe stable slug for routing, collectionName is the human-readable leaf folder name such as BJ_001_0001_OsmoNano_2026-03-19_14-51-43, and relativePath preserves the full BOS-relative directory such as raw/BJ_001_0001_OsmoNano_2026-03-19_14-51-43. The frontend groups these source collections together with completed package outputs inside a single 数据资产 module, and drives project/workflow scoping via query filters instead of separate first-level menus.

GET /api/packages

Returns finished data packages aggregated from real packages-manifest.json run artifacts. Each item includes the source projectId, workflowId, runId, package metadata, embedded quality / annotation summaries, and a previewArtifact when a previewable file such as the clip video exists.

GET /api/sampling-tasks

Sampling tasks are synced from real audit-samples.json run artifacts. The API persists review state separately from the originating run so reviewers can submit decisions later without mutating the original workflow outputs. Synced rule summaries must match the runtime rule contract: non-empty unique id, non-empty field, supported comparison operator, and finite numeric (0, 100] samplePercent; malformed rules, boolean percentages, and non-numeric percentages are skipped instead of being persisted as review tasks.

Audit

  • GET /api/audit-tasks
  • POST /api/audit-tasks/:id/approve
  • POST /api/audit-tasks/:id/reject

Users and Settings

  • GET /api/users
  • PUT /api/users/:id
  • GET /api/storage/connections
  • POST /api/storage/connections
  • GET /api/storage/connections/:id
  • PUT /api/storage/connections/:id
  • DELETE /api/storage/connections/:id
  • GET /api/external/business-db/tokens
  • POST /api/external/business-db/tokens
  • PUT /api/external/business-db/tokens/:id
  • POST /api/external/business-db/tokens/:id/rotate
  • DELETE /api/external/business-db/tokens/:id
  • GET /api/settings
  • PUT /api/settings

POST /api/storage/connections

{
  "name": "BOS 主连接",
  "description": "X-platform 扫描使用",
  "provider": "BOS",
  "endpoint": "",
  "accessKeyId": "ak-...",
  "secretAccessKey": "sk-..."
}

GET /api/storage/connections

The response never returns plaintext AK/SK. It only exposes hasAccessKeyId, hasSecretAccessKey, and masked previews so the UI can confirm a real secret is already stored.

GET /api/settings

{
  "id": "system",
  "k8s": {
    "enabled": true,
    "endpoint": "https://<api-server>:6443",
    "namespace": "droboticflow",
    "hasKubeconfig": true
  },
  "worker": {
    "pollIntervalMs": 2000,
    "maxConcurrentRuns": 2
  },
  "ai": {
    "provider": "dashscope-openai-compatible",
    "baseURL": "https://coding.dashscope.aliyuncs.com/v1",
    "model": "glm-5",
    "hasAPIKey": true
  },
  "timezone": "Asia/Shanghai"
}

PUT /api/settings

{
  "timezone": "Asia/Shanghai",
  "worker": {
    "maxConcurrentRuns": 2
  },
  "ai": {
    "provider": "dashscope-openai-compatible",
    "baseURL": "https://coding.dashscope.aliyuncs.com/v1",
    "model": "glm-5",
    "apiKey": "sk-..."
  }
}

Settings updates validate runtime defaults before persistence. worker.pollIntervalMs and worker.maxConcurrentRuns must be positive integers when present. defaults.cpuRequest, defaults.cpuLimit, defaults.memoryRequest, and defaults.memoryLimit are trimmed and must be valid positive Kubernetes resource quantities when present. defaults.gpuLimit is optional, but any explicit value must be a positive integer.

Database Browser

  • GET /api/database
  • GET /api/database/collections/:name/documents
  • GET /api/database/collections/:name/indexes
  • Business Mongo write endpoints under /api/database/collections/*

Document browsing accepts optional limit and skip query parameters. Empty values use defaults, explicit limit values must be positive integers, explicit skip values must be non-negative integers, and oversized values are capped by the API before hitting MongoDB or the in-memory store.

External Business DB Write API

For a copy-paste-ready partner handoff guide, see docs/external-business-db-write-api-integration.md.

Use this surface when a third party needs to write into a preconfigured business Mongo connection without seeing the raw Mongo URI. The external caller must use a server-generated API key in header X-API-Key.

Internal token-management routes:

  • GET /api/external/business-db/tokens
  • POST /api/external/business-db/tokens
  • PUT /api/external/business-db/tokens/:id
  • POST /api/external/business-db/tokens/:id/rotate
  • DELETE /api/external/business-db/tokens/:id

POST /api/external/business-db/tokens

{
  "name": "partner-a",
  "description": "订单同步写入",
  "enabled": true,
  "scopes": [
    {
      "connectionId": "mongo_abc123",
      "collections": ["orders", "customers"]
    }
  ]
}

Successful create and rotate responses return the plaintext apiKey once. The server stores only the key hash and preview, so callers must save the key from the create or rotate response:

{
  "token": {
    "id": "exttoken_abc123",
    "name": "partner-a",
    "enabled": true,
    "tokenPreview": "dbw_****7f3a",
    "scopes": [
      {
        "connectionId": "mongo_abc123",
        "collections": ["customers", "orders"]
      }
    ]
  },
  "apiKey": "dbw_9f..."
}

Third-party write routes:

  • POST /api/external/business-db/documents
  • PATCH /api/external/business-db/documents/:documentId
  • POST /api/external/business-db/upserts

All of these routes require header:

X-API-Key: dbw_...

POST /api/external/business-db/documents

{
  "connectionId": "mongo_abc123",
  "collection": "orders",
  "document": {
    "externalId": "EXT-1001",
    "status": "new",
    "amount": 20
  }
}

PATCH /api/external/business-db/documents/:documentId

{
  "connectionId": "mongo_abc123",
  "collection": "orders",
  "patch": {
    "status": "done",
    "amount": 25
  }
}

POST /api/external/business-db/upserts

{
  "connectionId": "mongo_abc123",
  "collection": "orders",
  "match": {
    "externalId": "EXT-1001"
  },
  "patch": {
    "status": "done",
    "amount": 25
  },
  "setOnInsert": {
    "source": "partner-a"
  }
}

External write responses use:

  • operation: "created" for inserts and create-path upserts
  • operation: "updated" for patch and update-path upserts

External write validation intentionally does not expose raw Mongo update operators. document, patch, match, and setOnInsert must all be plain JSON objects.

Workflow create/update payloads also accept workflow-level execution settings:

  • 0: inherit the global worker default
  • 1+: workflow-specific maximum active runs

Dispatch enforces both the global worker limit and the workflow-specific limit.

  • runTimeoutSeconds
    • 0: disable workflow-level timeout
    • 1+: fail the current run if total execution time exceeds this many seconds
  • nodeTimeoutSeconds
    • 0: disable workflow-level per-node timeout
    • 1+: fail the current node if a single operator execution exceeds this many seconds
  • maxRetryAttempts
    • 0: disable automatic retry
    • 1+: automatically enqueue up to N retry runs after a failed attempt
  • retryBackoffSeconds
    • 0: enqueue retry runs immediately
    • 1+: delay retry dispatch by the configured number of seconds

Run responses may also include:

  • attempt: current attempt number, starting from 1
  • parentRunId: the failed run that spawned this retry run
  • availableAt: when this queued run becomes eligible for dispatch; retry backoff uses this field

Storage

  • GET /api/storage/buckets?sourcePath=<node SOURCE_PATH>
  • GET /api/storage/objects?sourcePath=<node SOURCE_PATH>&bucket=<bucket>&prefix=<prefix>

Worker API

Base path: /worker-api

  • GET /worker-api/health: worker health check
  • GET /worker-api/stats: current queue/running/audit counts
  • GET /worker-api/metrics: Prometheus text metrics for queue depth, dispatch counts, node execution counts, cumulative durations, and duration histograms
  • POST /worker-api/process-once: process one queued run immediately
  • POST /worker-api/runs/:id/retry: reset a run to queued and clear prior execution state

Response Shape Notes

  • list endpoints return { "items": [...] }
  • delete endpoints return 204 No Content
  • validation failures return 400
  • missing or invalid external API key returns 401
  • authenticated external token without the requested connection or collection permission returns 403
  • missing resources return 404
  • AI service unavailable returns 503
  • upstream AI gateway failures return 502
  • unhandled server errors return 500

Implementation Notes

  • /api/* is served by the Go API binary in cmd/api.
  • /worker-api/* is served by the Go worker binary in cmd/worker.
  • Workflow execution and storage models are defined in internal/domain and internal/store.
  • AI generation is implemented by internal/ai and is called only from the Go API, never directly from the browser.
  • GET /api/settings never returns the saved AI key in plaintext. It only exposes ai.hasAPIKey so the browser can show whether a server-side key is already configured.
  • timezone uses IANA timezone identifiers such as Asia/Shanghai or America/Los_Angeles. The frontend formats timestamps with second precision in that timezone.
  • Monitoring previews are rendered from the persisted artifact payload itself, so text/table/robotics previews keep working after the Kubernetes Job has exited and its temporary filesystem is gone.
  • OSS 相关配置不通过 /api/settings 管理。系统设置页只维护 K8s、worker、AI 和默认资源;OSS bucket/prefix 等参数由具体工作流节点维护。
  • /api/storage/* 也不再读取系统级存储根目录。浏览请求必须显式传入当前 OSS 节点超参里的 SOURCE_PATH,API 只在这个节点上下文内列 bucket 和对象。
  • Kubernetes Jobs launched for operator nodes use the Python entrypoint at backend/operator-runner/main.py.