🔐 Support authenticated data reconciliation
Some checks are pending
CI / verify (push) Waiting to run
CI / docker-build (push) Waiting to run

This commit is contained in:
eust-w 2026-06-16 17:44:22 +08:00
parent 447ac14bbd
commit 5d20029c57
14 changed files with 141 additions and 77 deletions

View File

@ -68,7 +68,7 @@ Drobodata is a workflow orchestration platform for embodied AI and vision pipeli
- `npm run smoke`: local smoke test against the running API and worker
- `npm run gpu:smoke`: real Kubernetes GPU scheduling and `nvidia-smi` smoke test
- `npm run k8s:cleanup-jobs`: delete old finished operator-runner Jobs that predate TTL cleanup
- `npm run reconcile:data -- <api-base-url>`: remove legacy demo data, reset settings, and bootstrap built-in operators through the real API
- `npm run reconcile:data -- <api-base-url>`: remove legacy demo data, reset settings, and bootstrap built-in operators through the real API; honors `DROBOTICFLOW_API_TOKEN` and optional session login env vars for protected deployments
- `python3 scripts/wjzh_operator_regression.py --preflight-only`: preflight the production-style WJZH real-data operator regression against a live API
## Runtime Configuration
@ -99,6 +99,7 @@ Key environment variables:
- `DROBOTICFLOW_API_TOKEN`: optional shared API token for non-browser API clients. When set, non-public API calls require `Authorization: Bearer <token>` or `X-Droboticflow-API-Token` unless a valid session cookie is already present while session auth is enabled.
- `DROBOTICFLOW_REQUIRE_SESSION_AUTH`: optional cookie-session auth enforcement for business APIs, default `false` locally; public LoadBalancer deploys force it to `true`
- `DROBOTICFLOW_SESSION_COOKIE_SECURE`: sets the `Secure` attribute on session cookies, default `false` for local development; enable it behind HTTPS
- `DROBOTICFLOW_SMOKE_EMAIL` / `DROBOTICFLOW_SMOKE_PASSWORD`: optional admin/operator credentials used by smoke and reconcile scripts to log in, persist cookies, and send CSRF headers against session-authenticated deployments
- `DROBOTICFLOW_TRUSTED_PROXY_CIDRS`: optional comma/space-separated proxy CIDRs whose `X-Forwarded-For` or `X-Real-IP` headers may be trusted for rate limiting; empty means trust no forwarded headers
- `API_SERVICE_TYPE`: Kubernetes API service type for rendered manifests, `NodePort` by default and `ClusterIP` when exposure is handled by LoadBalancer or ingress
- `AI_API_BASE_URL`: OpenAI-compatible DashScope endpoint used by server-side AI generation, default `https://coding.dashscope.aliyuncs.com/v1`
@ -109,7 +110,7 @@ Key environment variables:
The frontend never reads the AI key directly. Workflow generation and operator generation are proxied through the Go API so secrets stay on the server side.
The settings page can update K8s access details, the worker defaults, the global `timezone`, `provider`, `baseURL`, `model`, and the server-side AI key. `GET /api/settings` only returns whether secrets exist via `k8s.hasKubeconfig` / `ai.hasAPIKey`; it does not echo plaintext kubeconfig or API keys back to the browser. Timestamps in the UI are rendered in the configured IANA timezone with compact minute-level formatting for cards, lists, and summaries, while logs keep finer granularity when needed. OSS-related bucket and path settings belong on OSS operators in workflows, not in the global system settings page. Scan workflows can now reference reusable BOS storage connections via `scanConfig.connectionId`, so endpoint and AK/SK are stored once in `/api/storage/connections` and resolved by the API/worker at preview and execution time instead of being duplicated into each workflow.
The API now includes a cookie-session account foundation for organization-based access. Auth endpoints cover bootstrap, login, logout, current session, and super-admin organization/user management. Existing local deployments remain compatible by default; public LoadBalancer deploys require `DROBOTICFLOW_REQUIRE_SESSION_AUTH=true` so browser business APIs keep role checks. Session-authenticated unsafe requests use a double-submit `droboticflow_csrf` cookie plus `X-Droboticflow-CSRF-Token` header. Worker control endpoints under `/worker-api` require an admin/operator session, or an API token when session auth is not enforced. Do not embed `DROBOTICFLOW_API_TOKEN` into frontend builds; it is intended for machine-to-machine calls only.
The API now includes a cookie-session account foundation for organization-based access. Auth endpoints cover bootstrap, login, logout, current session, and super-admin organization/user management. Existing local deployments remain compatible by default; public LoadBalancer deploys require `DROBOTICFLOW_REQUIRE_SESSION_AUTH=true` so browser business APIs keep role checks. Session-authenticated unsafe requests use a double-submit `droboticflow_csrf` cookie plus `X-Droboticflow-CSRF-Token` header. Worker control endpoints under `/worker-api` require an admin/operator session, or an API token when session auth is not enforced. Bundled smoke and reconcile scripts can log in with `DROBOTICFLOW_SMOKE_EMAIL` / `DROBOTICFLOW_SMOKE_PASSWORD` and will send CSRF headers on writes. Do not embed `DROBOTICFLOW_API_TOKEN` into frontend builds; it is intended for machine-to-machine calls only.
The Kubernetes runtime now assumes a shared workspace PVC. Worker writes per-node `input-artifacts.json` files into the shared workspace, operator-runner Jobs read those files through `INPUT_ARTIFACTS_PATH`, then persist `result.json` and `stage-manifest.json` back into the same PVC. Large artifact lists no longer need to travel through giant env vars or stdout payloads. Node CPU and memory limits are applied to Kubernetes Job container resources; GPU requests use `nvidia.com/gpu` when the node or default resource config sets a positive GPU count. GPU scheduling requires a cluster with GPU nodes and the NVIDIA device plugin/runtime installed. Optional GPU node selector, tolerations, and runtime class settings are only injected into Jobs that request a positive GPU count. Invalid GPU selector or tolerations configuration fails worker startup instead of being ignored. Operator-runner Jobs also receive `ttlSecondsAfterFinished` from `OPERATOR_JOB_TTL_SECONDS`, and the cleanup script can remove old completed Jobs that were created before TTL was enabled. Run `npm run gpu:smoke` against a real GPU cluster to validate allocatable GPU resources, scheduling, runtime, and `nvidia-smi` execution. Runs that execute on NVIDIA hardware report GPU utilization, GPU memory, GPU count, and GPU model in the resource samples shown on the monitoring page.
@ -209,6 +210,11 @@ Reconcile persistent API data after deployment:
```bash
export K8S_ENDPOINT=https://<api-server>:6443
# Optional when the API is protected by DROBOTICFLOW_API_TOKEN and session auth is not enforced:
export DROBOTICFLOW_API_TOKEN=<shared-api-token>
# Required for public session-auth deployments; use an organization admin account:
export DROBOTICFLOW_SMOKE_EMAIL=<admin-email>
export DROBOTICFLOW_SMOKE_PASSWORD=<admin-password>
./scripts/reconcile-system-data.sh http://<node-ip>:30080
```

View File

@ -391,7 +391,7 @@ The response never returns plaintext AK/SK. It only exposes `hasAccessKeyId`, `h
### External Business DB Write API
For a copy-paste-ready partner handoff guide, see [docs/external-business-db-write-api-integration.md](/Users/longtaowu/workspace/droboticflow/docs/external-business-db-write-api-integration.md).
For a copy-paste-ready partner handoff guide, see [docs/external-business-db-write-api-integration.md](./external-business-db-write-api-integration.md).
Use this surface when a third party needs to write into a preconfigured business Mongo connection without seeing the raw Mongo URI. The external caller must use a server-generated API key in header `X-API-Key`.
@ -559,13 +559,13 @@ Base path: `/worker-api`
## Implementation Notes
- `/api/*` is served by the Go API binary in [`cmd/api`](/Users/longtaowu/workspace/droboticflow/cmd/api).
- `/worker-api/*` is served by the Go worker binary in [`cmd/worker`](/Users/longtaowu/workspace/droboticflow/cmd/worker).
- Workflow execution and storage models are defined in [`internal/domain`](/Users/longtaowu/workspace/droboticflow/internal/domain) and [`internal/store`](/Users/longtaowu/workspace/droboticflow/internal/store).
- AI generation is implemented by [`internal/ai`](/Users/longtaowu/workspace/droboticflow/internal/ai) and is called only from the Go API, never directly from the browser.
- `/api/*` is served by the Go API binary in [`cmd/api`](../cmd/api).
- `/worker-api/*` is served by the Go worker binary in [`cmd/worker`](../cmd/worker).
- Workflow execution and storage models are defined in [`internal/domain`](../internal/domain) and [`internal/store`](../internal/store).
- AI generation is implemented by [`internal/ai`](../internal/ai) and is called only from the Go API, never directly from the browser.
- `GET /api/settings` never returns the saved AI key in plaintext. It only exposes `ai.hasAPIKey` so the browser can show whether a server-side key is already configured.
- `timezone` uses IANA timezone identifiers such as `Asia/Shanghai` or `America/Los_Angeles`. The frontend formats timestamps with second precision in that timezone.
- Monitoring previews are rendered from the persisted artifact payload itself, so text/table/robotics previews keep working after the Kubernetes Job has exited and its temporary filesystem is gone.
- OSS 相关配置不通过 `/api/settings` 管理。系统设置页只维护 K8s、worker、AI 和默认资源;OSS bucket/prefix 等参数由具体工作流节点维护。
- `/api/storage/*` 也不再读取系统级存储根目录。浏览请求必须显式传入当前 OSS 节点超参里的 `SOURCE_PATH`,API 只在这个节点上下文内列 bucket 和对象。
- Kubernetes Jobs launched for operator nodes use the Python entrypoint at [`backend/operator-runner/main.py`](/Users/longtaowu/workspace/droboticflow/backend/operator-runner/main.py).
- Kubernetes Jobs launched for operator nodes use the Python entrypoint at [`backend/operator-runner/main.py`](../backend/operator-runner/main.py).

View File

@ -1,6 +1,6 @@
# Drobodata 内建算子手册
本文档不是“把每个算子再抄一遍”的长列表,而是当前内建算子体系的总览、使用方法和验收规则。单算子细节请进入分册目录 [`docs/operators/README.md`](/Users/longtaowu/workspace/droboticflow/docs/operators/README.md)。
本文档不是“把每个算子再抄一遍”的长列表,而是当前内建算子体系的总览、使用方法和验收规则。单算子细节请进入分册目录 [`docs/operators/README.md`](./operators/README.md)。
截至 `2026-04-21`,当前线上算子目录中可见 `25` 个 `builtin` 算子、`0` 个 `custom` 实例。
@ -9,9 +9,9 @@
## 1. 文档结构
- 总手册:说明平台级规则、编排方式、通用坑点和验收口径
- 分册目录:[`docs/operators/README.md`](/Users/longtaowu/workspace/droboticflow/docs/operators/README.md)
- 自定义算子:[`docs/custom-operator-manual.zh-CN.md`](/Users/longtaowu/workspace/droboticflow/docs/custom-operator-manual.zh-CN.md)
- 算子开发参考:[`docs/operator-authoring.md`](/Users/longtaowu/workspace/droboticflow/docs/operator-authoring.md)
- 分册目录:[`docs/operators/README.md`](./operators/README.md)
- 自定义算子:[`docs/custom-operator-manual.zh-CN.md`](./custom-operator-manual.zh-CN.md)
- 算子开发参考:[`docs/operator-authoring.md`](./operator-authoring.md)
建议阅读顺序:
@ -79,16 +79,16 @@ sequenceDiagram
- 接口和产物结构还在频繁变化
- 需要快速验证单次项目需求
这类场景更适合先走 [`docs/custom-operator-manual.zh-CN.md`](/Users/longtaowu/workspace/droboticflow/docs/custom-operator-manual.zh-CN.md) 里的自定义路径。
这类场景更适合先走 [`docs/custom-operator-manual.zh-CN.md`](./custom-operator-manual.zh-CN.md) 里的自定义路径。
## 5. 当前算子分类地图
### 5.1 IO / 源与汇
- [BOS 增量扫描](/Users/longtaowu/workspace/droboticflow/docs/operators/BOS-增量扫描.md)
- [真实数据源](/Users/longtaowu/workspace/droboticflow/docs/operators/真实数据源.md)
- [目录扫描](/Users/longtaowu/workspace/droboticflow/docs/operators/目录扫描.md)
- [结果上传](/Users/longtaowu/workspace/droboticflow/docs/operators/结果上传.md)
- [BOS 增量扫描](./operators/BOS-增量扫描.md)
- [真实数据源](./operators/真实数据源.md)
- [目录扫描](./operators/目录扫描.md)
- [结果上传](./operators/结果上传.md)
适合解决:
@ -99,8 +99,8 @@ sequenceDiagram
### 5.2 DATASET / 数据集整理
- [X 平台数据集索引](/Users/longtaowu/workspace/droboticflow/docs/operators/X-平台数据集索引.md)
- [采集目录校验](/Users/longtaowu/workspace/droboticflow/docs/operators/采集目录校验.md)
- [X 平台数据集索引](./operators/X-平台数据集索引.md)
- [采集目录校验](./operators/采集目录校验.md)
适合解决:
@ -110,12 +110,12 @@ sequenceDiagram
### 5.3 DATA / 结构化处理中间层
- [CSV 转 JSON](/Users/longtaowu/workspace/droboticflow/docs/operators/CSV-转-JSON.md)
- [JSON 转 CSV](/Users/longtaowu/workspace/droboticflow/docs/operators/JSON-转-CSV.md)
- [JSON 字段提取](/Users/longtaowu/workspace/droboticflow/docs/operators/JSON-字段提取.md)
- [JSON 过滤](/Users/longtaowu/workspace/droboticflow/docs/operators/JSON-过滤.md)
- [JSON 映射重命名](/Users/longtaowu/workspace/droboticflow/docs/operators/JSON-映射重命名.md)
- [MongoDB 写入](/Users/longtaowu/workspace/droboticflow/docs/operators/MongoDB-写入.md)
- [CSV 转 JSON](./operators/CSV-转-JSON.md)
- [JSON 转 CSV](./operators/JSON-转-CSV.md)
- [JSON 字段提取](./operators/JSON-字段提取.md)
- [JSON 过滤](./operators/JSON-过滤.md)
- [JSON 映射重命名](./operators/JSON-映射重命名.md)
- [MongoDB 写入](./operators/MongoDB-写入.md)
适合解决:
@ -126,11 +126,11 @@ sequenceDiagram
### 5.4 VIDEO / 视频主链路
- [视频质检](/Users/longtaowu/workspace/droboticflow/docs/operators/视频质检.md)
- [数据过滤](/Users/longtaowu/workspace/droboticflow/docs/operators/数据过滤.md)
- [视频切片](/Users/longtaowu/workspace/droboticflow/docs/operators/视频切片.md)
- [视频标注](/Users/longtaowu/workspace/droboticflow/docs/operators/视频标注.md)
- [成品打包](/Users/longtaowu/workspace/droboticflow/docs/operators/成品打包.md)
- [视频质检](./operators/视频质检.md)
- [数据过滤](./operators/数据过滤.md)
- [视频切片](./operators/视频切片.md)
- [视频标注](./operators/视频标注.md)
- [成品打包](./operators/成品打包.md)
适合解决:
@ -142,14 +142,14 @@ sequenceDiagram
### 5.5 QA / HUMAN / LOGIC / UTIL / AI / ROBOTICS
- [抽检采样](/Users/longtaowu/workspace/droboticflow/docs/operators/抽检采样.md)
- [人工审核](/Users/longtaowu/workspace/droboticflow/docs/operators/人工审核.md)
- [条件分流](/Users/longtaowu/workspace/droboticflow/docs/operators/条件分流.md)
- [文本模板渲染](/Users/longtaowu/workspace/droboticflow/docs/operators/文本模板渲染.md)
- [Shell 命令执行](/Users/longtaowu/workspace/droboticflow/docs/operators/Shell-命令执行.md)
- [模型推理](/Users/longtaowu/workspace/droboticflow/docs/operators/模型推理.md)
- [LeRobot 数据集概览](/Users/longtaowu/workspace/droboticflow/docs/operators/LeRobot-数据集概览.md)
- [ROSBag 数据概览](/Users/longtaowu/workspace/droboticflow/docs/operators/ROSBag-数据概览.md)
- [抽检采样](./operators/抽检采样.md)
- [人工审核](./operators/人工审核.md)
- [条件分流](./operators/条件分流.md)
- [文本模板渲染](./operators/文本模板渲染.md)
- [Shell 命令执行](./operators/Shell-命令执行.md)
- [模型推理](./operators/模型推理.md)
- [LeRobot 数据集概览](./operators/LeRobot-数据集概览.md)
- [ROSBag 数据概览](./operators/ROSBag-数据概览.md)
这几类算子承担的是:
@ -260,7 +260,7 @@ flowchart LR
当你要修改工作流或给 AI 生成工作流时,建议这样使用这些文档:
1. 先在本手册中确认平台级约束
2. 到 [`docs/operators/README.md`](/Users/longtaowu/workspace/droboticflow/docs/operators/README.md) 选中目标算子
2. 到 [`docs/operators/README.md`](./operators/README.md) 选中目标算子
3. 阅读该算子的单册,确认端口、超参数、排障和验收
4. 再回到工作流设计和回归
@ -272,7 +272,7 @@ flowchart LR
## 10. 相关文档
- [平台使用手册](/Users/longtaowu/workspace/droboticflow/docs/platform-usage-manual.zh-CN.md)
- [内建算子分册索引](/Users/longtaowu/workspace/droboticflow/docs/operators/README.md)
- [自定义算子手册](/Users/longtaowu/workspace/droboticflow/docs/custom-operator-manual.zh-CN.md)
- [算子开发文档](/Users/longtaowu/workspace/droboticflow/docs/operator-authoring.md)
- [平台使用手册](./platform-usage-manual.zh-CN.md)
- [内建算子分册索引](./operators/README.md)
- [自定义算子手册](./custom-operator-manual.zh-CN.md)
- [算子开发文档](./operator-authoring.md)

View File

@ -4,10 +4,10 @@
配套文档:
- [内建算子手册](/Users/longtaowu/workspace/droboticflow/docs/builtin-operators-manual.zh-CN.md)
- [内建算子分册索引](/Users/longtaowu/workspace/droboticflow/docs/operators/README.md)
- [自定义算子分册体系](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/README.md)
- [算子开发文档](/Users/longtaowu/workspace/droboticflow/docs/operator-authoring.md)
- [内建算子手册](./builtin-operators-manual.zh-CN.md)
- [内建算子分册索引](./operators/README.md)
- [自定义算子分册体系](./custom-operators/README.md)
- [算子开发文档](./operator-authoring.md)
相关实现入口:
@ -44,8 +44,8 @@
最常见的起点是:
- [Shell 命令执行](/Users/longtaowu/workspace/droboticflow/docs/operators/Shell-命令执行.md)
- [模型推理](/Users/longtaowu/workspace/droboticflow/docs/operators/模型推理.md)
- [Shell 命令执行](./operators/Shell-命令执行.md)
- [模型推理](./operators/模型推理.md)
适合:
@ -164,7 +164,7 @@ ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow/custom-<operator-name>:<tag>
如果你当前接的是百度云 CCR `xspark` 命名空间,并希望让用户自己推镜像再在算子里填写 `image`,请直接参考:
- [百度云 CCR `xspark` 自定义算子接入指南](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/百度云-CCR-xspark-自定义算子接入指南.md)
- [百度云 CCR `xspark` 自定义算子接入指南](./custom-operators/百度云-CCR-xspark-自定义算子接入指南.md)
这份分册包含:
@ -310,8 +310,8 @@ path = artifact["path"]
这次仓库把自定义算子文档拆成三层:
1. 本手册:平台级规则
2. 模板与示例:[`docs/custom-operators/README.md`](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/README.md)
3. 实例级分册:[`docs/custom-operators/instances/README.md`](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/instances/README.md)
2. 模板与示例:[`docs/custom-operators/README.md`](./custom-operators/README.md)
3. 实例级分册:[`docs/custom-operators/instances/README.md`](./custom-operators/instances/README.md)
你应该为每个长期存在的自定义算子单独维护一篇 Markdown,至少包括:
@ -328,7 +328,7 @@ path = artifact["path"]
### 8.1 第一步:先写文档,再写实现
在动手之前,先用 [`自定义算子设计模板`](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/templates/自定义算子设计模板.md) 写清楚:
在动手之前,先用 [`自定义算子设计模板`](./custom-operators/templates/自定义算子设计模板.md) 写清楚:
- 它解决什么问题
- 它的输入是什么
@ -510,7 +510,7 @@ path = artifact["path"]
## 13. 相关资料
- [内建算子手册](/Users/longtaowu/workspace/droboticflow/docs/builtin-operators-manual.zh-CN.md)
- [自定义算子分册体系](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/README.md)
- [算子开发文档](/Users/longtaowu/workspace/droboticflow/docs/operator-authoring.md)
- [平台使用手册](/Users/longtaowu/workspace/droboticflow/docs/platform-usage-manual.zh-CN.md)
- [内建算子手册](./builtin-operators-manual.zh-CN.md)
- [自定义算子分册体系](./custom-operators/README.md)
- [算子开发文档](./operator-authoring.md)
- [平台使用手册](./platform-usage-manual.zh-CN.md)

View File

@ -12,8 +12,8 @@
当前内建算子里有:
- [视频切片](/Users/longtaowu/workspace/droboticflow/docs/operators/视频切片.md)
- [视频标注](/Users/longtaowu/workspace/droboticflow/docs/operators/视频标注.md)
- [视频切片](../../operators/视频切片.md)
- [视频标注](../../operators/视频标注.md)
但没有一个“只做关键帧抽取”的轻量算子。它更适合作为:

View File

@ -8,7 +8,7 @@
## 2. 为什么不用内建视频切片
内建 [视频切片](/Users/longtaowu/workspace/droboticflow/docs/operators/视频切片.md) 解决的是标准固定切片问题;本算子则强调“窗口采样”:
内建 [视频切片](../../operators/视频切片.md) 解决的是标准固定切片问题;本算子则强调“窗口采样”:
- 可能不是全量切
- 可能按滑窗或步长抽样

View File

@ -279,6 +279,6 @@ open /workspace/runs/<run-id>/nodes/<node-id>/stage-manifest.json: no such file
## 11. 相关文档
- [自定义算子总手册](/Users/longtaowu/workspace/droboticflow/docs/custom-operator-manual.zh-CN.md)
- [自定义算子 Markdown 体系索引](/Users/longtaowu/workspace/droboticflow/docs/custom-operators/README.md)
- [算子开发文档](/Users/longtaowu/workspace/droboticflow/docs/operator-authoring.md)
- [自定义算子总手册](../custom-operator-manual.zh-CN.md)
- [自定义算子 Markdown 体系索引](./README.md)
- [算子开发文档](../operator-authoring.md)

View File

@ -211,6 +211,7 @@ http://<node-ip>:30080
- Session-authenticated unsafe requests require `X-Droboticflow-CSRF-Token` to match the readable `droboticflow_csrf` cookie. The bundled frontend sets this header automatically.
- `IMAGE_TAG` defaults to the current git short SHA in build, deploy, and render scripts. Set it explicitly for release names or rollback-friendly tags.
- `scripts/smoke-test.sh` can exercise session-authenticated deployments when `DROBOTICFLOW_SMOKE_EMAIL` and `DROBOTICFLOW_SMOKE_PASSWORD` are set; it logs in, persists cookies, and sends the CSRF header on write requests.
- `scripts/reconcile-system-data.sh` uses the same protected-client behavior. Set `DROBOTICFLOW_API_TOKEN` for token-protected APIs when session auth is not enforced. For public session-auth deployments, set `DROBOTICFLOW_SMOKE_EMAIL` and `DROBOTICFLOW_SMOKE_PASSWORD` to an organization admin account because the reconcile flow updates settings, users, templates, and operators.
- `npm run gpu:smoke` creates a temporary GPU Job in the target namespace and runs `nvidia-smi`. Override `GPU_SMOKE_IMAGE`, `GPU_RESOURCE_NAME`, `GPU_RESOURCE_COUNT`, and the `OPERATOR_GPU_*` scheduling variables when the cluster uses custom GPU images or labels.
- GitHub Actions includes a manual `Acceptance` workflow for deployed-environment checks. Configure `KUBE_CONFIG_B64` for Kubernetes access and optionally `DROBOTICFLOW_API_TOKEN`, `DROBOTICFLOW_SMOKE_EMAIL`, and `DROBOTICFLOW_SMOKE_PASSWORD` for authenticated smoke tests.
- If you use DashScope-compatible models, prefer a backend-allowed API key. Do not deploy tool-only Coding Plan keys into the cluster.

View File

@ -337,11 +337,11 @@ BASE_URL=http://114.111.24.223:30080 node scripts/playwright-functional-check.mj
脚本文件位置:
- [scripts/playwright-functional-check.mjs](/Users/longtaowu/workspace/droboticflow/scripts/playwright-functional-check.mjs)
- [scripts/playwright-functional-check.mjs](../scripts/playwright-functional-check.mjs)
产物目录:
- [output/playwright](/Users/longtaowu/workspace/droboticflow/output/playwright)
- [output/playwright](../output/playwright)
## 5. 线上环境快照
@ -368,7 +368,7 @@ BASE_URL=http://114.111.24.223:30080 node scripts/playwright-functional-check.mj
Playwright 录制的视频已经生成:
- [playwright-functional-check.webm](/Users/longtaowu/workspace/droboticflow/output/playwright/playwright-functional-check.webm)
- [playwright-functional-check.webm](../output/playwright/playwright-functional-check.webm)
如果当前查看器支持内嵌视频,可直接播放:

View File

@ -117,12 +117,17 @@ STORAGE_CONNECTION_ID=<storage-connection-id> \
python3 scripts/wjzh_operator_regression.py --preflight-only
```
See the full runbook in [`docs/wjzh-operator-regression.md`](/Users/longtaowu/workspace/droboticflow/docs/wjzh-operator-regression.md).
See the full runbook in [`docs/wjzh-operator-regression.md`](wjzh-operator-regression.md).
Reconcile persistent system data and remove legacy demo records:
```bash
export K8S_ENDPOINT=https://<api-server>:6443
# Optional for token-protected APIs when DROBOTICFLOW_REQUIRE_SESSION_AUTH=false:
export DROBOTICFLOW_API_TOKEN=<shared-api-token>
# Required for session-authenticated public APIs; use an organization admin:
export DROBOTICFLOW_SMOKE_EMAIL=<admin-email>
export DROBOTICFLOW_SMOKE_PASSWORD=<admin-password>
./scripts/reconcile-system-data.sh http://127.0.0.1:3001
```
@ -188,5 +193,7 @@ kubectl -n droboticflow exec deploy/droboticflow-worker -- ls -R /workspace/runs
### Old demo data still appears after an upgrade
- run `K8S_ENDPOINT=https://<api-server>:6443 ./scripts/reconcile-system-data.sh http://<node-ip>:30080`
- include `DROBOTICFLOW_API_TOKEN` when token auth is enabled without required session auth
- include `DROBOTICFLOW_SMOKE_EMAIL` and `DROBOTICFLOW_SMOKE_PASSWORD` for public session-auth deployments, using an organization admin account
- confirm `/api/users`, `/api/projects`, and `/api/templates` no longer return legacy seeded records
- confirm `/api/operators` only returns the current built-in operator catalog with the active operator-runner image tag

View File

@ -532,10 +532,10 @@ def builtin_my_operator(
## 17. 相关文件
- [backend/operator-runner/main.py](/Users/longtaowu/workspace/droboticflow/backend/operator-runner/main.py)
- [internal/worker/executor.go](/Users/longtaowu/workspace/droboticflow/internal/worker/executor.go)
- [internal/worker/runtime.go](/Users/longtaowu/workspace/droboticflow/internal/worker/runtime.go)
- [internal/k8s/launcher.go](/Users/longtaowu/workspace/droboticflow/internal/k8s/launcher.go)
- [internal/domain/types.go](/Users/longtaowu/workspace/droboticflow/internal/domain/types.go)
- [docs/api.md](/Users/longtaowu/workspace/droboticflow/docs/api.md)
- [docs/operations.md](/Users/longtaowu/workspace/droboticflow/docs/operations.md)
- [backend/operator-runner/main.py](../backend/operator-runner/main.py)
- [internal/worker/executor.go](../internal/worker/executor.go)
- [internal/worker/runtime.go](../internal/worker/runtime.go)
- [internal/k8s/launcher.go](../internal/k8s/launcher.go)
- [internal/domain/types.go](../internal/domain/types.go)
- [docs/api.md](./api.md)
- [docs/operations.md](./operations.md)

View File

@ -9,6 +9,7 @@ This note tracks the production-facing safeguards that must stay true for Drobod
- Worker proxy health, stats, metrics, and retry calls use short upstream timeouts. Manual `process-once` uses a long timeout because it can execute real queued work.
- Explicit shell operators are disabled by default through `OPERATOR_ALLOW_SHELL_COMMANDS=false`; allowlist and denylist regexes are platform-controlled and cannot be overridden by workflow hyperparams.
- Environment-specific values such as `K8S_ENDPOINT`, registry username, and workspace BOS bucket have no private repository defaults; provide them from local deploy secrets or CI secrets.
- Data reconciliation uses real API calls and supports protected deployments through `DROBOTICFLOW_API_TOKEN` or session login credentials plus CSRF headers. Public session-auth environments must use an organization admin account for reconcile because it updates settings and management resources.
## Kubernetes Jobs
@ -46,6 +47,12 @@ Run deployed smoke after rollout:
./scripts/smoke-test.sh http://<node-ip>:30080
```
Reconcile persistent system data after rollout:
```bash
K8S_ENDPOINT=https://<api-server>:6443 ./scripts/reconcile-system-data.sh http://<node-ip>:30080
```
Run GPU smoke on a real GPU cluster:
```bash

View File

@ -2225,7 +2225,7 @@ func TestSettingsAIConfigurationDrivesAIStatus(t *testing.T) {
settingsPayload := []byte(`{
"k8s": {
"endpoint": "https://120.48.161.230:6443",
"endpoint": "https://192.0.2.10:6443",
"namespace": "droboticflow",
"workspacePVCName": "droboticflow-workspace",
"workspaceMountPath": "/workspace",

View File

@ -16,9 +16,11 @@ fi
python3 - "$API_BASE_URL" "$K8S_ENDPOINT" "$K8S_NAMESPACE" "$OPERATOR_RUNNER_IMAGE" <<'PY'
import http.client
import http.cookiejar
import json
import time
import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
@ -27,18 +29,40 @@ api_base_url = sys.argv[1].rstrip("/")
k8s_endpoint = sys.argv[2]
k8s_namespace = sys.argv[3]
operator_runner_image = sys.argv[4]
api_token = os.getenv("DROBOTICFLOW_API_TOKEN", "").strip()
smoke_email = os.getenv("DROBOTICFLOW_SMOKE_EMAIL", "").strip()
smoke_password = os.getenv("DROBOTICFLOW_SMOKE_PASSWORD", "")
cookie_jar = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cookie_jar))
if bool(smoke_email) != bool(smoke_password):
raise SystemExit("DROBOTICFLOW_SMOKE_EMAIL and DROBOTICFLOW_SMOKE_PASSWORD must be set together")
def csrf_token():
for cookie in cookie_jar:
if cookie.name == "droboticflow_csrf":
return cookie.value
return ""
def request(method, path, payload=None, expected=(200, 201, 202, 204)):
method = method.upper()
body = None
headers = {"Content-Type": "application/json"}
if api_token:
headers["Authorization"] = f"Bearer {api_token}"
if method not in {"GET", "HEAD", "OPTIONS"}:
token = csrf_token()
if token:
headers["X-Droboticflow-CSRF-Token"] = token
if payload is not None:
body = json.dumps(payload).encode("utf-8")
last_error = None
for attempt in range(1, 7):
req = urllib.request.Request(f"{api_base_url}{path}", data=body, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=20) as response:
with opener.open(req, timeout=20) as response:
status = response.getcode()
raw = response.read().decode("utf-8")
if status not in expected:
@ -57,6 +81,21 @@ def request(method, path, payload=None, expected=(200, 201, 202, 204)):
raise SystemExit(f"{method} {path} failed after retries: {last_error}")
def login_if_configured():
if not smoke_email:
return
response = request(
"POST",
"/api/auth/login",
{"email": smoke_email, "password": smoke_password},
expected=(200,),
)
identifier = smoke_email
if isinstance(response, dict):
identifier = response.get("email") or response.get("name") or identifier
print(f"logged in reconcile user: {identifier}")
def delete_each(path_prefix, items, predicate):
for item in items:
if not predicate(item):
@ -65,6 +104,10 @@ def delete_each(path_prefix, items, predicate):
request("DELETE", f"{path_prefix}/{urllib.parse.quote(item['id'])}", expected=(204, 404))
if api_token:
print("using API token auth for reconcile requests")
login_if_configured()
users = request("GET", "/api/users")["items"]
projects = request("GET", "/api/projects")["items"]
templates = request("GET", "/api/templates")["items"]