forked from longtao.wu/droboticflow
356 lines
16 KiB
Markdown
356 lines
16 KiB
Markdown
# WJZH 真实数据算子回归
|
||
|
||
这个回归入口面向真实生产环境,不再使用临时 shell 适配脚本里的明文 `AK/SK`,而是优先复用系统里的 BOS `storage connection`。
|
||
|
||
入口脚本:
|
||
|
||
```bash
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_ID=<storage-connection-id> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='[{"id":"clarity-gate","field":"quality.clarity","operator":"gte","value":0.3,"samplePercent":100}]' \
|
||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||
```
|
||
|
||
## 回归内容
|
||
|
||
脚本会创建 1 个回归项目,并依次执行完整真实数据回归工作流:
|
||
|
||
1. `00-源数据目录推理`
|
||
2. `01-索引校验双上传`
|
||
3. `02-JSON工具链多分支`
|
||
4. `03-人工审核通过`
|
||
5. `04-人工审核拒绝`
|
||
6. `05-Mongo写入`
|
||
7. `06-中大型视频切片`
|
||
8. `07-超大视频全链路`
|
||
9. `08-多目录批量窗口化`
|
||
10. `09-Robotics真实概览`
|
||
11. `10-多目录多视频批量质检`
|
||
12. `11-多目录多视频批量全链路`
|
||
13. `12-增量嗅探循环处理`
|
||
|
||
数据源必须显式提供:
|
||
|
||
```text
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix>
|
||
```
|
||
|
||
输出前缀必须显式提供:
|
||
|
||
```text
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix>
|
||
```
|
||
|
||
脚本不再内置任何真实 bucket、路径、连接名、视频文件名、设备号、日期范围、Robotics 数据路径、增量目录元数据、质检阈值、视频处理时长或抽检规则。执行者必须按目标环境传入 `SOURCE_PATH`、`UPLOAD_ROOT_PREFIX`、`STORAGE_CONNECTION_ID` 或 `STORAGE_CONNECTION_NAME`、`MEDIUM_VIDEO`、`HUGE_VIDEO`、`ROUTE_DEVICE_ID`、`FILTER_DATE_START`、`FILTER_DATE_END`、`LEROBOT_SOURCE_PATH`、`ROSBAG_SOURCE_PATH`、`INCREMENTAL_PLACE`、`INCREMENTAL_CAMERA_MODEL`、`INCREMENTAL_VIDEO_SUBDIR`、`QUALITY_MIN_CLARITY`、`QUALITY_MAX_SHAKE`、`QUALITY_MAX_HAND_OCCLUSION`、`HUGE_WINDOW_SECONDS`、`MEDIUM_WINDOW_SECONDS`、`MEDIUM_SEGMENT_DURATION_SECONDS`、`HUGE_SEGMENT_DURATION_SECONDS`、`BATCH_WINDOW_SECONDS`、`BATCH_QUALITY_WINDOW_SECONDS`、`BATCH_SEGMENT_DURATION_SECONDS`、`INCREMENTAL_WINDOW_SECONDS`、`INCREMENTAL_SEGMENT_DURATION_SECONDS`、`ANNOTATION_FRAME_INTERVAL_SECONDS`、`SAMPLING_RULES_JSON`。`AUDIT_VIDEO` 可单独指定;未指定时复用 `MEDIUM_VIDEO`。受保护部署还需要 `DROBOTICFLOW_API_TOKEN`,或 `DROBOTICFLOW_ACCEPTANCE_EMAIL` / `DROBOTICFLOW_ACCEPTANCE_PASSWORD` 组织管理员登录凭据。
|
||
|
||
`SAMPLING_RULES_JSON` 必须是非空 JSON 数组。每条规则必须包含 `id`、`field`、`operator`、`value`、`samplePercent`,且 `samplePercent` 必须是有限数字,取值范围是 `(0, 100]`;布尔值不会被当作 `1` / `0` 接受。`operator` 支持 `eq`、`ne`、`gt`、`gte`、`lt`、`lte`、`contains`、`in`、`exists`、`missing`。这些规则会写入 `07-超大视频全链路`、`11-多目录多视频批量全链路`、`12-增量嗅探循环处理` 以及多项目入口的 `11B-人工抽检采样`,脚本不会替调用方选择默认阈值。
|
||
|
||
`QUALITY_MIN_CLARITY`、`QUALITY_MAX_SHAKE`、`QUALITY_MAX_HAND_OCCLUSION` 必须是 `[0, 1]` 范围内的有限归一化阈值,会写入所有 `视频质检` 节点以及带日期过滤的 `数据过滤` 节点。调用方需要根据目标真实数据集选择阈值;脚本不会使用 `0.0/1.0/1.0` 这类宽松默认值作为验收标准,也不会接受超出归一化指标范围或 `NaN/Inf` 这类非有限值作为生产验收证据。
|
||
|
||
窗口、切片和标注 timing 配置必须显式提供。`HUGE_WINDOW_SECONDS`、`MEDIUM_WINDOW_SECONDS`、`MEDIUM_SEGMENT_DURATION_SECONDS`、`HUGE_SEGMENT_DURATION_SECONDS`、`BATCH_WINDOW_SECONDS`、`BATCH_QUALITY_WINDOW_SECONDS`、`BATCH_SEGMENT_DURATION_SECONDS`、`INCREMENTAL_WINDOW_SECONDS`、`INCREMENTAL_SEGMENT_DURATION_SECONDS` 必须是 exact safe positive integer,拒绝符号、前导零、小数、单位和 unsafe integer;`ANNOTATION_FRAME_INTERVAL_SECONDS` 必须是大于 0 的有限数字。这些值会写入 `Shell 命令执行` 的窗口采样命令、`视频切片` 的 `SEGMENT_DURATION_SECONDS` 和 `视频标注` 的 `FRAME_INTERVAL_SECONDS`。
|
||
|
||
## 生产前置
|
||
|
||
必须满足下面 4 项,否则脚本会在 preflight 或 runtime acceptance 阶段直接失败:
|
||
|
||
1. API / worker 已部署为当前仓库版本
|
||
关键标志:`/api/settings` 返回里应包含 `k8s.workspacePVCName`。
|
||
|
||
2. 共享工作区 PVC 已配置
|
||
至少要有:
|
||
- `WORKSPACE_PVC_NAME`
|
||
- `WORKSPACE_MOUNT_PATH`
|
||
|
||
3. 已存在可用的 BOS storage connection
|
||
连接需要指向 `SOURCE_PATH` 和 `UPLOAD_ROOT_PREFIX` 使用的 bucket。若两个路径跨 bucket,需要准备能访问两个 bucket 的连接,或拆分为相同 bucket 下的输入输出前缀。
|
||
|
||
4. Robotics 数据路径对 operator-runner 可见
|
||
`LEROBOT_SOURCE_PATH` 和 `ROSBAG_SOURCE_PATH` 必须是 Kubernetes Job 容器内可读取的真实路径,例如共享 PVC 下的 `/workspace/datasets/...`。
|
||
|
||
## 推荐执行顺序
|
||
|
||
先做纯配置预检:
|
||
|
||
```bash
|
||
API_BASE_URL=http://<api-host>:30080 \
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_ID=<storage-connection-id> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||
```
|
||
|
||
预检通过后,再做带运行时烟测的正式回归:
|
||
|
||
```bash
|
||
API_BASE_URL=http://<api-host>:30080 \
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_ID=<storage-connection-id> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||
python3 scripts/wjzh_operator_regression.py --delete-regression-projects
|
||
```
|
||
|
||
如果确实要先清空所有项目,再重建回归项目,显式加:
|
||
|
||
```bash
|
||
--delete-all-projects
|
||
```
|
||
|
||
这个选项是破坏性的,默认不会启用。
|
||
|
||
## 连接配置方式
|
||
|
||
### 方式 1:使用已有 storage connection
|
||
|
||
推荐:
|
||
|
||
```bash
|
||
API_BASE_URL=http://<api-host>:30080 \
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_ID=<storage-connection-id> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||
```
|
||
|
||
也可以按名称查找:
|
||
|
||
```bash
|
||
API_BASE_URL=http://<api-host>:30080 \
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_NAME=<storage-connection-name> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||
```
|
||
|
||
### 方式 2:由脚本创建 storage connection
|
||
|
||
仅当环境里还没有现成连接时使用。`BOS_ENDPOINT` 可留空,由 API 运行时默认值解析;如果需要固定特定地域端点,再额外设置 `BOS_ENDPOINT=https://<region>.bcebos.com`。
|
||
|
||
```bash
|
||
API_BASE_URL=http://<api-host>:30080 \
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_NAME=<storage-connection-name> \
|
||
CREATE_CONNECTION_IF_MISSING=1 \
|
||
BOS_BUCKET=<bucket> \
|
||
BOS_ACCESS_KEY_ID=<ak> \
|
||
BOS_SECRET_ACCESS_KEY=<sk> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||
```
|
||
|
||
## 多项目回归入口
|
||
|
||
`scripts/wjzh_operator_multi_project_regression.py` 会把基类回归拆到多个项目中执行,并额外覆盖 `真实数据源` 的 `SOURCE_TEXT` / `SOURCE_JSON` 两个入口。两个入口不再使用仓库内置样本,必须由调用方提供真实 payload:
|
||
|
||
```bash
|
||
WJZH_REGRESSION_SOURCE_TEXT_FILE=/path/to/real-source-text.txt \
|
||
WJZH_REGRESSION_SOURCE_JSON_FILE=/path/to/real-source-payload.json \
|
||
API_BASE_URL=http://<api-host>:30080 \
|
||
SOURCE_PATH=bos://<bucket>/<input-prefix> \
|
||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix> \
|
||
STORAGE_CONNECTION_ID=<storage-connection-id> \
|
||
MEDIUM_VIDEO=<medium-video-filename> \
|
||
HUGE_VIDEO=<huge-video-filename> \
|
||
ROUTE_DEVICE_ID=<device-id-present-in-source> \
|
||
FILTER_DATE_START=<yyyy-mm-dd> \
|
||
FILTER_DATE_END=<yyyy-mm-dd> \
|
||
LEROBOT_SOURCE_PATH=/workspace/datasets/<real-lerobot-root> \
|
||
ROSBAG_SOURCE_PATH=/workspace/datasets/<real-rosbag-path> \
|
||
INCREMENTAL_PLACE=<real-place-label> \
|
||
INCREMENTAL_CAMERA_MODEL=<camera-model> \
|
||
INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||
QUALITY_MIN_CLARITY=<threshold> \
|
||
QUALITY_MAX_SHAKE=<threshold> \
|
||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||
HUGE_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
BATCH_WINDOW_SECONDS=<seconds> \
|
||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||
python3 scripts/wjzh_operator_multi_project_regression.py --preflight-only
|
||
```
|
||
|
||
如果不想把 payload 放在文件中,也可以改用 `WJZH_REGRESSION_SOURCE_TEXT` 和 `WJZH_REGRESSION_SOURCE_JSON`。同一个 payload 只能选择环境变量或文件路径其中一种方式。
|
||
|
||
## 鉴权配置
|
||
|
||
- `DROBOTICFLOW_API_TOKEN`:用于未强制 session auth 的 token-protected API。
|
||
- `DROBOTICFLOW_ACCEPTANCE_EMAIL` / `DROBOTICFLOW_ACCEPTANCE_PASSWORD`:用于公开 session-auth 环境。脚本会登录、保存 cookie,并在写请求中发送 CSRF header。
|
||
- 如果 API 同时要求共享 token 和 session 登录,可以同时设置两组变量。
|
||
|
||
## 脚本做了什么
|
||
|
||
1. 校验 `/api/settings` 是否满足当前版本的数据面要求。
|
||
2. 校验并解析 BOS storage connection。
|
||
3. 校验回归所需内置算子是否齐全。
|
||
4. 先创建一条最小 `connectionId` 烟测工作流:
|
||
`BOS 增量扫描 -> X 平台数据集索引 -> 采集目录校验 -> 结果上传`
|
||
5. 只有烟测通过后,才会继续创建完整真实数据回归工作流。
|
||
|
||
## 如果脚本失败
|
||
|
||
重点看两类错误:
|
||
|
||
1. `settings.k8s.workspacePVCName is missing`
|
||
说明线上还没部署到当前仓库这版,或者共享 PVC 没配好。
|
||
|
||
2. `runtime-acceptance-failed`
|
||
说明 API 里虽然已经存了 `connectionId`,但 worker / operator-runner 在真实运行时没有正确解析 BOS 连接,通常是线上部署版本落后于当前代码。
|
||
|
||
## 验收标准
|
||
|
||
1. 运行时烟测工作流成功。
|
||
2. 正式回归工作流全部 `completed`。
|
||
3. 工作流定义里不再保存明文 `BOS_ACCESS_KEY_ID` / `BOS_SECRET_ACCESS_KEY`。
|
||
4. 结果写入 `UPLOAD_ROOT_PREFIX` 下独立时间戳前缀。
|
||
5. 所有抽检节点的 `RULES_JSON` 与调用方传入的 `SAMPLING_RULES_JSON` 一致。
|