forked from longtao.wu/droboticflow
🧪 Require WJZH video timing
This commit is contained in:
parent
953341240b
commit
f30294b167
@ -71,7 +71,7 @@ Drobodata is a workflow orchestration platform for embodied AI and vision pipeli
|
||||
- `npm run reconcile:data -- <api-base-url>`: remove legacy demo data, reset settings, and bootstrap built-in operators through the real API; honors `DROBOTICFLOW_API_TOKEN` and optional session login env vars for protected deployments
|
||||
- `npm run acceptance:object-storage`: browser-driven object-storage acceptance against a configured live storage connection; requires `OBJECT_STORAGE_CONNECTION_ID`, `OBJECT_STORAGE_CONNECTION_NAME`, or `OBJECT_STORAGE_BUCKET`
|
||||
- `npm run acceptance:xplatform`: X-platform operator-chain acceptance against an explicit real dataset; requires `XPLATFORM_SOURCE_PATH`, explicit `XPLATFORM_RULES_JSON`, all `XPLATFORM_MIN_*` counts, quality/slicing thresholds, and a storage connection selector when the source or upload path is `bos://...`
|
||||
- `python3 scripts/wjzh_operator_regression.py --preflight-only`: preflight the production-style WJZH real-data operator regression against a live API; requires explicit `SOURCE_PATH`, `UPLOAD_ROOT_PREFIX`, storage connection selector, target video filenames, route/filter values, real Robotics dataset paths, incremental seed metadata, quality thresholds, and `SAMPLING_RULES_JSON`
|
||||
- `python3 scripts/wjzh_operator_regression.py --preflight-only`: preflight the production-style WJZH real-data operator regression against a live API; requires explicit `SOURCE_PATH`, `UPLOAD_ROOT_PREFIX`, storage connection selector, target video filenames, route/filter values, real Robotics dataset paths, incremental seed metadata, quality thresholds, video timing, and `SAMPLING_RULES_JSON`
|
||||
|
||||
Builtin operator metadata is declared once in `internal/api/builtin_operators_catalog.json`. The API embeds and validates that catalog before seeding `/api/operators`, while `scripts/generate_operator_docs.py` reads the same file for runtime handler hints. Runtime deployment fields such as the active operator-runner image are injected at startup or reconciliation time.
|
||||
|
||||
|
||||
@ -132,11 +132,21 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||||
```
|
||||
|
||||
The WJZH regression script does not provide built-in quality or sampling thresholds; `QUALITY_MIN_CLARITY`, `QUALITY_MAX_SHAKE`, `QUALITY_MAX_HAND_OCCLUSION`, and `SAMPLING_RULES_JSON` must match the target dataset before the run is accepted as evidence. See the full runbook in [`docs/wjzh-operator-regression.md`](wjzh-operator-regression.md).
|
||||
The WJZH regression script does not provide built-in quality thresholds, video timing, or sampling thresholds; `QUALITY_MIN_CLARITY`, `QUALITY_MAX_SHAKE`, `QUALITY_MAX_HAND_OCCLUSION`, the `*_SECONDS` timing values, and `SAMPLING_RULES_JSON` must match the target dataset before the run is accepted as evidence. See the full runbook in [`docs/wjzh-operator-regression.md`](wjzh-operator-regression.md).
|
||||
|
||||
Run browser object-storage acceptance against a live storage connection:
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@ This note tracks the production-facing safeguards that must stay true for Drobod
|
||||
- Explicit shell operators are disabled by default through `OPERATOR_ALLOW_SHELL_COMMANDS=false`; allowlist and denylist regexes are platform-controlled and cannot be overridden by workflow hyperparams.
|
||||
- Environment-specific values such as `K8S_ENDPOINT`, registry username, and workspace BOS bucket have no private repository defaults; provide them from local deploy secrets or CI secrets.
|
||||
- Data reconciliation uses real API calls and supports protected deployments through `DROBOTICFLOW_API_TOKEN` or session login credentials plus CSRF headers. Public session-auth environments must use an organization admin account for reconcile because it updates settings and management resources.
|
||||
- Python live-acceptance and WJZH regression scripts use the shared `DroboticflowClient` helper for API token headers, session-login cookies, CSRF propagation, JSON calls, and binary object-storage calls. WJZH quality thresholds are caller-supplied through `QUALITY_MIN_CLARITY`, `QUALITY_MAX_SHAKE`, and `QUALITY_MAX_HAND_OCCLUSION`; sampling thresholds are caller-supplied through `SAMPLING_RULES_JSON`. The scripts do not carry built-in audit rules. New Python acceptance scripts should reuse that helper instead of carrying their own auth/request stack.
|
||||
- Python live-acceptance and WJZH regression scripts use the shared `DroboticflowClient` helper for API token headers, session-login cookies, CSRF propagation, JSON calls, and binary object-storage calls. WJZH quality thresholds are caller-supplied through `QUALITY_MIN_CLARITY`, `QUALITY_MAX_SHAKE`, and `QUALITY_MAX_HAND_OCCLUSION`; window, slicing, and annotation timing are caller-supplied through the documented `*_SECONDS` values; sampling thresholds are caller-supplied through `SAMPLING_RULES_JSON`. The scripts do not carry built-in audit rules. New Python acceptance scripts should reuse that helper instead of carrying their own auth/request stack.
|
||||
- Node/Playwright acceptance scripts use the shared `createAcceptanceAPIClient` helper for bearer token headers, Playwright context CSRF propagation, JSON calls, retries, and optional session login. Functional and operator-preview checks also pass `DROBOTICFLOW_API_TOKEN` through browser contexts so protected UI requests exercise the same token-authenticated path as API setup calls.
|
||||
|
||||
## Kubernetes Jobs
|
||||
|
||||
@ -21,6 +21,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='[{"id":"clarity-gate","field":"quality.clarity","operator":"gte","value":0.3,"samplePercent":100}]' \
|
||||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||||
```
|
||||
@ -55,12 +65,14 @@ SOURCE_PATH=bos://<bucket>/<input-prefix>
|
||||
UPLOAD_ROOT_PREFIX=bos://<bucket>/<output-prefix>
|
||||
```
|
||||
|
||||
脚本不再内置任何真实 bucket、路径、连接名、视频文件名、设备号、日期范围、Robotics 数据路径、增量目录元数据、质检阈值或抽检规则。执行者必须按目标环境传入 `SOURCE_PATH`、`UPLOAD_ROOT_PREFIX`、`STORAGE_CONNECTION_ID` 或 `STORAGE_CONNECTION_NAME`、`MEDIUM_VIDEO`、`HUGE_VIDEO`、`ROUTE_DEVICE_ID`、`FILTER_DATE_START`、`FILTER_DATE_END`、`LEROBOT_SOURCE_PATH`、`ROSBAG_SOURCE_PATH`、`INCREMENTAL_PLACE`、`INCREMENTAL_CAMERA_MODEL`、`INCREMENTAL_VIDEO_SUBDIR`、`QUALITY_MIN_CLARITY`、`QUALITY_MAX_SHAKE`、`QUALITY_MAX_HAND_OCCLUSION`、`SAMPLING_RULES_JSON`。`AUDIT_VIDEO` 可单独指定;未指定时复用 `MEDIUM_VIDEO`。受保护部署还需要 `DROBOTICFLOW_API_TOKEN`,或 `DROBOTICFLOW_SMOKE_EMAIL` / `DROBOTICFLOW_SMOKE_PASSWORD` 组织管理员登录凭据。
|
||||
脚本不再内置任何真实 bucket、路径、连接名、视频文件名、设备号、日期范围、Robotics 数据路径、增量目录元数据、质检阈值、视频处理时长或抽检规则。执行者必须按目标环境传入 `SOURCE_PATH`、`UPLOAD_ROOT_PREFIX`、`STORAGE_CONNECTION_ID` 或 `STORAGE_CONNECTION_NAME`、`MEDIUM_VIDEO`、`HUGE_VIDEO`、`ROUTE_DEVICE_ID`、`FILTER_DATE_START`、`FILTER_DATE_END`、`LEROBOT_SOURCE_PATH`、`ROSBAG_SOURCE_PATH`、`INCREMENTAL_PLACE`、`INCREMENTAL_CAMERA_MODEL`、`INCREMENTAL_VIDEO_SUBDIR`、`QUALITY_MIN_CLARITY`、`QUALITY_MAX_SHAKE`、`QUALITY_MAX_HAND_OCCLUSION`、`HUGE_WINDOW_SECONDS`、`MEDIUM_WINDOW_SECONDS`、`MEDIUM_SEGMENT_DURATION_SECONDS`、`HUGE_SEGMENT_DURATION_SECONDS`、`BATCH_WINDOW_SECONDS`、`BATCH_QUALITY_WINDOW_SECONDS`、`BATCH_SEGMENT_DURATION_SECONDS`、`INCREMENTAL_WINDOW_SECONDS`、`INCREMENTAL_SEGMENT_DURATION_SECONDS`、`ANNOTATION_FRAME_INTERVAL_SECONDS`、`SAMPLING_RULES_JSON`。`AUDIT_VIDEO` 可单独指定;未指定时复用 `MEDIUM_VIDEO`。受保护部署还需要 `DROBOTICFLOW_API_TOKEN`,或 `DROBOTICFLOW_SMOKE_EMAIL` / `DROBOTICFLOW_SMOKE_PASSWORD` 组织管理员登录凭据。
|
||||
|
||||
`SAMPLING_RULES_JSON` 必须是非空 JSON 数组。每条规则必须包含 `id`、`field`、`operator`、`value`、`samplePercent`,且 `samplePercent` 取值范围是 `(0, 100]`。`operator` 支持 `eq`、`ne`、`gt`、`gte`、`lt`、`lte`、`contains`、`in`、`exists`、`missing`。这些规则会写入 `07-超大视频全链路`、`11-多目录多视频批量全链路`、`12-增量嗅探循环处理` 以及多项目入口的 `11B-人工抽检采样`,脚本不会替调用方选择默认阈值。
|
||||
|
||||
`QUALITY_MIN_CLARITY`、`QUALITY_MAX_SHAKE`、`QUALITY_MAX_HAND_OCCLUSION` 必须是非负数字,会写入所有 `视频质检` 节点以及带日期过滤的 `数据过滤` 节点。调用方需要根据目标真实数据集选择阈值;脚本不会使用 `0.0/1.0/1.0` 这类宽松默认值作为验收标准。
|
||||
|
||||
窗口、切片和标注 timing 配置必须显式提供。`HUGE_WINDOW_SECONDS`、`MEDIUM_WINDOW_SECONDS`、`MEDIUM_SEGMENT_DURATION_SECONDS`、`HUGE_SEGMENT_DURATION_SECONDS`、`BATCH_WINDOW_SECONDS`、`BATCH_QUALITY_WINDOW_SECONDS`、`BATCH_SEGMENT_DURATION_SECONDS`、`INCREMENTAL_WINDOW_SECONDS`、`INCREMENTAL_SEGMENT_DURATION_SECONDS` 必须是正整数;`ANNOTATION_FRAME_INTERVAL_SECONDS` 必须是大于 0 的数字。这些值会写入 `Shell 命令执行` 的窗口采样命令、`视频切片` 的 `SEGMENT_DURATION_SECONDS` 和 `视频标注` 的 `FRAME_INTERVAL_SECONDS`。
|
||||
|
||||
## 生产前置
|
||||
|
||||
必须满足下面 4 项,否则脚本会在 preflight 或 runtime smoke 阶段直接失败:
|
||||
@ -101,6 +113,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||||
```
|
||||
@ -125,6 +147,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_regression.py --delete-regression-projects
|
||||
```
|
||||
@ -161,6 +193,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||||
```
|
||||
@ -185,6 +227,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||||
```
|
||||
@ -216,6 +268,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_regression.py --preflight-only
|
||||
```
|
||||
@ -244,6 +306,16 @@ INCREMENTAL_VIDEO_SUBDIR=<video-subdir> \
|
||||
QUALITY_MIN_CLARITY=<threshold> \
|
||||
QUALITY_MAX_SHAKE=<threshold> \
|
||||
QUALITY_MAX_HAND_OCCLUSION=<threshold> \
|
||||
HUGE_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_WINDOW_SECONDS=<seconds> \
|
||||
MEDIUM_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
HUGE_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
BATCH_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_QUALITY_WINDOW_SECONDS=<seconds> \
|
||||
BATCH_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
INCREMENTAL_WINDOW_SECONDS=<seconds> \
|
||||
INCREMENTAL_SEGMENT_DURATION_SECONDS=<seconds> \
|
||||
ANNOTATION_FRAME_INTERVAL_SECONDS=<seconds> \
|
||||
SAMPLING_RULES_JSON='<sampling-rules-json-array>' \
|
||||
python3 scripts/wjzh_operator_multi_project_regression.py --preflight-only
|
||||
```
|
||||
|
||||
@ -251,7 +251,7 @@ def create_manual_sampling_workflow(
|
||||
"batch_pick",
|
||||
660,
|
||||
0,
|
||||
{"COMMAND": base.cap_duration_command(45, max_items=4, max_duration=500)},
|
||||
{"COMMAND": base.cap_duration_command(base.int_config(config.batch_window_seconds), max_items=4, max_duration=500)},
|
||||
label="人工抽检前抽样",
|
||||
),
|
||||
base.build_node(
|
||||
@ -266,14 +266,14 @@ def create_manual_sampling_workflow(
|
||||
"slice",
|
||||
1100,
|
||||
-60,
|
||||
{"SEGMENT_DURATION_SECONDS": "30"},
|
||||
base.segment_hyperparams(config.batch_segment_duration_seconds),
|
||||
),
|
||||
base.build_node(
|
||||
operators_by_name["视频标注"],
|
||||
"annotate",
|
||||
1320,
|
||||
-60,
|
||||
{"FRAME_INTERVAL_SECONDS": "1.0"},
|
||||
base.annotation_hyperparams(config),
|
||||
),
|
||||
base.build_node(operators_by_name["成品打包"], "package", 1540, -60),
|
||||
base.build_node(
|
||||
|
||||
@ -24,7 +24,6 @@ from lib.droboticflow_acceptance import DroboticflowClient
|
||||
|
||||
DEFAULT_API_BASE_URL = "http://127.0.0.1:3001"
|
||||
DEFAULT_PROJECT_PREFIX = "真实数据算子回归"
|
||||
DEFAULT_HUGE_WINDOW_SECONDS = 120
|
||||
CLIENT: DroboticflowClient | None = None
|
||||
|
||||
|
||||
@ -58,7 +57,7 @@ class Config:
|
||||
audit_video: str
|
||||
medium_video: str
|
||||
huge_video: str
|
||||
huge_window_seconds: int
|
||||
huge_window_seconds: str
|
||||
route_device_id: str
|
||||
filter_date_start: str
|
||||
filter_date_end: str
|
||||
@ -71,6 +70,15 @@ class Config:
|
||||
quality_min_clarity: str
|
||||
quality_max_shake: str
|
||||
quality_max_hand_occlusion: str
|
||||
medium_window_seconds: str
|
||||
medium_segment_duration_seconds: str
|
||||
huge_segment_duration_seconds: str
|
||||
batch_window_seconds: str
|
||||
batch_quality_window_seconds: str
|
||||
batch_segment_duration_seconds: str
|
||||
incremental_window_seconds: str
|
||||
incremental_segment_duration_seconds: str
|
||||
annotation_frame_interval_seconds: str
|
||||
|
||||
|
||||
def env_default(name: str, fallback: str = "") -> str:
|
||||
@ -145,6 +153,18 @@ def filter_hyperparams(config: Config) -> dict[str, str]:
|
||||
}
|
||||
|
||||
|
||||
def int_config(value: str) -> int:
|
||||
return int(str(value).strip())
|
||||
|
||||
|
||||
def segment_hyperparams(duration_seconds: str) -> dict[str, str]:
|
||||
return {"SEGMENT_DURATION_SECONDS": str(duration_seconds).strip()}
|
||||
|
||||
|
||||
def annotation_hyperparams(config: Config) -> dict[str, str]:
|
||||
return {"FRAME_INTERVAL_SECONDS": config.annotation_frame_interval_seconds}
|
||||
|
||||
|
||||
def validate_config(config: Config) -> None:
|
||||
issues: list[str] = []
|
||||
parsed_filter_start: time.struct_time | None = None
|
||||
@ -223,6 +243,37 @@ def validate_config(config: Config) -> None:
|
||||
continue
|
||||
if parsed_value < 0:
|
||||
issues.append(f"{label} must be greater than or equal to 0")
|
||||
for label, value in (
|
||||
("HUGE_WINDOW_SECONDS", config.huge_window_seconds),
|
||||
("MEDIUM_WINDOW_SECONDS", config.medium_window_seconds),
|
||||
("MEDIUM_SEGMENT_DURATION_SECONDS", config.medium_segment_duration_seconds),
|
||||
("HUGE_SEGMENT_DURATION_SECONDS", config.huge_segment_duration_seconds),
|
||||
("BATCH_WINDOW_SECONDS", config.batch_window_seconds),
|
||||
("BATCH_QUALITY_WINDOW_SECONDS", config.batch_quality_window_seconds),
|
||||
("BATCH_SEGMENT_DURATION_SECONDS", config.batch_segment_duration_seconds),
|
||||
("INCREMENTAL_WINDOW_SECONDS", config.incremental_window_seconds),
|
||||
("INCREMENTAL_SEGMENT_DURATION_SECONDS", config.incremental_segment_duration_seconds),
|
||||
):
|
||||
if not value:
|
||||
issues.append(f"{label} is required")
|
||||
continue
|
||||
try:
|
||||
parsed_value = int(value)
|
||||
except ValueError:
|
||||
issues.append(f"{label} must be an integer")
|
||||
continue
|
||||
if parsed_value <= 0:
|
||||
issues.append(f"{label} must be greater than 0")
|
||||
if not config.annotation_frame_interval_seconds:
|
||||
issues.append("ANNOTATION_FRAME_INTERVAL_SECONDS is required")
|
||||
else:
|
||||
try:
|
||||
annotation_interval = float(config.annotation_frame_interval_seconds)
|
||||
except ValueError:
|
||||
issues.append("ANNOTATION_FRAME_INTERVAL_SECONDS must be numeric")
|
||||
else:
|
||||
if annotation_interval <= 0:
|
||||
issues.append("ANNOTATION_FRAME_INTERVAL_SECONDS must be greater than 0")
|
||||
if issues:
|
||||
raise RegressionError(
|
||||
"invalid regression configuration:\n"
|
||||
@ -269,7 +320,7 @@ def parse_args() -> Config:
|
||||
parser.add_argument("--audit-video", default=env_default("AUDIT_VIDEO"))
|
||||
parser.add_argument("--medium-video", default=env_default("MEDIUM_VIDEO"))
|
||||
parser.add_argument("--huge-video", default=env_default("HUGE_VIDEO"))
|
||||
parser.add_argument("--huge-window-seconds", type=int, default=int(env_default("HUGE_WINDOW_SECONDS", str(DEFAULT_HUGE_WINDOW_SECONDS)) or DEFAULT_HUGE_WINDOW_SECONDS))
|
||||
parser.add_argument("--huge-window-seconds", default=env_default("HUGE_WINDOW_SECONDS"))
|
||||
parser.add_argument("--route-device-id", default=env_default("ROUTE_DEVICE_ID"))
|
||||
parser.add_argument("--filter-date-start", default=env_default("FILTER_DATE_START"))
|
||||
parser.add_argument("--filter-date-end", default=env_default("FILTER_DATE_END"))
|
||||
@ -282,6 +333,15 @@ def parse_args() -> Config:
|
||||
parser.add_argument("--quality-min-clarity", default=env_default("QUALITY_MIN_CLARITY"))
|
||||
parser.add_argument("--quality-max-shake", default=env_default("QUALITY_MAX_SHAKE"))
|
||||
parser.add_argument("--quality-max-hand-occlusion", default=env_default("QUALITY_MAX_HAND_OCCLUSION"))
|
||||
parser.add_argument("--medium-window-seconds", default=env_default("MEDIUM_WINDOW_SECONDS"))
|
||||
parser.add_argument("--medium-segment-duration-seconds", default=env_default("MEDIUM_SEGMENT_DURATION_SECONDS"))
|
||||
parser.add_argument("--huge-segment-duration-seconds", default=env_default("HUGE_SEGMENT_DURATION_SECONDS"))
|
||||
parser.add_argument("--batch-window-seconds", default=env_default("BATCH_WINDOW_SECONDS"))
|
||||
parser.add_argument("--batch-quality-window-seconds", default=env_default("BATCH_QUALITY_WINDOW_SECONDS"))
|
||||
parser.add_argument("--batch-segment-duration-seconds", default=env_default("BATCH_SEGMENT_DURATION_SECONDS"))
|
||||
parser.add_argument("--incremental-window-seconds", default=env_default("INCREMENTAL_WINDOW_SECONDS"))
|
||||
parser.add_argument("--incremental-segment-duration-seconds", default=env_default("INCREMENTAL_SEGMENT_DURATION_SECONDS"))
|
||||
parser.add_argument("--annotation-frame-interval-seconds", default=env_default("ANNOTATION_FRAME_INTERVAL_SECONDS"))
|
||||
args = parser.parse_args()
|
||||
return Config(
|
||||
api_base_url=args.api_base_url.rstrip("/"),
|
||||
@ -308,7 +368,7 @@ def parse_args() -> Config:
|
||||
audit_video=args.audit_video or args.medium_video,
|
||||
medium_video=args.medium_video,
|
||||
huge_video=args.huge_video,
|
||||
huge_window_seconds=max(args.huge_window_seconds, 1),
|
||||
huge_window_seconds=args.huge_window_seconds,
|
||||
route_device_id=args.route_device_id,
|
||||
filter_date_start=args.filter_date_start,
|
||||
filter_date_end=args.filter_date_end,
|
||||
@ -321,6 +381,15 @@ def parse_args() -> Config:
|
||||
quality_min_clarity=args.quality_min_clarity,
|
||||
quality_max_shake=args.quality_max_shake,
|
||||
quality_max_hand_occlusion=args.quality_max_hand_occlusion,
|
||||
medium_window_seconds=args.medium_window_seconds,
|
||||
medium_segment_duration_seconds=args.medium_segment_duration_seconds,
|
||||
huge_segment_duration_seconds=args.huge_segment_duration_seconds,
|
||||
batch_window_seconds=args.batch_window_seconds,
|
||||
batch_quality_window_seconds=args.batch_quality_window_seconds,
|
||||
batch_segment_duration_seconds=args.batch_segment_duration_seconds,
|
||||
incremental_window_seconds=args.incremental_window_seconds,
|
||||
incremental_segment_duration_seconds=args.incremental_segment_duration_seconds,
|
||||
annotation_frame_interval_seconds=args.annotation_frame_interval_seconds,
|
||||
)
|
||||
|
||||
|
||||
@ -1255,8 +1324,15 @@ def create_regression_workflows(
|
||||
build_node(validate_operator, "validate", 440, 0, {"REQUIRE_INTRINSICS": "false"}),
|
||||
build_node(json_filter_operator, "pick_medium", 660, 0, {"FIELD_PATH": "filename", "OPERATOR": "eq", "EXPECTED": config.medium_video}),
|
||||
build_node(operators_by_name["视频质检"], "quality", 880, -60, quality_hyperparams(config)),
|
||||
build_node(shell_operator, "window_medium", 1100, -60, {"COMMAND": cap_duration_command(45, max_items=1, max_duration=120)}, label="中视频窗口采样"),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1320, -60, {"SEGMENT_DURATION_SECONDS": "45"}),
|
||||
build_node(
|
||||
shell_operator,
|
||||
"window_medium",
|
||||
1100,
|
||||
-60,
|
||||
{"COMMAND": cap_duration_command(int_config(config.medium_window_seconds), max_items=1, max_duration=int_config(config.huge_window_seconds))},
|
||||
label="中视频窗口采样",
|
||||
),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1320, -60, segment_hyperparams(config.medium_segment_duration_seconds)),
|
||||
named_upload("quality_upload", 1540, -140, "06-medium-video/quality"),
|
||||
named_upload("slice_upload", 1540, 20, "06-medium-video/slice"),
|
||||
]
|
||||
@ -1289,9 +1365,9 @@ def create_regression_workflows(
|
||||
0,
|
||||
filter_hyperparams(config),
|
||||
),
|
||||
build_node(shell_operator, "window", 1320, 0, {"COMMAND": cap_duration_command(config.huge_window_seconds)}, label="大文件窗口采样"),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1540, 0, {"SEGMENT_DURATION_SECONDS": "120"}),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1760, 0, {"FRAME_INTERVAL_SECONDS": "1.0"}),
|
||||
build_node(shell_operator, "window", 1320, 0, {"COMMAND": cap_duration_command(int_config(config.huge_window_seconds))}, label="大文件窗口采样"),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1540, 0, segment_hyperparams(config.huge_segment_duration_seconds)),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1760, 0, annotation_hyperparams(config)),
|
||||
build_node(operators_by_name["成品打包"], "package", 1980, 0),
|
||||
build_node(
|
||||
operators_by_name["抽检采样"],
|
||||
@ -1338,10 +1414,10 @@ def create_regression_workflows(
|
||||
build_node(index_operator, "index", 220, 0),
|
||||
build_node(validate_operator, "validate", 440, 0, {"REQUIRE_INTRINSICS": "false"}),
|
||||
build_node(json_filter_operator, "pick_batch", 660, 0, {"FIELD_PATH": "duration", "OPERATOR": "lte", "EXPECTED_JSON": "400"}),
|
||||
build_node(shell_operator, "window_batch", 880, 0, {"COMMAND": cap_duration_command(45, max_items=4)}, label="批量窗口采样"),
|
||||
build_node(shell_operator, "window_batch", 880, 0, {"COMMAND": cap_duration_command(int_config(config.batch_window_seconds), max_items=4)}, label="批量窗口采样"),
|
||||
build_node(operators_by_name["视频质检"], "quality", 1100, 0, quality_hyperparams(config)),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1320, 0, {"SEGMENT_DURATION_SECONDS": "30"}),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1540, 0, {"FRAME_INTERVAL_SECONDS": "1.0"}),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1320, 0, segment_hyperparams(config.batch_segment_duration_seconds)),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1540, 0, annotation_hyperparams(config)),
|
||||
build_node(operators_by_name["成品打包"], "package", 1760, 0),
|
||||
named_upload("batch_package_upload", 1980, -60, "08-batch-windowed/package"),
|
||||
named_upload("batch_annotation_upload", 1980, 60, "08-batch-windowed/annotate"),
|
||||
@ -1384,7 +1460,14 @@ def create_regression_workflows(
|
||||
scan_source(),
|
||||
build_node(index_operator, "index", 220, 0),
|
||||
build_node(validate_operator, "validate", 440, 0, {"REQUIRE_INTRINSICS": "false"}),
|
||||
build_node(shell_operator, "batch_pick", 660, 0, {"COMMAND": cap_duration_command(90, max_items=8, max_duration=500)}, label="多目录多视频抽样"),
|
||||
build_node(
|
||||
shell_operator,
|
||||
"batch_pick",
|
||||
660,
|
||||
0,
|
||||
{"COMMAND": cap_duration_command(int_config(config.batch_quality_window_seconds), max_items=8, max_duration=500)},
|
||||
label="多目录多视频抽样",
|
||||
),
|
||||
build_node(operators_by_name["视频质检"], "quality", 900, 0, quality_hyperparams(config)),
|
||||
named_upload("batch_pick_upload", 1140, -60, "10-batch-quality/pick"),
|
||||
named_upload("quality_upload", 1140, 60, "10-batch-quality/quality"),
|
||||
@ -1408,11 +1491,11 @@ def create_regression_workflows(
|
||||
scan_source(),
|
||||
build_node(index_operator, "index", 220, 0),
|
||||
build_node(validate_operator, "validate", 440, 0, {"REQUIRE_INTRINSICS": "false"}),
|
||||
build_node(shell_operator, "batch_pick", 660, 0, {"COMMAND": cap_duration_command(45, max_items=6, max_duration=500)}, label="多目录全链路抽样"),
|
||||
build_node(shell_operator, "batch_pick", 660, 0, {"COMMAND": cap_duration_command(int_config(config.batch_window_seconds), max_items=6, max_duration=500)}, label="多目录全链路抽样"),
|
||||
build_node(operators_by_name["视频质检"], "quality", 900, -80, quality_hyperparams(config)),
|
||||
build_node(operators_by_name["数据过滤"], "filter", 1120, -80, filter_hyperparams(config)),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1340, -80, {"SEGMENT_DURATION_SECONDS": "30"}),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1560, -80, {"FRAME_INTERVAL_SECONDS": "1.0"}),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1340, -80, segment_hyperparams(config.batch_segment_duration_seconds)),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1560, -80, annotation_hyperparams(config)),
|
||||
build_node(operators_by_name["成品打包"], "package", 1780, -80),
|
||||
build_node(
|
||||
operators_by_name["抽检采样"],
|
||||
@ -1458,9 +1541,16 @@ def create_regression_workflows(
|
||||
build_node(validate_operator, "validate", 440, 0, {"REQUIRE_INTRINSICS": "false"}),
|
||||
build_node(operators_by_name["视频质检"], "quality", 660, -80, quality_hyperparams(config)),
|
||||
build_node(operators_by_name["数据过滤"], "filter", 880, -80, filter_hyperparams(config)),
|
||||
build_node(shell_operator, "window", 1100, -80, {"COMMAND": cap_duration_command(30, max_items=8, max_duration=120)}, label="增量视频窗口抽样"),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1320, -80, {"SEGMENT_DURATION_SECONDS": "30"}),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1540, -80, {"FRAME_INTERVAL_SECONDS": "1.0"}),
|
||||
build_node(
|
||||
shell_operator,
|
||||
"window",
|
||||
1100,
|
||||
-80,
|
||||
{"COMMAND": cap_duration_command(int_config(config.incremental_window_seconds), max_items=8, max_duration=int_config(config.huge_window_seconds))},
|
||||
label="增量视频窗口抽样",
|
||||
),
|
||||
build_node(operators_by_name["视频切片"], "slice", 1320, -80, segment_hyperparams(config.incremental_segment_duration_seconds)),
|
||||
build_node(operators_by_name["视频标注"], "annotate", 1540, -80, annotation_hyperparams(config)),
|
||||
build_node(operators_by_name["成品打包"], "package", 1760, -80),
|
||||
build_node(
|
||||
operators_by_name["抽检采样"],
|
||||
|
||||
@ -75,6 +75,15 @@ def valid_config(**overrides):
|
||||
"quality_min_clarity": "0.35",
|
||||
"quality_max_shake": "0.2",
|
||||
"quality_max_hand_occlusion": "0.25",
|
||||
"medium_window_seconds": "45",
|
||||
"medium_segment_duration_seconds": "45",
|
||||
"huge_segment_duration_seconds": "120",
|
||||
"batch_window_seconds": "45",
|
||||
"batch_quality_window_seconds": "90",
|
||||
"batch_segment_duration_seconds": "30",
|
||||
"incremental_window_seconds": "30",
|
||||
"incremental_segment_duration_seconds": "30",
|
||||
"annotation_frame_interval_seconds": "1.0",
|
||||
}
|
||||
values.update(overrides)
|
||||
return BASE.Config(**values)
|
||||
@ -107,6 +116,15 @@ class WJZHOperatorRegressionTest(unittest.TestCase):
|
||||
with self.assertRaisesRegex(BASE.RegressionError, "QUALITY_MAX_SHAKE"):
|
||||
BASE.validate_config(config)
|
||||
|
||||
def test_validate_config_requires_explicit_video_timing(self):
|
||||
config = valid_config(batch_segment_duration_seconds="", annotation_frame_interval_seconds="0")
|
||||
|
||||
with self.assertRaisesRegex(BASE.RegressionError, "BATCH_SEGMENT_DURATION_SECONDS"):
|
||||
BASE.validate_config(config)
|
||||
|
||||
with self.assertRaisesRegex(BASE.RegressionError, "ANNOTATION_FRAME_INTERVAL_SECONDS"):
|
||||
BASE.validate_config(config)
|
||||
|
||||
def test_normalize_sampling_rules_json_validates_rule_shape(self):
|
||||
with self.assertRaisesRegex(BASE.RegressionError, "samplePercent"):
|
||||
BASE.normalize_sampling_rules_json(
|
||||
@ -185,6 +203,12 @@ class WJZHOperatorRegressionTest(unittest.TestCase):
|
||||
self.assertEqual(quality_params["MIN_CLARITY"], "0.35")
|
||||
self.assertEqual(quality_params["MAX_SHAKE"], "0.2")
|
||||
self.assertEqual(quality_params["MAX_HAND_OCCLUSION"], "0.25")
|
||||
slice_node = next(node for node in created_payloads[0]["nodes"] if node["id"] == "slice")
|
||||
slice_params = {item["key"]: item["value"] for item in slice_node["data"]["hyperparams"]}
|
||||
self.assertEqual(slice_params["SEGMENT_DURATION_SECONDS"], "30")
|
||||
annotate_node = next(node for node in created_payloads[0]["nodes"] if node["id"] == "annotate")
|
||||
annotate_params = {item["key"]: item["value"] for item in annotate_node["data"]["hyperparams"]}
|
||||
self.assertEqual(annotate_params["FRAME_INTERVAL_SECONDS"], "1.0")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user