diff --git a/README.md b/README.md index 35e55142..aa12c8a5 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ Drobodata is a workflow orchestration platform for embodied AI and vision pipeli - `npm run acceptance:xplatform`: X-platform operator-chain acceptance against an explicit real dataset; requires `XPLATFORM_SOURCE_PATH`, explicit `XPLATFORM_RULES_JSON`, all `XPLATFORM_MIN_*` counts, quality/slicing thresholds, and a storage connection selector when the source or upload path is `bos://...` - `python3 scripts/wjzh_operator_regression.py --preflight-only`: preflight the production-style WJZH real-data operator regression against a live API; requires explicit `SOURCE_PATH`, `UPLOAD_ROOT_PREFIX`, storage connection selector, target video filenames, route/filter values, real Robotics dataset paths, incremental seed metadata, quality thresholds, video timing, and `SAMPLING_RULES_JSON` -Builtin operator metadata is declared once in `internal/api/builtin_operators_catalog.json`. The API embeds and validates that catalog before seeding `/api/operators`, while `scripts/generate_operator_docs.py` reads the same file for runtime handler hints. Runtime deployment fields such as the active operator-runner image are injected at startup or reconciliation time. +Builtin operator metadata is declared once in `internal/api/builtin_operators_catalog.json`. The API embeds and validates that catalog before seeding `/api/operators`, while `scripts/generate_operator_docs.py` defaults to that same local catalog and only contacts a live API when `--api-base-url` is provided. Runtime deployment fields such as the active operator-runner image are injected at startup or reconciliation time. ## Runtime Configuration diff --git a/docs/custom-operators/AI-生成到正式算子文档流程.md b/docs/custom-operators/AI-生成到正式算子文档流程.md index a80e23a0..1fec3c43 100644 --- a/docs/custom-operators/AI-生成到正式算子文档流程.md +++ b/docs/custom-operators/AI-生成到正式算子文档流程.md @@ -67,6 +67,8 @@ python3 scripts/generate_operator_docs.py --input-json /path/to/generated-operat - 纯数组:`[ {...}, {...} ]` - 单个 operator 对象:`{ ... }` +不传参数时,脚本只读取仓库内的 `internal/api/builtin_operators_catalog.json`;需要连接真实 API 时必须显式传 `--api-base-url http://:30080`。 + 对于单个 `custom` operator,生成结果会出现在: - `docs/custom-operators/instances/` diff --git a/docs/custom-operators/README.md b/docs/custom-operators/README.md index adc968c9..1e13f817 100644 --- a/docs/custom-operators/README.md +++ b/docs/custom-operators/README.md @@ -38,6 +38,12 @@ 生成命令示例: +```bash +python3 scripts/generate_operator_docs.py +``` + +默认输入是仓库内的 `internal/api/builtin_operators_catalog.json`,不会隐式连接本机 API。如果要从某个真实环境导出自定义算子实例,显式传入目标 API: + ```bash python3 scripts/generate_operator_docs.py --api-base-url http://:30080 ``` diff --git a/docs/operations.md b/docs/operations.md index 5f069823..e436e25b 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -245,7 +245,7 @@ export DROBOTICFLOW_SMOKE_PASSWORD= `scripts/reconcile-system-data.sh` is only the environment wrapper. The Python implementation reads the current builtin operator catalog from `/api/operators` and updates those live records to the requested `OPERATOR_RUNNER_IMAGE`; it does not carry a second static builtin-operator list. The cleanup step skips builtin operators even when their image tag is `:latest`, so old custom/demo operators can be removed without tombstoning platform-provided builtins. -The source catalog for platform-provided builtin operators is `internal/api/builtin_operators_catalog.json`. The API embeds this file, validates required fields and duplicate IDs/names at startup, and injects only runtime deployment fields such as the current operator-runner image and timestamps. `scripts/generate_operator_docs.py` reads the same file for runtime handler hints, so adding or changing a builtin operator should update the catalog first instead of editing separate Go and Python maps. +The source catalog for platform-provided builtin operators is `internal/api/builtin_operators_catalog.json`. The API embeds this file, validates required fields and duplicate IDs/names at startup, and injects only runtime deployment fields such as the current operator-runner image and timestamps. `scripts/generate_operator_docs.py` defaults to the same local file and contacts a live API only when `--api-base-url` is provided, so adding or changing a builtin operator should update the catalog first instead of editing separate Go and Python maps. ## Kubernetes Verification diff --git a/docs/production-readiness.md b/docs/production-readiness.md index 0f171717..06fa1364 100644 --- a/docs/production-readiness.md +++ b/docs/production-readiness.md @@ -116,7 +116,7 @@ K8S_ENDPOINT=https://:6443 ./scripts/reconcile-system-data.sh http:/ The reconcile wrapper delegates to `scripts/reconcile_system_data.py`. It removes legacy demo records, updates cluster settings through the real API, and derives builtin operators from the live `/api/operators` catalog before refreshing their runner image; it no longer maintains a separate hardcoded builtin-operator catalog. -Builtin operator metadata is declared in `internal/api/builtin_operators_catalog.json` and embedded by the API. Startup validation rejects malformed or duplicate catalog entries before they can drift into the database, and documentation generation reads the same catalog for runtime handler hints. +Builtin operator metadata is declared in `internal/api/builtin_operators_catalog.json` and embedded by the API. Startup validation rejects malformed or duplicate catalog entries before they can drift into the database, and documentation generation defaults to the same local catalog; live API export requires an explicit `--api-base-url`. Run GPU smoke on a real GPU cluster: diff --git a/scripts/generate_operator_docs.py b/scripts/generate_operator_docs.py index 2e2ef0bb..9a97d341 100644 --- a/scripts/generate_operator_docs.py +++ b/scripts/generate_operator_docs.py @@ -10,7 +10,6 @@ from pathlib import Path from typing import Any -DEFAULT_API_BASE_URL = "http://127.0.0.1:3001" DEFAULT_BUILTIN_OUTPUT_DIR = "docs/operators" DEFAULT_CUSTOM_OUTPUT_DIR = "docs/custom-operators/instances" DEFAULT_BUILTIN_CATALOG_PATH = "internal/api/builtin_operators_catalog.json" @@ -55,7 +54,7 @@ TRANSMISSION_HINTS = { def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser(description="Generate detailed markdown manuals for builtin and custom operators") - parser.add_argument("--api-base-url", default=DEFAULT_API_BASE_URL) + parser.add_argument("--api-base-url", default="", help="Fetch operator payload from a specific live API /api/operators endpoint") parser.add_argument("--input-json", default="", help="Read operator payload from a local JSON file instead of calling /api/operators") parser.add_argument("--builtin-output-dir", default=DEFAULT_BUILTIN_OUTPUT_DIR) parser.add_argument("--custom-output-dir", default=DEFAULT_CUSTOM_OUTPUT_DIR) @@ -67,12 +66,25 @@ def request_json(url: str) -> Any: return json.loads(resp.read().decode("utf-8")) -def load_operator_items(args: argparse.Namespace) -> list[dict[str, Any]]: +def load_operator_payload(args: argparse.Namespace) -> tuple[Any, str]: + if args.input_json and args.api_base_url: + raise SystemExit("set only one of --input-json or --api-base-url") if args.input_json: - payload = json.loads(Path(args.input_json).read_text(encoding="utf-8")) - else: + input_path = Path(args.input_json) + return json.loads(input_path.read_text(encoding="utf-8")), str(input_path) + if args.api_base_url: api_base_url = args.api_base_url.rstrip("/") - payload = request_json(f"{api_base_url}/api/operators") + return request_json(f"{api_base_url}/api/operators"), f"{api_base_url}/api/operators" + catalog_path = Path(DEFAULT_BUILTIN_CATALOG_PATH) + return json.loads(catalog_path.read_text(encoding="utf-8")), str(catalog_path) + + +def load_operator_items(args: argparse.Namespace) -> list[dict[str, Any]]: + payload, _ = load_operator_payload(args) + return operator_items_from_payload(payload) + + +def operator_items_from_payload(payload: Any) -> list[dict[str, Any]]: if isinstance(payload, dict) and isinstance(payload.get("items"), list): items = payload.get("items", []) elif isinstance(payload, list): @@ -584,7 +596,8 @@ def write_custom_docs(items: list[dict[str, Any]], output_dir: Path) -> int: def main() -> int: args = parse_args() - items = load_operator_items(args) + payload, source = load_operator_payload(args) + items = operator_items_from_payload(payload) builtin_items = choose_builtin_records(items) custom_items = choose_custom_records(items) builtin_count = write_builtin_docs(builtin_items, Path(args.builtin_output_dir)) @@ -597,6 +610,7 @@ def main() -> int: "customCount": custom_count, "builtinOutputDir": args.builtin_output_dir, "customOutputDir": args.custom_output_dir, + "source": source, }, ensure_ascii=False, ) diff --git a/tests/backend/operator_docs_generation_test.py b/tests/backend/operator_docs_generation_test.py index 43888cf5..6365d08a 100644 --- a/tests/backend/operator_docs_generation_test.py +++ b/tests/backend/operator_docs_generation_test.py @@ -2,6 +2,7 @@ import json import pathlib import sys import unittest +from argparse import Namespace ROOT = pathlib.Path(__file__).resolve().parents[2] @@ -24,6 +25,21 @@ class OperatorDocsGenerationTest(unittest.TestCase): ) self.assertEqual(sorted(hints), sorted(builtin_ids)) + def test_default_operator_payload_comes_from_local_catalog(self): + args = Namespace(input_json="", api_base_url="") + payload, source = generate_operator_docs.load_operator_payload(args) + items = generate_operator_docs.operator_items_from_payload(payload) + + self.assertEqual(source, "internal/api/builtin_operators_catalog.json") + self.assertGreater(len(items), 20) + self.assertTrue(any(item.get("name") == "视频质检" for item in items)) + + def test_operator_payload_source_must_be_explicitly_unique(self): + args = Namespace(input_json="operators.json", api_base_url="http://api.example.test") + + with self.assertRaisesRegex(SystemExit, "set only one"): + generate_operator_docs.load_operator_payload(args) + if __name__ == "__main__": unittest.main()