chore: establish Real2Sim platform baseline
This commit is contained in:
commit
ad41f0fe72
21
.gitignore
vendored
Normal file
21
.gitignore
vendored
Normal file
@ -0,0 +1,21 @@
|
||||
.DS_Store
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.log
|
||||
config.toml
|
||||
workspace/
|
||||
acceptance_runs/
|
||||
dist/
|
||||
build/
|
||||
*.egg-info/
|
||||
certs/
|
||||
secrets/
|
||||
models/
|
||||
checkpoints/
|
||||
|
||||
89
README.md
Normal file
89
README.md
Normal file
@ -0,0 +1,89 @@
|
||||
# Real2Sim Platform
|
||||
|
||||
管理视频/图片到 3D 场景重建、Scene Builder 场景、专家策略采集、数据集、训练和推理任务的单机平台。
|
||||
|
||||
## 本机启动
|
||||
|
||||
```bash
|
||||
cd /Users/d-robotics/workSpace/real2sim2model/real2sim-platform
|
||||
PYTHONPATH=src python -m real2sim_platform.server --host 127.0.0.1 --port 18126
|
||||
```
|
||||
|
||||
打开:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:18126/
|
||||
```
|
||||
|
||||
## 任务类型
|
||||
|
||||
- `hf_import`: 从 Hugging Face dataset 下载指定 revision / allow patterns,落到 `workspace/datasets/{dataset_id}` 并生成 `dataset_manifest.json`。
|
||||
- `media_prepare`: 把 HuggingFace parquet/video URL、本地 video、图片目录统一准备成 `prepared_media` dataset,默认按 `sample_fps=2`、`max_frames_per_video=80` 抽帧并生成 `prepared_manifest.json`。
|
||||
- `reconstruction_preflight`: 对 prepared frames / 图片目录做重建前检查,默认要求 `min_frames=30`,输出 `capture_preflight_report.json`。
|
||||
- `scene_reconstruction`: 视频/图片/RGBD 到 Real2Sim 场景包,默认走 `gs_playground/scripts/real2sim_video_motrix_scene.py`,支持 `mujoco` 和 `motrixsim` 产物。
|
||||
- `simulator_validation`: 按 `smoke`、`runtime_if_available` 或 `runtime_required` 校验场景包中的 MuJoCo / MotrixSim 产物,输出 `validation_report.json`。
|
||||
- `expert_collection`: 调用 `real2sim_expert.collect`,输出 `rollouts.jsonl`、`manifest.json` 和可视化帧。
|
||||
- `training`: 调用 `real2sim_train.train`,输出 `policy_head.json`、`model_card.json`。
|
||||
- `inference`: 调用 `real2sim_infer.run`,输出 `prediction_rollout.json` 和 `viewer.html`。
|
||||
- `renderer_benchmark`: 记录 Aholo Viewer 与 Spark 的集成选择,默认 Spark,Aholo 用作大规模 LOD 备选。
|
||||
|
||||
## 关键接口
|
||||
|
||||
- `GET /api/version`
|
||||
- `GET /api/datasets/{id}/manifest`
|
||||
- `GET /api/scenes/{id}/manifest`
|
||||
- `GET /api/models/{id}/card`
|
||||
- `GET /api/tasks/{id}/reports`
|
||||
- `GET /api/artifacts/{id}/content`
|
||||
|
||||
前端固定页面:总览、验收、数据、场景、3D 预览、采集、训练、推理、任务。
|
||||
3D 预览页读取 scene manifest,优先用 vendored `@sparkjsdev/spark` 加载 PLY/SPZ/SPLAT/KSPLAT/SOG,失败或不可用时明确标为 Three.js fallback;Spark 和 Three.js runtime 都随静态文件部署,不依赖浏览器访问外部 CDN。Manifest 会给资产补充 `size_bytes`、`autoload`、`viewer_hint`,超过默认阈值的大 PLY/点云不会自动加载,页面会提示手动点击加载。
|
||||
|
||||
## 重建运行时
|
||||
|
||||
无影默认运行时配置:
|
||||
|
||||
```text
|
||||
runtime_root = /opt/real2sim_paper_runtime
|
||||
anysplat_root = /opt/real2sim_paper_runtime/AnySplat
|
||||
anysplat_python = /opt/real2sim_paper_runtime/envs/anysplat/bin/python
|
||||
cuda_home = /usr/local/cuda-12.1
|
||||
```
|
||||
|
||||
`scene_reconstruction` 会向上游脚本显式传入 `--anysplat-root` 和
|
||||
`--anysplat-python`,并导出 `REAL2SIM_ANYSPLAT_ROOT`、
|
||||
`REAL2SIM_ANY_SPLAT_PYTHON`、`REAL2SIM_COLMAP_BIN` 等环境变量。
|
||||
`reconstruction_preflight` 报告包含 `readiness_score` 和 `next_action`,用于 UI 向导区分“可以重建”“需要补帧/重采”“运行时配置缺失”等状态。
|
||||
|
||||
## 无影部署
|
||||
|
||||
默认通过 SSH 隧道访问,不把服务直接暴露公网。部署脚本会同步整个
|
||||
`real2sim2model` 工作区到 `/opt/real2sim2model`,并保留远端
|
||||
`real2sim-platform/workspace/` 中的历史任务、数据集和模型。
|
||||
|
||||
```bash
|
||||
bash scripts/deploy_wuying.sh --restart
|
||||
ssh -i /Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519 \
|
||||
-L 18126:127.0.0.1:18126 root@39.106.101.95
|
||||
```
|
||||
|
||||
另开一个本地终端建立隧道:
|
||||
|
||||
```bash
|
||||
ssh -i /Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519 \
|
||||
-L 18126:127.0.0.1:18126 root@39.106.101.95
|
||||
```
|
||||
|
||||
无影侧服务地址:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:18126/
|
||||
```
|
||||
|
||||
部署脚本会安装 `huggingface_hub`、`pyarrow`、`pillow`、`imageio`、`opencv-python-headless`、`mujoco` 等依赖,写入 systemd unit,并检查 `ffmpeg`、Spark bundle、Three.js bundle 和 AnySplat runtime。`images23dgs` 缺失只作为 `/api/doctor` warning。
|
||||
|
||||
远端验收:
|
||||
|
||||
```bash
|
||||
bash scripts/acceptance_wuying.sh
|
||||
```
|
||||
46
docs/IMPLEMENTATION_DETAIL.md
Normal file
46
docs/IMPLEMENTATION_DETAIL.md
Normal file
@ -0,0 +1,46 @@
|
||||
# Real2Sim Platform 实现细案
|
||||
|
||||
## 目标闭环
|
||||
|
||||
1. 场景:上传或导入视频、图片、RGBD 数据,创建 `scene_reconstruction` 任务,产出 3DGS/点云/mesh/collider、`scene.json`、MuJoCo/MotrixSim 可用文件和预览。
|
||||
2. 数据:创建 `hf_import` 任务,从 Hugging Face dataset 下载小批量验收数据,生成 `dataset_manifest.json`。
|
||||
3. 准备:创建 `media_prepare` 任务,把 HF parquet/video URL、本地 video、图片目录统一抽帧成 `prepared_media` 数据集,输出 `prepared_manifest.json` 和每个视频的帧贡献。
|
||||
4. 预检:创建 `reconstruction_preflight` 任务,检查帧数、分辨率、帧间变化和 AnySplat/COLMAP/ffmpeg runtime,输出 `capture_preflight_report.json`。
|
||||
5. 校验:创建 `simulator_validation` 任务,按 `smoke` / `runtime_if_available` / `runtime_required` 生成 MuJoCo/MotrixSim `validation_report.json`。
|
||||
6. 采集:选择场景、机器人配置、任务配置和 episode/step 参数,创建 `expert_collection` 任务,产出专家数据集。
|
||||
7. 训练:选择采集数据集,创建 `training` 任务,产出 `policy_head.json`、`model_card.json`。
|
||||
8. 推理:选择场景和模型,创建 `inference` 任务,产出 `prediction_rollout.json` 和 `viewer.html`。
|
||||
|
||||
## 渲染器选择
|
||||
|
||||
- 默认:Spark。理由是它直接集成 Three.js,适合把 3DGS、Mesh、MuJoCo/MotrixSim proxy、轨迹和 UI overlay 放进同一个前端视图。平台随包部署 `@sparkjsdev/spark` 静态 ESM bundle。
|
||||
- 回退:Three.js fallback。用于 Spark bundle 不存在、Spark runtime 加载失败或只是普通点云预览时,界面会明确显示 `Three.js fallback`,不再冒充 Spark;Three.js ESM、controls 和 loaders 随静态文件部署。
|
||||
- 大资产策略:scene manifest 会在读取时补齐 `size_bytes`、`autoload` 和 `viewer_hint`。超过默认阈值的大 PLY/点云只显示手动加载入口,避免页面长时间停在 loading。
|
||||
- 备选:Aholo Viewer。理由是它强调高性能 3DGS 和 chunked streaming LOD,更适合大场景或移动端流式加载,但需要把资产转成 Aholo 最适合的 packed/LOD 格式。
|
||||
- 平台 API 保留 `renderer` 参数:`spark | aholo | auto`。首版 `auto` 在没有 Aholo LOD package 时等价于 `spark`,当 scene package 内存在 Aholo LOD 资产时切到 `aholo`。
|
||||
|
||||
## 仿真器支持
|
||||
|
||||
- `mujoco`:保留 `scene_mujoco.xml`、V-HACD/convex collision、MJCF smoke/validation artifact。
|
||||
- `motrixsim`:保留 `motrix_scene.json`、`motrix_import.json`、预览 mesh/pointcloud/3DGS manifest。
|
||||
- UI 中每个场景记录 `simulators: ["mujoco", "motrixsim"]`,任务不声称两者物理完全一致,分别展示各自产物和验证结果。
|
||||
|
||||
## API
|
||||
|
||||
- `POST /api/datasets/import-path`:导入服务器已有路径。
|
||||
- `POST /api/datasets/upload`:上传 zip/video/RGBD。
|
||||
- `POST /api/scenes/import`:导入已有 `scene.json`、`motrix_import.json` 或 `project_manifest.json`。
|
||||
- `POST /api/tasks`:统一创建任务,`type` 为 `hf_import`、`media_prepare`、`reconstruction_preflight`、`scene_reconstruction`、`simulator_validation`、`expert_collection`、`training`、`inference`、`renderer_benchmark`。
|
||||
- `GET /api/tasks/{id}/logs`、`GET /api/tasks/{id}/artifacts`、`GET /api/artifacts/{id}/content`:统一验收与下载。
|
||||
- `GET /api/tasks/{id}/reports`:读取结构化报告索引和可读失败原因。
|
||||
- `reconstruction_preflight` report 包含 `readiness_score` 和 `next_action`,任务详情页会直接展示失败门槛、关键指标和重采建议。
|
||||
- `GET /api/datasets/{id}/manifest`、`GET /api/scenes/{id}/manifest`、`GET /api/models/{id}/card`:数据、场景、模型治理信息。
|
||||
- `GET /api/version`:返回平台版本、构建时间、部署路径、功能开关和当前 renderer backend。
|
||||
|
||||
## 无影策略
|
||||
|
||||
- 无影作为重建、训练和部署主环境:`39.106.101.95`,root 登录,GPU、COLMAP、Real2Sim 和 gsplat Python 已验证。
|
||||
- 服务默认绑定 `127.0.0.1:18126`,通过 SSH 隧道访问,避免公网暴露上传和模型产物。
|
||||
- 缺失 Node 不影响后端运行;Spark bundle 在本地通过 `scripts/vendor_spark.sh` 生成,Three.js bundle 通过 `scripts/vendor_three.sh` 生成,二者都随静态文件部署。
|
||||
- 无影 `.venv` 显式安装 `huggingface_hub`、`pyarrow`、`pillow`、`imageio`、`opencv-python-headless`、`mujoco`,`/api/doctor` 检查 `ffmpeg`、Spark bundle、Three.js bundle、媒体依赖、MuJoCo Python、AnySplat root/python、CUDA 和核心路径。
|
||||
- `deploy_wuying.sh --restart` 写入 systemd unit 并重启服务;`acceptance_wuying.sh` 通过远端 API 跑 preflight、非 dry-run 重建和仿真校验。
|
||||
8
docs/prototypes/README.md
Normal file
8
docs/prototypes/README.md
Normal file
@ -0,0 +1,8 @@
|
||||
# Real2Sim product prototypes
|
||||
|
||||
The prototype board in this directory is the visual contract for the POC
|
||||
productization branch. It covers login, the seven product pages, the task
|
||||
drawer, and the key mobile layouts.
|
||||
|
||||
Implementation screenshots are stored under acceptance output and are not
|
||||
committed to Git.
|
||||
BIN
docs/prototypes/real2sim-poc-product-board-v1.png
Normal file
BIN
docs/prototypes/real2sim-poc-product-board-v1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.7 MiB |
38
pyproject.toml
Normal file
38
pyproject.toml
Normal file
@ -0,0 +1,38 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "real2sim-platform"
|
||||
version = "0.1.0"
|
||||
description = "Real2Sim scene, collection, training, and inference management platform"
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"fastapi>=0.110",
|
||||
"huggingface_hub>=0.24",
|
||||
"imageio>=2.34",
|
||||
"opencv-python-headless>=4.9",
|
||||
"pillow>=10",
|
||||
"pyarrow>=15",
|
||||
"mujoco>=3.2",
|
||||
"pydantic>=2",
|
||||
"python-multipart>=0.0.9",
|
||||
"uvicorn[standard]>=0.27",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
real2sim-platform = "real2sim_platform.server:main"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
"real2sim_platform" = [
|
||||
"static/*",
|
||||
"static/vendor/*",
|
||||
"static/vendor/three/*",
|
||||
"static/vendor/three/addons/controls/*",
|
||||
"static/vendor/three/addons/loaders/*",
|
||||
"static/vendor/three/addons/postprocessing/*",
|
||||
"static/vendor/three/addons/utils/*",
|
||||
]
|
||||
BIN
qa/frontend/lively-workbench-concept.png
Normal file
BIN
qa/frontend/lively-workbench-concept.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
BIN
qa/frontend/workbench-concept.png
Normal file
BIN
qa/frontend/workbench-concept.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
74
scripts/acceptance_wuying.sh
Executable file
74
scripts/acceptance_wuying.sh
Executable file
@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
|
||||
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
|
||||
FIXTURE="${REAL2SIM_ACCEPTANCE_FIXTURE:-/opt/real2sim_paper_runtime/sam-3d-objects/notebook/images/shutterstock_modern_colorful_Interior_2620125197}"
|
||||
MIN_FRAMES="${REAL2SIM_ACCEPTANCE_MIN_FRAMES:-30}"
|
||||
TIMEOUT_SECONDS="${REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS:-3600}"
|
||||
|
||||
ssh -i "$KEY" -o BatchMode=yes -o StrictHostKeyChecking=accept-new "$HOST" \
|
||||
"REAL2SIM_ACCEPTANCE_FIXTURE='$FIXTURE' REAL2SIM_ACCEPTANCE_MIN_FRAMES='$MIN_FRAMES' REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS='$TIMEOUT_SECONDS' python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE = 'http://127.0.0.1:18126'
|
||||
FIXTURE = os.environ['REAL2SIM_ACCEPTANCE_FIXTURE']
|
||||
MIN_FRAMES = int(os.environ['REAL2SIM_ACCEPTANCE_MIN_FRAMES'])
|
||||
TIMEOUT = int(os.environ['REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS'])
|
||||
|
||||
|
||||
def api(path, payload=None):
|
||||
data = None
|
||||
headers = {}
|
||||
if payload is not None:
|
||||
data = json.dumps(payload).encode('utf-8')
|
||||
headers['Content-Type'] = 'application/json'
|
||||
req = urllib.request.Request(BASE + path, data=data, headers=headers)
|
||||
with urllib.request.urlopen(req, timeout=30) as response:
|
||||
return json.loads(response.read().decode('utf-8'))
|
||||
|
||||
|
||||
def create_task(task_type, params):
|
||||
task = api('/api/tasks', {'type': task_type, 'params': params})
|
||||
print(f'{task_type}: {task[\"id\"]}', flush=True)
|
||||
return wait_task(task['id'])
|
||||
|
||||
|
||||
def wait_task(task_id):
|
||||
deadline = time.time() + TIMEOUT
|
||||
while time.time() < deadline:
|
||||
task = api(f'/api/tasks/{task_id}')
|
||||
if task['status'] in {'succeeded', 'failed', 'canceled', 'failed_recovered'}:
|
||||
print(json.dumps({'task_id': task_id, 'type': task['type'], 'status': task['status'], 'result': task.get('result'), 'error': task.get('error')}, ensure_ascii=False), flush=True)
|
||||
return task
|
||||
time.sleep(5)
|
||||
raise SystemExit(f'timeout waiting for {task_id}')
|
||||
|
||||
|
||||
doctor = api('/api/doctor')
|
||||
version = api('/api/version')
|
||||
print(json.dumps({'doctor_ok': doctor['ok'], 'warnings': doctor.get('warnings'), 'version': version.get('version'), 'build_id': version.get('build_id')}, ensure_ascii=False), flush=True)
|
||||
|
||||
preflight = create_task('reconstruction_preflight', {'input_path': FIXTURE, 'min_frames': MIN_FRAMES})
|
||||
if not preflight.get('result', {}).get('ok'):
|
||||
raise SystemExit('preflight failed; inspect reports for task ' + preflight['id'])
|
||||
|
||||
reconstruction = create_task('scene_reconstruction', {
|
||||
'input_path': FIXTURE,
|
||||
'dry_run': False,
|
||||
'renderer': 'spark',
|
||||
'simulators': ['mujoco', 'motrixsim'],
|
||||
'min_frames': MIN_FRAMES,
|
||||
'quality_preset': 'capture-high',
|
||||
})
|
||||
if reconstruction['status'] != 'succeeded':
|
||||
raise SystemExit('scene_reconstruction failed; inspect reports for task ' + reconstruction['id'])
|
||||
|
||||
scene_id = reconstruction['result']['scene_id']
|
||||
validation = create_task('simulator_validation', {'scene_id': scene_id, 'simulators': ['mujoco', 'motrixsim'], 'level': 'runtime_if_available'})
|
||||
print(json.dumps({'scene_id': scene_id, 'validation_task': validation['id'], 'validation_ok': validation.get('result', {}).get('ok')}, ensure_ascii=False), flush=True)
|
||||
PY"
|
||||
89
scripts/convert_3dgs_ply_to_spz.mjs
Normal file
89
scripts/convert_3dgs_ply_to_spz.mjs
Normal file
@ -0,0 +1,89 @@
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
function parseArgs(argv) {
|
||||
const args = {};
|
||||
for (let index = 2; index < argv.length; index += 1) {
|
||||
const item = argv[index];
|
||||
if (!item.startsWith("--")) continue;
|
||||
const key = item.slice(2);
|
||||
const next = argv[index + 1];
|
||||
if (!next || next.startsWith("--")) args[key] = true;
|
||||
else {
|
||||
args[key] = next;
|
||||
index += 1;
|
||||
}
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
async function ensureSparkImport(vendorRoot) {
|
||||
const tmpRoot = path.join("/tmp", `real2sim-spark-node-${process.pid}`);
|
||||
const threeRoot = path.join(tmpRoot, "node_modules", "three");
|
||||
await fs.mkdir(path.join(threeRoot, "addons", "postprocessing"), { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(threeRoot, "package.json"),
|
||||
JSON.stringify({ name: "three", type: "module", main: "./three.module.js", exports: { ".": "./three.module.js", "./addons/*": "./addons/*" } }),
|
||||
);
|
||||
await fs.copyFile(path.join(vendorRoot, "three", "three.module.js"), path.join(threeRoot, "three.module.js"));
|
||||
await fs.copyFile(path.join(vendorRoot, "three", "three.core.js"), path.join(threeRoot, "three.core.js"));
|
||||
await fs.copyFile(path.join(vendorRoot, "three", "addons", "postprocessing", "Pass.js"), path.join(threeRoot, "addons", "postprocessing", "Pass.js"));
|
||||
await fs.copyFile(path.join(vendorRoot, "spark.module.min.js"), path.join(tmpRoot, "spark.module.min.js"));
|
||||
return import(pathToFileURL(path.join(tmpRoot, "spark.module.min.js")).href);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = parseArgs(process.argv);
|
||||
const input = args.input;
|
||||
const output = args.output;
|
||||
const vendorRoot = args["vendor-root"];
|
||||
const reportPath = args.report;
|
||||
if (!input || !output || !vendorRoot || !reportPath) {
|
||||
throw new Error("usage: node convert_3dgs_ply_to_spz.mjs --input scene.ply --output scene.spz --vendor-root static/vendor --report report.json");
|
||||
}
|
||||
|
||||
const started = Date.now();
|
||||
const spark = await ensureSparkImport(vendorRoot);
|
||||
const fileBytes = new Uint8Array(await fs.readFile(input));
|
||||
const reader = new spark.PlyReader({ fileBytes });
|
||||
await reader.parseHeader();
|
||||
await reader.parseData(() => {});
|
||||
const numSplats = Number(reader.numSplats || 0);
|
||||
if (!numSplats) throw new Error("Spark PlyReader returned zero splats");
|
||||
|
||||
const writer = new spark.SpzWriter({ numSplats, shDegree: 0 });
|
||||
let written = 0;
|
||||
reader.parseSplats((index, x, y, z, scaleX, scaleY, scaleZ, quatX, quatY, quatZ, quatW, opacity, red, green, blue) => {
|
||||
writer.setCenter(index, x, y, z);
|
||||
writer.setScale(index, Math.max(scaleX, 1e-6), Math.max(scaleY, 1e-6), Math.max(scaleZ, 1e-6));
|
||||
writer.setQuat(index, quatX, quatY, quatZ, quatW);
|
||||
writer.setAlpha(index, Math.max(0, Math.min(1, Number(opacity ?? 1))));
|
||||
writer.setRgb(index, Math.max(0, Math.min(1, red)), Math.max(0, Math.min(1, green)), Math.max(0, Math.min(1, blue)));
|
||||
written += 1;
|
||||
});
|
||||
const encoded = await writer.finalize();
|
||||
await fs.mkdir(path.dirname(output), { recursive: true });
|
||||
await fs.writeFile(output, encoded);
|
||||
const inputStat = await fs.stat(input);
|
||||
const outputStat = await fs.stat(output);
|
||||
const report = {
|
||||
schema: "real2sim.spz_export_report.v1",
|
||||
input,
|
||||
output,
|
||||
splats: numSplats,
|
||||
written_splats: written,
|
||||
input_size_bytes: inputStat.size,
|
||||
output_size_bytes: outputStat.size,
|
||||
compression_ratio: Number((outputStat.size / Math.max(inputStat.size, 1)).toFixed(4)),
|
||||
elapsed_ms: Date.now() - started,
|
||||
clipped_count: writer.clippedCount || 0,
|
||||
};
|
||||
await fs.writeFile(reportPath, `${JSON.stringify(report, null, 2)}\n`);
|
||||
console.log(JSON.stringify(report));
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error(error?.stack || error?.message || String(error));
|
||||
process.exit(1);
|
||||
});
|
||||
127
scripts/deploy_wuying.sh
Executable file
127
scripts/deploy_wuying.sh
Executable file
@ -0,0 +1,127 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
PROJECT_ROOT="$(cd "$ROOT/.." && pwd)"
|
||||
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
|
||||
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
|
||||
SSH_CONNECT_TIMEOUT="${REAL2SIM_SSH_CONNECT_TIMEOUT:-20}"
|
||||
REMOTE_BASE="${REAL2SIM_REMOTE_BASE:-/opt/real2sim2model}"
|
||||
REMOTE="$REMOTE_BASE/real2sim-platform"
|
||||
SERVICE="real2sim-platform.service"
|
||||
BUILD_TIME="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
BUILD_ID="${REAL2SIM_PLATFORM_BUILD_ID:-$BUILD_TIME}"
|
||||
RUNTIME_ROOT="${REAL2SIM_RUNTIME_ROOT:-/opt/real2sim_paper_runtime}"
|
||||
ANYSPLAT_ROOT="${REAL2SIM_ANYSPLAT_ROOT:-$RUNTIME_ROOT/AnySplat}"
|
||||
ANYSPLAT_PYTHON="${REAL2SIM_ANY_SPLAT_PYTHON:-$RUNTIME_ROOT/envs/anysplat/bin/python}"
|
||||
CUDA_HOME_REMOTE="${CUDA_HOME_REMOTE:-/usr/local/cuda-12.1}"
|
||||
RESTART=0
|
||||
STATUS_ONLY=0
|
||||
SETUP_DESKTOP=0
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--restart) RESTART=1 ;;
|
||||
--status) STATUS_ONLY=1 ;;
|
||||
--desktop-launcher) SETUP_DESKTOP=1 ;;
|
||||
*) echo "unknown argument: $arg" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
ssh_opts=(-i "$KEY" -o BatchMode=yes -o ConnectTimeout="$SSH_CONNECT_TIMEOUT" -o ServerAliveInterval=10 -o ServerAliveCountMax=2 -o StrictHostKeyChecking=accept-new)
|
||||
ssh_cmd=(ssh "${ssh_opts[@]}" "$HOST")
|
||||
|
||||
if [[ "$STATUS_ONLY" == "1" ]]; then
|
||||
"${ssh_cmd[@]}" "systemctl status '$SERVICE' --no-pager || true; curl -fsS http://127.0.0.1:18126/api/health || true; echo; curl -fsS http://127.0.0.1:18126/api/doctor || true"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ ! -f "$ROOT/src/real2sim_platform/static/vendor/spark.module.min.js" ]]; then
|
||||
bash "$ROOT/scripts/vendor_spark.sh"
|
||||
fi
|
||||
if [[ ! -f "$ROOT/src/real2sim_platform/static/vendor/three/three.module.js" ]]; then
|
||||
bash "$ROOT/scripts/vendor_three.sh"
|
||||
fi
|
||||
|
||||
"${ssh_cmd[@]}" "mkdir -p '$REMOTE_BASE'"
|
||||
rsync -az --delete \
|
||||
--exclude '.git/' \
|
||||
--exclude 'real2sim-platform/workspace/' \
|
||||
--exclude 'real2sim-platform/.venv/' \
|
||||
--exclude '.pytest_cache/' \
|
||||
--exclude '__pycache__/' \
|
||||
-e "ssh -i $KEY -o BatchMode=yes -o ConnectTimeout=$SSH_CONNECT_TIMEOUT -o ServerAliveInterval=10 -o ServerAliveCountMax=2 -o StrictHostKeyChecking=accept-new" \
|
||||
"$PROJECT_ROOT/" "$HOST:$REMOTE_BASE/"
|
||||
|
||||
"${ssh_cmd[@]}" "cd '$REMOTE' && python3 -m venv .venv && .venv/bin/python -m pip install -q --upgrade pip && .venv/bin/python -m pip install -q -e ."
|
||||
"${ssh_cmd[@]}" "cd '$REMOTE' && .venv/bin/python - <<'PY'
|
||||
import importlib.util
|
||||
required = ['huggingface_hub', 'pyarrow', 'PIL', 'imageio', 'cv2', 'mujoco']
|
||||
missing = [name for name in required if importlib.util.find_spec(name) is None]
|
||||
if missing:
|
||||
raise SystemExit('missing runtime deps: ' + ', '.join(missing))
|
||||
PY
|
||||
command -v ffmpeg >/dev/null
|
||||
test -f '$REMOTE/src/real2sim_platform/static/vendor/spark.module.min.js'
|
||||
test -f '$REMOTE/src/real2sim_platform/static/vendor/three/three.module.js'
|
||||
test -d '$ANYSPLAT_ROOT'
|
||||
test -x '$ANYSPLAT_PYTHON'"
|
||||
|
||||
"${ssh_cmd[@]}" "cat > '$REMOTE/config.toml' <<'EOF'
|
||||
app_root = \"$REMOTE\"
|
||||
workspace_dir = \"$REMOTE/workspace\"
|
||||
host = \"127.0.0.1\"
|
||||
port = 18126
|
||||
default_renderer = \"spark\"
|
||||
real2sim2model_root = \"$REMOTE_BASE\"
|
||||
expert_policy_root = \"$REMOTE_BASE/real2sim-expert-policy\"
|
||||
training_root = \"$REMOTE_BASE/real2sim-policy-training\"
|
||||
inference_root = \"$REMOTE_BASE/real2sim-policy-inference\"
|
||||
discoverse_root = \"/opt/gs_playground_real2sim_48q/DISCOVERSE\"
|
||||
gs_playground_root = \"/opt/gs_playground_real2sim_48q\"
|
||||
images23dgs_root = \"/opt/images23dgs_app/src\"
|
||||
colmap_binary = \"/usr/local/bin/colmap\"
|
||||
gsplat_python = \"$ANYSPLAT_PYTHON\"
|
||||
runtime_root = \"$RUNTIME_ROOT\"
|
||||
anysplat_root = \"$ANYSPLAT_ROOT\"
|
||||
anysplat_python = \"$ANYSPLAT_PYTHON\"
|
||||
cuda_home = \"$CUDA_HOME_REMOTE\"
|
||||
runtime_env_script = \"/opt/gs_playground_real2sim_48q/scripts/real2sim_wuying_paper_env.sh\"
|
||||
allowed_path_roots = \"$REMOTE_BASE:$REMOTE/workspace:/opt/gs_playground_real2sim_48q:$RUNTIME_ROOT:$ANYSPLAT_ROOT:/tmp\"
|
||||
EOF"
|
||||
|
||||
"${ssh_cmd[@]}" "cat > /etc/systemd/system/$SERVICE <<'EOF'
|
||||
[Unit]
|
||||
Description=Real2Sim Platform
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=$REMOTE
|
||||
Environment=REAL2SIM_PLATFORM_CONFIG=$REMOTE/config.toml
|
||||
Environment=REAL2SIM_PLATFORM_BUILD_TIME=$BUILD_TIME
|
||||
Environment=REAL2SIM_PLATFORM_BUILD_ID=$BUILD_ID
|
||||
Environment=MUJOCO_GL=egl
|
||||
ExecStart=$REMOTE/.venv/bin/real2sim-platform
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
systemctl daemon-reload
|
||||
systemctl enable '$SERVICE' >/dev/null"
|
||||
|
||||
if [[ "$RESTART" == "1" ]]; then
|
||||
"${ssh_cmd[@]}" "systemctl stop '$SERVICE' || true; port_pid=\$(ss -ltnp 2>/dev/null | sed -n 's/.*127\\.0\\.0\\.1:18126.*pid=\\([0-9][0-9]*\\).*/\\1/p' | head -1); if [ -n \"\$port_pid\" ]; then kill \"\$port_pid\" || true; fi; sleep 2; systemctl restart '$SERVICE'; sleep 2; systemctl status '$SERVICE' --no-pager --lines=20"
|
||||
fi
|
||||
|
||||
if [[ "$SETUP_DESKTOP" == "1" ]]; then
|
||||
"${ssh_cmd[@]}" "bash '$REMOTE/scripts/setup_wuying_desktop_launcher.sh' --local --project-root '$REMOTE' --service '$SERVICE' --port 18126"
|
||||
fi
|
||||
|
||||
echo "deployed to $HOST:$REMOTE"
|
||||
echo "service: $SERVICE"
|
||||
echo "health: ssh -i $KEY -L 18126:127.0.0.1:18126 $HOST"
|
||||
echo "restart: bash scripts/deploy_wuying.sh --restart"
|
||||
echo "desktop launcher: bash scripts/deploy_wuying.sh --restart --desktop-launcher"
|
||||
303
scripts/frontend_acceptance_local.mjs
Executable file
303
scripts/frontend_acceptance_local.mjs
Executable file
@ -0,0 +1,303 @@
|
||||
#!/usr/bin/env node
|
||||
import { spawn } from "node:child_process";
|
||||
import { createRequire } from "node:module";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
const key = process.env.REAL2SIM_WUYING_KEY || "/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519";
|
||||
const host = process.env.REAL2SIM_WUYING_HOST || "root@39.106.101.95";
|
||||
const remotePort = Number(process.env.REAL2SIM_REMOTE_PORT || "18126");
|
||||
const requestedBase = process.env.REAL2SIM_FRONTEND_BASE_URL || "";
|
||||
const outRoot = process.env.REAL2SIM_ACCEPTANCE_OUT || path.resolve("real2sim-platform", "workspace", "acceptance_runs", "frontend-local");
|
||||
const timestamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..*/, "Z");
|
||||
const outDir = path.join(outRoot, timestamp);
|
||||
const reportPath = path.join(outDir, "frontend_acceptance_report.json");
|
||||
const screenshots = [];
|
||||
const failures = [];
|
||||
const warnings = [];
|
||||
const consoleMessages = [];
|
||||
const pageChecks = [];
|
||||
let tunnel = null;
|
||||
|
||||
fs.mkdirSync(outDir, { recursive: true });
|
||||
|
||||
function fail(message, context = {}) {
|
||||
failures.push({ message, ...context });
|
||||
console.error("FAIL:", message, context);
|
||||
}
|
||||
|
||||
function saveReport(extra = {}) {
|
||||
const report = {
|
||||
schema: "real2sim.frontend_acceptance_report.v1",
|
||||
started_at: timestamp,
|
||||
finished_at: new Date().toISOString(),
|
||||
base_url: extra.baseUrl || requestedBase,
|
||||
screenshots,
|
||||
console_messages: consoleMessages,
|
||||
warnings,
|
||||
page_checks: pageChecks,
|
||||
failures,
|
||||
pass: failures.length === 0,
|
||||
...extra,
|
||||
};
|
||||
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
|
||||
return report;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { chromium } = await importPlaywright();
|
||||
const baseUrl = requestedBase || await startTunnel();
|
||||
const browser = await chromium.launch({ headless: true });
|
||||
try {
|
||||
const page = await browser.newPage({ viewport: { width: 1440, height: 900 }, deviceScaleFactor: 1 });
|
||||
page.on("console", (message) => {
|
||||
const entry = { type: message.type(), text: message.text() };
|
||||
consoleMessages.push(entry);
|
||||
if (message.type() === "error") fail("browser console error", entry);
|
||||
});
|
||||
page.on("pageerror", (error) => fail("browser page error", { error: error.message }));
|
||||
|
||||
await page.goto(baseUrl, { waitUntil: "domcontentloaded", timeout: 60000 });
|
||||
await page.waitForSelector(".app-shell", { timeout: 30000 });
|
||||
await neutralizeEmbeddedViewers(page);
|
||||
await expectText(page, "Real2Sim", "brand");
|
||||
await expectAbsentText(page, "验收", "old acceptance nav");
|
||||
await expectAbsentText(page, "3D预览", "old viewer nav");
|
||||
await expectAbsentText(page, "workspace", "old workspace footer");
|
||||
await expectAbsentText(page, "打开云桌面", "old cloud desktop button");
|
||||
|
||||
const pages = [
|
||||
{ tab: "overview", title: "总览", shot: "desktop-overview.png", selectors: ["#journeySteps", "#recentTaskList"] },
|
||||
{ tab: "data", title: "数据管理", shot: "desktop-data.png", selectors: ["#datasetList", "#dataTaskList"] },
|
||||
{ tab: "reconstruct", title: "场景重建", shot: "desktop-reconstruct.png", selectors: ["#reconstructInput", "#sceneTaskList"] },
|
||||
{ tab: "scenes", title: "场景管理", shot: "", selectors: ["#sceneList", "#scenePreview"] },
|
||||
{ tab: "collect", title: "数据采集", shot: "desktop-collect.png", selectors: ["#collectScene", "#collectTaskList"] },
|
||||
{ tab: "train", title: "模型训练", shot: "desktop-train.png", selectors: ["#trainDataset", "#trainTaskList", "#modelList"] },
|
||||
{ tab: "infer", title: "在线推理", shot: "desktop-infer.png", selectors: ["#inferModel", "#inferScene", "#inferTaskList"] },
|
||||
];
|
||||
for (const item of pages) {
|
||||
await openTab(page, item.tab);
|
||||
await expectText(page, item.title, `${item.tab} title`);
|
||||
for (const selector of item.selectors) await expectSelector(page, selector, `${item.tab} ${selector}`);
|
||||
await assertNoOverflow(page, `desktop ${item.tab}`);
|
||||
if (item.shot) await screenshot(page, item.shot, item.shotOptions || {});
|
||||
else warnings.push(`${item.tab} screenshot skipped; page contains embedded WebGL iframe that can hang headless screenshot capture`);
|
||||
pageChecks.push({ tab: item.tab, viewport: "desktop", ok: true });
|
||||
}
|
||||
|
||||
await openTab(page, "reconstruct");
|
||||
const taskCount = await page.locator("#sceneTaskList [data-task]").count();
|
||||
if (taskCount === 0) {
|
||||
warnings.push("no scene tasks visible for drawer/report validation");
|
||||
} else {
|
||||
await page.locator("#sceneTaskList [data-task]").first().click({ timeout: 10000 });
|
||||
await page.waitForTimeout(1500);
|
||||
await expectSelector(page, "#taskDrawer[aria-hidden=\"false\"]", "task drawer opened");
|
||||
await expectSelector(page, "#drawerBody", "task drawer body");
|
||||
await screenshot(page, "desktop-task-drawer.png");
|
||||
}
|
||||
|
||||
await validateSceneViewerManifestGate(baseUrl);
|
||||
warnings.push("live scene viewer probe is excluded from the main frontend gate; SPZ manifest/autoload is validated here and live WebGL should be checked with the dedicated viewer probe.");
|
||||
|
||||
await page.setViewportSize({ width: 390, height: 900 });
|
||||
await neutralizeEmbeddedViewers(page);
|
||||
for (const item of [pages[0], pages[1], pages[2], pages[3]]) {
|
||||
await openTab(page, item.tab);
|
||||
await expectText(page, item.title, `mobile ${item.tab} title`);
|
||||
await assertNoOverflow(page, `mobile ${item.tab}`);
|
||||
if (item.shot) await screenshot(page, item.shot.replace("desktop", "mobile"), item.shotOptions || {});
|
||||
else warnings.push(`mobile ${item.tab} screenshot skipped; page contains embedded WebGL iframe that can hang headless screenshot capture`);
|
||||
pageChecks.push({ tab: item.tab, viewport: "mobile", ok: true });
|
||||
}
|
||||
|
||||
const report = saveReport({ baseUrl });
|
||||
console.log(JSON.stringify({ pass: report.pass, report: reportPath, screenshots }, null, 2));
|
||||
process.exitCode = report.pass ? 0 : 1;
|
||||
} finally {
|
||||
await browser.close();
|
||||
if (tunnel) tunnel.kill("SIGTERM");
|
||||
}
|
||||
}
|
||||
|
||||
async function importPlaywright() {
|
||||
const moduleDir = process.env.REAL2SIM_PLAYWRIGHT_NODE_MODULES || "";
|
||||
if (moduleDir) {
|
||||
const require = createRequire(path.join(moduleDir, "real2sim-playwright-loader.cjs"));
|
||||
return require("playwright");
|
||||
}
|
||||
try {
|
||||
return await import("playwright");
|
||||
} catch (error) {
|
||||
throw new Error(`playwright is required. Install it locally with: npm install playwright, or set REAL2SIM_PLAYWRIGHT_NODE_MODULES=/path/to/node_modules. Original error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function startTunnel() {
|
||||
const port = Number(process.env.REAL2SIM_LOCAL_TUNNEL_PORT || await findPort(18130));
|
||||
tunnel = spawn("ssh", [
|
||||
"-i", key,
|
||||
"-o", "BatchMode=yes",
|
||||
"-o", "StrictHostKeyChecking=accept-new",
|
||||
"-N",
|
||||
"-L", `${port}:127.0.0.1:${remotePort}`,
|
||||
host,
|
||||
], { stdio: ["ignore", "pipe", "pipe"] });
|
||||
tunnel.stderr.on("data", (chunk) => process.stderr.write(chunk));
|
||||
tunnel.stdout.on("data", (chunk) => process.stdout.write(chunk));
|
||||
const baseUrl = `http://127.0.0.1:${port}`;
|
||||
const deadline = Date.now() + 30000;
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
await fetchJson(baseUrl, "/api/health");
|
||||
return baseUrl;
|
||||
} catch {
|
||||
await sleep(500);
|
||||
}
|
||||
}
|
||||
throw new Error(`SSH tunnel did not become ready at ${baseUrl}`);
|
||||
}
|
||||
|
||||
async function findPort(start) {
|
||||
const net = await import("node:net");
|
||||
for (let port = start; port < start + 100; port += 1) {
|
||||
const ok = await new Promise((resolve) => {
|
||||
const server = net.createServer();
|
||||
server.once("error", () => resolve(false));
|
||||
server.once("listening", () => server.close(() => resolve(true)));
|
||||
server.listen(port, "127.0.0.1");
|
||||
});
|
||||
if (ok) return port;
|
||||
}
|
||||
throw new Error("no local tunnel port available");
|
||||
}
|
||||
|
||||
async function fetchJson(baseUrl, apiPath) {
|
||||
const response = await fetch(`${baseUrl}${apiPath}`);
|
||||
if (!response.ok) throw new Error(`${apiPath} failed with ${response.status}: ${await response.text()}`);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async function openTab(page, tab) {
|
||||
await page.evaluate((targetTab) => {
|
||||
const button = document.querySelector(`[data-tab="${targetTab}"]`);
|
||||
if (!button) throw new Error(`tab not found: ${targetTab}`);
|
||||
button.click();
|
||||
}, tab);
|
||||
await page.waitForFunction((targetTab) => document.getElementById(targetTab)?.classList.contains("active"), tab, { timeout: 10000 });
|
||||
}
|
||||
|
||||
async function neutralizeEmbeddedViewers(page) {
|
||||
await page.evaluate(() => {
|
||||
document.querySelectorAll("iframe").forEach((iframe) => {
|
||||
iframe.dataset.acceptanceSrc = iframe.getAttribute("src") || "";
|
||||
iframe.setAttribute("src", "about:blank");
|
||||
});
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
async function assertNoOverflow(page, label) {
|
||||
const metrics = await page.evaluate(() => ({
|
||||
scrollWidth: document.documentElement.scrollWidth,
|
||||
clientWidth: document.documentElement.clientWidth,
|
||||
bodyScrollWidth: document.body.scrollWidth,
|
||||
bodyClientWidth: document.body.clientWidth,
|
||||
}));
|
||||
if (metrics.scrollWidth > metrics.clientWidth + 2 || metrics.bodyScrollWidth > metrics.bodyClientWidth + 2) {
|
||||
fail("horizontal overflow", { label, metrics });
|
||||
}
|
||||
}
|
||||
|
||||
async function expectText(page, text, label) {
|
||||
const visible = await page.getByText(text, { exact: false }).first().isVisible().catch(() => false);
|
||||
if (!visible) fail("expected text not visible", { label, text });
|
||||
}
|
||||
|
||||
async function expectAbsentText(page, text, label) {
|
||||
const visible = await page.getByText(text, { exact: false }).first().isVisible().catch(() => false);
|
||||
if (visible) fail("legacy/debug text should not be visible", { label, text });
|
||||
}
|
||||
|
||||
async function expectSelector(page, selector, label) {
|
||||
const count = await page.locator(selector).count().catch(() => 0);
|
||||
if (count <= 0) fail("expected selector not found", { label, selector });
|
||||
}
|
||||
|
||||
async function pickPreviewableScene(baseUrl) {
|
||||
const scenes = await fetchJson(baseUrl, "/api/scenes");
|
||||
if (!Array.isArray(scenes) || scenes.length === 0) return null;
|
||||
const scored = [];
|
||||
for (const scene of scenes) {
|
||||
try {
|
||||
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(scene.id)}/manifest`);
|
||||
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
|
||||
let score = 0;
|
||||
if (assets.some((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")))) score += 100;
|
||||
if (assets.some((asset) => /pose_init_trained_scene\.ply$/i.test(String(asset.name || asset.path || "")))) score += 50;
|
||||
if (assets.some((asset) => /\.(ply|glb|gltf|obj)$/i.test(String(asset.name || asset.path || "")))) score += 20;
|
||||
scored.push({ scene, score });
|
||||
} catch (error) {
|
||||
warnings.push(`scene manifest unavailable for ${scene.id}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
scored.sort((left, right) => right.score - left.score);
|
||||
return scored[0]?.scene || scenes[0];
|
||||
}
|
||||
|
||||
async function validateSceneViewerManifestGate(baseUrl) {
|
||||
const scene = await pickPreviewableScene(baseUrl);
|
||||
if (!scene) {
|
||||
fail("no scenes available for 3D viewer manifest validation");
|
||||
return;
|
||||
}
|
||||
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(scene.id)}/manifest`);
|
||||
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
|
||||
const optimized = assets.find((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")));
|
||||
if (!optimized) {
|
||||
fail("scene manifest has no optimized splat asset for browser preview", { scene_id: scene.id });
|
||||
return;
|
||||
}
|
||||
if (!optimized.url) fail("optimized splat asset has no URL", { scene_id: scene.id, asset: optimized });
|
||||
if (optimized.autoload === false) fail("optimized splat asset is not marked autoload", { scene_id: scene.id, asset: optimized });
|
||||
pageChecks.push({ tab: "scene-viewer", viewport: "desktop", ok: true, asset: optimized.name || optimized.path, size: optimized.size_label || optimized.size_bytes || "" });
|
||||
}
|
||||
|
||||
async function screenshot(page, name, options = {}) {
|
||||
const target = path.join(outDir, name);
|
||||
const timeoutMs = Number(options.timeoutMs || 20000);
|
||||
let styleHandle = null;
|
||||
try {
|
||||
if (options.hideIframes) {
|
||||
styleHandle = await page.evaluateHandle(() => {
|
||||
const style = document.createElement("style");
|
||||
style.dataset.acceptanceScreenshot = "true";
|
||||
style.textContent = "iframe, canvas { visibility: hidden !important; }";
|
||||
document.head.appendChild(style);
|
||||
return style;
|
||||
});
|
||||
await page.waitForTimeout(150);
|
||||
}
|
||||
await page.screenshot({ path: target, fullPage: options.fullPage ?? true, timeout: timeoutMs });
|
||||
screenshots.push(target);
|
||||
} catch (error) {
|
||||
const message = `screenshot skipped for ${name}: ${error.message}`;
|
||||
warnings.push(message);
|
||||
} finally {
|
||||
if (styleHandle) {
|
||||
await page.evaluate(() => document.querySelector("[data-acceptance-screenshot='true']")?.remove()).catch(() => {});
|
||||
await styleHandle.dispose().catch(() => {});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function sleep(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
fail("frontend acceptance runner crashed", { error: error.message, stack: error.stack });
|
||||
const report = saveReport();
|
||||
console.error(JSON.stringify({ pass: false, report: reportPath, error: error.message }, null, 2));
|
||||
if (tunnel) tunnel.kill("SIGTERM");
|
||||
process.exit(1);
|
||||
});
|
||||
254
scripts/generate_motrixsim_preview.py
Executable file
254
scripts/generate_motrixsim_preview.py
Executable file
@ -0,0 +1,254 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
import time
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import trimesh
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
APP_ROOT = SCRIPT_DIR.parent
|
||||
SRC_ROOT = APP_ROOT / "src"
|
||||
if str(SRC_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(SRC_ROOT))
|
||||
|
||||
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
|
||||
from real2sim_platform.store import PlatformStore # noqa: E402
|
||||
|
||||
|
||||
SCENE_ID = "1f2618bdfcac4e45456569980716405a"
|
||||
PACKAGE_SLUG = "parlour_rm65_pick_place"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Generate a browser GLB preview from the MotrixSim parlour + RM65 scaffold package.")
|
||||
parser.add_argument("--scene-id", default=SCENE_ID)
|
||||
parser.add_argument("--max-parlour-meshes", type=int, default=240, help="Limit parlour meshes to keep the browser preview responsive.")
|
||||
parser.add_argument("--skip-rm65", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
config = load_config()
|
||||
ensure_workspace(config)
|
||||
store = PlatformStore(config.db_path)
|
||||
scene = store.get_scene(args.scene_id)
|
||||
package_dir = Path(str((scene.get("metadata") or {}).get("package") or config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG))
|
||||
manifest_path = Path(str(scene.get("manifest_path") or package_dir / "motrixsim_scene_manifest.json"))
|
||||
if not manifest_path.is_file():
|
||||
raise SystemExit(f"manifest not found: {manifest_path}")
|
||||
|
||||
output_dir = package_dir / "viewer_assets"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
output_path = output_dir / "motrixsim_parlour_rm65_preview.glb"
|
||||
report_path = output_dir / "motrixsim_preview_report.json"
|
||||
|
||||
scene_glb = trimesh.Scene()
|
||||
report: dict[str, Any] = {
|
||||
"schema": "real2sim.motrixsim_preview_report.v1",
|
||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"inputs": [],
|
||||
"warnings": [],
|
||||
}
|
||||
|
||||
parlour_xml = package_dir / "assets/parlour/parlour.xml"
|
||||
parlour_stats = add_mjcf_visuals(scene_glb, parlour_xml, max_meshes=args.max_parlour_meshes, name_prefix="parlour")
|
||||
report["inputs"].append({"role": "parlour", "xml": str(parlour_xml), **parlour_stats})
|
||||
|
||||
if not args.skip_rm65:
|
||||
rm65_xml = package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml"
|
||||
rm65_base = translation_matrix(np.array([0.8, 0.0, 0.55]))
|
||||
rm65_stats = add_mjcf_visuals(scene_glb, rm65_xml, root_transform=rm65_base, max_meshes=80, name_prefix="rm65")
|
||||
report["inputs"].append({"role": "rm65", "xml": str(rm65_xml), **rm65_stats})
|
||||
|
||||
if not scene_glb.geometry:
|
||||
raise SystemExit("no mesh geometry was loaded from MotrixSim package")
|
||||
|
||||
data = scene_glb.export(file_type="glb")
|
||||
output_path.write_bytes(data)
|
||||
report["output"] = {"path": str(output_path), "size_bytes": output_path.stat().st_size, "geometry_count": len(scene_glb.geometry)}
|
||||
report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
artifact = store.add_artifact(task_id=None, name=output_path.name, path=output_path, kind="mesh")
|
||||
report_artifact = store.add_artifact(task_id=None, name=report_path.name, path=report_path, kind="json")
|
||||
update_manifest(manifest_path, output_path, artifact["id"], report_path, report_artifact["id"], report)
|
||||
print(json.dumps({"ok": True, "scene_id": args.scene_id, "glb": str(output_path), "report": str(report_path), "geometry_count": len(scene_glb.geometry), "size_bytes": output_path.stat().st_size}, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def add_mjcf_visuals(
|
||||
scene: trimesh.Scene,
|
||||
xml_path: Path,
|
||||
*,
|
||||
root_transform: np.ndarray | None = None,
|
||||
max_meshes: int,
|
||||
name_prefix: str,
|
||||
) -> dict[str, Any]:
|
||||
root = ET.parse(xml_path).getroot()
|
||||
asset_dir = xml_path.parent / (root.find("compiler").attrib.get("assetdir", "") if root.find("compiler") is not None else "")
|
||||
mesh_defs = {
|
||||
item.attrib["name"]: asset_dir / item.attrib["file"]
|
||||
for item in root.findall("./asset/mesh")
|
||||
if item.attrib.get("name") and item.attrib.get("file")
|
||||
}
|
||||
stats = {"mesh_defs": len(mesh_defs), "loaded": 0, "missing": 0, "skipped": 0}
|
||||
world = root.find("worldbody")
|
||||
if world is None:
|
||||
return stats
|
||||
root_matrix = root_transform if root_transform is not None else np.eye(4)
|
||||
for node in list(world):
|
||||
traverse_mjcf_node(scene, node, mesh_defs, root_matrix, stats, max_meshes=max_meshes, name_prefix=name_prefix)
|
||||
return stats
|
||||
|
||||
|
||||
def traverse_mjcf_node(
|
||||
scene: trimesh.Scene,
|
||||
node: ET.Element,
|
||||
mesh_defs: dict[str, Path],
|
||||
parent_transform: np.ndarray,
|
||||
stats: dict[str, int],
|
||||
*,
|
||||
max_meshes: int,
|
||||
name_prefix: str,
|
||||
) -> None:
|
||||
current = parent_transform
|
||||
if node.tag == "body":
|
||||
current = parent_transform @ node_transform(node.attrib)
|
||||
elif node.tag == "geom":
|
||||
mesh_name = node.attrib.get("mesh")
|
||||
geom_class = node.attrib.get("class", "")
|
||||
if mesh_name and mesh_name in mesh_defs and (geom_class != "collision" or node.attrib.get("group") == "2"):
|
||||
if stats["loaded"] >= max_meshes:
|
||||
stats["skipped"] += 1
|
||||
else:
|
||||
mesh_path = mesh_defs[mesh_name]
|
||||
if mesh_path.is_file():
|
||||
try:
|
||||
mesh = load_mesh(mesh_path)
|
||||
mesh.apply_transform(parent_transform @ node_transform(node.attrib))
|
||||
scene.add_geometry(mesh, geom_name=f"{name_prefix}_{mesh_name}_{stats['loaded']}")
|
||||
stats["loaded"] += 1
|
||||
except Exception:
|
||||
stats["skipped"] += 1
|
||||
else:
|
||||
stats["missing"] += 1
|
||||
for child in list(node):
|
||||
traverse_mjcf_node(scene, child, mesh_defs, current, stats, max_meshes=max_meshes, name_prefix=name_prefix)
|
||||
|
||||
|
||||
def load_mesh(path: Path) -> trimesh.Trimesh:
|
||||
loaded = trimesh.load(path, force="mesh", process=False)
|
||||
if isinstance(loaded, trimesh.Scene):
|
||||
return trimesh.util.concatenate(tuple(loaded.geometry.values()))
|
||||
return loaded
|
||||
|
||||
|
||||
def node_transform(attrs: dict[str, str]) -> np.ndarray:
|
||||
transform = np.eye(4)
|
||||
if "pos" in attrs:
|
||||
transform = transform @ translation_matrix(parse_vec(attrs["pos"], 3))
|
||||
if "quat" in attrs:
|
||||
transform = transform @ quat_matrix(parse_vec(attrs["quat"], 4))
|
||||
elif "euler" in attrs:
|
||||
transform = transform @ euler_matrix(parse_vec(attrs["euler"], 3))
|
||||
if "scale" in attrs:
|
||||
scale = parse_vec(attrs["scale"], 3)
|
||||
transform = transform @ np.diag([scale[0], scale[1], scale[2], 1.0])
|
||||
return transform
|
||||
|
||||
|
||||
def parse_vec(value: str, count: int) -> np.ndarray:
|
||||
parts = [float(item) for item in value.split()]
|
||||
if len(parts) == 1 and count == 3:
|
||||
parts = parts * 3
|
||||
while len(parts) < count:
|
||||
parts.append(0.0)
|
||||
return np.array(parts[:count], dtype=float)
|
||||
|
||||
|
||||
def translation_matrix(vec: np.ndarray) -> np.ndarray:
|
||||
matrix = np.eye(4)
|
||||
matrix[:3, 3] = vec[:3]
|
||||
return matrix
|
||||
|
||||
|
||||
def quat_matrix(q: np.ndarray) -> np.ndarray:
|
||||
w, x, y, z = q
|
||||
norm = math.sqrt(w * w + x * x + y * y + z * z)
|
||||
if norm <= 0:
|
||||
return np.eye(4)
|
||||
w, x, y, z = w / norm, x / norm, y / norm, z / norm
|
||||
matrix = np.eye(4)
|
||||
matrix[:3, :3] = np.array(
|
||||
[
|
||||
[1 - 2 * y * y - 2 * z * z, 2 * x * y - 2 * z * w, 2 * x * z + 2 * y * w],
|
||||
[2 * x * y + 2 * z * w, 1 - 2 * x * x - 2 * z * z, 2 * y * z - 2 * x * w],
|
||||
[2 * x * z - 2 * y * w, 2 * y * z + 2 * x * w, 1 - 2 * x * x - 2 * y * y],
|
||||
]
|
||||
)
|
||||
return matrix
|
||||
|
||||
|
||||
def euler_matrix(euler: np.ndarray) -> np.ndarray:
|
||||
x, y, z = euler
|
||||
cx, cy, cz = math.cos(x), math.cos(y), math.cos(z)
|
||||
sx, sy, sz = math.sin(x), math.sin(y), math.sin(z)
|
||||
rx = np.array([[1, 0, 0, 0], [0, cx, -sx, 0], [0, sx, cx, 0], [0, 0, 0, 1]])
|
||||
ry = np.array([[cy, 0, sy, 0], [0, 1, 0, 0], [-sy, 0, cy, 0], [0, 0, 0, 1]])
|
||||
rz = np.array([[cz, -sz, 0, 0], [sz, cz, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]])
|
||||
return rz @ ry @ rx
|
||||
|
||||
|
||||
def update_manifest(manifest_path: Path, glb_path: Path, artifact_id: str, report_path: Path, report_artifact_id: str, report: dict[str, Any]) -> None:
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and item.get("name") == glb_path.name)]
|
||||
glb_asset = {
|
||||
"name": glb_path.name,
|
||||
"role": "viewer_preview",
|
||||
"kind": "mesh",
|
||||
"path": str(glb_path),
|
||||
"relative_path": str(glb_path.relative_to(manifest_path.parent)),
|
||||
"url": f"/api/artifacts/{artifact_id}/content",
|
||||
"exists": True,
|
||||
"size_bytes": glb_path.stat().st_size,
|
||||
"size_label": format_bytes(glb_path.stat().st_size),
|
||||
"autoload": True,
|
||||
"generated": True,
|
||||
}
|
||||
report_asset = {
|
||||
"name": report_path.name,
|
||||
"role": "preview_report",
|
||||
"kind": "metadata",
|
||||
"path": str(report_path),
|
||||
"relative_path": str(report_path.relative_to(manifest_path.parent)),
|
||||
"url": f"/api/artifacts/{report_artifact_id}/content",
|
||||
"exists": True,
|
||||
"size_bytes": report_path.stat().st_size,
|
||||
"size_label": format_bytes(report_path.stat().st_size),
|
||||
"generated": True,
|
||||
}
|
||||
manifest["assets"] = [glb_asset, report_asset, *assets]
|
||||
manifest["viewer_assets"] = {"primary": glb_asset, "report": report_asset, "generator": "generate_motrixsim_preview.py"}
|
||||
manifest["preview_generation"] = report
|
||||
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def format_bytes(value: int) -> str:
|
||||
units = ["B", "KB", "MB", "GB"]
|
||||
size = float(value)
|
||||
for unit in units:
|
||||
if size < 1024 or unit == units[-1]:
|
||||
return f"{size:.1f} {unit}" if unit != "B" else f"{int(size)} B"
|
||||
size /= 1024
|
||||
return f"{value} B"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
287
scripts/generate_motrixsim_runtime_scene.py
Executable file
287
scripts/generate_motrixsim_runtime_scene.py
Executable file
@ -0,0 +1,287 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
APP_ROOT = SCRIPT_DIR.parent
|
||||
SRC_ROOT = APP_ROOT / "src"
|
||||
if str(SRC_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(SRC_ROOT))
|
||||
|
||||
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
|
||||
from real2sim_platform.store import PlatformStore # noqa: E402
|
||||
|
||||
|
||||
SCENE_ID = "1f2618bdfcac4e45456569980716405a"
|
||||
PACKAGE_SLUG = "parlour_rm65_pick_place"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Generate a runnable RM65 + coffee-table pick-place smoke MJCF for the MotrixSim asset package.")
|
||||
parser.add_argument("--scene-id", default=SCENE_ID)
|
||||
parser.add_argument("--steps", type=int, default=120)
|
||||
parser.add_argument("--motrixsim-python", default="")
|
||||
args = parser.parse_args()
|
||||
|
||||
config = load_config()
|
||||
ensure_workspace(config)
|
||||
store = PlatformStore(config.db_path)
|
||||
scene = store.get_scene(args.scene_id)
|
||||
package_dir = Path(str((scene.get("metadata") or {}).get("package") or config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG))
|
||||
manifest_path = Path(str(scene.get("manifest_path") or package_dir / "motrixsim_scene_manifest.json"))
|
||||
sim_dir = package_dir / "simulator_assets"
|
||||
sim_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
runtime_rm65 = sim_dir / "rmjx_rm65_runtime.xml"
|
||||
source_rm65 = package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml"
|
||||
runtime_xml = source_rm65.read_text(encoding="utf-8").replace(
|
||||
'assetdir="assets"',
|
||||
'assetdir="../assets/rm65_open_cabinet/assets"',
|
||||
)
|
||||
# The upstream pad capsules are visual-only diagnostics. Enable collision on
|
||||
# the runtime copy so pick-place validation measures real bilateral contact.
|
||||
runtime_xml = runtime_xml.replace(
|
||||
'contype="0" conaffinity="0" density="0" group="2"/>',
|
||||
'contype="1" conaffinity="1" density="0" friction="1.2 0.01 0.001" group="3"/>',
|
||||
)
|
||||
runtime_rm65.write_text(runtime_xml, encoding="utf-8")
|
||||
|
||||
combined = sim_dir / "rm65_table_pick_place.xml"
|
||||
combined.write_text(runtime_scene_xml(), encoding="utf-8")
|
||||
|
||||
mujoco_report = validate_mujoco(combined, steps=args.steps)
|
||||
motrixsim_python = find_motrixsim_python(args.motrixsim_python)
|
||||
motrixsim_report = validate_motrixsim(combined, steps=args.steps, python=motrixsim_python)
|
||||
report = {
|
||||
"schema": "real2sim.motrixsim_runtime_scene_validation.v2",
|
||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"combined_mjcf": str(combined),
|
||||
"steps_requested": args.steps,
|
||||
"ok": bool(mujoco_report.get("ok") and motrixsim_report.get("ok")),
|
||||
"runtimes": {"mujoco": mujoco_report, "motrixsim": motrixsim_report},
|
||||
}
|
||||
report_path = sim_dir / "rm65_table_pick_place_validation.json"
|
||||
report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
scene_artifact = store.add_artifact(task_id=None, name=combined.name, path=combined, kind="simulator")
|
||||
rm65_artifact = store.add_artifact(task_id=None, name=runtime_rm65.name, path=runtime_rm65, kind="simulator")
|
||||
report_artifact = store.add_artifact(task_id=None, name=report_path.name, path=report_path, kind="json")
|
||||
update_manifest(manifest_path, combined, scene_artifact["id"], runtime_rm65, rm65_artifact["id"], report_path, report_artifact["id"], report)
|
||||
runtime_ready = bool(report["ok"])
|
||||
metadata = dict(scene.get("metadata") or {})
|
||||
metadata.update(
|
||||
{
|
||||
"status": "runtime_validated" if runtime_ready else "runtime_validation_failed",
|
||||
"adapter_pending": not runtime_ready,
|
||||
"runtime_validation": {"mujoco": bool(mujoco_report.get("ok")), "motrixsim": bool(motrixsim_report.get("ok"))},
|
||||
}
|
||||
)
|
||||
tags = [tag for tag in scene.get("tags", []) if tag not in {"adapter-pending", "runtime-validated", "motrixsim-ready"}]
|
||||
if runtime_ready:
|
||||
tags.extend(["runtime-validated", "motrixsim-ready"])
|
||||
else:
|
||||
tags.append("adapter-pending")
|
||||
store.update_scene(args.scene_id, metadata=metadata, tags=tags)
|
||||
|
||||
print(json.dumps({"ok": report["ok"], "scene_id": args.scene_id, "combined_mjcf": str(combined), "report": str(report_path), "validation": report}, ensure_ascii=False, indent=2))
|
||||
return 0 if report["ok"] else 1
|
||||
|
||||
|
||||
def runtime_scene_xml() -> str:
|
||||
return """<mujoco model="rm65 table pick place smoke">
|
||||
<include file="rmjx_rm65_runtime.xml"/>
|
||||
<visual>
|
||||
<global offwidth="960" offheight="540"/>
|
||||
</visual>
|
||||
<worldbody>
|
||||
<body name="coffee_table" pos="0.65 0 0.25">
|
||||
<geom name="coffee_table_top" type="box" size="0.45 0.32 0.03" rgba="0.45 0.30 0.18 1" friction="0.8 0.01 0.001"/>
|
||||
<geom name="coffee_table_leg_1" type="box" pos="0.36 0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
|
||||
<geom name="coffee_table_leg_2" type="box" pos="-0.36 0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
|
||||
<geom name="coffee_table_leg_3" type="box" pos="0.36 -0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
|
||||
<geom name="coffee_table_leg_4" type="box" pos="-0.36 -0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
|
||||
</body>
|
||||
<body name="pick_cube" pos="0.65 0 0.33">
|
||||
<freejoint name="pick_cube_free"/>
|
||||
<geom name="pick_cube_geom" type="box" size="0.03 0.03 0.03" rgba="0.9 0.2 0.12 1" mass="0.05" friction="0.9 0.01 0.001"/>
|
||||
</body>
|
||||
<body name="place_target" pos="0.43 0.18 0.335">
|
||||
<geom name="place_target_marker" type="cylinder" size="0.055 0.004" rgba="0.1 0.8 0.6 0.35" contype="0" conaffinity="0"/>
|
||||
</body>
|
||||
<camera name="overview" pos="1.7 -1.6 1.25" xyaxes="0.68 0.73 0 -0.37 0.34 0.86"/>
|
||||
</worldbody>
|
||||
<keyframe>
|
||||
<key name="home" qpos="0 0 0 -1.57079 0 1.57079 -0.7853 0 0 0 0 0 0.65 0 0.33 1 0 0 0" ctrl="0 0 0 -1.57079 0 1.57079 -0.7853"/>
|
||||
</keyframe>
|
||||
</mujoco>
|
||||
"""
|
||||
|
||||
|
||||
def validate_mujoco(path: Path, *, steps: int) -> dict[str, Any]:
|
||||
started = time.time()
|
||||
report: dict[str, Any] = {
|
||||
"schema": "real2sim.motrixsim_runtime_scene_validation.v1",
|
||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"combined_mjcf": str(path),
|
||||
"steps_requested": steps,
|
||||
"ok": False,
|
||||
}
|
||||
try:
|
||||
import mujoco
|
||||
|
||||
model = mujoco.MjModel.from_xml_path(str(path))
|
||||
data = mujoco.MjData(model)
|
||||
for _ in range(max(0, steps)):
|
||||
mujoco.mj_step(model, data)
|
||||
report.update(
|
||||
{
|
||||
"ok": True,
|
||||
"runtime": "mujoco",
|
||||
"nq": int(model.nq),
|
||||
"nv": int(model.nv),
|
||||
"nbody": int(model.nbody),
|
||||
"ngeom": int(model.ngeom),
|
||||
"nkey": int(model.nkey),
|
||||
"sim_time": float(data.time),
|
||||
"elapsed_ms": round((time.time() - started) * 1000),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
report.update({"ok": False, "error": f"{type(exc).__name__}: {exc}", "elapsed_ms": round((time.time() - started) * 1000)})
|
||||
return report
|
||||
|
||||
|
||||
def find_motrixsim_python(explicit: str) -> Path | None:
|
||||
candidates = [
|
||||
explicit,
|
||||
os.environ.get("REAL2SIM_MOTRIXSIM_PYTHON", ""),
|
||||
"/Users/d-robotics/workSpace/motrixsim-docs/.venv/bin/python",
|
||||
"/opt/Motphys/motrixsim-docs/.venv/bin/python",
|
||||
]
|
||||
for candidate in candidates:
|
||||
if candidate and Path(candidate).is_file():
|
||||
return Path(candidate)
|
||||
return None
|
||||
|
||||
|
||||
def validate_motrixsim(path: Path, *, steps: int, python: Path | None) -> dict[str, Any]:
|
||||
started = time.time()
|
||||
report: dict[str, Any] = {"ok": False, "runtime": "motrixsim", "python": str(python or "")}
|
||||
if python is None:
|
||||
report.update({"status": "runtime_unavailable", "error": "motrixsim python environment was not found"})
|
||||
return report
|
||||
probe = """
|
||||
import json, sys
|
||||
from motrixsim import SceneData, load_model, step
|
||||
model = load_model(sys.argv[1])
|
||||
data = SceneData(model)
|
||||
for _ in range(int(sys.argv[2])):
|
||||
step(model, data)
|
||||
print("REAL2SIM_RESULT=" + json.dumps({
|
||||
"timestep": float(model.options.timestep),
|
||||
"nq": len(data.low.dof_positions),
|
||||
"nv": len(data.low.dof_velocities),
|
||||
}))
|
||||
"""
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[str(python), "-c", probe, str(path), str(steps)],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
)
|
||||
result_line = next(line for line in completed.stdout.splitlines() if line.startswith("REAL2SIM_RESULT="))
|
||||
result = json.loads(result_line.split("=", 1)[1])
|
||||
report.update(
|
||||
{
|
||||
"ok": True,
|
||||
"status": "loaded_and_stepped",
|
||||
"steps": steps,
|
||||
**result,
|
||||
"elapsed_ms": round((time.time() - started) * 1000),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
stderr = getattr(exc, "stderr", "") or ""
|
||||
report.update(
|
||||
{
|
||||
"ok": False,
|
||||
"status": "runtime_failed",
|
||||
"error": f"{type(exc).__name__}: {exc}",
|
||||
"stderr": stderr[-2000:],
|
||||
"elapsed_ms": round((time.time() - started) * 1000),
|
||||
}
|
||||
)
|
||||
return report
|
||||
|
||||
|
||||
def update_manifest(
|
||||
manifest_path: Path,
|
||||
combined: Path,
|
||||
combined_artifact_id: str,
|
||||
runtime_rm65: Path,
|
||||
runtime_rm65_artifact_id: str,
|
||||
report_path: Path,
|
||||
report_artifact_id: str,
|
||||
report: dict[str, Any],
|
||||
) -> None:
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
generated_names = {combined.name, runtime_rm65.name, report_path.name}
|
||||
assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and item.get("name") in generated_names)]
|
||||
generated = [
|
||||
asset_entry(combined, combined_artifact_id, "runtime_scene", "simulator", "可运行 RM65 茶几 pick-place smoke MJCF", manifest_path),
|
||||
asset_entry(runtime_rm65, runtime_rm65_artifact_id, "runtime_robot", "simulator", "RM65 runtime assetdir 修正版", manifest_path),
|
||||
asset_entry(report_path, report_artifact_id, "runtime_validation_report", "metadata", "运行时加载校验报告", manifest_path),
|
||||
]
|
||||
manifest["assets"] = [*generated, *assets]
|
||||
manifest["runtime_scene"] = {
|
||||
"combined_mjcf": generated[0],
|
||||
"runtime_rm65": generated[1],
|
||||
"validation_report": generated[2],
|
||||
"validation": report,
|
||||
"status": "mujoco_and_motrixsim_smoke_passed" if report.get("ok") else "failed",
|
||||
"scope": "combined_rm65_table_scene_runtime_load_and_step",
|
||||
}
|
||||
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def asset_entry(path: Path, artifact_id: str, role: str, kind: str, name: str, manifest_path: Path) -> dict[str, Any]:
|
||||
size = path.stat().st_size
|
||||
return {
|
||||
"name": path.name,
|
||||
"display_name": name,
|
||||
"role": role,
|
||||
"kind": kind,
|
||||
"path": str(path),
|
||||
"relative_path": str(path.relative_to(manifest_path.parent)),
|
||||
"url": f"/api/artifacts/{artifact_id}/content",
|
||||
"exists": True,
|
||||
"size_bytes": size,
|
||||
"size_label": format_bytes(size),
|
||||
"generated": True,
|
||||
}
|
||||
|
||||
|
||||
def format_bytes(value: int) -> str:
|
||||
units = ["B", "KB", "MB", "GB"]
|
||||
size = float(value)
|
||||
for unit in units:
|
||||
if size < 1024 or unit == units[-1]:
|
||||
return f"{size:.1f} {unit}" if unit != "B" else f"{int(size)} B"
|
||||
size /= 1024
|
||||
return f"{value} B"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
328
scripts/import_motrixsim_assets.py
Executable file
328
scripts/import_motrixsim_assets.py
Executable file
@ -0,0 +1,328 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
APP_ROOT = SCRIPT_DIR.parent
|
||||
SRC_ROOT = APP_ROOT / "src"
|
||||
if str(SRC_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(SRC_ROOT))
|
||||
|
||||
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
|
||||
from real2sim_platform.store import PlatformStore # noqa: E402
|
||||
|
||||
|
||||
DEFAULT_SOURCE = Path("/Users/d-robotics/workSpace/motrixsim-docs")
|
||||
PACKAGE_SLUG = "parlour_rm65_pick_place"
|
||||
SCENE_ID = hashlib.sha1(f"motrixsim:{PACKAGE_SLUG}".encode("utf-8")).hexdigest()[:32]
|
||||
|
||||
|
||||
REQUIRED_FILES = {
|
||||
"parlour_scene": Path("examples/assets/parlour/parlour.xml"),
|
||||
"rm65_robot": Path("examples/assets/rm65_open_cabinet/mjx_rm65.xml"),
|
||||
"rm65_reference_scene": Path("examples/assets/rm65_open_cabinet/scene.xml"),
|
||||
"rm65_reference_controller": Path("examples/control/rm65_open_cabinet.py"),
|
||||
"pick_place_template": Path("examples/assets/franka_emika_panda/scene_pick_cube.xml"),
|
||||
"parlour_locomotion_example": Path("examples/control/robot_locomotion.py"),
|
||||
}
|
||||
|
||||
DIRECTORIES_TO_COPY = {
|
||||
"parlour": Path("examples/assets/parlour"),
|
||||
"rm65_open_cabinet": Path("examples/assets/rm65_open_cabinet"),
|
||||
"franka_emika_panda": Path("examples/assets/franka_emika_panda"),
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Import MotrixSim parlour + RM65 pick-place scaffold into Real2Sim Platform.")
|
||||
parser.add_argument("--source", type=Path, default=DEFAULT_SOURCE, help="Local MotrixSim docs repository path.")
|
||||
parser.add_argument("--name", default="MotrixSim 客厅 RM65 Pick-Place", help="Scene name shown in Real2Sim.")
|
||||
parser.add_argument("--replace-index", action="store_true", help="Replace the existing deterministic scene index row.")
|
||||
parser.add_argument("--skip-copy", action="store_true", help="Only refresh manifest/index using existing package files.")
|
||||
parser.add_argument("--allow-missing-source", action="store_true", help="Allow registering an already-copied package when the MotrixSim repo is not present.")
|
||||
args = parser.parse_args()
|
||||
|
||||
source = args.source.expanduser().resolve()
|
||||
config = load_config()
|
||||
ensure_workspace(config)
|
||||
store = PlatformStore(config.db_path)
|
||||
package_dir = config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG
|
||||
if source.is_dir():
|
||||
_assert_source(source)
|
||||
source_commit = _git_commit(source)
|
||||
elif args.skip_copy and args.allow_missing_source:
|
||||
source_commit = _existing_source_commit(package_dir) or ""
|
||||
else:
|
||||
raise SystemExit(f"MotrixSim docs repo not found: {source}")
|
||||
|
||||
if not args.skip_copy:
|
||||
_copy_package(source, package_dir)
|
||||
_assert_package(package_dir)
|
||||
|
||||
scene_descriptor = package_dir / f"{PACKAGE_SLUG}.scene.json"
|
||||
manifest_path = package_dir / "motrixsim_scene_manifest.json"
|
||||
scene_descriptor.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema": "real2sim.scene_descriptor.v1",
|
||||
"name": args.name,
|
||||
"package": str(package_dir),
|
||||
"manifest_path": str(manifest_path),
|
||||
"simulator": "motrixsim",
|
||||
"status": "asset_imported_adapter_pending",
|
||||
},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
+ "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
artifacts = _register_file_artifacts(store, package_dir)
|
||||
manifest = _build_manifest(
|
||||
name=args.name,
|
||||
source=source,
|
||||
source_commit=source_commit,
|
||||
package_dir=package_dir,
|
||||
scene_descriptor=scene_descriptor,
|
||||
artifact_urls=artifacts,
|
||||
)
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
scene = _upsert_scene(
|
||||
store=store,
|
||||
name=args.name,
|
||||
package_dir=package_dir,
|
||||
scene_descriptor=scene_descriptor,
|
||||
manifest_path=manifest_path,
|
||||
manifest=manifest,
|
||||
replace_index=args.replace_index,
|
||||
)
|
||||
|
||||
print(json.dumps({"ok": True, "scene_id": scene["id"], "scene_name": scene["name"], "package_dir": str(package_dir), "manifest_path": str(manifest_path)}, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _assert_source(source: Path) -> None:
|
||||
if not source.is_dir():
|
||||
raise SystemExit(f"MotrixSim docs repo not found: {source}")
|
||||
missing = [str(source / rel) for rel in REQUIRED_FILES.values() if not (source / rel).is_file()]
|
||||
missing.extend(str(source / rel) for rel in DIRECTORIES_TO_COPY.values() if not (source / rel).is_dir())
|
||||
if missing:
|
||||
raise SystemExit("Missing required MotrixSim assets:\n" + "\n".join(missing))
|
||||
|
||||
|
||||
def _copy_package(source: Path, package_dir: Path) -> None:
|
||||
if package_dir.exists():
|
||||
shutil.rmtree(package_dir)
|
||||
(package_dir / "assets").mkdir(parents=True, exist_ok=True)
|
||||
(package_dir / "examples" / "control").mkdir(parents=True, exist_ok=True)
|
||||
for name, rel in DIRECTORIES_TO_COPY.items():
|
||||
shutil.copytree(source / rel, package_dir / "assets" / name)
|
||||
for rel in [Path("examples/control/rm65_open_cabinet.py"), Path("examples/control/robot_locomotion.py")]:
|
||||
shutil.copy2(source / rel, package_dir / "examples" / "control" / rel.name)
|
||||
|
||||
|
||||
def _assert_package(package_dir: Path) -> None:
|
||||
required = [
|
||||
package_dir / "assets/parlour/parlour.xml",
|
||||
package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml",
|
||||
package_dir / "assets/rm65_open_cabinet/scene.xml",
|
||||
package_dir / "assets/franka_emika_panda/scene_pick_cube.xml",
|
||||
package_dir / "examples/control/rm65_open_cabinet.py",
|
||||
package_dir / "examples/control/robot_locomotion.py",
|
||||
]
|
||||
missing = [str(path) for path in required if not path.is_file()]
|
||||
if missing:
|
||||
raise SystemExit("Package is incomplete:\n" + "\n".join(missing))
|
||||
|
||||
|
||||
def _register_file_artifacts(store: PlatformStore, package_dir: Path) -> dict[str, str]:
|
||||
files = {
|
||||
"scene_descriptor": package_dir / f"{PACKAGE_SLUG}.scene.json",
|
||||
"parlour_scene": package_dir / "assets/parlour/parlour.xml",
|
||||
"rm65_robot": package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml",
|
||||
"rm65_reference_scene": package_dir / "assets/rm65_open_cabinet/scene.xml",
|
||||
"rm65_controller": package_dir / "examples/control/rm65_open_cabinet.py",
|
||||
"pick_place_template": package_dir / "assets/franka_emika_panda/scene_pick_cube.xml",
|
||||
"franka_preview": package_dir / "assets/franka_emika_panda/panda.png",
|
||||
}
|
||||
urls: dict[str, str] = {}
|
||||
for role, path in files.items():
|
||||
if not path.is_file():
|
||||
continue
|
||||
artifact = store.add_artifact(task_id=None, name=path.name, path=path, kind=_artifact_kind(path))
|
||||
urls[role] = f"/api/artifacts/{artifact['id']}/content"
|
||||
return urls
|
||||
|
||||
|
||||
def _build_manifest(
|
||||
*,
|
||||
name: str,
|
||||
source: Path,
|
||||
source_commit: str,
|
||||
package_dir: Path,
|
||||
scene_descriptor: Path,
|
||||
artifact_urls: dict[str, str],
|
||||
) -> dict[str, Any]:
|
||||
ready_assets = [
|
||||
_asset(package_dir, "assets/parlour/parlour.xml", "parlour_scene", "simulator", "MotrixSim parlour 客厅场景", artifact_urls.get("parlour_scene")),
|
||||
_asset(package_dir, "assets/rm65_open_cabinet/mjx_rm65.xml", "rm65_robot", "robot", "RM65 机械臂 MJCF", artifact_urls.get("rm65_robot")),
|
||||
_asset(package_dir, "assets/rm65_open_cabinet/scene.xml", "rm65_reference_scene", "simulator", "RM65 开柜门参考场景", artifact_urls.get("rm65_reference_scene")),
|
||||
_asset(package_dir, "examples/control/rm65_open_cabinet.py", "rm65_controller", "controller", "RM65 参考控制脚本", artifact_urls.get("rm65_controller")),
|
||||
_asset(package_dir, "assets/franka_emika_panda/scene_pick_cube.xml", "pick_place_template", "simulator", "Franka pick-cube 任务模板", artifact_urls.get("pick_place_template")),
|
||||
]
|
||||
preview = _asset(package_dir, "assets/franka_emika_panda/panda.png", "reference_preview", "image", "机械臂参考预览图", artifact_urls.get("franka_preview"))
|
||||
assets = [preview, *ready_assets] if preview["exists"] else ready_assets
|
||||
total_size = _directory_size(package_dir)
|
||||
return {
|
||||
"schema": "real2sim.motrixsim_asset_package.v1",
|
||||
"name": name,
|
||||
"status": "asset_imported_adapter_pending",
|
||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"package_dir": str(package_dir),
|
||||
"scene_descriptor": str(scene_descriptor),
|
||||
"source": {
|
||||
"repo": "https://github.com/Motphys/motrixsim-docs",
|
||||
"docs": "https://motrixsim.readthedocs.io/zh-cn/latest/user_guide/overview/examples.html",
|
||||
"local_path": str(source),
|
||||
"commit": source_commit,
|
||||
},
|
||||
"ready_assets": ready_assets,
|
||||
"assets": assets,
|
||||
"adapter_plan": {
|
||||
"goal": "将 parlour 客厅场景中的人形机器人示例替换为 RM65 机械臂,并在茶几区域完成 pick-place。",
|
||||
"base_scene": "assets/parlour/parlour.xml",
|
||||
"robot_asset": "assets/rm65_open_cabinet/mjx_rm65.xml",
|
||||
"reference_controller": "examples/control/rm65_open_cabinet.py",
|
||||
"pick_place_template": "assets/franka_emika_panda/scene_pick_cube.xml",
|
||||
"recommended_steps": [
|
||||
"在 parlour 中定位茶几 mesh/collision,并确定表面高度与可达工作区。",
|
||||
"把 RM65 base 固定到茶几旁或茶几边缘,完成 z-up 坐标、尺度和碰撞组检查。",
|
||||
"把 Franka pick-cube 的目标物、home/grasp/lift 阶段迁移到 RM65 关节空间。",
|
||||
"将 RM65 开柜门示例中的 7 维动作和 gripper 控制改成 cube pick-place controller。",
|
||||
"生成 combined MJCF 后再进入 simulator_validation runtime_required。",
|
||||
],
|
||||
},
|
||||
"runbook": {
|
||||
"source_parlour_demo": "uv run examples/control/robot_locomotion.py --robot g1 --scene parlour",
|
||||
"source_rm65_demo": "uv run examples/control/rm65_open_cabinet.py",
|
||||
"source_pick_template": "uv run examples/bench/grasp/shake_test_mx.py --object cube",
|
||||
},
|
||||
"known_gaps": [
|
||||
"当前包是资产接入 scaffold,不是可直接运行的 combined MJCF。",
|
||||
"尚未完成 parlour 茶几 mesh 的语义定位和 RM65 base 位姿标定。",
|
||||
"尚未把 Franka pick-cube keyframe/controller 迁移到 RM65。",
|
||||
],
|
||||
"metrics": {
|
||||
"asset_count": sum(1 for path in package_dir.rglob("*") if path.is_file()),
|
||||
"total_size_bytes": total_size,
|
||||
"copied_directories": sorted(DIRECTORIES_TO_COPY),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _asset(package_dir: Path, rel: str, role: str, kind: str, name: str, url: str | None = None) -> dict[str, Any]:
|
||||
path = package_dir / rel
|
||||
return {
|
||||
"name": name,
|
||||
"role": role,
|
||||
"kind": kind,
|
||||
"path": str(path),
|
||||
"relative_path": rel,
|
||||
"url": url or "",
|
||||
"exists": path.is_file(),
|
||||
"size_bytes": path.stat().st_size if path.is_file() else 0,
|
||||
}
|
||||
|
||||
|
||||
def _upsert_scene(
|
||||
*,
|
||||
store: PlatformStore,
|
||||
name: str,
|
||||
package_dir: Path,
|
||||
scene_descriptor: Path,
|
||||
manifest_path: Path,
|
||||
manifest: dict[str, Any],
|
||||
replace_index: bool,
|
||||
) -> dict[str, Any]:
|
||||
try:
|
||||
existing = store.get_scene(SCENE_ID)
|
||||
except KeyError:
|
||||
existing = None
|
||||
if existing and not replace_index:
|
||||
store.update_scene(SCENE_ID, name=name, tags=_tags(), version=str(manifest["source"]["commit"]))
|
||||
return store.get_scene(SCENE_ID)
|
||||
if existing and replace_index:
|
||||
store.delete_scene(SCENE_ID)
|
||||
return store.create_scene(
|
||||
item_id=SCENE_ID,
|
||||
name=name,
|
||||
path=scene_descriptor,
|
||||
metadata={
|
||||
"package": str(package_dir),
|
||||
"manifest_path": str(manifest_path),
|
||||
"simulator": "motrixsim",
|
||||
"status": "asset_imported_adapter_pending",
|
||||
"adapter_pending": True,
|
||||
},
|
||||
source_type="motrixsim_docs",
|
||||
source_uri="https://github.com/Motphys/motrixsim-docs",
|
||||
version=str(manifest["source"]["commit"]),
|
||||
manifest_path=manifest_path,
|
||||
metrics=manifest["metrics"],
|
||||
tags=_tags(),
|
||||
)
|
||||
|
||||
|
||||
def _tags() -> list[str]:
|
||||
return ["motrixsim", "parlour", "rm65", "pick-place", "adapter-pending"]
|
||||
|
||||
|
||||
def _git_commit(source: Path) -> str:
|
||||
try:
|
||||
return subprocess.check_output(["git", "-C", str(source), "rev-parse", "HEAD"], text=True).strip()
|
||||
except (OSError, subprocess.CalledProcessError):
|
||||
return ""
|
||||
|
||||
|
||||
def _existing_source_commit(package_dir: Path) -> str:
|
||||
manifest_path = package_dir / "motrixsim_scene_manifest.json"
|
||||
if not manifest_path.is_file():
|
||||
return ""
|
||||
try:
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError:
|
||||
return ""
|
||||
return str((manifest.get("source") or {}).get("commit") or "")
|
||||
|
||||
|
||||
def _directory_size(path: Path) -> int:
|
||||
return sum(item.stat().st_size for item in path.rglob("*") if item.is_file())
|
||||
|
||||
|
||||
def _artifact_kind(path: Path) -> str:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix in {".png", ".jpg", ".jpeg", ".webp"}:
|
||||
return "image"
|
||||
if suffix == ".xml":
|
||||
return "simulator"
|
||||
if suffix == ".py":
|
||||
return "controller"
|
||||
if suffix == ".json":
|
||||
return "metadata"
|
||||
return suffix.lstrip(".") or "file"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
441
scripts/production_acceptance_wuying.sh
Executable file
441
scripts/production_acceptance_wuying.sh
Executable file
@ -0,0 +1,441 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
|
||||
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
|
||||
BASE_URL="${REAL2SIM_ACCEPTANCE_BASE_URL:-http://127.0.0.1:18126}"
|
||||
MODE="${REAL2SIM_ACCEPTANCE_MODE:-production}"
|
||||
SCALE="${REAL2SIM_ACCEPTANCE_SCALE:-pressure}"
|
||||
OUT="${REAL2SIM_ACCEPTANCE_OUT:-/opt/real2sim2model/real2sim-platform/workspace/acceptance_runs}"
|
||||
HF_REPO="${REAL2SIM_ACCEPTANCE_HF_REPO:-shashuo0104/xarm7_pusht}"
|
||||
HF_PATTERNS="${REAL2SIM_ACCEPTANCE_HF_PATTERNS:-videos/chunk-000/observation.images.front/*.mp4,data/chunk-000/*.parquet,meta/**,README.md}"
|
||||
RECON_FIXTURE="${REAL2SIM_ACCEPTANCE_RECON_FIXTURE:-/opt/real2sim_paper_runtime/sam-3d-objects/notebook/images/shutterstock_modern_colorful_Interior_2620125197}"
|
||||
RECON_FALLBACKS="${REAL2SIM_ACCEPTANCE_RECON_FALLBACKS:-/opt/real2sim_paper_runtime/sam2/notebooks/videos/bedroom}"
|
||||
MIN_RECON_FRAMES="${REAL2SIM_ACCEPTANCE_MIN_RECON_FRAMES:-80}"
|
||||
TARGET_HF_FRAMES="${REAL2SIM_ACCEPTANCE_TARGET_HF_FRAMES:-240}"
|
||||
MIN_HF_FRAMES="${REAL2SIM_ACCEPTANCE_MIN_HF_FRAMES:-120}"
|
||||
TIMEOUT_SECONDS="${REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS:-7200}"
|
||||
REQUIRE_MOTRIXSIM="${REAL2SIM_REQUIRE_MOTRIXSIM:-0}"
|
||||
REPEAT_RECONSTRUCTIONS="${REAL2SIM_ACCEPTANCE_REPEAT_RECONSTRUCTIONS:-2}"
|
||||
PRESSURE_TASKS="${REAL2SIM_ACCEPTANCE_PRESSURE_TASKS:-12}"
|
||||
CONTINUE_AFTER_RECON_QUALITY_FAIL="${REAL2SIM_ACCEPTANCE_CONTINUE_AFTER_RECON_QUALITY_FAIL:-1}"
|
||||
|
||||
ssh -i "$KEY" -o BatchMode=yes -o StrictHostKeyChecking=accept-new "$HOST" \
|
||||
env \
|
||||
REAL2SIM_ACCEPTANCE_BASE_URL="$BASE_URL" \
|
||||
REAL2SIM_ACCEPTANCE_MODE="$MODE" \
|
||||
REAL2SIM_ACCEPTANCE_SCALE="$SCALE" \
|
||||
REAL2SIM_ACCEPTANCE_OUT="$OUT" \
|
||||
REAL2SIM_ACCEPTANCE_HF_REPO="$HF_REPO" \
|
||||
REAL2SIM_ACCEPTANCE_HF_PATTERNS="$HF_PATTERNS" \
|
||||
REAL2SIM_ACCEPTANCE_RECON_FIXTURE="$RECON_FIXTURE" \
|
||||
REAL2SIM_ACCEPTANCE_RECON_FALLBACKS="$RECON_FALLBACKS" \
|
||||
REAL2SIM_ACCEPTANCE_MIN_RECON_FRAMES="$MIN_RECON_FRAMES" \
|
||||
REAL2SIM_ACCEPTANCE_TARGET_HF_FRAMES="$TARGET_HF_FRAMES" \
|
||||
REAL2SIM_ACCEPTANCE_MIN_HF_FRAMES="$MIN_HF_FRAMES" \
|
||||
REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS="$TIMEOUT_SECONDS" \
|
||||
REAL2SIM_REQUIRE_MOTRIXSIM="$REQUIRE_MOTRIXSIM" \
|
||||
REAL2SIM_ACCEPTANCE_REPEAT_RECONSTRUCTIONS="$REPEAT_RECONSTRUCTIONS" \
|
||||
REAL2SIM_ACCEPTANCE_PRESSURE_TASKS="$PRESSURE_TASKS" \
|
||||
REAL2SIM_ACCEPTANCE_CONTINUE_AFTER_RECON_QUALITY_FAIL="$CONTINUE_AFTER_RECON_QUALITY_FAIL" \
|
||||
python3 -s <<'PY'
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
BASE = os.environ["REAL2SIM_ACCEPTANCE_BASE_URL"].rstrip("/")
|
||||
MODE = os.environ["REAL2SIM_ACCEPTANCE_MODE"]
|
||||
SCALE = os.environ["REAL2SIM_ACCEPTANCE_SCALE"]
|
||||
OUT_ROOT = Path(os.environ["REAL2SIM_ACCEPTANCE_OUT"])
|
||||
HF_REPO = os.environ["REAL2SIM_ACCEPTANCE_HF_REPO"]
|
||||
HF_PATTERNS = [item.strip() for item in os.environ["REAL2SIM_ACCEPTANCE_HF_PATTERNS"].split(",") if item.strip()]
|
||||
RECON_FIXTURE = os.environ["REAL2SIM_ACCEPTANCE_RECON_FIXTURE"]
|
||||
RECON_FALLBACKS = [item.strip() for item in os.environ["REAL2SIM_ACCEPTANCE_RECON_FALLBACKS"].split(":") if item.strip()]
|
||||
MIN_RECON_FRAMES = int(os.environ["REAL2SIM_ACCEPTANCE_MIN_RECON_FRAMES"])
|
||||
TARGET_HF_FRAMES = int(os.environ["REAL2SIM_ACCEPTANCE_TARGET_HF_FRAMES"])
|
||||
MIN_HF_FRAMES = int(os.environ["REAL2SIM_ACCEPTANCE_MIN_HF_FRAMES"])
|
||||
TIMEOUT = int(os.environ["REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS"])
|
||||
REQUIRE_MOTRIXSIM = os.environ["REAL2SIM_REQUIRE_MOTRIXSIM"] == "1"
|
||||
REPEAT_RECONSTRUCTIONS = int(os.environ["REAL2SIM_ACCEPTANCE_REPEAT_RECONSTRUCTIONS"])
|
||||
PRESSURE_TASKS = int(os.environ["REAL2SIM_ACCEPTANCE_PRESSURE_TASKS"])
|
||||
CONTINUE_AFTER_RECON_QUALITY_FAIL = os.environ["REAL2SIM_ACCEPTANCE_CONTINUE_AFTER_RECON_QUALITY_FAIL"] == "1"
|
||||
TERMINAL_STATUSES = {"succeeded", "failed", "canceled", "failed_recovered"}
|
||||
|
||||
started_at = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
run_dir = OUT_ROOT / time.strftime("%Y%m%dT%H%M%SZ", time.gmtime())
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
report: dict[str, Any] = {
|
||||
"schema": "real2sim.acceptance_report.v1",
|
||||
"mode": MODE,
|
||||
"scale": SCALE,
|
||||
"started_at": started_at,
|
||||
"finished_at": "",
|
||||
"base_url": BASE,
|
||||
"inputs": {
|
||||
"hf_repo": HF_REPO,
|
||||
"hf_patterns": HF_PATTERNS,
|
||||
"reconstruction_fixture": RECON_FIXTURE,
|
||||
"reconstruction_fallbacks": RECON_FALLBACKS,
|
||||
"min_reconstruction_frames": MIN_RECON_FRAMES,
|
||||
"target_hf_frames": TARGET_HF_FRAMES,
|
||||
"min_hf_frames": MIN_HF_FRAMES,
|
||||
"repeat_reconstructions": REPEAT_RECONSTRUCTIONS,
|
||||
"pressure_tasks": PRESSURE_TASKS,
|
||||
"require_motrixsim": REQUIRE_MOTRIXSIM,
|
||||
"continue_after_reconstruction_quality_fail": CONTINUE_AFTER_RECON_QUALITY_FAIL,
|
||||
},
|
||||
"version": {},
|
||||
"doctor": {},
|
||||
"tasks": [],
|
||||
"datasets": [],
|
||||
"scenes": [],
|
||||
"models": [],
|
||||
"browser": {},
|
||||
"viewer_probe": {
|
||||
"status": "not_run",
|
||||
"reason": "Run scripts/scene_viewer_probe_local.mjs separately; production API acceptance does not launch local Playwright.",
|
||||
},
|
||||
"screenshots": [],
|
||||
"artifacts": [],
|
||||
"warnings": [],
|
||||
"failures": [],
|
||||
"pass": False,
|
||||
}
|
||||
|
||||
|
||||
def api(path: str, payload: Any | None = None, timeout: int = 60) -> Any:
|
||||
data = None
|
||||
headers = {}
|
||||
if payload is not None:
|
||||
data = json.dumps(payload).encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(BASE + path, data=data, headers=headers)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as response:
|
||||
body = response.read().decode("utf-8")
|
||||
if not body:
|
||||
return None
|
||||
ctype = response.headers.get("content-type", "")
|
||||
return json.loads(body) if "application/json" in ctype else body
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", errors="replace")
|
||||
raise RuntimeError(f"{path} failed with HTTP {exc.code}: {detail}") from exc
|
||||
|
||||
|
||||
def record_failure(message: str, **context: Any) -> None:
|
||||
failure = {"message": message, **context}
|
||||
print("FAIL:", json.dumps(failure, ensure_ascii=False), flush=True)
|
||||
report["failures"].append(failure)
|
||||
|
||||
|
||||
def save_report() -> None:
|
||||
report["finished_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
(run_dir / "acceptance_report.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
lines = [
|
||||
"# Real2Sim Production Acceptance",
|
||||
"",
|
||||
f"- pass: {str(report['pass']).lower()}",
|
||||
f"- started_at: {report['started_at']}",
|
||||
f"- finished_at: {report['finished_at']}",
|
||||
f"- build_id: {report.get('version', {}).get('build_id', '')}",
|
||||
f"- tasks: {len(report['tasks'])}",
|
||||
f"- failures: {len(report['failures'])}",
|
||||
f"- warnings: {len(report['warnings'])}",
|
||||
f"- viewer_probe: {report.get('viewer_probe', {}).get('status', 'not_run')}",
|
||||
"",
|
||||
"## Task IDs",
|
||||
]
|
||||
for task in report["tasks"]:
|
||||
lines.append(f"- {task.get('type')} {task.get('id')} {task.get('status')}")
|
||||
if report["failures"]:
|
||||
lines.extend(["", "## Failures"])
|
||||
for failure in report["failures"]:
|
||||
lines.append(f"- {failure.get('message')} {json.dumps(failure, ensure_ascii=False)[:600]}")
|
||||
if report["warnings"]:
|
||||
lines.extend(["", "## Warnings"])
|
||||
for warning in report["warnings"]:
|
||||
lines.append(f"- {warning}")
|
||||
(run_dir / "summary.md").write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def task_snapshot(task_id: str) -> dict[str, Any]:
|
||||
task = api(f"/api/tasks/{task_id}")
|
||||
try:
|
||||
reports = api(f"/api/tasks/{task_id}/reports")
|
||||
except Exception as exc:
|
||||
reports = {"error": str(exc), "reports": []}
|
||||
try:
|
||||
artifacts = api(f"/api/tasks/{task_id}/artifacts")
|
||||
except Exception as exc:
|
||||
artifacts = [{"error": str(exc)}]
|
||||
try:
|
||||
logs = api(f"/api/tasks/{task_id}/logs")
|
||||
except Exception as exc:
|
||||
logs = f"log unavailable: {exc}"
|
||||
snapshot = {
|
||||
"id": task["id"],
|
||||
"type": task["type"],
|
||||
"status": task["status"],
|
||||
"params": task.get("params"),
|
||||
"result": task.get("result"),
|
||||
"error": task.get("error"),
|
||||
"run_dir": task.get("run_dir"),
|
||||
"reports": reports,
|
||||
"artifacts": artifacts,
|
||||
"log_tail": str(logs)[-5000:],
|
||||
}
|
||||
(run_dir / f"task_{task_id}.json").write_text(json.dumps(snapshot, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
return snapshot
|
||||
|
||||
|
||||
def wait_task(task_id: str, *, required: bool = True) -> dict[str, Any]:
|
||||
deadline = time.time() + TIMEOUT
|
||||
while time.time() < deadline:
|
||||
task = api(f"/api/tasks/{task_id}")
|
||||
if task["status"] in TERMINAL_STATUSES:
|
||||
snapshot = task_snapshot(task_id)
|
||||
report["tasks"].append(snapshot)
|
||||
if required and task["status"] != "succeeded":
|
||||
record_failure(f"required task {task['type']} did not succeed", task_id=task_id, status=task["status"], error=task.get("error"))
|
||||
print(json.dumps({"task_id": task_id, "type": task["type"], "status": task["status"]}, ensure_ascii=False), flush=True)
|
||||
return snapshot
|
||||
time.sleep(5)
|
||||
snapshot = task_snapshot(task_id)
|
||||
report["tasks"].append(snapshot)
|
||||
record_failure("task timed out", task_id=task_id)
|
||||
if required:
|
||||
raise TimeoutError(task_id)
|
||||
return snapshot
|
||||
|
||||
|
||||
def create_task(task_type: str, params: dict[str, Any], *, required: bool = True) -> dict[str, Any]:
|
||||
task = api("/api/tasks", {"type": task_type, "params": params})
|
||||
print(f"{task_type}: {task['id']}", flush=True)
|
||||
return wait_task(task["id"], required=required)
|
||||
|
||||
|
||||
def assert_artifacts(task: dict[str, Any], required_names: list[str] | None = None) -> None:
|
||||
artifacts = task.get("artifacts") or []
|
||||
names = {item.get("name") for item in artifacts}
|
||||
if not artifacts:
|
||||
record_failure("succeeded task has no artifacts", task_id=task.get("id"), task_type=task.get("type"))
|
||||
for name in required_names or []:
|
||||
if name not in names:
|
||||
record_failure("required artifact missing", task_id=task.get("id"), task_type=task.get("type"), artifact=name, available=sorted(n for n in names if n))
|
||||
for item in artifacts:
|
||||
if item.get("name"):
|
||||
report["artifacts"].append({"task_id": task.get("id"), "name": item.get("name"), "path": item.get("path"), "kind": item.get("kind")})
|
||||
|
||||
|
||||
def refresh_entities() -> None:
|
||||
report["datasets"] = api("/api/datasets")
|
||||
report["scenes"] = api("/api/scenes")
|
||||
report["models"] = api("/api/models")
|
||||
|
||||
|
||||
def check_doctor() -> None:
|
||||
report["version"] = api("/api/version")
|
||||
doctor = api("/api/doctor")
|
||||
report["doctor"] = doctor
|
||||
checks = doctor.get("checks", {})
|
||||
required = ["anysplat_root", "anysplat_python", "spark_bundle", "mujoco_python", "media_prepare_deps", "service_process"]
|
||||
for name in required:
|
||||
if not checks.get(name):
|
||||
record_failure("doctor required check failed", check=name, value=checks.get(name))
|
||||
for warning in doctor.get("warnings", []) or []:
|
||||
report["warnings"].append(f"doctor: {warning}")
|
||||
if not doctor.get("ok"):
|
||||
record_failure("doctor returned ok=false", doctor=doctor)
|
||||
|
||||
|
||||
def latest_scene_id(task: dict[str, Any]) -> str:
|
||||
scene_id = (((task.get("result") or {}).get("scene_id")) or "")
|
||||
if not scene_id:
|
||||
record_failure("scene task did not return scene_id", task_id=task.get("id"))
|
||||
return scene_id
|
||||
|
||||
|
||||
def run_pressure_tasks(scene_id: str, expert_dataset_id: str, model_id: str, reconstruction_input: str) -> None:
|
||||
queued = []
|
||||
templates = [
|
||||
("simulator_validation", {"scene_id": scene_id, "simulators": ["mujoco", "motrixsim"], "level": "runtime_if_available"}),
|
||||
("expert_collection", {"scene_id": scene_id, "episodes": 3, "steps": 24}),
|
||||
("training", {"dataset_id": expert_dataset_id, "epochs": 1}),
|
||||
("inference", {"scene_id": scene_id, "model_id": model_id, "steps": 40}),
|
||||
("reconstruction_preflight", {"input_path": reconstruction_input, "min_frames": MIN_RECON_FRAMES}),
|
||||
]
|
||||
for index in range(max(PRESSURE_TASKS, 12)):
|
||||
task_type, params = templates[index % len(templates)]
|
||||
task = api("/api/tasks", {"type": task_type, "params": params})
|
||||
queued.append(task)
|
||||
print(f"pressure {index + 1}: {task_type} {task['id']}", flush=True)
|
||||
for task in queued:
|
||||
wait_task(task["id"], required=True)
|
||||
active = [task for task in api("/api/tasks") if task.get("status") in {"queued", "running", "canceling"}]
|
||||
if active:
|
||||
record_failure("active tasks remain after pressure batch", active=active[:20])
|
||||
|
||||
|
||||
def select_reconstruction_input() -> tuple[str, dict[str, Any]]:
|
||||
candidates = [RECON_FIXTURE, *RECON_FALLBACKS]
|
||||
snapshots = []
|
||||
for index, candidate in enumerate(candidates):
|
||||
preflight = create_task("reconstruction_preflight", {
|
||||
"input_path": candidate,
|
||||
"min_frames": MIN_RECON_FRAMES,
|
||||
})
|
||||
assert_artifacts(preflight, ["capture_preflight_report.json"])
|
||||
snapshots.append(preflight)
|
||||
result = preflight.get("result") or {}
|
||||
if result.get("ok"):
|
||||
if index > 0:
|
||||
report["warnings"].append(f"primary reconstruction fixture failed preflight; using fallback: {candidate}")
|
||||
return candidate, preflight
|
||||
report["warnings"].append(f"reconstruction fixture candidate failed preflight: {candidate}")
|
||||
record_failure("no reconstruction fixture passed preflight", candidates=candidates)
|
||||
return candidates[0], snapshots[-1] if snapshots else {}
|
||||
|
||||
|
||||
def reconstruction_failed_on_quality_gate(task: dict[str, Any]) -> bool:
|
||||
reports = task.get("reports") or {}
|
||||
readable = str(reports.get("readable_error") or task.get("error") or "")
|
||||
for report_item in reports.get("reports", []) or []:
|
||||
payload = report_item.get("payload") or {}
|
||||
readable += " " + str(payload.get("readable_error") or payload.get("decision") or "")
|
||||
readable += " " + " ".join(str(item) for item in payload.get("blockers", []) or [])
|
||||
return any(marker in readable for marker in ["allow-low-quality", "capture acceptance failed", "recapture_required", "registered_ratio", "sparse_points"])
|
||||
|
||||
|
||||
def run_reconstruction_with_optional_continuation(reconstruction_input: str, repeat_index: int) -> dict[str, Any]:
|
||||
params = {
|
||||
"input_path": reconstruction_input,
|
||||
"dry_run": False,
|
||||
"renderer": "spark",
|
||||
"simulators": ["mujoco", "motrixsim"],
|
||||
"min_frames": MIN_RECON_FRAMES,
|
||||
"quality_preset": "capture-high",
|
||||
}
|
||||
strict = create_task("scene_reconstruction", params)
|
||||
if strict.get("status") == "succeeded":
|
||||
assert_artifacts(strict, ["environment.json", "reconstruction_command.json", "capture_preflight_report.json", "reconstruction_report.json", "scene_manifest.json"])
|
||||
return strict
|
||||
assert_artifacts(strict, ["capture_preflight_report.json", "reconstruction_report.json"])
|
||||
if not (CONTINUE_AFTER_RECON_QUALITY_FAIL and reconstruction_failed_on_quality_gate(strict)):
|
||||
raise RuntimeError(f"scene_reconstruction failed: {strict.get('id')}")
|
||||
report["warnings"].append(f"strict reconstruction {strict.get('id')} failed quality gate; continuing with allow_low_quality for downstream coverage")
|
||||
continuation_params = {**params, "allow_low_quality": True, "name": f"acceptance-continuation-{repeat_index}"}
|
||||
continuation = create_task("scene_reconstruction", continuation_params)
|
||||
if continuation.get("status") != "succeeded":
|
||||
assert_artifacts(continuation, ["capture_preflight_report.json", "reconstruction_report.json"])
|
||||
raise RuntimeError(f"allow_low_quality scene_reconstruction failed: {continuation.get('id')}")
|
||||
assert_artifacts(continuation, ["environment.json", "reconstruction_command.json", "capture_preflight_report.json", "reconstruction_report.json", "scene_manifest.json"])
|
||||
return continuation
|
||||
|
||||
|
||||
def main() -> int:
|
||||
try:
|
||||
check_doctor()
|
||||
hf_import = create_task("hf_import", {
|
||||
"repo_id": HF_REPO,
|
||||
"repo_type": "dataset",
|
||||
"allow_patterns": HF_PATTERNS,
|
||||
"local_name": "acceptance-xarm7-pusht",
|
||||
})
|
||||
assert_artifacts(hf_import)
|
||||
hf_dataset_id = (hf_import.get("result") or {}).get("dataset_id")
|
||||
if not hf_dataset_id:
|
||||
record_failure("hf_import did not return dataset_id", task_id=hf_import.get("id"))
|
||||
|
||||
media = create_task("media_prepare", {
|
||||
"dataset_id": hf_dataset_id,
|
||||
"camera_label": "observation.images.front",
|
||||
"sample_fps": 2,
|
||||
"max_frames_per_video": 80,
|
||||
"max_videos": 16,
|
||||
"target_frame_count": TARGET_HF_FRAMES,
|
||||
"min_total_frames": MIN_HF_FRAMES,
|
||||
"output_name": "acceptance-xarm7-pusht-prepared",
|
||||
})
|
||||
assert_artifacts(media)
|
||||
prepared_id = (media.get("result") or {}).get("dataset_id")
|
||||
frame_count = int((media.get("result") or {}).get("frame_count") or 0)
|
||||
if frame_count < MIN_HF_FRAMES:
|
||||
record_failure("media_prepare produced too few frames", frame_count=frame_count, minimum=MIN_HF_FRAMES)
|
||||
|
||||
hf_preflight = create_task("reconstruction_preflight", {
|
||||
"dataset_id": prepared_id,
|
||||
"min_frames": MIN_RECON_FRAMES,
|
||||
"allow_low_quality": True,
|
||||
})
|
||||
assert_artifacts(hf_preflight, ["capture_preflight_report.json"])
|
||||
|
||||
reconstruction_input, fixture_preflight = select_reconstruction_input()
|
||||
if not ((fixture_preflight.get("result") or {}).get("ok")):
|
||||
raise RuntimeError("no reconstruction input passed preflight")
|
||||
|
||||
reconstruction_tasks = []
|
||||
for index in range(max(REPEAT_RECONSTRUCTIONS, 2)):
|
||||
reconstruction_tasks.append(run_reconstruction_with_optional_continuation(reconstruction_input, index + 1))
|
||||
|
||||
scene_id = latest_scene_id(reconstruction_tasks[-1])
|
||||
if not scene_id:
|
||||
raise RuntimeError("scene_reconstruction did not produce scene_id")
|
||||
validation = create_task("simulator_validation", {
|
||||
"scene_id": scene_id,
|
||||
"simulators": ["mujoco", "motrixsim"],
|
||||
"level": "runtime_required" if REQUIRE_MOTRIXSIM else "runtime_if_available",
|
||||
})
|
||||
assert_artifacts(validation, ["validation_report.json"])
|
||||
validation_report = (validation.get("result") or {}).get("report") or {}
|
||||
motrix = (validation_report.get("simulators") or {}).get("motrixsim") or {}
|
||||
if REQUIRE_MOTRIXSIM and not motrix.get("ok"):
|
||||
record_failure("MotrixSim runtime is required but validation failed", motrix=motrix)
|
||||
elif not motrix.get("ok"):
|
||||
report["warnings"].append(f"motrixsim validation not fully ok: {motrix.get('reason') or motrix.get('status')}")
|
||||
|
||||
expert = create_task("expert_collection", {"scene_id": scene_id, "episodes": 12, "steps": 48, "name": "acceptance-expert-rollouts"})
|
||||
assert_artifacts(expert, ["manifest.json", "rollouts.jsonl"])
|
||||
expert_dataset_id = (expert.get("result") or {}).get("dataset_id")
|
||||
if not expert_dataset_id:
|
||||
record_failure("expert_collection did not return dataset_id", task_id=expert.get("id"))
|
||||
|
||||
training = create_task("training", {"dataset_id": expert_dataset_id, "epochs": 3, "name": "acceptance-policy"})
|
||||
assert_artifacts(training, ["model_card.json", "policy_head.json"])
|
||||
model_id = (training.get("result") or {}).get("model_id")
|
||||
if not model_id:
|
||||
record_failure("training did not return model_id", task_id=training.get("id"))
|
||||
|
||||
inference = create_task("inference", {"scene_id": scene_id, "model_id": model_id, "steps": 80})
|
||||
assert_artifacts(inference, ["prediction_rollout.json", "viewer.html"])
|
||||
if int((inference.get("result") or {}).get("rollout_steps") or 0) <= 0:
|
||||
record_failure("inference produced empty rollout", task_id=inference.get("id"), result=inference.get("result"))
|
||||
|
||||
run_pressure_tasks(scene_id, expert_dataset_id, model_id, reconstruction_input)
|
||||
refresh_entities()
|
||||
report["pass"] = len(report["failures"]) == 0
|
||||
save_report()
|
||||
print(json.dumps({"pass": report["pass"], "run_dir": str(run_dir), "report": str(run_dir / "acceptance_report.json")}, ensure_ascii=False), flush=True)
|
||||
return 0 if report["pass"] else 1
|
||||
except Exception as exc:
|
||||
record_failure("acceptance runner crashed", error=str(exc), traceback=traceback.format_exc())
|
||||
try:
|
||||
refresh_entities()
|
||||
except Exception:
|
||||
pass
|
||||
report["pass"] = False
|
||||
save_report()
|
||||
print(json.dumps({"pass": False, "run_dir": str(run_dir), "report": str(run_dir / "acceptance_report.json"), "error": str(exc)}, ensure_ascii=False), flush=True)
|
||||
return 1
|
||||
|
||||
|
||||
sys.exit(main())
|
||||
PY
|
||||
356
scripts/run_motrixsim_pick_place_rollout.py
Executable file
356
scripts/run_motrixsim_pick_place_rollout.py
Executable file
@ -0,0 +1,356 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import imageio.v2 as imageio
|
||||
import numpy as np
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
APP_ROOT = SCRIPT_DIR.parent
|
||||
SRC_ROOT = APP_ROOT / "src"
|
||||
if str(SRC_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(SRC_ROOT))
|
||||
|
||||
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
|
||||
from real2sim_platform.store import PlatformStore # noqa: E402
|
||||
|
||||
|
||||
SCENE_ID = "1f2618bdfcac4e45456569980716405a"
|
||||
PACKAGE_SLUG = "parlour_rm65_pick_place"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Run a scripted RM65 pick-place rollout on the generated smoke MJCF.")
|
||||
parser.add_argument("--scene-id", default=SCENE_ID)
|
||||
parser.add_argument("--fps", type=int, default=20)
|
||||
parser.add_argument("--width", type=int, default=960)
|
||||
parser.add_argument("--height", type=int, default=540)
|
||||
parser.add_argument("--no-render", action="store_true")
|
||||
parser.add_argument(
|
||||
"--grasp-mode",
|
||||
choices=("physical", "kinematic_attachment"),
|
||||
default="physical",
|
||||
help="Use contact physics by default; kinematic attachment is retained only for diagnostics.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
config = load_config()
|
||||
ensure_workspace(config)
|
||||
store = PlatformStore(config.db_path)
|
||||
scene = store.get_scene(args.scene_id)
|
||||
package_dir = Path(str((scene.get("metadata") or {}).get("package") or config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG))
|
||||
manifest_path = Path(str(scene.get("manifest_path") or package_dir / "motrixsim_scene_manifest.json"))
|
||||
sim_path = package_dir / "simulator_assets/rm65_table_pick_place.xml"
|
||||
if not sim_path.is_file():
|
||||
raise SystemExit(f"runtime scene not found: {sim_path}")
|
||||
|
||||
rollout_dir = package_dir / "rollouts" / time.strftime("%Y%m%d_%H%M%S")
|
||||
rollout_dir.mkdir(parents=True, exist_ok=True)
|
||||
result = run_rollout(
|
||||
sim_path,
|
||||
rollout_dir,
|
||||
fps=args.fps,
|
||||
width=args.width,
|
||||
height=args.height,
|
||||
render=not args.no_render,
|
||||
grasp_mode=args.grasp_mode,
|
||||
)
|
||||
|
||||
artifacts = {}
|
||||
for key, path in result["artifact_paths"].items():
|
||||
if Path(path).is_file():
|
||||
artifacts[key] = store.add_artifact(task_id=None, name=Path(path).name, path=Path(path), kind=artifact_kind(Path(path)))
|
||||
update_manifest(manifest_path, rollout_dir, result, artifacts)
|
||||
print(json.dumps({"ok": result["ok"], "scene_id": args.scene_id, "rollout_dir": str(rollout_dir), "summary": result["summary"], "artifacts": result["artifact_paths"]}, ensure_ascii=False, indent=2))
|
||||
return 0 if result["ok"] else 1
|
||||
|
||||
|
||||
def run_rollout(
|
||||
sim_path: Path,
|
||||
out_dir: Path,
|
||||
*,
|
||||
fps: int,
|
||||
width: int,
|
||||
height: int,
|
||||
render: bool,
|
||||
grasp_mode: str = "physical",
|
||||
) -> dict[str, Any]:
|
||||
import mujoco
|
||||
|
||||
model = mujoco.MjModel.from_xml_path(str(sim_path))
|
||||
data = mujoco.MjData(model)
|
||||
mujoco.mj_resetDataKeyframe(model, data, 0)
|
||||
mujoco.mj_forward(model, data)
|
||||
|
||||
site_id = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_SITE, "gripper")
|
||||
cube_joint = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "pick_cube_free")
|
||||
cube_qadr = int(model.jnt_qposadr[cube_joint])
|
||||
cube_geom = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_GEOM, "pick_cube_geom")
|
||||
finger_geoms = {
|
||||
mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_GEOM, "left_finger_pad_z_axis"),
|
||||
mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_GEOM, "right_finger_pad_z_axis"),
|
||||
}
|
||||
ctrl_ranges = model.actuator_ctrlrange.copy()
|
||||
gripper_joint = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "gripper_Left_1_Joint")
|
||||
gripper_qadr = int(model.jnt_qposadr[gripper_joint])
|
||||
joint_ids = [mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, f"joint_{i}") for i in range(1, 7)]
|
||||
qpos_adrs = [int(model.jnt_qposadr[jid]) for jid in joint_ids]
|
||||
dof_adrs = [int(model.jnt_dofadr[jid]) for jid in joint_ids]
|
||||
|
||||
duration_scale = 1 if grasp_mode == "kinematic_attachment" else 8
|
||||
phases = [
|
||||
("home", np.array([0.42, 0.00, 0.72]), 0.0, 35 * duration_scale, False),
|
||||
("approach", np.array([0.71, 0.015, 0.42]), 0.0, 55 * duration_scale, False),
|
||||
("descend", np.array([0.71, 0.015, 0.24]), 0.0, 40 * duration_scale, False),
|
||||
("grasp", np.array([0.71, 0.015, 0.24]), -0.91, 45 * duration_scale, True),
|
||||
("lift", np.array([0.71, 0.015, 0.60]), -0.91, 200 * duration_scale, True),
|
||||
("place", np.array([0.43, 0.18, 0.40]), -0.91, 65 * duration_scale, True),
|
||||
("release", np.array([0.43, 0.18, 0.40]), 0.0, 35 * duration_scale, False),
|
||||
("retreat", np.array([0.42, 0.00, 0.72]), 0.0, 45 * duration_scale, False),
|
||||
]
|
||||
|
||||
trajectory = []
|
||||
frames = []
|
||||
renderer = None
|
||||
if render:
|
||||
renderer = mujoco.Renderer(model, height=height, width=width)
|
||||
|
||||
attached = False
|
||||
contact_steps = 0
|
||||
bilateral_contact_steps = 0
|
||||
phase_contacts = {phase: {"contact_steps": 0, "bilateral_contact_steps": 0} for phase, *_ in phases}
|
||||
contact_force_metrics = {"max_normal": 0.0, "max_tangent": 0.0, "max_normal_z": 0.0, "samples": 0}
|
||||
gripper_qpos_range = [float(data.qpos[gripper_qadr]), float(data.qpos[gripper_qadr])]
|
||||
max_cube_z = float(data.qpos[cube_qadr + 2])
|
||||
initial_cube = data.qpos[cube_qadr : cube_qadr + 3].copy()
|
||||
settled_cube_z = float(initial_cube[2])
|
||||
started = time.time()
|
||||
frame_stride = max(1, round((1.0 / model.opt.timestep) / fps))
|
||||
step_index = 0
|
||||
for phase, target, gripper_ctrl, count, attach_cube in phases:
|
||||
if phase == "grasp":
|
||||
settled_cube_z = float(data.qpos[cube_qadr + 2])
|
||||
max_cube_z = settled_cube_z
|
||||
q_target = solve_ik(model, data, site_id, target, qpos_adrs, dof_adrs, ctrl_ranges)
|
||||
for local in range(count):
|
||||
alpha = (local + 1) / count
|
||||
current = data.ctrl[:6].copy()
|
||||
data.ctrl[:6] = current * (1 - alpha) + q_target * alpha
|
||||
data.ctrl[6] = gripper_ctrl
|
||||
if grasp_mode == "kinematic_attachment" and (attach_cube or attached):
|
||||
attached = attach_cube
|
||||
if attached:
|
||||
set_cube_pose(data, cube_qadr, data.site_xpos[site_id] + np.array([0.0, 0.0, -0.095]))
|
||||
mujoco.mj_step(model, data)
|
||||
if grasp_mode == "kinematic_attachment" and attached:
|
||||
set_cube_pose(data, cube_qadr, data.site_xpos[site_id] + np.array([0.0, 0.0, -0.095]))
|
||||
mujoco.mj_forward(model, data)
|
||||
touching_fingers = cube_finger_contacts(data, cube_geom, finger_geoms)
|
||||
accumulate_contact_metrics(model, data, cube_geom, finger_geoms, contact_force_metrics)
|
||||
if touching_fingers:
|
||||
contact_steps += 1
|
||||
phase_contacts[phase]["contact_steps"] += 1
|
||||
if len(touching_fingers) == 2:
|
||||
bilateral_contact_steps += 1
|
||||
phase_contacts[phase]["bilateral_contact_steps"] += 1
|
||||
gripper_qpos_range[0] = min(gripper_qpos_range[0], float(data.qpos[gripper_qadr]))
|
||||
gripper_qpos_range[1] = max(gripper_qpos_range[1], float(data.qpos[gripper_qadr]))
|
||||
max_cube_z = max(max_cube_z, float(data.qpos[cube_qadr + 2]))
|
||||
if step_index % frame_stride == 0:
|
||||
trajectory.append(sample_state(model, data, phase, step_index, site_id, cube_qadr))
|
||||
if renderer is not None:
|
||||
renderer.update_scene(data, camera="overview")
|
||||
frames.append(renderer.render().copy())
|
||||
step_index += 1
|
||||
|
||||
if renderer is not None:
|
||||
renderer.close()
|
||||
|
||||
trajectory_path = out_dir / "rm65_pick_place_trajectory.json"
|
||||
report_path = out_dir / "rm65_pick_place_rollout_report.json"
|
||||
gif_path = out_dir / "rm65_pick_place_rollout.gif"
|
||||
keyframe_path = out_dir / "rm65_pick_place_final.png"
|
||||
trajectory_path.write_text(json.dumps({"schema": "real2sim.rm65_pick_place_trajectory.v1", "samples": trajectory}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
artifact_paths = {"trajectory": str(trajectory_path)}
|
||||
if frames:
|
||||
imageio.mimsave(gif_path, frames, fps=fps)
|
||||
imageio.imwrite(keyframe_path, frames[-1])
|
||||
artifact_paths["gif"] = str(gif_path)
|
||||
artifact_paths["final_frame"] = str(keyframe_path)
|
||||
|
||||
final = trajectory[-1] if trajectory else {}
|
||||
final_cube = np.asarray(final.get("cube_pos") or initial_cube, dtype=float)
|
||||
target_cube = np.array([0.43, 0.18, 0.31])
|
||||
lift_height = max_cube_z - settled_cube_z
|
||||
place_error = float(np.linalg.norm(final_cube - target_cube))
|
||||
physical_success = bilateral_contact_steps > 0 and lift_height >= 0.10 and place_error <= 0.12
|
||||
summary = {
|
||||
"mode": "scripted_ik_physical_contact" if grasp_mode == "physical" else "scripted_ik_with_cube_attachment",
|
||||
"steps": step_index,
|
||||
"samples": len(trajectory),
|
||||
"sim_time": float(data.time),
|
||||
"final_gripper": final.get("gripper_pos"),
|
||||
"final_cube": final.get("cube_pos"),
|
||||
"initial_cube": [float(x) for x in initial_cube],
|
||||
"settled_cube_z": settled_cube_z,
|
||||
"max_cube_z": max_cube_z,
|
||||
"lift_height": lift_height,
|
||||
"place_target": [float(x) for x in target_cube],
|
||||
"place_error": place_error,
|
||||
"contact_steps": contact_steps,
|
||||
"bilateral_contact_steps": bilateral_contact_steps,
|
||||
"phase_contacts": phase_contacts,
|
||||
"gripper_qpos_range": gripper_qpos_range,
|
||||
"contact_force_metrics": contact_force_metrics,
|
||||
"physical_success": physical_success if grasp_mode == "physical" else None,
|
||||
"elapsed_ms": round((time.time() - started) * 1000),
|
||||
}
|
||||
report = {
|
||||
"schema": "real2sim.rm65_pick_place_rollout_report.v1",
|
||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"ok": physical_success if grasp_mode == "physical" else True,
|
||||
"runtime": "mujoco",
|
||||
"scene": str(sim_path),
|
||||
"summary": summary,
|
||||
"phases": [phase for phase, *_ in phases],
|
||||
"note": (
|
||||
"Scripted IK with contact-physics grasp; the cube pose is never overwritten. This is not a learned policy."
|
||||
if grasp_mode == "physical"
|
||||
else "Diagnostic-only rollout. Cube follows the gripper during grasp/lift/place; this is not a learned policy."
|
||||
),
|
||||
}
|
||||
report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
artifact_paths["report"] = str(report_path)
|
||||
return {"ok": bool(report["ok"]), "summary": summary, "artifact_paths": artifact_paths}
|
||||
|
||||
|
||||
def cube_finger_contacts(data: Any, cube_geom: int, finger_geoms: set[int]) -> set[int]:
|
||||
touching: set[int] = set()
|
||||
for index in range(data.ncon):
|
||||
contact = data.contact[index]
|
||||
pair = {int(contact.geom1), int(contact.geom2)}
|
||||
if cube_geom not in pair:
|
||||
continue
|
||||
touching.update(pair & finger_geoms)
|
||||
return touching
|
||||
|
||||
|
||||
def accumulate_contact_metrics(model: Any, data: Any, cube_geom: int, finger_geoms: set[int], metrics: dict[str, Any]) -> None:
|
||||
import mujoco
|
||||
|
||||
for index in range(data.ncon):
|
||||
contact = data.contact[index]
|
||||
pair = {int(contact.geom1), int(contact.geom2)}
|
||||
if cube_geom not in pair or not (pair & finger_geoms):
|
||||
continue
|
||||
force = np.zeros(6)
|
||||
mujoco.mj_contactForce(model, data, index, force)
|
||||
metrics["max_normal"] = max(float(metrics["max_normal"]), abs(float(force[0])))
|
||||
metrics["max_tangent"] = max(float(metrics["max_tangent"]), float(np.linalg.norm(force[1:3])))
|
||||
metrics["max_normal_z"] = max(float(metrics["max_normal_z"]), abs(float(contact.frame[2])))
|
||||
metrics["samples"] = int(metrics["samples"]) + 1
|
||||
|
||||
|
||||
def solve_ik(model: Any, data: Any, site_id: int, target: np.ndarray, qpos_adrs: list[int], dof_adrs: list[int], ctrl_ranges: np.ndarray) -> np.ndarray:
|
||||
import mujoco
|
||||
|
||||
q_backup = data.qpos.copy()
|
||||
q = data.qpos[qpos_adrs].copy()
|
||||
damping = 1e-3
|
||||
for _ in range(80):
|
||||
data.qpos[qpos_adrs] = q
|
||||
mujoco.mj_forward(model, data)
|
||||
err = target - data.site_xpos[site_id]
|
||||
if np.linalg.norm(err) < 0.012:
|
||||
break
|
||||
jacp = np.zeros((3, model.nv))
|
||||
jacr = np.zeros((3, model.nv))
|
||||
mujoco.mj_jacSite(model, data, jacp, jacr, site_id)
|
||||
j = jacp[:, dof_adrs]
|
||||
dq = j.T @ np.linalg.solve(j @ j.T + damping * np.eye(3), err)
|
||||
q = q + np.clip(dq, -0.08, 0.08)
|
||||
for i in range(6):
|
||||
q[i] = float(np.clip(q[i], ctrl_ranges[i, 0], ctrl_ranges[i, 1]))
|
||||
data.qpos[:] = q_backup
|
||||
mujoco.mj_forward(model, data)
|
||||
return q
|
||||
|
||||
|
||||
def set_cube_pose(data: Any, qadr: int, pos: np.ndarray) -> None:
|
||||
data.qpos[qadr : qadr + 3] = pos
|
||||
data.qpos[qadr + 3 : qadr + 7] = np.array([1.0, 0.0, 0.0, 0.0])
|
||||
data.qvel[qadr : qadr + 6] = 0
|
||||
|
||||
|
||||
def sample_state(model: Any, data: Any, phase: str, step: int, site_id: int, cube_qadr: int) -> dict[str, Any]:
|
||||
return {
|
||||
"step": step,
|
||||
"time": float(data.time),
|
||||
"phase": phase,
|
||||
"qpos_arm": [float(x) for x in data.qpos[:6]],
|
||||
"gripper_ctrl": float(data.ctrl[6]),
|
||||
"gripper_pos": [float(x) for x in data.site_xpos[site_id]],
|
||||
"cube_pos": [float(x) for x in data.qpos[cube_qadr : cube_qadr + 3]],
|
||||
}
|
||||
|
||||
|
||||
def update_manifest(manifest_path: Path, rollout_dir: Path, result: dict[str, Any], artifacts: dict[str, dict[str, Any]]) -> None:
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
new_assets = []
|
||||
for key, artifact in artifacts.items():
|
||||
path = Path(artifact["path"])
|
||||
new_assets.append(
|
||||
{
|
||||
"name": path.name,
|
||||
"role": f"rollout_{key}",
|
||||
"kind": artifact_kind(path),
|
||||
"path": str(path),
|
||||
"relative_path": str(path.relative_to(manifest_path.parent)),
|
||||
"url": f"/api/artifacts/{artifact['id']}/content",
|
||||
"exists": True,
|
||||
"size_bytes": path.stat().st_size,
|
||||
"size_label": format_bytes(path.stat().st_size),
|
||||
"generated": True,
|
||||
}
|
||||
)
|
||||
old_assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and str(item.get("role", "")).startswith("rollout_"))]
|
||||
manifest["assets"] = [*new_assets, *old_assets]
|
||||
manifest["latest_rollout"] = {
|
||||
"dir": str(rollout_dir),
|
||||
"summary": result["summary"],
|
||||
"artifacts": new_assets,
|
||||
"status": "scripted_rollout_succeeded",
|
||||
}
|
||||
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def artifact_kind(path: Path) -> str:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix in {".png", ".jpg", ".jpeg", ".webp", ".gif"}:
|
||||
return "image"
|
||||
if suffix == ".json":
|
||||
return "metadata"
|
||||
return suffix.lstrip(".") or "file"
|
||||
|
||||
|
||||
def format_bytes(value: int) -> str:
|
||||
units = ["B", "KB", "MB", "GB"]
|
||||
size = float(value)
|
||||
for unit in units:
|
||||
if size < 1024 or unit == units[-1]:
|
||||
return f"{size:.1f} {unit}" if unit != "B" else f"{int(size)} B"
|
||||
size /= 1024
|
||||
return f"{value} B"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
244
scripts/scene_viewer_probe_local.mjs
Normal file
244
scripts/scene_viewer_probe_local.mjs
Normal file
@ -0,0 +1,244 @@
|
||||
#!/usr/bin/env node
|
||||
import { spawn } from "node:child_process";
|
||||
import { createRequire } from "node:module";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
const key = process.env.REAL2SIM_WUYING_KEY || "/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519";
|
||||
const host = process.env.REAL2SIM_WUYING_HOST || "root@39.106.101.95";
|
||||
const remotePort = Number(process.env.REAL2SIM_REMOTE_PORT || "18126");
|
||||
const requestedBase = process.env.REAL2SIM_FRONTEND_BASE_URL || "";
|
||||
const outRoot = process.env.REAL2SIM_ACCEPTANCE_OUT || path.resolve("real2sim-platform", "workspace", "acceptance_runs", "viewer-probe");
|
||||
const liveRequired = process.env.REAL2SIM_VIEWER_LIVE_REQUIRED === "1";
|
||||
const liveTimeoutMs = Number(process.env.REAL2SIM_VIEWER_LIVE_TIMEOUT_MS || "150000");
|
||||
const sceneIdOverride = process.env.REAL2SIM_VIEWER_SCENE_ID || "";
|
||||
const timestamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..*/, "Z");
|
||||
const outDir = path.join(outRoot, timestamp);
|
||||
const reportPath = path.join(outDir, "scene_viewer_probe_report.json");
|
||||
const failures = [];
|
||||
const warnings = [];
|
||||
let tunnel = null;
|
||||
|
||||
fs.mkdirSync(outDir, { recursive: true });
|
||||
|
||||
function fail(message, context = {}) {
|
||||
failures.push({ message, ...context });
|
||||
console.error("FAIL:", message, context);
|
||||
}
|
||||
|
||||
function warn(message, context = {}) {
|
||||
warnings.push({ message, ...context });
|
||||
console.warn("WARN:", message, context);
|
||||
}
|
||||
|
||||
function writeReport(report) {
|
||||
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
|
||||
return report;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const startedAt = new Date().toISOString();
|
||||
const baseUrl = requestedBase || await startTunnel();
|
||||
const scene = sceneIdOverride ? { id: sceneIdOverride } : await pickPreviewableScene(baseUrl);
|
||||
let manifestGate = null;
|
||||
let liveProbe = null;
|
||||
|
||||
if (!scene) {
|
||||
fail("no scene available for viewer probe");
|
||||
} else {
|
||||
manifestGate = await validateManifestGate(baseUrl, scene.id);
|
||||
liveProbe = await runLiveProbe(baseUrl, scene.id).catch((error) => {
|
||||
const payload = { ok: false, error: error.message, stack: error.stack };
|
||||
if (liveRequired) fail("live viewer probe failed", payload);
|
||||
else warn("live viewer probe failed in non-required mode", payload);
|
||||
return payload;
|
||||
});
|
||||
}
|
||||
|
||||
const report = writeReport({
|
||||
schema: "real2sim.scene_viewer_probe_report.v1",
|
||||
started_at: startedAt,
|
||||
finished_at: new Date().toISOString(),
|
||||
base_url: baseUrl,
|
||||
scene_id: scene?.id || "",
|
||||
live_required: liveRequired,
|
||||
manifest_gate: manifestGate,
|
||||
live_probe: liveProbe,
|
||||
failures,
|
||||
warnings,
|
||||
pass: failures.length === 0,
|
||||
});
|
||||
console.log(JSON.stringify({ pass: report.pass, report: reportPath, scene_id: report.scene_id, live_probe: report.live_probe }, null, 2));
|
||||
if (tunnel) tunnel.kill("SIGTERM");
|
||||
process.exitCode = report.pass ? 0 : 1;
|
||||
}
|
||||
|
||||
async function runLiveProbe(baseUrl, sceneId) {
|
||||
const { chromium } = await importPlaywright();
|
||||
const browser = await chromium.launch({ headless: true });
|
||||
const page = await browser.newPage({ viewport: { width: 1440, height: 900 }, deviceScaleFactor: 1 });
|
||||
const consoleErrors = [];
|
||||
page.on("console", (message) => {
|
||||
if (message.type() === "error") consoleErrors.push(message.text());
|
||||
});
|
||||
try {
|
||||
const viewerUrl = `${baseUrl}/static/scene-viewer.html?scene_id=${encodeURIComponent(sceneId)}`;
|
||||
await withTimeout((async () => {
|
||||
await page.goto(viewerUrl, { waitUntil: "domcontentloaded", timeout: 30000 });
|
||||
await page.waitForSelector("canvas", { timeout: 20000 });
|
||||
await page.waitForFunction(() => {
|
||||
const debug = window.real2simViewerDebug;
|
||||
return debug && debug.canvas && debug.canvas.width > 0 && debug.canvas.height > 0 && (debug.loaded || debug.backend === "placeholder" || debug.errors?.length);
|
||||
}, null, { timeout: Math.max(30000, liveTimeoutMs - 30000) });
|
||||
await page.mouse.move(720, 450);
|
||||
await page.mouse.down();
|
||||
await page.mouse.move(900, 530, { steps: 8 });
|
||||
await page.mouse.up();
|
||||
})(), liveTimeoutMs, `live viewer probe timed out after ${liveTimeoutMs}ms`);
|
||||
const debug = await page.evaluate(() => window.real2simViewerDebug.snapshot?.() || window.real2simViewerDebug);
|
||||
const ok = !!debug && !debug.errors?.length && (debug.loaded || debug.backend === "placeholder");
|
||||
if (liveRequired && !ok) fail("live viewer debug is not acceptable", { debug, console_errors: consoleErrors });
|
||||
return { ok, url: viewerUrl, debug, console_errors: consoleErrors };
|
||||
} finally {
|
||||
await page.close().catch(() => {});
|
||||
await browser.close().catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
async function validateManifestGate(baseUrl, sceneId) {
|
||||
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(sceneId)}/manifest`);
|
||||
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
|
||||
const optimized = assets.find((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")));
|
||||
if (!optimized) {
|
||||
fail("scene manifest has no optimized splat asset", { scene_id: sceneId });
|
||||
return { ok: false, scene_id: sceneId };
|
||||
}
|
||||
if (!optimized.url) fail("optimized splat asset has no URL", { scene_id: sceneId, asset: optimized });
|
||||
if (optimized.autoload === false) fail("optimized splat asset is not autoload", { scene_id: sceneId, asset: optimized });
|
||||
return {
|
||||
ok: failures.length === 0,
|
||||
scene_id: sceneId,
|
||||
asset: {
|
||||
name: optimized.name || "",
|
||||
url: optimized.url || "",
|
||||
size_label: optimized.size_label || "",
|
||||
size_bytes: optimized.size_bytes || 0,
|
||||
autoload: optimized.autoload !== false,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function pickPreviewableScene(baseUrl) {
|
||||
const scenes = await fetchJson(baseUrl, "/api/scenes");
|
||||
if (!Array.isArray(scenes) || scenes.length === 0) return null;
|
||||
const scored = [];
|
||||
for (const scene of scenes) {
|
||||
try {
|
||||
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(scene.id)}/manifest`);
|
||||
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
|
||||
let score = 0;
|
||||
if (assets.some((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")))) score += 100;
|
||||
if (assets.some((asset) => /pose_init_trained_scene\.ply$/i.test(String(asset.name || asset.path || "")))) score += 50;
|
||||
if (assets.some((asset) => /\.(ply|glb|gltf|obj)$/i.test(String(asset.name || asset.path || "")))) score += 20;
|
||||
scored.push({ scene, score });
|
||||
} catch (error) {
|
||||
warn("scene manifest unavailable", { scene_id: scene.id, error: error.message });
|
||||
}
|
||||
}
|
||||
scored.sort((left, right) => right.score - left.score);
|
||||
return scored[0]?.scene || scenes[0];
|
||||
}
|
||||
|
||||
async function importPlaywright() {
|
||||
const moduleDir = process.env.REAL2SIM_PLAYWRIGHT_NODE_MODULES || "";
|
||||
if (moduleDir) {
|
||||
const require = createRequire(path.join(moduleDir, "real2sim-playwright-loader.cjs"));
|
||||
return require("playwright");
|
||||
}
|
||||
try {
|
||||
return await import("playwright");
|
||||
} catch (error) {
|
||||
throw new Error(`playwright is required. Set REAL2SIM_PLAYWRIGHT_NODE_MODULES=/path/to/node_modules. Original error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function startTunnel() {
|
||||
const port = Number(process.env.REAL2SIM_LOCAL_TUNNEL_PORT || await findPort(18140));
|
||||
tunnel = spawn("ssh", [
|
||||
"-i", key,
|
||||
"-o", "BatchMode=yes",
|
||||
"-o", "StrictHostKeyChecking=accept-new",
|
||||
"-N",
|
||||
"-L", `${port}:127.0.0.1:${remotePort}`,
|
||||
host,
|
||||
], { stdio: ["ignore", "pipe", "pipe"] });
|
||||
tunnel.stderr.on("data", (chunk) => process.stderr.write(chunk));
|
||||
tunnel.stdout.on("data", (chunk) => process.stdout.write(chunk));
|
||||
const baseUrl = `http://127.0.0.1:${port}`;
|
||||
const deadline = Date.now() + 30000;
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
await fetchJson(baseUrl, "/api/health");
|
||||
return baseUrl;
|
||||
} catch {
|
||||
await sleep(500);
|
||||
}
|
||||
}
|
||||
throw new Error(`SSH tunnel did not become ready at ${baseUrl}`);
|
||||
}
|
||||
|
||||
async function findPort(start) {
|
||||
const net = await import("node:net");
|
||||
for (let port = start; port < start + 100; port += 1) {
|
||||
const ok = await new Promise((resolve) => {
|
||||
const server = net.createServer();
|
||||
server.once("error", () => resolve(false));
|
||||
server.once("listening", () => server.close(() => resolve(true)));
|
||||
server.listen(port, "127.0.0.1");
|
||||
});
|
||||
if (ok) return port;
|
||||
}
|
||||
throw new Error("no local tunnel port available");
|
||||
}
|
||||
|
||||
async function fetchJson(baseUrl, apiPath) {
|
||||
const response = await fetch(`${baseUrl}${apiPath}`);
|
||||
if (!response.ok) throw new Error(`${apiPath} failed with ${response.status}: ${await response.text()}`);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async function withTimeout(promise, timeoutMs, message) {
|
||||
let timer;
|
||||
try {
|
||||
return await Promise.race([
|
||||
promise,
|
||||
new Promise((_, reject) => {
|
||||
timer = setTimeout(() => reject(new Error(message)), timeoutMs);
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
function sleep(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
fail("scene viewer probe runner crashed", { error: error.message, stack: error.stack });
|
||||
const report = writeReport({
|
||||
schema: "real2sim.scene_viewer_probe_report.v1",
|
||||
started_at: timestamp,
|
||||
finished_at: new Date().toISOString(),
|
||||
base_url: requestedBase,
|
||||
scene_id: sceneIdOverride,
|
||||
live_required: liveRequired,
|
||||
failures,
|
||||
warnings,
|
||||
pass: false,
|
||||
});
|
||||
console.error(JSON.stringify({ pass: false, report: reportPath, error: error.message }, null, 2));
|
||||
if (tunnel) tunnel.kill("SIGTERM");
|
||||
process.exit(report.pass ? 0 : 1);
|
||||
});
|
||||
331
scripts/setup_wuying_desktop_launcher.sh
Executable file
331
scripts/setup_wuying_desktop_launcher.sh
Executable file
@ -0,0 +1,331 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
|
||||
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
|
||||
PROJECT_ROOT="${REAL2SIM_REMOTE_PLATFORM_ROOT:-/opt/real2sim2model/real2sim-platform}"
|
||||
SERVICE="${REAL2SIM_PLATFORM_SERVICE:-real2sim-platform.service}"
|
||||
PORT="${REAL2SIM_PLATFORM_PORT:-18126}"
|
||||
DESKTOP_NAME="${REAL2SIM_DESKTOP_NAME:-Real2Sim 平台}"
|
||||
DESKTOP_FILE_NAME="${REAL2SIM_DESKTOP_FILE_NAME:-Real2Sim Platform.desktop}"
|
||||
MODE="local"
|
||||
CLEAN_DESKTOP=1
|
||||
CREATE_LAUNCHER=1
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
scripts/setup_wuying_desktop_launcher.sh --remote
|
||||
scripts/setup_wuying_desktop_launcher.sh --local
|
||||
|
||||
Options:
|
||||
--remote Run the setup on the Wuying host over SSH.
|
||||
--local Run on the current Linux desktop host.
|
||||
--no-clean Do not move existing desktop icons into backup.
|
||||
--no-launcher Do not create/update the Real2Sim launcher.
|
||||
--project-root PATH Remote Real2Sim platform root. Default: /opt/real2sim2model/real2sim-platform
|
||||
--service NAME systemd service name. Default: real2sim-platform.service
|
||||
--port PORT Local service port. Default: 18126
|
||||
|
||||
The cleanup is reversible: existing desktop entries are moved into a hidden
|
||||
.real2sim_desktop_backup_<timestamp> directory on the Desktop, not deleted.
|
||||
EOF
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--remote) MODE="remote"; shift ;;
|
||||
--local) MODE="local"; shift ;;
|
||||
--no-clean) CLEAN_DESKTOP=0; shift ;;
|
||||
--no-launcher) CREATE_LAUNCHER=0; shift ;;
|
||||
--project-root) PROJECT_ROOT="$2"; shift 2 ;;
|
||||
--service) SERVICE="$2"; shift 2 ;;
|
||||
--port) PORT="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ "$MODE" == "remote" ]]; then
|
||||
ssh -i "$KEY" \
|
||||
-o BatchMode=yes \
|
||||
-o ConnectTimeout=20 \
|
||||
-o StrictHostKeyChecking=accept-new \
|
||||
"$HOST" \
|
||||
"REAL2SIM_REMOTE_PLATFORM_ROOT='$PROJECT_ROOT' REAL2SIM_PLATFORM_SERVICE='$SERVICE' REAL2SIM_PLATFORM_PORT='$PORT' REAL2SIM_DESKTOP_NAME='$DESKTOP_NAME' REAL2SIM_DESKTOP_FILE_NAME='$DESKTOP_FILE_NAME' bash -s -- --local $([[ "$CLEAN_DESKTOP" == "0" ]] && printf '%s' '--no-clean') $([[ "$CREATE_LAUNCHER" == "0" ]] && printf '%s' '--no-launcher')" < "$0"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ "$(uname -s)" != "Linux" ]]; then
|
||||
echo "This setup must run on the Wuying Linux desktop host." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
timestamp="$(date +%Y%m%d-%H%M%S)"
|
||||
|
||||
add_desktop_dir() {
|
||||
local dir="$1"
|
||||
[[ -n "$dir" ]] || return 0
|
||||
if [[ ! -d "$dir" ]]; then
|
||||
mkdir -p "$dir"
|
||||
fi
|
||||
printf '%s\n' "$dir"
|
||||
}
|
||||
|
||||
discover_desktop_dirs() {
|
||||
local dirs=()
|
||||
if [[ -n "${XDG_DESKTOP_DIR:-}" ]]; then
|
||||
dirs+=("$XDG_DESKTOP_DIR")
|
||||
fi
|
||||
dirs+=("$HOME/Desktop" "$HOME/桌面")
|
||||
if [[ "$(id -u)" == "0" ]]; then
|
||||
dirs+=("/root/Desktop" "/root/桌面")
|
||||
local home_dir
|
||||
for home_dir in /home/*; do
|
||||
[[ -d "$home_dir" ]] || continue
|
||||
dirs+=("$home_dir/Desktop" "$home_dir/桌面")
|
||||
done
|
||||
fi
|
||||
|
||||
local seen="" dir
|
||||
for dir in "${dirs[@]}"; do
|
||||
[[ -n "$dir" ]] || continue
|
||||
case ":$seen:" in
|
||||
*":$dir:"*) continue ;;
|
||||
esac
|
||||
seen="$seen:$dir"
|
||||
add_desktop_dir "$dir"
|
||||
done
|
||||
}
|
||||
|
||||
owner_for_path() {
|
||||
local path="$1"
|
||||
stat -c '%U:%G' "$path" 2>/dev/null || printf 'root:root\n'
|
||||
}
|
||||
|
||||
write_launcher_script() {
|
||||
local home_dir="$1"
|
||||
local owner_group="$2"
|
||||
local bin_dir="$home_dir/.local/bin"
|
||||
local state_dir="$home_dir/.local/state"
|
||||
local launcher="$bin_dir/real2sim-platform-launch"
|
||||
|
||||
mkdir -p "$bin_dir" "$state_dir"
|
||||
cat > "$launcher" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
set -u
|
||||
|
||||
PROJECT_ROOT="$PROJECT_ROOT"
|
||||
SERVICE="$SERVICE"
|
||||
PORT="$PORT"
|
||||
URL="http://127.0.0.1:$PORT/"
|
||||
LOG="\${XDG_STATE_HOME:-\$HOME/.local/state}/real2sim-launcher.log"
|
||||
mkdir -p "\$(dirname "\$LOG")"
|
||||
{
|
||||
echo "==== \$(date '+%Y-%m-%d %H:%M:%S') Real2Sim launcher ===="
|
||||
|
||||
run_with_timeout() {
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
timeout 25 "\$@"
|
||||
else
|
||||
"\$@"
|
||||
fi
|
||||
}
|
||||
|
||||
health_ok() {
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
curl -fsS "http://127.0.0.1:$PORT/api/health" >/dev/null 2>&1
|
||||
else
|
||||
python3 - <<'PY' >/dev/null 2>&1
|
||||
import urllib.request
|
||||
urllib.request.urlopen('http://127.0.0.1:$PORT/api/health', timeout=2).read()
|
||||
PY
|
||||
fi
|
||||
}
|
||||
|
||||
wait_for_health() {
|
||||
local i
|
||||
for i in \$(seq 1 40); do
|
||||
health_ok && return 0
|
||||
sleep 1
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
if command -v systemctl >/dev/null 2>&1 && systemctl list-unit-files "$SERVICE" >/dev/null 2>&1; then
|
||||
if systemctl is-failed --quiet "$SERVICE"; then
|
||||
run_with_timeout systemctl restart "$SERVICE" || true
|
||||
else
|
||||
run_with_timeout systemctl start "$SERVICE" || true
|
||||
fi
|
||||
fi
|
||||
|
||||
if ! wait_for_health; then
|
||||
if [[ -x "\$PROJECT_ROOT/.venv/bin/real2sim-platform" ]]; then
|
||||
if ! pgrep -f "\$PROJECT_ROOT/.venv/bin/real2sim-platform" >/dev/null 2>&1; then
|
||||
cd "\$PROJECT_ROOT" || exit 1
|
||||
REAL2SIM_PLATFORM_CONFIG="\$PROJECT_ROOT/config.toml" nohup "\$PROJECT_ROOT/.venv/bin/real2sim-platform" >> "\$LOG" 2>&1 &
|
||||
fi
|
||||
wait_for_health || true
|
||||
fi
|
||||
fi
|
||||
|
||||
launch_chrome_app() {
|
||||
local browser="\$1"
|
||||
mkdir -p "\$HOME/.config/real2sim-platform-chrome"
|
||||
nohup "\$browser" \\
|
||||
--app="\$URL" \\
|
||||
--new-window \\
|
||||
--user-data-dir="\$HOME/.config/real2sim-platform-chrome" \\
|
||||
--no-first-run \\
|
||||
--disable-sync \\
|
||||
--disable-translate \\
|
||||
--disable-background-networking \\
|
||||
--disable-component-update \\
|
||||
--disable-domain-reliability \\
|
||||
--disable-client-side-phishing-detection \\
|
||||
--no-default-browser-check \\
|
||||
--no-service-autorun \\
|
||||
--disable-features=Translate,TranslateUI,AutofillServerCommunication \\
|
||||
--window-size=1440,900 \\
|
||||
--force-device-scale-factor=1 \\
|
||||
>/dev/null 2>&1 &
|
||||
}
|
||||
|
||||
if command -v google-chrome >/dev/null 2>&1; then
|
||||
launch_chrome_app google-chrome
|
||||
elif command -v chromium-browser >/dev/null 2>&1; then
|
||||
launch_chrome_app chromium-browser
|
||||
elif command -v xdg-open >/dev/null 2>&1; then
|
||||
nohup xdg-open "\$URL" >/dev/null 2>&1 &
|
||||
elif command -v firefox >/dev/null 2>&1; then
|
||||
nohup firefox "\$URL" >/dev/null 2>&1 &
|
||||
else
|
||||
echo "No browser opener found. Open \$URL manually."
|
||||
fi
|
||||
|
||||
if ! health_ok; then
|
||||
echo "Real2Sim did not become healthy. Check: \$LOG"
|
||||
if command -v x-terminal-emulator >/dev/null 2>&1; then
|
||||
x-terminal-emulator -e "bash -lc 'tail -n 120 \"\$LOG\"; echo; read -p \"Press Enter to close\"'" &
|
||||
elif command -v gnome-terminal >/dev/null 2>&1; then
|
||||
gnome-terminal -- bash -lc "tail -n 120 \"\$LOG\"; echo; read -p 'Press Enter to close'" &
|
||||
fi
|
||||
fi
|
||||
} >> "\$LOG" 2>&1
|
||||
EOF
|
||||
chmod 0755 "$launcher"
|
||||
chown "$owner_group" "$launcher" "$bin_dir" "$state_dir" 2>/dev/null || true
|
||||
printf '%s\n' "$launcher"
|
||||
}
|
||||
|
||||
write_icon() {
|
||||
local home_dir="$1"
|
||||
local owner_group="$2"
|
||||
local icon_dir="$home_dir/.local/share/icons"
|
||||
local icon_file="$icon_dir/real2sim-platform.svg"
|
||||
mkdir -p "$icon_dir"
|
||||
cat > "$icon_file" <<'EOF'
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
|
||||
<defs>
|
||||
<linearGradient id="g" x1="16" y1="16" x2="112" y2="112" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#66ead4"/>
|
||||
<stop offset="0.55" stop-color="#23bca4"/>
|
||||
<stop offset="1" stop-color="#6aa8ff"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
<rect width="128" height="128" rx="28" fill="#071119"/>
|
||||
<path d="M64 16 104 39v50L64 112 24 89V39L64 16Z" fill="none" stroke="url(#g)" stroke-width="8" stroke-linejoin="round"/>
|
||||
<path d="M64 36 84 47.5v25L64 84 44 72.5v-25L64 36Z" fill="none" stroke="#9ff7ea" stroke-width="6" stroke-linejoin="round"/>
|
||||
<path d="M64 36v48M44 48l40 24M84 48 44 72" stroke="#2fdbc4" stroke-width="4" stroke-linecap="round"/>
|
||||
</svg>
|
||||
EOF
|
||||
chmod 0644 "$icon_file"
|
||||
chown "$owner_group" "$icon_file" "$icon_dir" 2>/dev/null || true
|
||||
printf '%s\n' "$icon_file"
|
||||
}
|
||||
|
||||
clean_desktop_dir() {
|
||||
local desktop_dir="$1"
|
||||
local keep_file="$desktop_dir/$DESKTOP_FILE_NAME"
|
||||
local backup_dir="$desktop_dir/.real2sim_desktop_backup_$timestamp"
|
||||
local moved=0
|
||||
shopt -s nullglob dotglob
|
||||
local item base
|
||||
for item in "$desktop_dir"/* "$desktop_dir"/.[!.]* "$desktop_dir"/..?*; do
|
||||
[[ -e "$item" ]] || continue
|
||||
base="$(basename "$item")"
|
||||
case "$base" in
|
||||
"$DESKTOP_FILE_NAME"|".directory"|".real2sim_desktop_backup_"*) continue ;;
|
||||
esac
|
||||
if [[ "$item" == "$keep_file" ]]; then
|
||||
continue
|
||||
fi
|
||||
mkdir -p "$backup_dir"
|
||||
mv "$item" "$backup_dir/"
|
||||
moved=$((moved + 1))
|
||||
done
|
||||
shopt -u nullglob dotglob
|
||||
if [[ "$moved" == "0" && -d "$backup_dir" ]]; then
|
||||
rmdir "$backup_dir" 2>/dev/null || true
|
||||
fi
|
||||
printf '%s\n' "$moved"
|
||||
}
|
||||
|
||||
write_desktop_file() {
|
||||
local desktop_dir="$1"
|
||||
local owner_group="$2"
|
||||
local launcher="$3"
|
||||
local icon_file="$4"
|
||||
local desktop_file="$desktop_dir/$DESKTOP_FILE_NAME"
|
||||
|
||||
cat > "$desktop_file" <<EOF
|
||||
[Desktop Entry]
|
||||
Type=Application
|
||||
Name=$DESKTOP_NAME
|
||||
Comment=启动 Real2Sim 服务并打开平台
|
||||
Exec=$launcher
|
||||
Icon=$icon_file
|
||||
Terminal=false
|
||||
Categories=Development;Science;
|
||||
StartupNotify=true
|
||||
EOF
|
||||
chmod 0755 "$desktop_file"
|
||||
chown "$owner_group" "$desktop_file" 2>/dev/null || true
|
||||
if command -v gio >/dev/null 2>&1; then
|
||||
gio set "$desktop_file" metadata::trusted true >/dev/null 2>&1 || true
|
||||
fi
|
||||
printf '%s\n' "$desktop_file"
|
||||
}
|
||||
|
||||
main() {
|
||||
if [[ ! -d "$PROJECT_ROOT" ]]; then
|
||||
echo "warning: project root does not exist yet: $PROJECT_ROOT" >&2
|
||||
fi
|
||||
|
||||
local desktop_dir owner_group home_dir launcher icon_file moved desktop_file
|
||||
local count=0
|
||||
while IFS= read -r desktop_dir; do
|
||||
[[ -d "$desktop_dir" ]] || continue
|
||||
owner_group="$(owner_for_path "$desktop_dir")"
|
||||
home_dir="$(dirname "$desktop_dir")"
|
||||
if [[ "$CREATE_LAUNCHER" == "1" ]]; then
|
||||
launcher="$(write_launcher_script "$home_dir" "$owner_group")"
|
||||
icon_file="$(write_icon "$home_dir" "$owner_group")"
|
||||
desktop_file="$(write_desktop_file "$desktop_dir" "$owner_group" "$launcher" "$icon_file")"
|
||||
else
|
||||
desktop_file="$desktop_dir/$DESKTOP_FILE_NAME"
|
||||
fi
|
||||
if [[ "$CLEAN_DESKTOP" == "1" ]]; then
|
||||
moved="$(clean_desktop_dir "$desktop_dir")"
|
||||
else
|
||||
moved=0
|
||||
fi
|
||||
echo "desktop=$desktop_dir launcher=$desktop_file moved_to_backup=$moved"
|
||||
count=$((count + 1))
|
||||
done < <(discover_desktop_dirs)
|
||||
|
||||
echo "Real2Sim desktop launcher setup complete. desktop_dirs=$count"
|
||||
}
|
||||
|
||||
main
|
||||
4
scripts/start_local.sh
Normal file
4
scripts/start_local.sh
Normal file
@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
PYTHONPATH=src python -m real2sim_platform.server --host 127.0.0.1 --port 18126
|
||||
16
scripts/vendor_spark.sh
Executable file
16
scripts/vendor_spark.sh
Executable file
@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
VERSION="${SPARK_VERSION:-2.1.0}"
|
||||
TMP="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
cd "$TMP"
|
||||
npm pack "@sparkjsdev/spark@$VERSION" >/dev/null
|
||||
tar -xzf "sparkjsdev-spark-$VERSION.tgz"
|
||||
mkdir -p "$ROOT/src/real2sim_platform/static/vendor"
|
||||
cp package/dist/spark.module.min.js "$ROOT/src/real2sim_platform/static/vendor/spark.module.min.js"
|
||||
cp package/LICENSE "$ROOT/src/real2sim_platform/static/vendor/spark.LICENSE"
|
||||
printf '%s\n' "$VERSION" > "$ROOT/src/real2sim_platform/static/vendor/spark.VERSION"
|
||||
echo "vendored @sparkjsdev/spark@$VERSION"
|
||||
20
scripts/vendor_three.sh
Executable file
20
scripts/vendor_three.sh
Executable file
@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
VERSION="${THREE_VERSION:-0.185.1}"
|
||||
VENDOR="$ROOT/src/real2sim_platform/static/vendor/three"
|
||||
BASE="https://unpkg.com/three@$VERSION"
|
||||
|
||||
mkdir -p "$VENDOR/addons/controls" "$VENDOR/addons/loaders" "$VENDOR/addons/postprocessing" "$VENDOR/addons/utils"
|
||||
curl -fsSL "$BASE/build/three.module.js" -o "$VENDOR/three.module.js"
|
||||
curl -fsSL "$BASE/build/three.core.js" -o "$VENDOR/three.core.js"
|
||||
curl -fsSL "$BASE/examples/jsm/controls/OrbitControls.js" -o "$VENDOR/addons/controls/OrbitControls.js"
|
||||
curl -fsSL "$BASE/examples/jsm/loaders/OBJLoader.js" -o "$VENDOR/addons/loaders/OBJLoader.js"
|
||||
curl -fsSL "$BASE/examples/jsm/loaders/PLYLoader.js" -o "$VENDOR/addons/loaders/PLYLoader.js"
|
||||
curl -fsSL "$BASE/examples/jsm/loaders/GLTFLoader.js" -o "$VENDOR/addons/loaders/GLTFLoader.js"
|
||||
curl -fsSL "$BASE/examples/jsm/postprocessing/Pass.js" -o "$VENDOR/addons/postprocessing/Pass.js"
|
||||
curl -fsSL "$BASE/examples/jsm/utils/BufferGeometryUtils.js" -o "$VENDOR/addons/utils/BufferGeometryUtils.js"
|
||||
curl -fsSL "$BASE/examples/jsm/utils/SkeletonUtils.js" -o "$VENDOR/addons/utils/SkeletonUtils.js"
|
||||
printf '%s\n' "$VERSION" > "$ROOT/src/real2sim_platform/static/vendor/three.VERSION"
|
||||
echo "vendored three@$VERSION"
|
||||
3
src/real2sim_platform/__init__.py
Normal file
3
src/real2sim_platform/__init__.py
Normal file
@ -0,0 +1,3 @@
|
||||
__all__ = ["__version__"]
|
||||
|
||||
__version__ = "0.1.0"
|
||||
2476
src/real2sim_platform/adapters.py
Normal file
2476
src/real2sim_platform/adapters.py
Normal file
File diff suppressed because it is too large
Load Diff
186
src/real2sim_platform/config.py
Normal file
186
src/real2sim_platform/config.py
Normal file
@ -0,0 +1,186 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
APP_ROOT = Path(__file__).resolve().parents[2]
|
||||
REAL2SIM2MODEL_ROOT = APP_ROOT.parent
|
||||
WORKSPACE_ROOT = Path(os.environ.get("REAL2SIM_PLATFORM_WORKSPACE", APP_ROOT / "workspace"))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PlatformConfig:
|
||||
app_root: Path = APP_ROOT
|
||||
workspace_dir: Path = WORKSPACE_ROOT
|
||||
host: str = "127.0.0.1"
|
||||
port: int = 18126
|
||||
default_renderer: str = "spark"
|
||||
real2sim2model_root: Path = REAL2SIM2MODEL_ROOT
|
||||
expert_policy_root: Path = REAL2SIM2MODEL_ROOT / "real2sim-expert-policy"
|
||||
training_root: Path = REAL2SIM2MODEL_ROOT / "real2sim-policy-training"
|
||||
inference_root: Path = REAL2SIM2MODEL_ROOT / "real2sim-policy-inference"
|
||||
discoverse_root: Path = Path("/Users/d-robotics/workSpace/DISCOVERSE")
|
||||
gs_playground_root: Path = Path("/Users/d-robotics/workSpace/gs_playground")
|
||||
images23dgs_root: Path = Path("/Users/d-robotics/workSpace/images23dgs")
|
||||
colmap_binary: Path = Path("/usr/local/bin/colmap")
|
||||
gsplat_python: Path = Path("/opt/real2sim_paper_runtime/envs/anysplat/bin/python")
|
||||
runtime_root: Path = Path("/opt/real2sim_paper_runtime")
|
||||
anysplat_root: Path = Path("/opt/real2sim_paper_runtime/AnySplat")
|
||||
anysplat_python: Path = Path("/opt/real2sim_paper_runtime/envs/anysplat/bin/python")
|
||||
cuda_home: Path = Path("/usr/local/cuda-12.1")
|
||||
runtime_env_script: Path = Path("/opt/gs_playground_real2sim_48q/scripts/real2sim_wuying_paper_env.sh")
|
||||
allowed_path_roots: tuple[Path, ...] = ()
|
||||
|
||||
@property
|
||||
def db_path(self) -> Path:
|
||||
return self.workspace_dir / "state.db"
|
||||
|
||||
@property
|
||||
def uploads_dir(self) -> Path:
|
||||
return self.workspace_dir / "uploads"
|
||||
|
||||
@property
|
||||
def datasets_dir(self) -> Path:
|
||||
return self.workspace_dir / "datasets"
|
||||
|
||||
@property
|
||||
def scenes_dir(self) -> Path:
|
||||
return self.workspace_dir / "scenes"
|
||||
|
||||
@property
|
||||
def runs_dir(self) -> Path:
|
||||
return self.workspace_dir / "runs"
|
||||
|
||||
@property
|
||||
def models_dir(self) -> Path:
|
||||
return self.workspace_dir / "models"
|
||||
|
||||
@property
|
||||
def manifests_dir(self) -> Path:
|
||||
return self.workspace_dir / "manifests"
|
||||
|
||||
@property
|
||||
def effective_allowed_path_roots(self) -> tuple[Path, ...]:
|
||||
if self.allowed_path_roots:
|
||||
return self.allowed_path_roots
|
||||
return (
|
||||
self.workspace_dir,
|
||||
self.real2sim2model_root,
|
||||
self.gs_playground_root,
|
||||
self.discoverse_root,
|
||||
self.runtime_root,
|
||||
self.anysplat_root,
|
||||
Path("/opt/gs_playground_real2sim_48q"),
|
||||
Path("/tmp"),
|
||||
)
|
||||
|
||||
|
||||
def load_config(path: Path | None = None) -> PlatformConfig:
|
||||
config_path = path or Path(os.environ.get("REAL2SIM_PLATFORM_CONFIG", ""))
|
||||
if not config_path or not config_path.is_file():
|
||||
return PlatformConfig()
|
||||
values = _parse_simple_toml(config_path.read_text(encoding="utf-8"))
|
||||
app_root = Path(str(values.get("app_root", APP_ROOT)))
|
||||
workspace_dir = Path(str(values.get("workspace_dir", WORKSPACE_ROOT)))
|
||||
return PlatformConfig(
|
||||
app_root=app_root,
|
||||
workspace_dir=workspace_dir,
|
||||
host=str(values.get("host", "127.0.0.1")),
|
||||
port=int(values.get("port", 18126)),
|
||||
default_renderer=str(values.get("default_renderer", "spark")),
|
||||
real2sim2model_root=Path(str(values.get("real2sim2model_root", REAL2SIM2MODEL_ROOT))),
|
||||
expert_policy_root=Path(str(values.get("expert_policy_root", REAL2SIM2MODEL_ROOT / "real2sim-expert-policy"))),
|
||||
training_root=Path(str(values.get("training_root", REAL2SIM2MODEL_ROOT / "real2sim-policy-training"))),
|
||||
inference_root=Path(str(values.get("inference_root", REAL2SIM2MODEL_ROOT / "real2sim-policy-inference"))),
|
||||
discoverse_root=Path(str(values.get("discoverse_root", "/Users/d-robotics/workSpace/DISCOVERSE"))),
|
||||
gs_playground_root=Path(str(values.get("gs_playground_root", "/Users/d-robotics/workSpace/gs_playground"))),
|
||||
images23dgs_root=Path(str(values.get("images23dgs_root", "/Users/d-robotics/workSpace/images23dgs"))),
|
||||
colmap_binary=Path(str(values.get("colmap_binary", "/usr/local/bin/colmap"))),
|
||||
gsplat_python=Path(str(values.get("gsplat_python", "/opt/real2sim_paper_runtime/envs/anysplat/bin/python"))),
|
||||
runtime_root=Path(str(values.get("runtime_root", "/opt/real2sim_paper_runtime"))),
|
||||
anysplat_root=Path(str(values.get("anysplat_root", "/opt/real2sim_paper_runtime/AnySplat"))),
|
||||
anysplat_python=Path(str(values.get("anysplat_python", values.get("gsplat_python", "/opt/real2sim_paper_runtime/envs/anysplat/bin/python")))),
|
||||
cuda_home=Path(str(values.get("cuda_home", "/usr/local/cuda-12.1"))),
|
||||
runtime_env_script=Path(str(values.get("runtime_env_script", "/opt/gs_playground_real2sim_48q/scripts/real2sim_wuying_paper_env.sh"))),
|
||||
allowed_path_roots=_parse_path_tuple(str(values.get("allowed_path_roots", ""))),
|
||||
)
|
||||
|
||||
|
||||
def ensure_workspace(config: PlatformConfig) -> None:
|
||||
for path in [config.workspace_dir, config.uploads_dir, config.datasets_dir, config.scenes_dir, config.runs_dir, config.models_dir, config.manifests_dir]:
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def write_default_config(path: Path, config: PlatformConfig | None = None) -> None:
|
||||
cfg = config or PlatformConfig(app_root=path.parent, workspace_dir=path.parent / "workspace")
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
lines = [
|
||||
f'app_root = "{cfg.app_root.as_posix()}"',
|
||||
f'workspace_dir = "{cfg.workspace_dir.as_posix()}"',
|
||||
f'host = "{cfg.host}"',
|
||||
f"port = {cfg.port}",
|
||||
f'default_renderer = "{cfg.default_renderer}"',
|
||||
f'real2sim2model_root = "{cfg.real2sim2model_root.as_posix()}"',
|
||||
f'expert_policy_root = "{cfg.expert_policy_root.as_posix()}"',
|
||||
f'training_root = "{cfg.training_root.as_posix()}"',
|
||||
f'inference_root = "{cfg.inference_root.as_posix()}"',
|
||||
f'discoverse_root = "{cfg.discoverse_root.as_posix()}"',
|
||||
f'gs_playground_root = "{cfg.gs_playground_root.as_posix()}"',
|
||||
f'images23dgs_root = "{cfg.images23dgs_root.as_posix()}"',
|
||||
f'colmap_binary = "{cfg.colmap_binary.as_posix()}"',
|
||||
f'gsplat_python = "{cfg.gsplat_python.as_posix()}"',
|
||||
f'runtime_root = "{cfg.runtime_root.as_posix()}"',
|
||||
f'anysplat_root = "{cfg.anysplat_root.as_posix()}"',
|
||||
f'anysplat_python = "{cfg.anysplat_python.as_posix()}"',
|
||||
f'cuda_home = "{cfg.cuda_home.as_posix()}"',
|
||||
f'runtime_env_script = "{cfg.runtime_env_script.as_posix()}"',
|
||||
f'allowed_path_roots = "{_format_path_tuple(cfg.effective_allowed_path_roots)}"',
|
||||
"",
|
||||
]
|
||||
path.write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
|
||||
def is_path_allowed(path: Path, config: PlatformConfig) -> bool:
|
||||
try:
|
||||
resolved = path.expanduser().resolve()
|
||||
except OSError:
|
||||
resolved = path.expanduser().absolute()
|
||||
for root in config.effective_allowed_path_roots:
|
||||
try:
|
||||
resolved.relative_to(root.expanduser().resolve())
|
||||
return True
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def _parse_simple_toml(text: str) -> dict[str, Any]:
|
||||
values: dict[str, Any] = {}
|
||||
for raw in text.splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if value.startswith('"') and value.endswith('"'):
|
||||
values[key] = value[1:-1]
|
||||
else:
|
||||
try:
|
||||
values[key] = int(value)
|
||||
except ValueError:
|
||||
values[key] = value
|
||||
return values
|
||||
|
||||
|
||||
def _parse_path_tuple(value: str) -> tuple[Path, ...]:
|
||||
if not value:
|
||||
return ()
|
||||
return tuple(Path(part) for part in value.split(":") if part)
|
||||
|
||||
|
||||
def _format_path_tuple(paths: tuple[Path, ...]) -> str:
|
||||
return ":".join(path.as_posix() for path in paths)
|
||||
901
src/real2sim_platform/server.py
Normal file
901
src/real2sim_platform/server.py
Normal file
@ -0,0 +1,901 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import FastAPI, File, HTTPException, UploadFile
|
||||
from fastapi.responses import FileResponse, PlainTextResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel
|
||||
|
||||
from . import __version__
|
||||
from .adapters import SPARK_VENDOR_PATH, collect_task_reports, enrich_scene_manifest_assets, run_doctor, scan_dataset
|
||||
from .config import PlatformConfig, ensure_workspace, is_path_allowed, load_config, write_default_config
|
||||
from .store import PlatformStore
|
||||
from .worker import TaskWorker
|
||||
|
||||
|
||||
class ImportPathRequest(BaseModel):
|
||||
path: str
|
||||
name: str | None = None
|
||||
|
||||
|
||||
class ImportSceneRequest(BaseModel):
|
||||
path: str
|
||||
name: str | None = None
|
||||
metadata: dict[str, Any] = {}
|
||||
|
||||
|
||||
class SceneAssetRequest(BaseModel):
|
||||
path: str
|
||||
name: str | None = None
|
||||
role: str = "added_asset"
|
||||
kind: str | None = None
|
||||
transform: dict[str, Any] = {}
|
||||
|
||||
|
||||
class CreateTaskRequest(BaseModel):
|
||||
type: str
|
||||
params: dict[str, Any] = {}
|
||||
|
||||
|
||||
class UpdateEntityRequest(BaseModel):
|
||||
name: str | None = None
|
||||
tags: list[str] | None = None
|
||||
version: str | None = None
|
||||
|
||||
|
||||
class UpdateTaskRequest(BaseModel):
|
||||
params: dict[str, Any]
|
||||
|
||||
|
||||
class BulkActionRequest(BaseModel):
|
||||
action: str
|
||||
ids: list[str]
|
||||
params: dict[str, Any] = {}
|
||||
|
||||
|
||||
def create_app(config: PlatformConfig | None = None) -> FastAPI:
|
||||
cfg = config or load_config()
|
||||
ensure_workspace(cfg)
|
||||
store = PlatformStore(cfg.db_path)
|
||||
worker = TaskWorker(store, cfg)
|
||||
app = FastAPI(title="Real2Sim Platform", version="0.1.0")
|
||||
static_dir = Path(__file__).parent / "static"
|
||||
app.mount("/static", StaticFiles(directory=static_dir), name="static")
|
||||
app.mount("/datasets", StaticFiles(directory=cfg.datasets_dir), name="datasets")
|
||||
|
||||
@app.on_event("startup")
|
||||
def _startup() -> None:
|
||||
store.recover_interrupted_tasks()
|
||||
worker.start()
|
||||
|
||||
@app.on_event("shutdown")
|
||||
def _shutdown() -> None:
|
||||
worker.stop()
|
||||
|
||||
@app.get("/")
|
||||
def index():
|
||||
return FileResponse(static_dir / "index.html")
|
||||
|
||||
@app.get("/api/health")
|
||||
def health() -> dict[str, Any]:
|
||||
return {"ok": True, "workspace": str(cfg.workspace_dir)}
|
||||
|
||||
@app.get("/api/version")
|
||||
def version() -> dict[str, Any]:
|
||||
spark_available = SPARK_VENDOR_PATH.is_file()
|
||||
return {
|
||||
"version": __version__,
|
||||
"build_time": os.environ.get("REAL2SIM_PLATFORM_BUILD_TIME", "dev"),
|
||||
"app_root": str(cfg.app_root),
|
||||
"workspace": str(cfg.workspace_dir),
|
||||
"deploy_path": str(cfg.app_root),
|
||||
"build_id": os.environ.get("REAL2SIM_PLATFORM_BUILD_ID", os.environ.get("REAL2SIM_PLATFORM_BUILD_TIME", "dev")),
|
||||
"config_digest": _config_digest(cfg),
|
||||
"git_commit": _git_commit(cfg.app_root),
|
||||
"source_snapshot": _source_snapshot(cfg.app_root),
|
||||
"runtime_paths": _runtime_paths(cfg),
|
||||
"renderer_backend": "spark" if spark_available else "three_fallback",
|
||||
"features": {
|
||||
"hf_import": True,
|
||||
"media_prepare": True,
|
||||
"reconstruction_preflight": True,
|
||||
"scene_reconstruction": True,
|
||||
"spark_viewer": spark_available,
|
||||
"three_fallback_viewer": True,
|
||||
"aholo_optional": True,
|
||||
"mujoco": True,
|
||||
"motrixsim": True,
|
||||
},
|
||||
}
|
||||
|
||||
@app.get("/api/doctor")
|
||||
def doctor() -> dict[str, Any]:
|
||||
return run_doctor(cfg)
|
||||
|
||||
@app.get("/api/datasets")
|
||||
def list_datasets() -> list[dict[str, Any]]:
|
||||
return [_with_view_model(item, "dataset") for item in store.list_datasets()]
|
||||
|
||||
@app.post("/api/datasets/import-path")
|
||||
def import_dataset(payload: ImportPathRequest) -> dict[str, Any]:
|
||||
path = Path(payload.path)
|
||||
if not path.exists():
|
||||
raise HTTPException(status_code=404, detail="path not found")
|
||||
if not is_path_allowed(path, cfg):
|
||||
raise HTTPException(status_code=403, detail="path is outside configured import whitelist")
|
||||
scan = scan_dataset(path)
|
||||
return store.create_dataset(name=payload.name or path.name, path=path, scan=scan, source_type="local_path", source_uri=str(path))
|
||||
|
||||
@app.post("/api/datasets/upload")
|
||||
def upload_dataset(file: UploadFile = File(...)) -> dict[str, Any]:
|
||||
target = cfg.uploads_dir / Path(file.filename or "upload.bin").name
|
||||
with target.open("wb") as handle:
|
||||
shutil.copyfileobj(file.file, handle)
|
||||
dataset_dir = cfg.datasets_dir / target.stem
|
||||
if dataset_dir.exists():
|
||||
shutil.rmtree(dataset_dir)
|
||||
dataset_dir.mkdir(parents=True, exist_ok=True)
|
||||
if target.suffix.lower() == ".zip":
|
||||
with zipfile.ZipFile(target) as archive:
|
||||
_safe_extract(archive, dataset_dir)
|
||||
path = dataset_dir
|
||||
else:
|
||||
shutil.copy2(target, dataset_dir / target.name)
|
||||
path = dataset_dir
|
||||
scan = scan_dataset(path)
|
||||
manifest_path = dataset_dir / "dataset_manifest.json"
|
||||
manifest_path.write_text(json.dumps({"schema": "real2sim.dataset_manifest.v1", "name": dataset_dir.name, "source_type": "upload", "scan": scan}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
return store.create_dataset(name=dataset_dir.name, path=path, scan=scan, source_type="upload", source_uri=str(target), manifest_path=manifest_path)
|
||||
|
||||
@app.get("/api/scenes")
|
||||
def list_scenes() -> list[dict[str, Any]]:
|
||||
return [_with_view_model(item, "scene") for item in store.list_scenes()]
|
||||
|
||||
@app.post("/api/scenes/import")
|
||||
def import_scene(payload: ImportSceneRequest) -> dict[str, Any]:
|
||||
path = Path(payload.path)
|
||||
if not path.is_file():
|
||||
raise HTTPException(status_code=404, detail="scene file not found")
|
||||
if not is_path_allowed(path, cfg):
|
||||
raise HTTPException(status_code=403, detail="scene path is outside configured import whitelist")
|
||||
return store.create_scene(name=payload.name or path.stem, path=path, metadata=payload.metadata, source_type="local_path", source_uri=str(path), manifest_path=Path(str(payload.metadata.get("manifest_path", ""))) if payload.metadata.get("manifest_path") else None)
|
||||
|
||||
@app.get("/api/datasets/{dataset_id}/manifest")
|
||||
def dataset_manifest(dataset_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
dataset = store.get_dataset(dataset_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="dataset not found") from exc
|
||||
return _entity_manifest(dataset, fallback_kind="dataset")
|
||||
|
||||
@app.patch("/api/datasets/{dataset_id}")
|
||||
def update_dataset(dataset_id: str, payload: UpdateEntityRequest) -> dict[str, Any]:
|
||||
try:
|
||||
return store.update_dataset(dataset_id, name=payload.name, tags=payload.tags, version=payload.version)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="dataset not found") from exc
|
||||
|
||||
@app.delete("/api/datasets/{dataset_id}")
|
||||
def delete_dataset(dataset_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
store.delete_dataset(dataset_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="dataset not found") from exc
|
||||
return {"ok": True}
|
||||
|
||||
@app.get("/api/scenes/{scene_id}/manifest")
|
||||
def scene_manifest(scene_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
scene = store.get_scene(scene_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="scene not found") from exc
|
||||
return _entity_manifest(scene, fallback_kind="scene")
|
||||
|
||||
@app.post("/api/scenes/{scene_id}/assets")
|
||||
def add_scene_asset(scene_id: str, payload: SceneAssetRequest) -> dict[str, Any]:
|
||||
try:
|
||||
scene = store.get_scene(scene_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="scene not found") from exc
|
||||
asset_path = Path(payload.path).expanduser()
|
||||
if not asset_path.is_file():
|
||||
raise HTTPException(status_code=404, detail="asset file not found")
|
||||
if not is_path_allowed(asset_path, cfg):
|
||||
raise HTTPException(status_code=403, detail="asset path is outside configured import whitelist")
|
||||
manifest_path = _scene_manifest_path_for_write(scene)
|
||||
manifest = _read_manifest_or_default(scene, manifest_path)
|
||||
artifact = store.add_artifact(task_id=None, name=payload.name or asset_path.name, path=asset_path, kind=(payload.kind or _scene_asset_kind(asset_path)))
|
||||
asset = _scene_asset_entry(
|
||||
asset_path,
|
||||
url=f"/api/artifacts/{artifact['id']}/content",
|
||||
name=payload.name,
|
||||
kind=payload.kind,
|
||||
role=payload.role,
|
||||
transform=payload.transform,
|
||||
)
|
||||
assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and str(item.get("path") or "") == str(asset_path))]
|
||||
manifest["assets"] = [asset] + assets
|
||||
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
manifest["asset_editing"] = {"supports_added_assets": True, "last_added_asset": asset["name"]}
|
||||
manifest_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
tags = list(dict.fromkeys([*(scene.get("tags") or []), "added_asset"]))
|
||||
store.update_scene(scene_id, tags=tags)
|
||||
return {"ok": True, "scene": store.get_scene(scene_id), "asset": asset, "manifest_path": str(manifest_path)}
|
||||
|
||||
@app.patch("/api/scenes/{scene_id}")
|
||||
def update_scene(scene_id: str, payload: UpdateEntityRequest) -> dict[str, Any]:
|
||||
try:
|
||||
return store.update_scene(scene_id, name=payload.name, tags=payload.tags, version=payload.version)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="scene not found") from exc
|
||||
|
||||
@app.delete("/api/scenes/{scene_id}")
|
||||
def delete_scene(scene_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
store.delete_scene(scene_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="scene not found") from exc
|
||||
return {"ok": True}
|
||||
|
||||
@app.get("/api/models")
|
||||
def list_models() -> list[dict[str, Any]]:
|
||||
return [_with_view_model(item, "model") for item in store.list_models()]
|
||||
|
||||
@app.get("/api/models/{model_id}/card")
|
||||
def model_card(model_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
model = store.get_model(model_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="model not found") from exc
|
||||
return _entity_manifest(model, fallback_kind="model")
|
||||
|
||||
@app.patch("/api/models/{model_id}")
|
||||
def update_model(model_id: str, payload: UpdateEntityRequest) -> dict[str, Any]:
|
||||
try:
|
||||
return store.update_model(model_id, name=payload.name, tags=payload.tags, version=payload.version)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="model not found") from exc
|
||||
|
||||
@app.delete("/api/models/{model_id}")
|
||||
def delete_model(model_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
store.delete_model(model_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="model not found") from exc
|
||||
return {"ok": True}
|
||||
|
||||
def _queue_task(task_type: str, params: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = CreateTaskRequest(type=task_type, params=params)
|
||||
_validate_task_payload(payload)
|
||||
_validate_task_resources(store, payload)
|
||||
if payload.type not in {"scene_reconstruction", "reconstruction_preflight", "media_prepare", "expert_collection", "training", "inference", "renderer_benchmark", "scene_cover_render", "scene_splat_optimize", "hf_import", "simulator_validation"}:
|
||||
raise ValueError(f"unknown task type: {payload.type}")
|
||||
run_dir = cfg.runs_dir / payload.type / _next_run_name(cfg.runs_dir / payload.type, payload.type)
|
||||
return store.create_task(task_type=payload.type, params=payload.params, run_dir=run_dir)
|
||||
|
||||
def _bulk_results(ids: list[str], handler) -> dict[str, Any]:
|
||||
results: list[dict[str, Any]] = []
|
||||
for item_id in ids:
|
||||
try:
|
||||
results.append({"id": item_id, "ok": True, "result": handler(item_id)})
|
||||
except Exception as exc:
|
||||
results.append({"id": item_id, "ok": False, "error": str(exc)})
|
||||
return {"ok": all(item["ok"] for item in results), "results": results}
|
||||
|
||||
@app.post("/api/tasks/bulk")
|
||||
def bulk_tasks(payload: BulkActionRequest) -> dict[str, Any]:
|
||||
if payload.action not in {"retry", "cancel", "delete"}:
|
||||
raise HTTPException(status_code=400, detail="tasks bulk action must be retry, cancel, or delete")
|
||||
|
||||
def handle(task_id: str) -> dict[str, Any]:
|
||||
if payload.action == "retry":
|
||||
return store.retry_task(task_id)
|
||||
if payload.action == "cancel":
|
||||
return store.cancel_task(task_id)
|
||||
store.delete_task(task_id)
|
||||
return {"deleted": True}
|
||||
|
||||
return _bulk_results(payload.ids, handle)
|
||||
|
||||
@app.post("/api/datasets/bulk")
|
||||
def bulk_datasets(payload: BulkActionRequest) -> dict[str, Any]:
|
||||
if payload.action not in {"prepare", "preflight", "delete"}:
|
||||
raise HTTPException(status_code=400, detail="datasets bulk action must be prepare, preflight, or delete")
|
||||
|
||||
def handle(dataset_id: str) -> dict[str, Any]:
|
||||
dataset = store.get_dataset(dataset_id)
|
||||
if payload.action == "prepare":
|
||||
params = {
|
||||
"dataset_id": dataset_id,
|
||||
"camera_label": payload.params.get("camera_label", "observation.images.front"),
|
||||
"sample_fps": payload.params.get("sample_fps", 2),
|
||||
"max_frames_per_video": payload.params.get("max_frames_per_video", 80),
|
||||
"max_videos": payload.params.get("max_videos", 1),
|
||||
"target_frame_count": payload.params.get("target_frame_count", 0),
|
||||
"min_total_frames": payload.params.get("min_total_frames", 0),
|
||||
}
|
||||
return _queue_task("media_prepare", params)
|
||||
if payload.action == "preflight":
|
||||
return _queue_task("reconstruction_preflight", {"dataset_id": dataset_id, "min_frames": payload.params.get("min_frames", 30)})
|
||||
store.delete_dataset(dataset["id"])
|
||||
return {"deleted": True}
|
||||
|
||||
return _bulk_results(payload.ids, handle)
|
||||
|
||||
@app.post("/api/scenes/bulk")
|
||||
def bulk_scenes(payload: BulkActionRequest) -> dict[str, Any]:
|
||||
if payload.action not in {"validate", "export", "delete"}:
|
||||
raise HTTPException(status_code=400, detail="scenes bulk action must be validate, export, or delete")
|
||||
|
||||
def handle(scene_id: str) -> dict[str, Any]:
|
||||
scene = store.get_scene(scene_id)
|
||||
if payload.action == "validate":
|
||||
return _queue_task(
|
||||
"simulator_validation",
|
||||
{
|
||||
"scene_id": scene_id,
|
||||
"simulators": payload.params.get("simulators", ["mujoco", "motrixsim"]),
|
||||
"level": payload.params.get("level", "runtime_if_available"),
|
||||
},
|
||||
)
|
||||
if payload.action == "export":
|
||||
return {"scene_path": scene["path"], "manifest_path": scene.get("manifest_path", ""), "package_path": scene.get("metadata", {}).get("package", str(Path(scene["path"]).parent))}
|
||||
store.delete_scene(scene["id"])
|
||||
return {"deleted": True}
|
||||
|
||||
return _bulk_results(payload.ids, handle)
|
||||
|
||||
@app.post("/api/models/bulk")
|
||||
def bulk_models(payload: BulkActionRequest) -> dict[str, Any]:
|
||||
if payload.action not in {"compare", "export", "delete"}:
|
||||
raise HTTPException(status_code=400, detail="models bulk action must be compare, export, or delete")
|
||||
|
||||
def handle(model_id: str) -> dict[str, Any]:
|
||||
model = store.get_model(model_id)
|
||||
if payload.action == "compare":
|
||||
return {"model_id": model_id, "name": model["name"], "metrics": model.get("metrics", {})}
|
||||
if payload.action == "export":
|
||||
return {"model_path": model["path"], "card_path": model.get("manifest_path", "")}
|
||||
store.delete_model(model["id"])
|
||||
return {"deleted": True}
|
||||
|
||||
return _bulk_results(payload.ids, handle)
|
||||
|
||||
@app.get("/api/tasks")
|
||||
def list_tasks() -> list[dict[str, Any]]:
|
||||
return [_with_task_view_model(item) for item in store.list_tasks()]
|
||||
|
||||
@app.post("/api/tasks")
|
||||
def create_task(payload: CreateTaskRequest) -> dict[str, Any]:
|
||||
try:
|
||||
return _queue_task(payload.type, payload.params)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
@app.get("/api/tasks/{task_id}")
|
||||
def get_task(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return store.get_task(task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
|
||||
@app.get("/api/tasks/{task_id}/logs")
|
||||
def get_task_logs(task_id: str):
|
||||
try:
|
||||
task = store.get_task(task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
path = Path(task["run_dir"]) / "logs" / "task.log"
|
||||
return PlainTextResponse(path.read_text(encoding="utf-8") if path.is_file() else "")
|
||||
|
||||
@app.get("/api/tasks/{task_id}/artifacts")
|
||||
def get_task_artifacts(task_id: str) -> list[dict[str, Any]]:
|
||||
try:
|
||||
store.get_task(task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
return [{**item, "url": f"/api/artifacts/{item['id']}/content"} for item in store.list_task_artifacts(task_id)]
|
||||
|
||||
@app.get("/api/tasks/{task_id}/reports")
|
||||
def get_task_reports(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return collect_task_reports(store, task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
|
||||
@app.post("/api/tasks/{task_id}/cancel")
|
||||
def cancel_task(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return store.cancel_task(task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
|
||||
@app.post("/api/tasks/{task_id}/retry")
|
||||
def retry_task(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return store.retry_task(task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
|
||||
@app.patch("/api/tasks/{task_id}")
|
||||
def update_task(task_id: str, payload: UpdateTaskRequest) -> dict[str, Any]:
|
||||
try:
|
||||
return store.update_task_params(task_id, params=payload.params)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
@app.delete("/api/tasks/{task_id}")
|
||||
def delete_task(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
store.delete_task(task_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="task not found") from exc
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
return {"ok": True}
|
||||
|
||||
@app.get("/api/artifacts/{artifact_id}/content")
|
||||
def artifact_content(artifact_id: str):
|
||||
try:
|
||||
artifact = store.get_artifact(artifact_id)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="artifact not found") from exc
|
||||
path = Path(artifact["path"])
|
||||
if not path.is_file():
|
||||
raise HTTPException(status_code=404, detail="artifact file not found")
|
||||
return FileResponse(path, filename=path.name)
|
||||
|
||||
@app.get("/api/config/default")
|
||||
def default_config() -> dict[str, Any]:
|
||||
return {"default_renderer": cfg.default_renderer, "simulators": ["mujoco", "motrixsim"], "port": cfg.port}
|
||||
|
||||
app.mount("/runs", StaticFiles(directory=cfg.runs_dir), name="runs")
|
||||
return app
|
||||
|
||||
|
||||
def _entity_manifest(entity: dict[str, Any], *, fallback_kind: str) -> dict[str, Any]:
|
||||
manifest_path = Path(entity.get("manifest_path") or "")
|
||||
if manifest_path.is_file():
|
||||
return _postprocess_manifest(json.loads(manifest_path.read_text(encoding="utf-8")), fallback_kind=fallback_kind)
|
||||
path = Path(entity["path"])
|
||||
candidates = []
|
||||
if fallback_kind == "dataset":
|
||||
candidates = [path / "dataset_manifest.json", path / "manifest.json"]
|
||||
elif fallback_kind == "scene":
|
||||
candidates = [path.parent / "scene_manifest.json", path.parent / "package_manifest.json", path]
|
||||
elif fallback_kind == "model":
|
||||
candidates = [path / "model_card.json", path / "manifest.json"]
|
||||
for candidate in candidates:
|
||||
if candidate.is_file():
|
||||
return _postprocess_manifest(json.loads(candidate.read_text(encoding="utf-8")), fallback_kind=fallback_kind)
|
||||
return _postprocess_manifest({"schema": f"real2sim.{fallback_kind}_manifest.synthetic.v1", "id": entity["id"], "name": entity["name"], "path": entity["path"], "source_type": entity.get("source_type"), "source_uri": entity.get("source_uri"), "metrics": entity.get("metrics", {})}, fallback_kind=fallback_kind)
|
||||
|
||||
|
||||
def _scene_manifest_path_for_write(scene: dict[str, Any]) -> Path:
|
||||
manifest_path = Path(scene.get("manifest_path") or "")
|
||||
if manifest_path.is_file():
|
||||
return manifest_path
|
||||
scene_path = Path(scene["path"])
|
||||
package_dir = scene_path if scene_path.is_dir() else scene_path.parent
|
||||
return package_dir / "scene_manifest.json"
|
||||
|
||||
|
||||
def _read_manifest_or_default(scene: dict[str, Any], manifest_path: Path) -> dict[str, Any]:
|
||||
if manifest_path.is_file():
|
||||
payload = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
if isinstance(payload, dict):
|
||||
payload.setdefault("assets", [])
|
||||
return payload
|
||||
return {
|
||||
"schema": "real2sim.scene_manifest.v1",
|
||||
"name": scene.get("name") or scene.get("id"),
|
||||
"input_path": scene.get("source_uri") or "",
|
||||
"renderer": scene.get("metadata", {}).get("renderer", "auto"),
|
||||
"renderer_status": {},
|
||||
"simulators": scene.get("metadata", {}).get("simulators", []),
|
||||
"dry_run": scene.get("metadata", {}).get("dry_run", False),
|
||||
"assets": [],
|
||||
"simulator_artifacts": {},
|
||||
}
|
||||
|
||||
|
||||
def _scene_asset_entry(path: Path, *, url: str, name: str | None, kind: str | None, role: str, transform: dict[str, Any]) -> dict[str, Any]:
|
||||
size_bytes = path.stat().st_size
|
||||
asset_kind = kind or _scene_asset_kind(path)
|
||||
autoload = asset_kind in {"splat_or_pointcloud", "mesh"} and size_bytes <= 25 * 1024 * 1024
|
||||
return {
|
||||
"name": name or path.name,
|
||||
"path": str(path),
|
||||
"url": url,
|
||||
"kind": asset_kind,
|
||||
"role": role or "added_asset",
|
||||
"format": path.suffix.lower().lstrip("."),
|
||||
"size_bytes": size_bytes,
|
||||
"autoload": autoload,
|
||||
"viewer_hint": "autoload" if autoload else "manual_load",
|
||||
"transform": transform or {},
|
||||
"added_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
}
|
||||
|
||||
|
||||
def _scene_asset_kind(path: Path) -> str:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix in {".ply", ".splat", ".spz", ".ksplat", ".sog", ".zip", ".rad"}:
|
||||
return "splat_or_pointcloud"
|
||||
if suffix in {".glb", ".gltf", ".obj"}:
|
||||
return "mesh"
|
||||
if suffix in {".png", ".jpg", ".jpeg", ".webp", ".bmp"}:
|
||||
return "texture"
|
||||
if suffix in {".xml"}:
|
||||
return "simulator"
|
||||
if suffix in {".json", ".jsonl"}:
|
||||
return "metadata"
|
||||
return "file"
|
||||
|
||||
|
||||
def _postprocess_manifest(manifest: dict[str, Any], *, fallback_kind: str) -> dict[str, Any]:
|
||||
if fallback_kind == "scene":
|
||||
return enrich_scene_manifest_assets(manifest)
|
||||
return manifest
|
||||
|
||||
|
||||
def _with_view_model(entity: dict[str, Any], kind: str) -> dict[str, Any]:
|
||||
item = dict(entity)
|
||||
manifest = _safe_entity_manifest(item, kind)
|
||||
primary_asset = _primary_asset(manifest)
|
||||
thumbnail = _thumbnail_from_manifest(item, manifest, primary_asset)
|
||||
metrics = item.get("metrics") or {}
|
||||
scan = item.get("scan") or {}
|
||||
item["view"] = {
|
||||
"display_name": item.get("name") or Path(str(item.get("path") or "")).stem or item.get("id", ""),
|
||||
"thumbnail_url": thumbnail,
|
||||
"preview_url": _preview_url(kind, item, primary_asset),
|
||||
"primary_asset": primary_asset,
|
||||
"metrics_summary": _metrics_summary(kind, metrics, scan, manifest),
|
||||
"subtitle": _resource_subtitle(kind, item, manifest),
|
||||
}
|
||||
return item
|
||||
|
||||
|
||||
def _with_task_view_model(task: dict[str, Any]) -> dict[str, Any]:
|
||||
item = dict(task)
|
||||
progress = int(float(item.get("progress") or 0))
|
||||
if item.get("status") == "succeeded":
|
||||
progress = 100
|
||||
item["view"] = {
|
||||
"display_name": _task_display_name(item),
|
||||
"stage_label": item.get("stage") or _task_type_label(str(item.get("type") or "")),
|
||||
"progress": max(0, min(100, progress)),
|
||||
"message": item.get("message") or item.get("error") or "",
|
||||
"updated_label": time.strftime("%m/%d %H:%M", time.localtime(float(item.get("updated_at") or item.get("created_at") or time.time()))),
|
||||
}
|
||||
return item
|
||||
|
||||
|
||||
def _safe_entity_manifest(entity: dict[str, Any], kind: str) -> dict[str, Any]:
|
||||
try:
|
||||
return _entity_manifest(entity, fallback_kind=kind)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def _primary_asset(manifest: dict[str, Any]) -> dict[str, Any] | None:
|
||||
assets = manifest.get("assets") if isinstance(manifest, dict) else []
|
||||
if not isinstance(assets, list):
|
||||
return None
|
||||
candidates = [asset for asset in assets if isinstance(asset, dict)]
|
||||
if not candidates:
|
||||
return None
|
||||
return sorted(candidates, key=_asset_view_rank)[0]
|
||||
|
||||
|
||||
def _asset_view_rank(asset: dict[str, Any]) -> tuple[int, int, str]:
|
||||
name = str(asset.get("name") or asset.get("path") or "").lower()
|
||||
path_text = str(asset.get("path") or "").lower()
|
||||
kind = str(asset.get("kind") or "").lower()
|
||||
size = int(asset.get("size_bytes") or 0)
|
||||
if asset.get("cover") or name == "scene_cover.png":
|
||||
rank = 0
|
||||
elif name.endswith("_threejs_rgb.ply"):
|
||||
rank = 1
|
||||
elif name.endswith((".splat", ".spz", ".ksplat", ".sog")):
|
||||
rank = 2
|
||||
elif name == "pose_init_trained_scene.ply":
|
||||
rank = 3
|
||||
elif name == "scene.ply" and "/3dgs/" in path_text:
|
||||
rank = 4
|
||||
elif name in {"pointcloud_projection_preview.png", "source_gs_preview.png"}:
|
||||
rank = 5
|
||||
elif name.endswith((".png", ".jpg", ".jpeg", ".webp")) and ("preview" in name or "render" in name):
|
||||
rank = 6
|
||||
elif name.startswith("thumb_") and name.endswith((".png", ".jpg", ".jpeg", ".webp")):
|
||||
rank = 7
|
||||
elif name == "scene.ply":
|
||||
rank = 8
|
||||
elif kind in {"mesh", "splat_or_pointcloud"} and "sparse" not in name:
|
||||
rank = 9
|
||||
elif name in {"background_image.png"}:
|
||||
rank = 10
|
||||
elif name.endswith((".png", ".jpg", ".jpeg", ".webp")):
|
||||
rank = 11
|
||||
elif "colmap_sparse" in name or "sparse_points" in name:
|
||||
rank = 12
|
||||
else:
|
||||
rank = 13
|
||||
return (rank, -size, name)
|
||||
|
||||
|
||||
def _thumbnail_from_manifest(entity: dict[str, Any], manifest: dict[str, Any], primary_asset: dict[str, Any] | None) -> str:
|
||||
assets = manifest.get("assets") if isinstance(manifest, dict) else []
|
||||
if isinstance(assets, list):
|
||||
image_assets = [asset for asset in assets if isinstance(asset, dict) and str(asset.get("name") or asset.get("path") or "").lower().endswith((".png", ".jpg", ".jpeg", ".webp"))]
|
||||
image_assets.sort(key=_asset_view_rank)
|
||||
for asset in image_assets:
|
||||
if asset.get("url"):
|
||||
return str(asset["url"])
|
||||
dataset_thumb = _dataset_thumbnail_url(entity)
|
||||
if dataset_thumb:
|
||||
return dataset_thumb
|
||||
return str(primary_asset.get("url") or "") if primary_asset and str(primary_asset.get("kind") or "") == "image" else ""
|
||||
|
||||
|
||||
def _dataset_thumbnail_url(entity: dict[str, Any]) -> str:
|
||||
root = Path(str(entity.get("path") or ""))
|
||||
candidates = [
|
||||
root / "thumbnails" / "thumb_01.jpg",
|
||||
root / "thumbnails" / "thumb_01.png",
|
||||
root / "thumbnail.jpg",
|
||||
root / "preview.jpg",
|
||||
root / "preview.png",
|
||||
]
|
||||
for path in candidates:
|
||||
if path.is_file():
|
||||
return _workspace_public_url(path)
|
||||
return ""
|
||||
|
||||
|
||||
def _workspace_public_url(path: Path) -> str:
|
||||
parts = path.parts
|
||||
for marker in ("runs", "datasets"):
|
||||
if marker in parts:
|
||||
index = parts.index(marker)
|
||||
return "/" + "/".join(parts[index:])
|
||||
return ""
|
||||
|
||||
|
||||
def _preview_url(kind: str, entity: dict[str, Any], primary_asset: dict[str, Any] | None) -> str:
|
||||
if kind == "scene":
|
||||
return f"/static/scene-viewer.html?scene_id={entity['id']}"
|
||||
return str(primary_asset.get("url") or "") if primary_asset else ""
|
||||
|
||||
|
||||
def _metrics_summary(kind: str, metrics: dict[str, Any], scan: dict[str, Any], manifest: dict[str, Any]) -> str:
|
||||
if kind == "dataset":
|
||||
frames = metrics.get("frame_count") or manifest.get("frame_count") or scan.get("image_count")
|
||||
videos = scan.get("video_count")
|
||||
samples = metrics.get("samples") or scan.get("samples")
|
||||
parts = []
|
||||
if frames:
|
||||
parts.append(f"{frames} 帧")
|
||||
if videos:
|
||||
parts.append(f"{videos} 视频")
|
||||
if samples:
|
||||
parts.append(f"{samples} 样本")
|
||||
return " · ".join(parts) or "等待扫描"
|
||||
if kind == "scene":
|
||||
assets = manifest.get("assets") if isinstance(manifest.get("assets"), list) else []
|
||||
return f"{len(assets)} 个资产" if assets else "暂无资产"
|
||||
score = metrics.get("success_rate") or metrics.get("score") or metrics.get("accuracy")
|
||||
loss = metrics.get("loss") or metrics.get("val_loss")
|
||||
parts = []
|
||||
if score is not None:
|
||||
parts.append(f"成功率 {float(score) * 100:.1f}%" if float(score) <= 1 else f"成功率 {float(score):.1f}%")
|
||||
if loss is not None:
|
||||
parts.append(f"loss {float(loss):.3f}")
|
||||
return " · ".join(parts) or "暂无指标"
|
||||
|
||||
|
||||
def _resource_subtitle(kind: str, entity: dict[str, Any], manifest: dict[str, Any]) -> str:
|
||||
source = str(entity.get("source_type") or manifest.get("source_type") or "")
|
||||
version = str(entity.get("version") or "")
|
||||
label = {"prepared_media": "已准备帧数据", "expert_collection": "专家采集数据", "scene_reconstruction": "场景重建", "huggingface": "HuggingFace", "upload": "上传文件", "local_path": "本地导入"}.get(source, source or kind)
|
||||
return f"{label}{' · ' + version if version else ''}"
|
||||
|
||||
|
||||
def _task_display_name(task: dict[str, Any]) -> str:
|
||||
params = task.get("params") or {}
|
||||
return str(params.get("scene_name") or params.get("output_name") or _task_type_label(str(task.get("type") or "")))
|
||||
|
||||
|
||||
def _task_type_label(task_type: str) -> str:
|
||||
return {
|
||||
"hf_import": "数据导入",
|
||||
"media_prepare": "媒体准备",
|
||||
"reconstruction_preflight": "重建预检",
|
||||
"scene_reconstruction": "场景重建",
|
||||
"simulator_validation": "仿真校验",
|
||||
"expert_collection": "专家采集",
|
||||
"training": "模型训练",
|
||||
"inference": "在线推理",
|
||||
"renderer_benchmark": "渲染测试",
|
||||
"scene_splat_optimize": "3DGS 优化",
|
||||
}.get(task_type, task_type)
|
||||
|
||||
|
||||
def _validate_task_payload(payload: CreateTaskRequest) -> None:
|
||||
params = payload.params
|
||||
if payload.type == "hf_import":
|
||||
repo_type = params.get("repo_type", "dataset")
|
||||
if repo_type != "dataset":
|
||||
raise HTTPException(status_code=400, detail="hf_import only supports repo_type='dataset'")
|
||||
if not params.get("repo_id"):
|
||||
raise HTTPException(status_code=400, detail="hf_import requires repo_id")
|
||||
if payload.type == "media_prepare":
|
||||
if not (params.get("dataset_id") or params.get("input_path")):
|
||||
raise HTTPException(status_code=400, detail="media_prepare requires dataset_id or input_path")
|
||||
if float(params.get("sample_fps", 2.0)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="sample_fps must be > 0")
|
||||
if int(params.get("max_frames_per_video", 80)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="max_frames_per_video must be > 0")
|
||||
if int(params.get("target_frame_count", 0) or 0) < 0:
|
||||
raise HTTPException(status_code=400, detail="target_frame_count must be >= 0")
|
||||
if int(params.get("min_total_frames", 0) or 0) < 0:
|
||||
raise HTTPException(status_code=400, detail="min_total_frames must be >= 0")
|
||||
if payload.type == "reconstruction_preflight":
|
||||
if not (params.get("dataset_id") or params.get("input_path")):
|
||||
raise HTTPException(status_code=400, detail="reconstruction_preflight requires dataset_id or input_path")
|
||||
if int(params.get("min_frames", 30)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="min_frames must be > 0")
|
||||
if payload.type == "scene_reconstruction":
|
||||
renderer = params.get("renderer", "auto")
|
||||
if renderer not in {"auto", "spark", "aholo"}:
|
||||
raise HTTPException(status_code=400, detail="renderer must be auto, spark, or aholo")
|
||||
simulators = set(params.get("simulators", ["mujoco", "motrixsim"]))
|
||||
if not simulators <= {"mujoco", "motrixsim"}:
|
||||
raise HTTPException(status_code=400, detail="simulators must be mujoco and/or motrixsim")
|
||||
if not (params.get("dataset_id") or params.get("input_path")):
|
||||
raise HTTPException(status_code=400, detail="scene_reconstruction requires dataset_id or input_path")
|
||||
if int(params.get("min_frames", 30)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="min_frames must be > 0")
|
||||
if payload.type == "simulator_validation":
|
||||
if not (params.get("scene_id") or params.get("scene_path")):
|
||||
raise HTTPException(status_code=400, detail="simulator_validation requires scene_id or scene_path")
|
||||
if params.get("level", "runtime_if_available") not in {"smoke", "runtime_if_available", "runtime_required"}:
|
||||
raise HTTPException(status_code=400, detail="level must be smoke, runtime_if_available, or runtime_required")
|
||||
if payload.type == "expert_collection":
|
||||
if not (params.get("scene_id") or params.get("scene_path")):
|
||||
raise HTTPException(status_code=400, detail="expert_collection requires scene_id or scene_path")
|
||||
if params.get("policy_preset", "pick_and_place_v1") != "pick_and_place_v1":
|
||||
raise HTTPException(status_code=400, detail="当前仅支持抓取放置策略;导航和精细操作策略尚未接入执行器")
|
||||
if int(params.get("workers", 1)) != 1:
|
||||
raise HTTPException(status_code=400, detail="当前专家采集执行器仅支持 workers=1")
|
||||
if int(params.get("episodes", 4)) <= 0 or int(params.get("steps", 24)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="episodes and steps must be > 0")
|
||||
if payload.type == "training":
|
||||
if not (params.get("dataset_id") or params.get("dataset_path")):
|
||||
raise HTTPException(status_code=400, detail="training requires dataset_id or dataset_path")
|
||||
if params.get("recipe", "bc_v2") not in {"bc_v2", "bc_fast"}:
|
||||
raise HTTPException(status_code=400, detail="当前仅支持行为克隆 v2 和快速验证;扩散策略尚未接入训练器")
|
||||
if int(params.get("epochs", 100)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="epochs must be > 0")
|
||||
if payload.type == "inference":
|
||||
if not (params.get("model_id") or params.get("model_path")):
|
||||
raise HTTPException(status_code=400, detail="inference requires model_id or model_path")
|
||||
if not (params.get("scene_id") or params.get("scene_path")):
|
||||
raise HTTPException(status_code=400, detail="inference requires scene_id or scene_path")
|
||||
if int(params.get("workers", 1)) != 1:
|
||||
raise HTTPException(status_code=400, detail="当前推理执行器仅支持 workers=1")
|
||||
if int(params.get("steps", 80)) <= 0:
|
||||
raise HTTPException(status_code=400, detail="steps must be > 0")
|
||||
|
||||
|
||||
def _validate_task_resources(store: PlatformStore, payload: CreateTaskRequest) -> None:
|
||||
params = payload.params
|
||||
try:
|
||||
if payload.type == "expert_collection" and params.get("scene_id"):
|
||||
store.get_scene(str(params["scene_id"]))
|
||||
if payload.type == "training" and params.get("dataset_id"):
|
||||
dataset = store.get_dataset(str(params["dataset_id"]))
|
||||
source = str(dataset.get("source_type") or "")
|
||||
input_kind = str(dataset.get("scan", {}).get("input_kind") or "")
|
||||
if source != "expert_collection" and input_kind != "expert_rollouts":
|
||||
raise HTTPException(status_code=400, detail="训练只能使用专家采集产生的 rollout 数据集")
|
||||
if payload.type == "inference":
|
||||
if params.get("model_id"):
|
||||
store.get_model(str(params["model_id"]))
|
||||
if params.get("scene_id"):
|
||||
store.get_scene(str(params["scene_id"]))
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=400, detail=f"任务引用的资源不存在: {exc.args[0]}") from exc
|
||||
|
||||
|
||||
def _runtime_paths(cfg: PlatformConfig) -> dict[str, str]:
|
||||
return {
|
||||
"gs_playground_root": str(cfg.gs_playground_root),
|
||||
"runtime_root": str(cfg.runtime_root),
|
||||
"anysplat_root": str(cfg.anysplat_root),
|
||||
"anysplat_python": str(cfg.anysplat_python),
|
||||
"gsplat_python": str(cfg.gsplat_python),
|
||||
"cuda_home": str(cfg.cuda_home),
|
||||
"runtime_env_script": str(cfg.runtime_env_script),
|
||||
}
|
||||
|
||||
|
||||
def _config_digest(cfg: PlatformConfig) -> str:
|
||||
payload = json.dumps(_runtime_paths(cfg) | {"workspace": str(cfg.workspace_dir), "renderer": cfg.default_renderer}, sort_keys=True)
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:16]
|
||||
|
||||
|
||||
def _git_commit(root: Path) -> str:
|
||||
try:
|
||||
return subprocess.check_output(["git", "-C", str(root), "rev-parse", "--short", "HEAD"], text=True, stderr=subprocess.DEVNULL).strip()
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def _source_snapshot(root: Path) -> str:
|
||||
files = [root / "pyproject.toml", root / "src/real2sim_platform/adapters.py", root / "src/real2sim_platform/server.py", root / "src/real2sim_platform/config.py"]
|
||||
digest = hashlib.sha256()
|
||||
for path in files:
|
||||
if path.is_file():
|
||||
digest.update(path.name.encode("utf-8"))
|
||||
digest.update(path.read_bytes())
|
||||
return digest.hexdigest()[:16]
|
||||
|
||||
|
||||
def _safe_extract(archive: zipfile.ZipFile, target_dir: Path) -> None:
|
||||
root = target_dir.resolve()
|
||||
for member in archive.infolist():
|
||||
destination = (target_dir / member.filename).resolve()
|
||||
try:
|
||||
destination.relative_to(root)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=f"unsafe zip member: {member.filename}") from exc
|
||||
archive.extractall(target_dir)
|
||||
|
||||
|
||||
def _next_run_name(root: Path, prefix: str) -> str:
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
index = 1
|
||||
while (root / f"{prefix}_{index:04d}").exists():
|
||||
index += 1
|
||||
return f"{prefix}_{index:04d}"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--config", type=Path)
|
||||
parser.add_argument("--write-config", type=Path)
|
||||
parser.add_argument("--host")
|
||||
parser.add_argument("--port", type=int)
|
||||
args = parser.parse_args()
|
||||
cfg = load_config(args.config)
|
||||
if args.host:
|
||||
cfg = PlatformConfig(**{**cfg.__dict__, "host": args.host})
|
||||
if args.port:
|
||||
cfg = PlatformConfig(**{**cfg.__dict__, "port": args.port})
|
||||
if args.write_config:
|
||||
write_default_config(args.write_config, cfg)
|
||||
print(args.write_config)
|
||||
return 0
|
||||
import uvicorn
|
||||
|
||||
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
1490
src/real2sim_platform/static/app.js
Normal file
1490
src/real2sim_platform/static/app.js
Normal file
File diff suppressed because it is too large
Load Diff
586
src/real2sim_platform/static/index.html
Normal file
586
src/real2sim_platform/static/index.html
Normal file
@ -0,0 +1,586 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Real2Sim</title>
|
||||
<link rel="icon" href="data:,">
|
||||
<link rel="stylesheet" href="/static/styles.css?v=20260710-motrix-runtime-v7">
|
||||
</head>
|
||||
<body>
|
||||
<div class="app-shell">
|
||||
<aside class="sidebar">
|
||||
<div class="brand">
|
||||
<strong>Real2Sim</strong>
|
||||
<span>真实数据到仿真智能体</span>
|
||||
</div>
|
||||
<nav class="side-nav" aria-label="主导航">
|
||||
<button class="tab active" data-tab="overview">总览</button>
|
||||
<button class="tab" data-tab="data">数据管理</button>
|
||||
<button class="tab" data-tab="reconstruct">场景重建</button>
|
||||
<button class="tab" data-tab="scenes">场景管理</button>
|
||||
<button class="tab" data-tab="collect">数据采集</button>
|
||||
<button class="tab" data-tab="train">模型训练</button>
|
||||
<button class="tab" data-tab="infer">在线推理</button>
|
||||
</nav>
|
||||
</aside>
|
||||
|
||||
<main class="main-surface">
|
||||
<section id="overview" class="page active page-dashboard">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>总览</h1>
|
||||
<p>按数据准备、场景重建、专家采集、训练和推理组织当前项目资源。</p>
|
||||
</div>
|
||||
<button id="nextStepAction" class="primary-button">继续下一步</button>
|
||||
</div>
|
||||
|
||||
<div class="overview-grid">
|
||||
<section class="panel journey-panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>我的流程</h2>
|
||||
<p>从真实数据到在线推理的当前进度</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="journey" id="journeySteps"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel next-step">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>推荐下一步</h2>
|
||||
<p id="nextStepText">正在读取当前资源状态</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="overviewMetrics" class="metric-strip"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>最近任务</h2>
|
||||
<p>显示真实阶段、进度和失败原因</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="recentTaskList" class="list-stack compact-list"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel wide scene-hero-panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>最近场景</h2>
|
||||
<p>重建完成后可直接进入预览和资产增强</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="overviewScenePreview" class="overview-scene"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>模型与推理</h2>
|
||||
<p>训练结果、模型指标和最近推理轨迹</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="latestModelCard" class="model-summary"></div>
|
||||
</section>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section id="data" class="page page-data">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>数据管理</h1>
|
||||
<p>导入数据、抽帧准备、预检质量,并把可用数据集交给重建或训练流程。</p>
|
||||
</div>
|
||||
<button class="primary-button" data-open-import="hf">新增数据</button>
|
||||
</div>
|
||||
|
||||
<div class="source-grid">
|
||||
<button class="source-card" data-open-import="hf">
|
||||
<strong>HuggingFace 样例</strong>
|
||||
<span>xarm7、3DGS fixture 和公开数据集</span>
|
||||
</button>
|
||||
<button class="source-card" data-open-import="upload">
|
||||
<strong>上传文件</strong>
|
||||
<span>zip、视频或图片包</span>
|
||||
</button>
|
||||
<button class="source-card" data-open-import="local">
|
||||
<strong>本地目录</strong>
|
||||
<span>从配置白名单导入已有数据</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>数据集</h2>
|
||||
<p>多选后可批量准备、预检或删除索引</p>
|
||||
</div>
|
||||
<div class="toolbar">
|
||||
<select id="datasetFilter" aria-label="筛选数据集">
|
||||
<option value="all">全部数据</option>
|
||||
<option value="prepared_media">已准备帧数据</option>
|
||||
<option value="expert_collection">采集数据</option>
|
||||
<option value="huggingface">HuggingFace</option>
|
||||
</select>
|
||||
<input id="datasetSearch" placeholder="搜索数据集名称">
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-resource-scope="datasetList">
|
||||
<label><input type="checkbox" data-select-all-resources="datasetList"> 全选</label>
|
||||
<span data-selected-count="datasetList">已选择 0 项</span>
|
||||
<button data-bulk-resource-action="prepare">批量准备</button>
|
||||
<button data-bulk-resource-action="preflight">批量预检</button>
|
||||
<button class="danger-button" data-bulk-resource-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div class="table-shell dataset-shell">
|
||||
<div class="table-header dataset-header">
|
||||
<span></span><span>缩略图</span><span>数据集名称</span><span>来源</span><span>规模</span><span>操作</span>
|
||||
</div>
|
||||
<div id="datasetList" class="resource-table"></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>准备输入</h2>
|
||||
<p>将视频或图片目录整理成可重建帧数据</p>
|
||||
</div>
|
||||
<button id="createMediaPrepare" class="primary-button">创建准备任务</button>
|
||||
</div>
|
||||
<div class="form-grid">
|
||||
<label class="span-2">原始数据集<select id="prepareDataset"></select></label>
|
||||
<label>相机视角<input id="prepareCamera" value="observation.images.front"></label>
|
||||
<label>每秒抽帧<input id="prepareFps" type="number" min="0.1" step="0.1" value="2"></label>
|
||||
<label>每段最多帧<input id="prepareMaxFrames" type="number" min="1" value="80"></label>
|
||||
<label>最多视频数<input id="prepareMaxVideos" type="number" min="1" value="1"></label>
|
||||
<label>目标总帧数<input id="prepareTargetFrames" type="number" min="0" value="0"></label>
|
||||
<label>最低总帧数<input id="prepareMinFrames" type="number" min="0" value="0"></label>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>数据任务</h2>
|
||||
<p>导入、上传和准备任务会在这里反馈进度</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-scope="dataTaskList">
|
||||
<label><input type="checkbox" data-select-all-tasks="dataTaskList"> 全选</label>
|
||||
<span data-selected-count="dataTaskList">已选择 0 项</span>
|
||||
<button data-bulk-task-action="retry">批量重试</button>
|
||||
<button data-bulk-task-action="cancel">批量取消</button>
|
||||
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="dataTaskList" class="list-stack"></div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
<section id="reconstruct" class="page page-reconstruct">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>场景重建</h1>
|
||||
<p>场景重建负责生成新的点云、网格、场景包和仿真导入产物;完成后进入场景管理继续预览和增强。</p>
|
||||
</div>
|
||||
<button id="createSceneTask" class="primary-button">开始重建</button>
|
||||
</div>
|
||||
|
||||
<div class="reconstruct-grid">
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>重建配置</h2>
|
||||
<p>先选 prepared 数据,再选择质量、几何和仿真策略</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="reconstructDatasetCards" class="selection-card-rail"></div>
|
||||
<div class="form-grid">
|
||||
<label class="span-2">场景名称<input id="sceneName" placeholder="例如 Playroom 高质量重建"></label>
|
||||
<label class="span-2">已准备数据集<select id="reconstructInput"></select></label>
|
||||
<label>重建预设<select id="qualityPreset"><option value="discoverse-high">DISCOVERSE 高质量</option><option value="motrix-high">仿真优先</option><option value="balanced">平衡</option></select></label>
|
||||
<label>渲染方式<select id="rendererSelect"><option value="auto">自动选择</option><option value="spark">Spark 预览</option><option value="aholo">Aholo LOD</option></select></label>
|
||||
<label>仿真器<select id="simulatorSelect"><option value="both">MuJoCo + MotrixSim</option><option value="mujoco">MuJoCo</option><option value="motrixsim">MotrixSim</option></select></label>
|
||||
<label>最少帧数<input id="sceneMinFrames" type="number" min="1" value="30"></label>
|
||||
<label>验收标准<select id="acceptanceLevel"><option value="balanced">平衡</option><option value="strict">严格</option><option value="preview">仅预览</option></select></label>
|
||||
</div>
|
||||
|
||||
<details class="option-group">
|
||||
<summary>高级重建参数</summary>
|
||||
<div class="form-grid">
|
||||
<label>相机模型<select id="cameraModel"><option value="auto">自动识别</option><option value="SIMPLE_RADIAL">SIMPLE_RADIAL</option><option value="PINHOLE">PINHOLE</option><option value="OPENCV">OPENCV</option></select></label>
|
||||
<label>匹配策略<select id="matcherSelect"><option value="exhaustive">全量匹配</option><option value="sequential">顺序匹配</option></select></label>
|
||||
<label>最大图像边<input id="maxImageSize" type="number" min="512" step="128" value="1600"></label>
|
||||
<label>SIFT 特征数<input id="siftFeatures" type="number" min="1000" step="1000" value="8192"></label>
|
||||
<label>顺序重叠帧<input id="sequentialOverlap" type="number" min="1" value="12"></label>
|
||||
<label>AnySplat 视角数<input id="anysplatMaxViews" type="number" min="8" step="8" value="80"></label>
|
||||
<label>训练步数<input id="poseTrainSteps" type="number" min="1000" step="1000" value="15000"></label>
|
||||
<label>目标高斯数<input id="targetGaussians" type="number" min="100000" step="100000" value="1500000"></label>
|
||||
<label>尺度参考<select id="scaleHint"><option value="auto">自动估计</option><option value="metric_room">房间尺度</option><option value="tabletop">桌面尺度</option><option value="robot_workspace">机器人工作区</option></select></label>
|
||||
<label>坐标系<select id="coordinateFrame"><option value="gs_metric_zup">Z-up 米制</option><option value="mujoco_zup">MuJoCo Z-up</option><option value="motrixsim_world">MotrixSim World</option></select></label>
|
||||
<label class="check"><input id="autoPrepareScene" type="checkbox"> 自动准备输入</label>
|
||||
<label class="check"><input id="allowLowQualityScene" type="checkbox"> 允许低质量继续</label>
|
||||
<label class="check"><input id="guidedMatching" type="checkbox" checked> 引导匹配</label>
|
||||
<label class="check"><input id="poseInitTrain" type="checkbox" checked> 位姿初始化训练</label>
|
||||
<label class="check"><input id="replaceSceneWithPoseTrain" type="checkbox" checked> 用训练结果替换主场景</label>
|
||||
<label class="check"><input id="exportTextures" type="checkbox" checked> 生成纹理资产</label>
|
||||
<label class="check"><input id="keepIntermediate" type="checkbox" checked> 保留中间产物</label>
|
||||
<label class="check muted-check"><input id="dryRunScene" type="checkbox"> 仅生成样例</label>
|
||||
</div>
|
||||
</details>
|
||||
|
||||
<div class="button-row">
|
||||
<button id="createPreflightTask">只做预检</button>
|
||||
<button id="rendererBenchmark">渲染器对比</button>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel preflight-panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>质量预检</h2>
|
||||
<p>帧数、输入类型、验收档位和运行建议</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="preflightSummary" class="quality-grid"></div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>重建任务与预览</h2>
|
||||
<p>任务创建后立即出现在这里;成功后可直接预览</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-scope="sceneTaskList">
|
||||
<label><input type="checkbox" data-select-all-tasks="sceneTaskList"> 全选</label>
|
||||
<span data-selected-count="sceneTaskList">已选择 0 项</span>
|
||||
<button data-bulk-task-action="retry">批量重试</button>
|
||||
<button data-bulk-task-action="cancel">批量取消</button>
|
||||
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="sceneTaskList" class="list-stack"></div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
<section id="scenes" class="page">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>场景管理</h1>
|
||||
<p>场景管理用于查看重建结果、添加资产、仿真校验、导出,并绑定后续采集和推理。</p>
|
||||
</div>
|
||||
<button id="validateSelectedScenes" class="primary-button">校验已选场景</button>
|
||||
</div>
|
||||
|
||||
<div class="scene-workbench">
|
||||
<section class="panel scene-library">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>场景库</h2>
|
||||
<p>多选后可校验、导出或删除索引</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-resource-scope="sceneList">
|
||||
<label><input type="checkbox" data-select-all-resources="sceneList"> 全选</label>
|
||||
<span data-selected-count="sceneList">已选择 0 项</span>
|
||||
<button data-bulk-resource-action="validate">批量校验</button>
|
||||
<button data-bulk-resource-action="export">批量导出</button>
|
||||
<button class="danger-button" data-bulk-resource-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="sceneList" class="resource-table scene-list"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel viewer-panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>场景预览</h2>
|
||||
<p id="viewerStatus">选择场景后加载 Spark/Three 预览</p>
|
||||
</div>
|
||||
<button id="openSceneViewerFull" class="ghost-button" type="button">最大化预览</button>
|
||||
</div>
|
||||
<iframe id="scenePreview" title="场景预览"></iframe>
|
||||
</section>
|
||||
|
||||
<section class="panel asset-editor-panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>场景增强</h2>
|
||||
<p>从当前场景包选择资产,或在高级入口导入白名单文件</p>
|
||||
</div>
|
||||
<button id="addSceneAsset" class="primary-button">添加资产</button>
|
||||
</div>
|
||||
<div class="form-grid">
|
||||
<label class="span-2">目标场景<select id="assetScene"></select></label>
|
||||
<label class="span-2">资产来源<select id="assetPreset"></select></label>
|
||||
<details class="span-2 option-group">
|
||||
<summary>高级:从白名单路径添加新资产</summary>
|
||||
<label>资产文件<input id="assetPath" placeholder="仅支持配置白名单内文件"></label>
|
||||
</details>
|
||||
<label>显示名称<input id="assetName" placeholder="例如柜体碰撞模型"></label>
|
||||
<label>资产类型<select id="assetKind"><option value="auto">自动识别</option><option value="mesh">网格 / 物体</option><option value="splat_or_pointcloud">点云 / Splat</option><option value="texture">纹理</option><option value="simulator">仿真文件</option><option value="metadata">配置 / 元数据</option></select></label>
|
||||
<label>用途<select id="assetRole"><option value="added_asset">新增资产</option><option value="collision">碰撞体</option><option value="visual">可视化模型</option><option value="texture">纹理贴图</option><option value="simulator_import">仿真导入</option></select></label>
|
||||
<label>尺度<input id="assetScale" type="number" min="0.001" step="0.001" value="1"></label>
|
||||
<label>位置 X<input id="assetPosX" type="number" step="0.01" value="0"></label>
|
||||
<label>位置 Y<input id="assetPosY" type="number" step="0.01" value="0"></label>
|
||||
<label>位置 Z<input id="assetPosZ" type="number" step="0.01" value="0"></label>
|
||||
<label>朝向 Yaw<input id="assetYaw" type="number" step="1" value="0"></label>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>资产清单</h2>
|
||||
<p>点云、网格、纹理、相机和仿真产物</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="assetRail" class="asset-grid"></div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
<section id="collect" class="page">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>数据采集</h1>
|
||||
<p>选择已重建场景和专家策略,批量生成可训练的 rollout 数据。</p>
|
||||
</div>
|
||||
<button id="createCollectTask" class="primary-button">开始采集</button>
|
||||
</div>
|
||||
|
||||
<div class="two-column">
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>采集配置</h2>
|
||||
<p>场景、策略和 rollout 规模</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-grid">
|
||||
<label class="span-2">选择场景<select id="collectScene"></select></label>
|
||||
<label>专家策略<select id="policyPreset"><option value="pick_and_place_v1">抓取放置策略</option><option value="navigation_v1" disabled>导航采集策略(待接入)</option><option value="manipulation_v2" disabled>精细操作策略(待接入)</option></select></label>
|
||||
<label>执行并发<input id="collectWorkers" type="number" min="1" max="1" value="1" disabled title="当前专家采集执行器仅支持单 worker"></label>
|
||||
<label>Episode 数<input id="episodes" type="number" value="4"></label>
|
||||
<label>每个 Episode 步数<input id="steps" type="number" value="24"></label>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>采集数据集</h2>
|
||||
<p>生成后可直接用于模型训练</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="collectionDatasetList" class="resource-table compact-table"></div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>采集任务</h2>
|
||||
<p>支持多选取消、重试、删除和查看产物</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-scope="collectTaskList">
|
||||
<label><input type="checkbox" data-select-all-tasks="collectTaskList"> 全选</label>
|
||||
<span data-selected-count="collectTaskList">已选择 0 项</span>
|
||||
<button data-bulk-task-action="retry">批量重试</button>
|
||||
<button data-bulk-task-action="cancel">批量取消</button>
|
||||
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="collectTaskList" class="list-stack"></div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
<section id="train" class="page">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>模型训练</h1>
|
||||
<p>基于专家采集数据训练策略模型,并追踪指标、模型卡和导出产物。</p>
|
||||
</div>
|
||||
<button id="createTrainTask" class="primary-button">开始训练</button>
|
||||
</div>
|
||||
|
||||
<div class="two-column">
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>训练配置</h2>
|
||||
<p>选择训练数据和配方</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-grid">
|
||||
<label class="span-2">训练数据集<select id="trainDataset"></select></label>
|
||||
<label>训练配方<select id="trainRecipe"><option value="bc_v2">行为克隆 v2</option><option value="bc_fast">快速验证</option><option value="diffusion_policy" disabled>扩散策略(待接入)</option></select></label>
|
||||
<label>Epoch 数<input id="epochs" type="number" value="100"></label>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>训练指标</h2>
|
||||
<p>成功率、损失和验证集表现</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="trainingChart" class="chart-panel"></div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>训练任务</h2>
|
||||
<p>查看训练进度和模型产物</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-scope="trainTaskList">
|
||||
<label><input type="checkbox" data-select-all-tasks="trainTaskList"> 全选</label>
|
||||
<span data-selected-count="trainTaskList">已选择 0 项</span>
|
||||
<button data-bulk-task-action="retry">批量重试</button>
|
||||
<button data-bulk-task-action="cancel">批量取消</button>
|
||||
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="trainTaskList" class="list-stack"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>模型列表</h2>
|
||||
<p>多选后可对比、导出或删除索引</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-resource-scope="modelList">
|
||||
<label><input type="checkbox" data-select-all-resources="modelList"> 全选</label>
|
||||
<span data-selected-count="modelList">已选择 0 项</span>
|
||||
<button data-bulk-resource-action="compare">批量对比</button>
|
||||
<button data-bulk-resource-action="export">批量导出</button>
|
||||
<button class="danger-button" data-bulk-resource-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="modelList" class="model-grid"></div>
|
||||
<div id="modelCompareResult" class="compare-result"></div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
<section id="infer" class="page">
|
||||
<div class="page-head">
|
||||
<div>
|
||||
<h1>在线推理</h1>
|
||||
<p>选择模型和场景运行在线推理,查看 rollout 播放、轨迹产物和对比结果。</p>
|
||||
</div>
|
||||
<button id="createInferTask" class="primary-button">开始推理</button>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>推理配置</h2>
|
||||
<p>模型、场景和 rollout 参数</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-grid">
|
||||
<label>选择模型<select id="inferModel"></select></label>
|
||||
<label>选择场景<select id="inferScene"></select></label>
|
||||
<label>步数上限<input id="inferSteps" type="number" min="1" value="80"></label>
|
||||
<label>执行并发<input id="inferWorkers" type="number" min="1" max="1" value="1" disabled title="当前推理执行器仅支持单 worker"></label>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<div class="two-column">
|
||||
<section class="panel rollout-panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>Rollout 回放</h2>
|
||||
<p>推理完成后展示 viewer 或轨迹预览</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="rolloutPreview" class="rollout-preview"></div>
|
||||
</section>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>轨迹产物</h2>
|
||||
<p>rollout、viewer 和模型输出</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="trajectoryList" class="list-stack"></div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section class="panel">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2>推理任务</h2>
|
||||
<p>支持多选处理、重试和删除</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bulk-bar" data-bulk-scope="inferTaskList">
|
||||
<label><input type="checkbox" data-select-all-tasks="inferTaskList"> 全选</label>
|
||||
<span data-selected-count="inferTaskList">已选择 0 项</span>
|
||||
<button data-bulk-task-action="retry">批量重试</button>
|
||||
<button data-bulk-task-action="cancel">批量取消</button>
|
||||
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
|
||||
</div>
|
||||
<div id="inferTaskList" class="list-stack"></div>
|
||||
</section>
|
||||
</section>
|
||||
</main>
|
||||
</div>
|
||||
|
||||
<aside id="taskDrawer" class="task-drawer" aria-hidden="true">
|
||||
<div class="drawer-head">
|
||||
<div>
|
||||
<span>任务详情</span>
|
||||
<h2 id="drawerTitle">未选择任务</h2>
|
||||
</div>
|
||||
<button id="closeDrawer" class="ghost-button">关闭</button>
|
||||
</div>
|
||||
<div id="drawerBody"></div>
|
||||
</aside>
|
||||
|
||||
<dialog id="importDialog">
|
||||
<form id="importForm" class="dialog-card">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2 id="importTitle">新增数据</h2>
|
||||
<p id="importSubtitle">选择来源后提交导入任务</p>
|
||||
</div>
|
||||
<button type="button" data-close-dialog="importDialog" class="ghost-button">关闭</button>
|
||||
</div>
|
||||
<div id="importFields" class="form-grid"></div>
|
||||
<menu>
|
||||
<button type="button" data-close-dialog="importDialog">取消</button>
|
||||
<button id="submitImport" type="submit" class="primary-button">提交</button>
|
||||
</menu>
|
||||
</form>
|
||||
</dialog>
|
||||
|
||||
<dialog id="editDialog">
|
||||
<form id="editForm" class="dialog-card">
|
||||
<div class="panel-head">
|
||||
<div>
|
||||
<h2 id="editTitle">编辑</h2>
|
||||
<p id="editSubtitle">更新名称、批次和标签</p>
|
||||
</div>
|
||||
<button type="button" data-close-dialog="editDialog" class="ghost-button">关闭</button>
|
||||
</div>
|
||||
<div id="editFields" class="form-grid"></div>
|
||||
<menu>
|
||||
<button type="button" data-close-dialog="editDialog">取消</button>
|
||||
<button id="saveEdit" type="submit" class="primary-button">保存</button>
|
||||
</menu>
|
||||
</form>
|
||||
</dialog>
|
||||
|
||||
<div id="toast" class="toast" role="status"></div>
|
||||
<script src="/static/app.js?v=20260710-motrix-runtime-v7"></script>
|
||||
</body>
|
||||
</html>
|
||||
560
src/real2sim_platform/static/scene-viewer.html
Normal file
560
src/real2sim_platform/static/scene-viewer.html
Normal file
@ -0,0 +1,560 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Real2Sim Scene Viewer</title>
|
||||
<link rel="icon" href="data:,">
|
||||
<style>
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; overflow: hidden; background: #080b0d; color: #e7edf2; font: 13px/1.4 system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
|
||||
#hud { position: fixed; z-index: 10; left: 12px; top: 12px; width: min(560px, calc(100vw - 24px)); min-width: 280px; min-height: 76px; max-width: calc(100vw - 24px); max-height: calc(100vh - 24px); padding: 10px 12px; border: 1px solid #2a343d; border-radius: 8px; background: rgba(12, 17, 22, 0.92); box-shadow: 0 12px 30px rgba(0, 0, 0, 0.25); overflow: hidden; }
|
||||
#hudTitlebar { display: flex; align-items: center; justify-content: space-between; gap: 12px; margin: -10px -12px 8px; padding: 9px 10px 7px 12px; border-bottom: 1px solid rgba(75, 91, 104, 0.45); cursor: grab; user-select: none; touch-action: none; }
|
||||
#hudTitlebar:active { cursor: grabbing; }
|
||||
#hud strong { display: block; margin: 0; }
|
||||
#hudToggle { flex: 0 0 auto; margin: 0; min-width: 28px; min-height: 26px; padding: 2px 7px; font-size: 15px; line-height: 1; }
|
||||
#hudContent { overflow: auto; max-height: calc(100vh - 116px); }
|
||||
#hud.is-collapsed { width: auto !important; min-width: 0; min-height: 0; height: auto !important; padding-bottom: 10px; resize: none; }
|
||||
#hud.is-collapsed #hudTitlebar { margin-bottom: -1px; border-bottom: 0; }
|
||||
#hud.is-collapsed #hudContent, #hud.is-collapsed #hudResizer { display: none; }
|
||||
#hudResizer { position: absolute; right: 1px; bottom: 1px; width: 17px; height: 17px; cursor: nwse-resize; touch-action: none; background: linear-gradient(135deg, transparent 46%, #52717a 47%, #52717a 55%, transparent 56%, transparent 64%, #52717a 65%, #52717a 73%, transparent 74%); opacity: 0.8; }
|
||||
#hud div { color: #9aa8b4; overflow-wrap: anywhere; }
|
||||
#hint { margin-top: 6px; color: #c4d0dc; }
|
||||
#assetList { margin-top: 8px; max-height: 120px; overflow: auto; }
|
||||
#assetList button[data-autoload="false"] { border-color: #6b5a33; }
|
||||
#assetList button[data-renderable="false"] { opacity: 0.72; border-color: #33404a; }
|
||||
#assetList button[data-quality="recommended"] { border-color: #28d8bd; box-shadow: inset 0 0 0 1px rgba(40, 216, 189, 0.18); }
|
||||
#assetList button[data-quality="full"] { border-color: #c9a85d; }
|
||||
#assetList button[data-quality="diagnostic"] { opacity: 0.72; }
|
||||
button { margin: 3px 4px 0 0; border: 1px solid #2a343d; border-radius: 6px; background: #17232b; color: #e7edf2; padding: 5px 8px; cursor: pointer; }
|
||||
canvas { display: block; width: 100vw; height: 100vh; }
|
||||
@media (max-width: 640px) {
|
||||
#hud { left: 8px; top: 8px; width: calc(100vw - 16px); min-width: 0; max-height: min(46vh, 360px); }
|
||||
#hudContent { max-height: calc(min(46vh, 360px) - 56px); }
|
||||
#hudResizer { display: none; }
|
||||
}
|
||||
</style>
|
||||
<script type="importmap">
|
||||
{
|
||||
"imports": {
|
||||
"three": "/static/vendor/three/three.module.js",
|
||||
"three/addons/": "/static/vendor/three/addons/"
|
||||
}
|
||||
}
|
||||
</script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="hud" aria-label="场景预览资源面板">
|
||||
<div id="hudTitlebar">
|
||||
<strong>Real2Sim Scene Viewer</strong>
|
||||
<button id="hudToggle" type="button" title="收起资源面板" aria-label="收起资源面板">−</button>
|
||||
</div>
|
||||
<div id="hudContent">
|
||||
<div id="status">loading</div>
|
||||
<div id="hint"></div>
|
||||
<div id="assetList"></div>
|
||||
</div>
|
||||
<div id="hudResizer" aria-label="调整面板大小" title="拖动调整面板大小"></div>
|
||||
</div>
|
||||
<script type="module">
|
||||
import * as THREE from "three";
|
||||
import { OrbitControls } from "three/addons/controls/OrbitControls.js";
|
||||
import { OBJLoader } from "three/addons/loaders/OBJLoader.js";
|
||||
import { PLYLoader } from "three/addons/loaders/PLYLoader.js";
|
||||
|
||||
const status = document.getElementById("status");
|
||||
const hint = document.getElementById("hint");
|
||||
const assetList = document.getElementById("assetList");
|
||||
const hud = document.getElementById("hud");
|
||||
const hudTitlebar = document.getElementById("hudTitlebar");
|
||||
const hudToggle = document.getElementById("hudToggle");
|
||||
const hudResizer = document.getElementById("hudResizer");
|
||||
const params = new URLSearchParams(location.search);
|
||||
const sceneId = params.get("scene_id");
|
||||
const embeddedMode = params.get("embedded") === "1";
|
||||
const coverMode = params.get("cover") === "1";
|
||||
const coverView = params.get("cover_view") || "hero";
|
||||
const probeFull = params.get("probe_full") === "1";
|
||||
if (coverMode) document.getElementById("hud").style.display = "none";
|
||||
setupHudPanel();
|
||||
if (embeddedMode || innerWidth <= 640) setHudCollapsed(true);
|
||||
const nativeWarn = console.warn.bind(console);
|
||||
console.warn = (...args) => {
|
||||
const message = args.map((item) => item?.message || String(item)).join(" ");
|
||||
if (message.includes("THREE.Clock: This module has been deprecated")) {
|
||||
console.debug(...args);
|
||||
return;
|
||||
}
|
||||
nativeWarn(...args);
|
||||
};
|
||||
|
||||
const renderer = new THREE.WebGLRenderer({ antialias: true, preserveDrawingBuffer: coverMode });
|
||||
const basePixelRatio = Math.min(devicePixelRatio, 2);
|
||||
renderer.setPixelRatio(basePixelRatio);
|
||||
renderer.setSize(innerWidth, innerHeight);
|
||||
renderer.setClearColor(0x080b0d, 1);
|
||||
document.body.appendChild(renderer.domElement);
|
||||
|
||||
const scene = new THREE.Scene();
|
||||
const camera = new THREE.PerspectiveCamera(55, innerWidth / innerHeight, 0.01, 1000);
|
||||
setCoverCamera();
|
||||
const controls = new OrbitControls(camera, renderer.domElement);
|
||||
controls.target.set(0, 0, 0);
|
||||
controls.enableDamping = true;
|
||||
|
||||
scene.add(new THREE.GridHelper(4, 16, 0x2f6d68, 0x24313a));
|
||||
scene.add(new THREE.AxesHelper(1.2));
|
||||
const light = new THREE.DirectionalLight(0xffffff, 1.5);
|
||||
light.position.set(3, 4, 2);
|
||||
scene.add(light);
|
||||
scene.add(new THREE.AmbientLight(0x9fb6c8, 0.8));
|
||||
|
||||
let loadedObject = null;
|
||||
let sparkRenderer = null;
|
||||
let sparkModule = null;
|
||||
const debug = {
|
||||
backend: "loading",
|
||||
loaded: false,
|
||||
assetName: "",
|
||||
pointCount: 0,
|
||||
errors: [],
|
||||
warnings: [],
|
||||
canvas: { width: 0, height: 0 },
|
||||
};
|
||||
window.real2simViewerDebug = debug;
|
||||
window.real2simViewerDebug.snapshot = () => ({ ...debug, canvas: { ...debug.canvas } });
|
||||
|
||||
function setupHudPanel() {
|
||||
let drag = null;
|
||||
let resize = null;
|
||||
const clamp = (value, min, max) => Math.max(min, Math.min(value, max));
|
||||
const clearPointerState = () => {
|
||||
drag = null;
|
||||
resize = null;
|
||||
};
|
||||
|
||||
hudToggle.addEventListener("click", (event) => {
|
||||
event.stopPropagation();
|
||||
setHudCollapsed(!hud.classList.contains("is-collapsed"));
|
||||
});
|
||||
|
||||
hudTitlebar.addEventListener("pointerdown", (event) => {
|
||||
if (event.target.closest("button") || hud.classList.contains("is-collapsed")) return;
|
||||
const rect = hud.getBoundingClientRect();
|
||||
drag = { offsetX: event.clientX - rect.left, offsetY: event.clientY - rect.top };
|
||||
hudTitlebar.setPointerCapture(event.pointerId);
|
||||
event.preventDefault();
|
||||
});
|
||||
hudResizer.addEventListener("pointerdown", (event) => {
|
||||
if (hud.classList.contains("is-collapsed")) return;
|
||||
const rect = hud.getBoundingClientRect();
|
||||
resize = { startX: event.clientX, startY: event.clientY, width: rect.width, height: rect.height };
|
||||
hudResizer.setPointerCapture(event.pointerId);
|
||||
event.preventDefault();
|
||||
});
|
||||
addEventListener("pointermove", (event) => {
|
||||
if (drag) {
|
||||
const rect = hud.getBoundingClientRect();
|
||||
const left = clamp(event.clientX - drag.offsetX, 8, innerWidth - rect.width - 8);
|
||||
const top = clamp(event.clientY - drag.offsetY, 8, innerHeight - rect.height - 8);
|
||||
hud.style.left = `${left}px`;
|
||||
hud.style.top = `${top}px`;
|
||||
hud.style.right = "auto";
|
||||
hud.style.bottom = "auto";
|
||||
}
|
||||
if (resize) {
|
||||
const width = clamp(resize.width + event.clientX - resize.startX, 280, innerWidth - hud.offsetLeft - 8);
|
||||
const height = clamp(resize.height + event.clientY - resize.startY, 76, innerHeight - hud.offsetTop - 8);
|
||||
hud.style.width = `${width}px`;
|
||||
hud.style.height = `${height}px`;
|
||||
document.getElementById("hudContent").style.maxHeight = `${Math.max(32, height - 56)}px`;
|
||||
}
|
||||
});
|
||||
addEventListener("pointerup", clearPointerState);
|
||||
addEventListener("pointercancel", clearPointerState);
|
||||
}
|
||||
|
||||
function setHudCollapsed(collapsed) {
|
||||
hud.classList.toggle("is-collapsed", collapsed);
|
||||
hudToggle.textContent = collapsed ? "+" : "−";
|
||||
hudToggle.title = collapsed ? "展开资源面板" : "收起资源面板";
|
||||
hudToggle.setAttribute("aria-label", hudToggle.title);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (!sceneId) {
|
||||
addPlaceholder("missing scene_id");
|
||||
return;
|
||||
}
|
||||
const manifest = await fetch(`/api/scenes/${encodeURIComponent(sceneId)}/manifest`).then((response) => {
|
||||
if (!response.ok) throw new Error(`manifest request failed: ${response.status}`);
|
||||
return response.json();
|
||||
});
|
||||
const assets = Array.isArray(manifest.assets) ? prioritizeAssets(manifest.assets) : [];
|
||||
const renderable = assets.filter(isRenderableAsset);
|
||||
assetList.innerHTML = assets.slice(0, 20).map((item, index) => {
|
||||
const quality = assetQuality(item, index);
|
||||
const renderable = isRenderableAsset(item);
|
||||
const label = [assetQualityLabel(quality), assetTypeLabel(item), item.name || item.path || "asset", item.size_label || "", item.autoload === false ? "手动加载" : ""].filter(Boolean).join(" · ");
|
||||
const title = renderable ? "加载此预览资产" : "查看资源说明,不会替换当前 3D 预览";
|
||||
return `<button data-index="${index}" data-renderable="${renderable}" data-autoload="${item.autoload !== false}" data-quality="${quality}" title="${escapeHtml(title)}">${escapeHtml(label)}</button>`;
|
||||
}).join("");
|
||||
assetList.addEventListener("click", (event) => {
|
||||
const button = event.target.closest("[data-index]");
|
||||
if (!button) return;
|
||||
if (button.dataset.renderable === "false") {
|
||||
showAssetInfo(assets[Number(button.dataset.index)]);
|
||||
return;
|
||||
}
|
||||
loadAsset(assets[Number(button.dataset.index)], { manual: true }).catch(showError);
|
||||
});
|
||||
status.textContent = rendererStatusText(manifest, assets.length);
|
||||
hint.textContent = viewerHint(assets, renderable);
|
||||
const preferred = renderable.find((item) => item.autoload !== false) || null;
|
||||
if (preferred) await loadAsset(preferred);
|
||||
else if (renderable[0]) addPlaceholder(`可渲染大文件未自动加载:${renderable[0].name || "asset"} · ${renderable[0].size_label || "unknown size"} · 点击按钮手动加载`);
|
||||
else addPlaceholder("当前清单没有可直接渲染的 PLY/SPZ/SPLAT/GLB/OBJ 资产");
|
||||
if (probeFull) {
|
||||
const full = assets.find((item) => {
|
||||
const name = String(item.name || item.path || "").toLowerCase();
|
||||
return name.endsWith(".spz") || name.endsWith(".splat") || name.endsWith(".ksplat") || name.endsWith(".sog");
|
||||
}) || assets.find((item) => assetQuality(item, 1) === "full" && String(item.name || item.path || "").toLowerCase().includes("scene.ply"));
|
||||
if (full) {
|
||||
const started = performance.now();
|
||||
try {
|
||||
await loadAsset(full, { manual: true });
|
||||
writeProbeResult({ loaded: debug.loaded, backend: debug.backend, assetName: debug.assetName, pointCount: debug.pointCount, elapsedMs: Math.round(performance.now() - started), canvas: debug.canvas, warnings: debug.warnings, errors: debug.errors });
|
||||
} catch (error) {
|
||||
writeProbeResult({ loaded: false, error: error.message || String(error), elapsedMs: Math.round(performance.now() - started), warnings: debug.warnings, errors: debug.errors });
|
||||
}
|
||||
} else {
|
||||
writeProbeResult({ loaded: false, error: "full 3DGS asset not found" });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function writeProbeResult(result) {
|
||||
let node = document.getElementById("probeResult");
|
||||
if (!node) {
|
||||
node = document.createElement("pre");
|
||||
node.id = "probeResult";
|
||||
node.style.display = "none";
|
||||
document.body.appendChild(node);
|
||||
}
|
||||
node.textContent = JSON.stringify(result);
|
||||
document.body.dataset.probeResult = node.textContent;
|
||||
}
|
||||
|
||||
async function loadAsset(asset, options = {}) {
|
||||
if (!asset?.url) {
|
||||
addPlaceholder("asset has no preview URL");
|
||||
return;
|
||||
}
|
||||
if (asset.autoload === false && !options.manual) {
|
||||
addPlaceholder(`large asset skipped: ${asset.name || "asset"} · ${asset.size_label || "unknown size"} · click to load`);
|
||||
return;
|
||||
}
|
||||
if (loadedObject) scene.remove(loadedObject);
|
||||
const suffix = (asset.name || asset.path || "").toLowerCase().split(".").pop();
|
||||
debug.loaded = false;
|
||||
debug.assetName = asset.name || asset.path || "asset";
|
||||
debug.pointCount = 0;
|
||||
setRenderQualityForAsset(asset);
|
||||
const size = asset.size_label ? ` · ${asset.size_label}` : "";
|
||||
if (suffix === "ply" && shouldUseSparkForPly(asset)) {
|
||||
await withLoadingTimeout(loadSparkOrFallback(asset, suffix), `loading timed out: ${asset.name || "asset"}${size}`, 90000);
|
||||
} else if (suffix === "ply") {
|
||||
await loadPlyFallback(asset);
|
||||
} else if (["spz", "splat", "ksplat", "sog", "zip", "rad"].includes(suffix)) {
|
||||
await withLoadingTimeout(loadSparkOrFallback(asset, suffix), `loading timed out: ${asset.name || "asset"}${size}`);
|
||||
} else if (suffix === "glb" || suffix === "gltf") {
|
||||
status.textContent = `Three.js fallback · loading ${asset.name}${size}`;
|
||||
const { GLTFLoader } = await import("three/addons/loaders/GLTFLoader.js");
|
||||
const gltf = await withLoadingTimeout(new GLTFLoader().loadAsync(asset.url), `loading timed out: ${asset.name || "asset"}${size}`);
|
||||
loadedObject = gltf.scene;
|
||||
normalizeObjectFromBox(loadedObject);
|
||||
scene.add(loadedObject);
|
||||
debug.backend = "three_fallback";
|
||||
debug.loaded = true;
|
||||
status.textContent = `Three.js fallback · ${asset.name}`;
|
||||
} else if (suffix === "obj") {
|
||||
status.textContent = `Three.js fallback · loading ${asset.name}${size}`;
|
||||
loadedObject = await withLoadingTimeout(new OBJLoader().loadAsync(asset.url), `loading timed out: ${asset.name || "asset"}${size}`);
|
||||
normalizeObjectFromBox(loadedObject);
|
||||
scene.add(loadedObject);
|
||||
debug.backend = "three_fallback";
|
||||
debug.loaded = true;
|
||||
status.textContent = `Three.js fallback · ${asset.name}`;
|
||||
} else if (["xml", "urdf", "mjcf"].includes(suffix)) {
|
||||
status.textContent = "这是仿真描述文件,用于 MotrixSim/MuJoCo 加载,不会替换当前 3D 预览";
|
||||
} else if (suffix === "py") {
|
||||
status.textContent = "这是控制脚本,用于运行策略或示例,不会替换当前 3D 预览";
|
||||
} else if (["png", "jpg", "jpeg", "webp"].includes(suffix)) {
|
||||
addImagePlane(asset);
|
||||
} else {
|
||||
addPlaceholder(`当前浏览器预览暂不支持 .${suffix || "file"} 文件`);
|
||||
}
|
||||
}
|
||||
|
||||
function isRenderableAsset(asset) {
|
||||
const suffix = String(asset?.name || asset?.path || "").toLowerCase().split(".").pop();
|
||||
return ["ply", "spz", "splat", "ksplat", "sog", "glb", "gltf", "obj", "png", "jpg", "jpeg", "webp"].includes(suffix);
|
||||
}
|
||||
|
||||
function assetTypeLabel(asset) {
|
||||
const suffix = String(asset?.name || asset?.path || "").toLowerCase().split(".").pop();
|
||||
const kind = String(asset?.kind || "");
|
||||
if (["ply", "spz", "splat", "ksplat", "sog", "glb", "gltf", "obj"].includes(suffix)) return "可预览";
|
||||
if (["png", "jpg", "jpeg", "webp"].includes(suffix)) return "图片";
|
||||
if (["xml", "urdf", "mjcf"].includes(suffix) || kind === "simulator") return "仿真文件";
|
||||
if (suffix === "py" || kind === "controller") return "控制脚本";
|
||||
return "资源";
|
||||
}
|
||||
|
||||
function viewerHint(assets, renderable) {
|
||||
if (renderable.some((asset) => !["png", "jpg", "jpeg", "webp"].includes(String(asset.name || asset.path || "").toLowerCase().split(".").pop()))) {
|
||||
return "可直接预览 PLY、SPZ/SPLAT、GLB/OBJ 等资产;XML 和 Python 只作为仿真/控制资源列出。";
|
||||
}
|
||||
if (assets.length) return "当前包主要是仿真 XML、控制脚本和参考图;需要生成 PLY/GLB/3DGS 后才能看到真实三维场景。";
|
||||
return "当前场景清单为空。";
|
||||
}
|
||||
|
||||
function showAssetInfo(asset) {
|
||||
const suffix = String(asset?.name || asset?.path || "").toLowerCase().split(".").pop();
|
||||
const name = asset?.name || asset?.path || "资源";
|
||||
const size = asset?.size_label ? ` · ${asset.size_label}` : "";
|
||||
const labels = {
|
||||
xml: "仿真描述文件,用于 MuJoCo/MotrixSim 加载",
|
||||
urdf: "机器人/场景描述文件,用于仿真器加载",
|
||||
mjcf: "MuJoCo MJCF 文件,用于仿真器加载",
|
||||
py: "控制脚本,用于运行策略或示例",
|
||||
json: "结构化报告或轨迹数据,可在任务/清单中查看",
|
||||
md: "说明文档",
|
||||
txt: "文本资源",
|
||||
gif: "rollout 回放资源,可在场景管理的资产区查看",
|
||||
mp4: "视频资源,可在场景管理的资产区查看",
|
||||
};
|
||||
status.textContent = `${labels[suffix] || "资源文件"} · ${name}${size}`;
|
||||
hint.textContent = "这个条目不会替换当前 3D 场景。真正可旋转预览的是 GLB/OBJ/PLY/3DGS 等几何资产。";
|
||||
}
|
||||
|
||||
function addImagePlane(asset) {
|
||||
if (loadedObject) scene.remove(loadedObject);
|
||||
const texture = new THREE.TextureLoader().load(asset.url);
|
||||
const geometry = new THREE.PlaneGeometry(1.6, 0.9);
|
||||
const material = new THREE.MeshBasicMaterial({ map: texture, side: THREE.DoubleSide });
|
||||
loadedObject = new THREE.Mesh(geometry, material);
|
||||
scene.add(loadedObject);
|
||||
debug.backend = "image_preview";
|
||||
debug.loaded = true;
|
||||
debug.pointCount = 0;
|
||||
status.textContent = `图片预览 · ${asset.name || "asset"}`;
|
||||
}
|
||||
|
||||
function setRenderQualityForAsset(asset) {
|
||||
const sizeBytes = Number(asset?.size_bytes || 0);
|
||||
const heavy = shouldUseSparkForPly(asset) || sizeBytes > 80 * 1024 * 1024;
|
||||
renderer.setPixelRatio(heavy ? 1 : basePixelRatio);
|
||||
renderer.setSize(innerWidth, innerHeight);
|
||||
}
|
||||
|
||||
function shouldUseSparkForPly(asset) {
|
||||
const name = String(asset?.name || "").toLowerCase();
|
||||
const path = String(asset?.path || asset?.url || "").toLowerCase();
|
||||
const sizeBytes = Number(asset?.size_bytes || 0);
|
||||
if (name.endsWith("_threejs_rgb.ply")) return false;
|
||||
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return false;
|
||||
if (name === "pose_init_trained_scene.ply") return false;
|
||||
return name === "scene.ply" && (path.includes("/3dgs/") || sizeBytes > 50 * 1024 * 1024);
|
||||
}
|
||||
|
||||
async function loadSparkOrFallback(asset, suffix) {
|
||||
try {
|
||||
const spark = await loadSparkModule();
|
||||
if (!sparkRenderer) {
|
||||
sparkRenderer = new spark.SparkRenderer({ renderer });
|
||||
scene.add(sparkRenderer);
|
||||
}
|
||||
status.textContent = `Spark · loading ${asset.name}${asset.size_label ? ` · ${asset.size_label}` : ""}`;
|
||||
const splatMesh = new spark.SplatMesh({ url: asset.url });
|
||||
loadedObject = splatMesh;
|
||||
scene.add(splatMesh);
|
||||
await splatMesh.initialized;
|
||||
normalizeObjectFromBox(splatMesh);
|
||||
const count = splatMesh.splats?.getNumSplats?.() || splatMesh.packedSplats?.numSplats || 0;
|
||||
debug.backend = "spark";
|
||||
debug.loaded = true;
|
||||
debug.pointCount = count;
|
||||
status.textContent = `Spark · ${asset.name}${count ? ` · ${count} splats` : ""}`;
|
||||
} catch (error) {
|
||||
console.info("Spark preview unavailable; using Three.js fallback", error);
|
||||
debug.warnings.push(error.message || String(error));
|
||||
if (loadedObject) {
|
||||
scene.remove(loadedObject);
|
||||
loadedObject = null;
|
||||
}
|
||||
if (suffix !== "ply") throw error;
|
||||
await loadPlyFallback(asset);
|
||||
}
|
||||
}
|
||||
|
||||
async function loadPlyFallback(asset) {
|
||||
status.textContent = `Three.js fallback · loading ${asset.name}${asset.size_label ? ` · ${asset.size_label}` : ""}`;
|
||||
const geometry = await withLoadingTimeout(new PLYLoader().loadAsync(asset.url), `loading timed out: ${asset.name || "asset"}${asset.size_label ? ` · ${asset.size_label}` : ""}`);
|
||||
geometry.computeBoundingSphere();
|
||||
geometry.computeVertexNormals();
|
||||
const count = geometry.getAttribute("position")?.count || 0;
|
||||
const material = new THREE.PointsMaterial({
|
||||
size: pointSizeFor(count, asset),
|
||||
sizeAttenuation: true,
|
||||
vertexColors: geometry.hasAttribute("color"),
|
||||
color: 0x58c4a8,
|
||||
});
|
||||
loadedObject = new THREE.Points(geometry, material);
|
||||
normalizeObject(loadedObject, geometry.boundingSphere);
|
||||
scene.add(loadedObject);
|
||||
debug.backend = "three_fallback";
|
||||
debug.loaded = true;
|
||||
debug.pointCount = count;
|
||||
status.textContent = `Three.js fallback · ${asset.name} · ${count} points`;
|
||||
}
|
||||
|
||||
function pointSizeFor(count, asset) {
|
||||
const name = String(asset?.name || asset?.path || "").toLowerCase();
|
||||
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return 0.022;
|
||||
if (count <= 100000) return 0.018;
|
||||
if (count <= 500000) return 0.011;
|
||||
return 0.006;
|
||||
}
|
||||
|
||||
function withLoadingTimeout(promise, message, timeoutMs = 45000) {
|
||||
let timer;
|
||||
return Promise.race([
|
||||
promise.finally(() => clearTimeout(timer)),
|
||||
new Promise((_, reject) => {
|
||||
timer = setTimeout(() => reject(new Error(message)), timeoutMs);
|
||||
}),
|
||||
]);
|
||||
}
|
||||
|
||||
async function loadSparkModule() {
|
||||
if (sparkModule) return sparkModule;
|
||||
sparkModule = await import("/static/vendor/spark.module.min.js");
|
||||
if (!sparkModule.SparkRenderer || !sparkModule.SplatMesh) throw new Error("Spark bundle missing SparkRenderer/SplatMesh exports");
|
||||
return sparkModule;
|
||||
}
|
||||
|
||||
function normalizeObject(object, sphere) {
|
||||
if (!sphere) return;
|
||||
object.position.sub(sphere.center);
|
||||
const scale = sphere.radius > 0 ? 1.6 / sphere.radius : 1;
|
||||
object.scale.setScalar(scale);
|
||||
controls.target.set(0, 0, 0);
|
||||
setCoverCamera();
|
||||
controls.update();
|
||||
}
|
||||
|
||||
function setCoverCamera() {
|
||||
if (!coverMode) {
|
||||
camera.position.set(2.8, 2.2, 2.8);
|
||||
return;
|
||||
}
|
||||
const views = {
|
||||
hero: [2.6, 1.8, 2.4],
|
||||
front: [0.0, 1.55, 3.2],
|
||||
side: [3.2, 1.55, 0.2],
|
||||
top: [0.9, 3.4, 1.1],
|
||||
};
|
||||
const pos = views[coverView] || views.hero;
|
||||
camera.position.set(pos[0], pos[1], pos[2]);
|
||||
}
|
||||
|
||||
function normalizeObjectFromBox(object) {
|
||||
const box = new THREE.Box3().setFromObject(object);
|
||||
if (!Number.isFinite(box.min.x) || !Number.isFinite(box.max.x)) return;
|
||||
const sphere = new THREE.Sphere();
|
||||
box.getBoundingSphere(sphere);
|
||||
normalizeObject(object, sphere);
|
||||
}
|
||||
|
||||
function addPlaceholder(message) {
|
||||
if (loadedObject) scene.remove(loadedObject);
|
||||
const geometry = new THREE.BoxGeometry(0.8, 0.8, 0.8);
|
||||
const material = new THREE.MeshStandardMaterial({ color: 0x58c4a8, roughness: 0.65 });
|
||||
loadedObject = new THREE.Mesh(geometry, material);
|
||||
scene.add(loadedObject);
|
||||
debug.backend = "placeholder";
|
||||
debug.loaded = false;
|
||||
status.textContent = message;
|
||||
}
|
||||
|
||||
function showError(error) {
|
||||
console.error(error);
|
||||
debug.errors.push(error.message || String(error));
|
||||
addPlaceholder(error.message || String(error));
|
||||
}
|
||||
|
||||
function rendererStatusText(manifest, assetCount) {
|
||||
const renderer = manifest.renderer || "auto";
|
||||
const rendererStatus = manifest.renderer_status || {};
|
||||
if (rendererStatus.aholo_available && rendererStatus.selected === "aholo") return `Aholo · ${assetCount} assets`;
|
||||
if (rendererStatus.spark_available) return `Spark · ${assetCount} assets · ${rendererStatus.aholo_reason || ""}`;
|
||||
return `Three.js fallback · ${assetCount} assets · ${rendererStatus.fallback_reason || "Spark bundle unavailable"}`;
|
||||
}
|
||||
|
||||
function prioritizeAssets(assets) {
|
||||
return assets.slice().sort((left, right) => assetPriority(right) - assetPriority(left));
|
||||
}
|
||||
|
||||
function assetPriority(asset) {
|
||||
const name = String(asset?.name || asset?.path || "").toLowerCase();
|
||||
const role = String(asset?.role || "").toLowerCase();
|
||||
if (role === "viewer_preview" && name.endsWith(".glb")) return 20;
|
||||
if (role === "rollout_gif" || role === "rollout_final_frame") return 14;
|
||||
if (name.endsWith("_threejs_rgb.ply")) return 12;
|
||||
if (name.endsWith(".spz") || name.endsWith(".splat") || name.endsWith(".ksplat") || name.endsWith(".sog")) return 11;
|
||||
if (name.endsWith("/pose_init_trained_scene.ply") || name === "pose_init_trained_scene.ply") return 10;
|
||||
if (name.endsWith("/scene.ply") || name === "scene.ply") return asset?.autoload === false ? 7 : 9;
|
||||
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return 2;
|
||||
if (["added_asset", "collision", "visual", "simulator_import"].includes(role)) return 4;
|
||||
const kind = String(asset?.kind || "");
|
||||
if (["mesh", "splat_or_pointcloud"].includes(kind)) return 3;
|
||||
if (kind === "simulator") return 2;
|
||||
return 1;
|
||||
}
|
||||
|
||||
function assetQuality(asset, index) {
|
||||
const name = String(asset?.name || asset?.path || "").toLowerCase();
|
||||
if (index === 0) return "recommended";
|
||||
if (name === "scene.ply" || name.endsWith("/3dgs/scene.ply") || name.endsWith(".spz") || name.endsWith(".splat") || name.endsWith(".ksplat") || name.endsWith(".sog")) return "full";
|
||||
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return "diagnostic";
|
||||
if (asset?.autoload === false) return "manual";
|
||||
return "normal";
|
||||
}
|
||||
|
||||
function assetQualityLabel(quality) {
|
||||
return ({ recommended: "推荐", full: "完整3DGS", diagnostic: "诊断", manual: "大文件" })[quality] || "";
|
||||
}
|
||||
|
||||
function escapeHtml(value) {
|
||||
return String(value).replace(/[&<>"']/g, (ch) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" }[ch]));
|
||||
}
|
||||
|
||||
function animate() {
|
||||
controls.update();
|
||||
debug.canvas = { width: renderer.domElement.width, height: renderer.domElement.height };
|
||||
renderer.render(scene, camera);
|
||||
requestAnimationFrame(animate);
|
||||
}
|
||||
|
||||
addEventListener("resize", () => {
|
||||
camera.aspect = innerWidth / innerHeight;
|
||||
camera.updateProjectionMatrix();
|
||||
renderer.setSize(innerWidth, innerHeight);
|
||||
});
|
||||
|
||||
main().catch(showError);
|
||||
animate();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
2032
src/real2sim_platform/static/styles.css
Normal file
2032
src/real2sim_platform/static/styles.css
Normal file
File diff suppressed because it is too large
Load Diff
21
src/real2sim_platform/static/vendor/spark.LICENSE
vendored
Normal file
21
src/real2sim_platform/static/vendor/spark.LICENSE
vendored
Normal file
@ -0,0 +1,21 @@
|
||||
The MIT License
|
||||
|
||||
Copyright © 2025 WORLD LABS TECHNOLOGIES, INC.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
1
src/real2sim_platform/static/vendor/spark.VERSION
vendored
Normal file
1
src/real2sim_platform/static/vendor/spark.VERSION
vendored
Normal file
@ -0,0 +1 @@
|
||||
2.1.0
|
||||
16520
src/real2sim_platform/static/vendor/spark.module.min.js
vendored
Normal file
16520
src/real2sim_platform/static/vendor/spark.module.min.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
src/real2sim_platform/static/vendor/three.VERSION
vendored
Normal file
1
src/real2sim_platform/static/vendor/three.VERSION
vendored
Normal file
@ -0,0 +1 @@
|
||||
0.185.1
|
||||
1963
src/real2sim_platform/static/vendor/three/addons/controls/OrbitControls.js
vendored
Normal file
1963
src/real2sim_platform/static/vendor/three/addons/controls/OrbitControls.js
vendored
Normal file
File diff suppressed because it is too large
Load Diff
4860
src/real2sim_platform/static/vendor/three/addons/loaders/GLTFLoader.js
vendored
Normal file
4860
src/real2sim_platform/static/vendor/three/addons/loaders/GLTFLoader.js
vendored
Normal file
File diff suppressed because it is too large
Load Diff
955
src/real2sim_platform/static/vendor/three/addons/loaders/OBJLoader.js
vendored
Normal file
955
src/real2sim_platform/static/vendor/three/addons/loaders/OBJLoader.js
vendored
Normal file
@ -0,0 +1,955 @@
|
||||
import {
|
||||
BufferGeometry,
|
||||
FileLoader,
|
||||
Float32BufferAttribute,
|
||||
Group,
|
||||
LineBasicMaterial,
|
||||
LineSegments,
|
||||
Loader,
|
||||
Material,
|
||||
Mesh,
|
||||
MeshPhongMaterial,
|
||||
Points,
|
||||
PointsMaterial,
|
||||
Vector3,
|
||||
Color,
|
||||
SRGBColorSpace
|
||||
} from 'three';
|
||||
|
||||
// o object_name | g group_name
|
||||
const _object_pattern = /^[og]\s*(.+)?/;
|
||||
// mtllib file_reference
|
||||
const _material_library_pattern = /^mtllib /;
|
||||
// usemtl material_name
|
||||
const _material_use_pattern = /^usemtl /;
|
||||
// usemap map_name
|
||||
const _map_use_pattern = /^usemap /;
|
||||
const _face_vertex_data_separator_pattern = /\s+/;
|
||||
|
||||
const _vA = new Vector3();
|
||||
const _vB = new Vector3();
|
||||
const _vC = new Vector3();
|
||||
|
||||
const _ab = new Vector3();
|
||||
const _cb = new Vector3();
|
||||
|
||||
const _color = new Color();
|
||||
|
||||
function ParserState() {
|
||||
|
||||
const state = {
|
||||
objects: [],
|
||||
object: {},
|
||||
|
||||
vertices: [],
|
||||
normals: [],
|
||||
colors: [],
|
||||
uvs: [],
|
||||
|
||||
materials: {},
|
||||
materialLibraries: [],
|
||||
|
||||
startObject: function ( name, fromDeclaration ) {
|
||||
|
||||
// If the current object (initial from reset) is not from a g/o declaration in the parsed
|
||||
// file. We need to use it for the first parsed g/o to keep things in sync.
|
||||
if ( this.object && this.object.fromDeclaration === false ) {
|
||||
|
||||
this.object.name = name;
|
||||
this.object.fromDeclaration = ( fromDeclaration !== false );
|
||||
return;
|
||||
|
||||
}
|
||||
|
||||
const previousMaterial = ( this.object && typeof this.object.currentMaterial === 'function' ? this.object.currentMaterial() : undefined );
|
||||
|
||||
if ( this.object && typeof this.object._finalize === 'function' ) {
|
||||
|
||||
this.object._finalize( true );
|
||||
|
||||
}
|
||||
|
||||
this.object = {
|
||||
name: name || '',
|
||||
fromDeclaration: ( fromDeclaration !== false ),
|
||||
|
||||
geometry: {
|
||||
vertices: [],
|
||||
normals: [],
|
||||
colors: [],
|
||||
uvs: [],
|
||||
hasUVIndices: false
|
||||
},
|
||||
materials: [],
|
||||
smooth: true,
|
||||
|
||||
startMaterial: function ( name, libraries ) {
|
||||
|
||||
const previous = this._finalize( false );
|
||||
|
||||
// New usemtl declaration overwrites an inherited material, except if faces were declared
|
||||
// after the material, then it must be preserved for proper MultiMaterial continuation.
|
||||
if ( previous && ( previous.inherited || previous.groupCount <= 0 ) ) {
|
||||
|
||||
this.materials.splice( previous.index, 1 );
|
||||
|
||||
}
|
||||
|
||||
const material = {
|
||||
index: this.materials.length,
|
||||
name: name || '',
|
||||
mtllib: ( Array.isArray( libraries ) && libraries.length > 0 ? libraries[ libraries.length - 1 ] : '' ),
|
||||
smooth: ( previous !== undefined ? previous.smooth : this.smooth ),
|
||||
groupStart: ( previous !== undefined ? previous.groupEnd : 0 ),
|
||||
groupEnd: - 1,
|
||||
groupCount: - 1,
|
||||
inherited: false,
|
||||
|
||||
clone: function ( index ) {
|
||||
|
||||
const cloned = {
|
||||
index: ( typeof index === 'number' ? index : this.index ),
|
||||
name: this.name,
|
||||
mtllib: this.mtllib,
|
||||
smooth: this.smooth,
|
||||
groupStart: 0,
|
||||
groupEnd: - 1,
|
||||
groupCount: - 1,
|
||||
inherited: false
|
||||
};
|
||||
cloned.clone = this.clone.bind( cloned );
|
||||
return cloned;
|
||||
|
||||
}
|
||||
};
|
||||
|
||||
this.materials.push( material );
|
||||
|
||||
return material;
|
||||
|
||||
},
|
||||
|
||||
currentMaterial: function () {
|
||||
|
||||
if ( this.materials.length > 0 ) {
|
||||
|
||||
return this.materials[ this.materials.length - 1 ];
|
||||
|
||||
}
|
||||
|
||||
return undefined;
|
||||
|
||||
},
|
||||
|
||||
_finalize: function ( end ) {
|
||||
|
||||
const lastMultiMaterial = this.currentMaterial();
|
||||
if ( lastMultiMaterial && lastMultiMaterial.groupEnd === - 1 ) {
|
||||
|
||||
lastMultiMaterial.groupEnd = this.geometry.vertices.length / 3;
|
||||
lastMultiMaterial.groupCount = lastMultiMaterial.groupEnd - lastMultiMaterial.groupStart;
|
||||
lastMultiMaterial.inherited = false;
|
||||
|
||||
}
|
||||
|
||||
// Ignore objects tail materials if no face declarations followed them before a new o/g started.
|
||||
if ( end && this.materials.length > 1 ) {
|
||||
|
||||
for ( let mi = this.materials.length - 1; mi >= 0; mi -- ) {
|
||||
|
||||
if ( this.materials[ mi ].groupCount <= 0 ) {
|
||||
|
||||
this.materials.splice( mi, 1 );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// Guarantee at least one empty material, this makes the creation later more straight forward.
|
||||
if ( end && this.materials.length === 0 ) {
|
||||
|
||||
this.materials.push( {
|
||||
name: '',
|
||||
smooth: this.smooth
|
||||
} );
|
||||
|
||||
}
|
||||
|
||||
return lastMultiMaterial;
|
||||
|
||||
}
|
||||
};
|
||||
|
||||
// Inherit previous objects material.
|
||||
// Spec tells us that a declared material must be set to all objects until a new material is declared.
|
||||
// If a usemtl declaration is encountered while this new object is being parsed, it will
|
||||
// overwrite the inherited material. Exception being that there was already face declarations
|
||||
// to the inherited material, then it will be preserved for proper MultiMaterial continuation.
|
||||
|
||||
if ( previousMaterial && previousMaterial.name && typeof previousMaterial.clone === 'function' ) {
|
||||
|
||||
const declared = previousMaterial.clone( 0 );
|
||||
declared.inherited = true;
|
||||
this.object.materials.push( declared );
|
||||
|
||||
}
|
||||
|
||||
this.objects.push( this.object );
|
||||
|
||||
},
|
||||
|
||||
finalize: function () {
|
||||
|
||||
if ( this.object && typeof this.object._finalize === 'function' ) {
|
||||
|
||||
this.object._finalize( true );
|
||||
|
||||
}
|
||||
|
||||
},
|
||||
|
||||
parseVertexIndex: function ( value, len ) {
|
||||
|
||||
const index = parseInt( value, 10 );
|
||||
return ( index >= 0 ? index - 1 : index + len / 3 ) * 3;
|
||||
|
||||
},
|
||||
|
||||
parseNormalIndex: function ( value, len ) {
|
||||
|
||||
const index = parseInt( value, 10 );
|
||||
return ( index >= 0 ? index - 1 : index + len / 3 ) * 3;
|
||||
|
||||
},
|
||||
|
||||
parseUVIndex: function ( value, len ) {
|
||||
|
||||
const index = parseInt( value, 10 );
|
||||
return ( index >= 0 ? index - 1 : index + len / 2 ) * 2;
|
||||
|
||||
},
|
||||
|
||||
addVertex: function ( a, b, c ) {
|
||||
|
||||
const src = this.vertices;
|
||||
const dst = this.object.geometry.vertices;
|
||||
|
||||
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
|
||||
dst.push( src[ b + 0 ], src[ b + 1 ], src[ b + 2 ] );
|
||||
dst.push( src[ c + 0 ], src[ c + 1 ], src[ c + 2 ] );
|
||||
|
||||
},
|
||||
|
||||
addVertexPoint: function ( a ) {
|
||||
|
||||
const src = this.vertices;
|
||||
const dst = this.object.geometry.vertices;
|
||||
|
||||
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
|
||||
|
||||
},
|
||||
|
||||
addVertexLine: function ( a ) {
|
||||
|
||||
const src = this.vertices;
|
||||
const dst = this.object.geometry.vertices;
|
||||
|
||||
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
|
||||
|
||||
},
|
||||
|
||||
addNormal: function ( a, b, c ) {
|
||||
|
||||
const src = this.normals;
|
||||
const dst = this.object.geometry.normals;
|
||||
|
||||
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
|
||||
dst.push( src[ b + 0 ], src[ b + 1 ], src[ b + 2 ] );
|
||||
dst.push( src[ c + 0 ], src[ c + 1 ], src[ c + 2 ] );
|
||||
|
||||
},
|
||||
|
||||
addFaceNormal: function ( a, b, c ) {
|
||||
|
||||
const src = this.vertices;
|
||||
const dst = this.object.geometry.normals;
|
||||
|
||||
_vA.fromArray( src, a );
|
||||
_vB.fromArray( src, b );
|
||||
_vC.fromArray( src, c );
|
||||
|
||||
_cb.subVectors( _vC, _vB );
|
||||
_ab.subVectors( _vA, _vB );
|
||||
_cb.cross( _ab );
|
||||
|
||||
_cb.normalize();
|
||||
|
||||
dst.push( _cb.x, _cb.y, _cb.z );
|
||||
dst.push( _cb.x, _cb.y, _cb.z );
|
||||
dst.push( _cb.x, _cb.y, _cb.z );
|
||||
|
||||
},
|
||||
|
||||
addColor: function ( a, b, c ) {
|
||||
|
||||
const src = this.colors;
|
||||
const dst = this.object.geometry.colors;
|
||||
|
||||
if ( src[ a ] !== undefined ) dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
|
||||
if ( src[ b ] !== undefined ) dst.push( src[ b + 0 ], src[ b + 1 ], src[ b + 2 ] );
|
||||
if ( src[ c ] !== undefined ) dst.push( src[ c + 0 ], src[ c + 1 ], src[ c + 2 ] );
|
||||
|
||||
},
|
||||
|
||||
addUV: function ( a, b, c ) {
|
||||
|
||||
const src = this.uvs;
|
||||
const dst = this.object.geometry.uvs;
|
||||
|
||||
dst.push( src[ a + 0 ], src[ a + 1 ] );
|
||||
dst.push( src[ b + 0 ], src[ b + 1 ] );
|
||||
dst.push( src[ c + 0 ], src[ c + 1 ] );
|
||||
|
||||
},
|
||||
|
||||
addDefaultUV: function () {
|
||||
|
||||
const dst = this.object.geometry.uvs;
|
||||
|
||||
dst.push( 0, 0 );
|
||||
dst.push( 0, 0 );
|
||||
dst.push( 0, 0 );
|
||||
|
||||
},
|
||||
|
||||
addUVLine: function ( a ) {
|
||||
|
||||
const src = this.uvs;
|
||||
const dst = this.object.geometry.uvs;
|
||||
|
||||
dst.push( src[ a + 0 ], src[ a + 1 ] );
|
||||
|
||||
},
|
||||
|
||||
addFace: function ( a, b, c, ua, ub, uc, na, nb, nc ) {
|
||||
|
||||
const vLen = this.vertices.length;
|
||||
|
||||
let ia = this.parseVertexIndex( a, vLen );
|
||||
let ib = this.parseVertexIndex( b, vLen );
|
||||
let ic = this.parseVertexIndex( c, vLen );
|
||||
|
||||
this.addVertex( ia, ib, ic );
|
||||
this.addColor( ia, ib, ic );
|
||||
|
||||
// normals
|
||||
|
||||
if ( na !== undefined && na !== '' ) {
|
||||
|
||||
const nLen = this.normals.length;
|
||||
|
||||
ia = this.parseNormalIndex( na, nLen );
|
||||
ib = this.parseNormalIndex( nb, nLen );
|
||||
ic = this.parseNormalIndex( nc, nLen );
|
||||
|
||||
this.addNormal( ia, ib, ic );
|
||||
|
||||
} else {
|
||||
|
||||
this.addFaceNormal( ia, ib, ic );
|
||||
|
||||
}
|
||||
|
||||
// uvs
|
||||
|
||||
if ( ua !== undefined && ua !== '' ) {
|
||||
|
||||
const uvLen = this.uvs.length;
|
||||
|
||||
ia = this.parseUVIndex( ua, uvLen );
|
||||
ib = this.parseUVIndex( ub, uvLen );
|
||||
ic = this.parseUVIndex( uc, uvLen );
|
||||
|
||||
this.addUV( ia, ib, ic );
|
||||
|
||||
this.object.geometry.hasUVIndices = true;
|
||||
|
||||
} else {
|
||||
|
||||
// add placeholder values (for inconsistent face definitions)
|
||||
|
||||
this.addDefaultUV();
|
||||
|
||||
}
|
||||
|
||||
},
|
||||
|
||||
addPointGeometry: function ( vertices ) {
|
||||
|
||||
this.object.geometry.type = 'Points';
|
||||
|
||||
const vLen = this.vertices.length;
|
||||
|
||||
for ( let vi = 0, l = vertices.length; vi < l; vi ++ ) {
|
||||
|
||||
const index = this.parseVertexIndex( vertices[ vi ], vLen );
|
||||
|
||||
this.addVertexPoint( index );
|
||||
this.addColor( index );
|
||||
|
||||
}
|
||||
|
||||
},
|
||||
|
||||
addLineGeometry: function ( vertices, uvs ) {
|
||||
|
||||
this.object.geometry.type = 'Line';
|
||||
|
||||
const vLen = this.vertices.length;
|
||||
const uvLen = this.uvs.length;
|
||||
|
||||
for ( let vi = 0, l = vertices.length; vi < l; vi ++ ) {
|
||||
|
||||
this.addVertexLine( this.parseVertexIndex( vertices[ vi ], vLen ) );
|
||||
|
||||
}
|
||||
|
||||
for ( let uvi = 0, l = uvs.length; uvi < l; uvi ++ ) {
|
||||
|
||||
this.addUVLine( this.parseUVIndex( uvs[ uvi ], uvLen ) );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
};
|
||||
|
||||
state.startObject( '', false );
|
||||
|
||||
return state;
|
||||
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* A loader for the OBJ format.
|
||||
*
|
||||
* The [OBJ format](https://en.wikipedia.org/wiki/Wavefront_.obj_file) is a simple data-format that
|
||||
* represents 3D geometry in a human readable format as the position of each vertex, the UV position of
|
||||
* each texture coordinate vertex, vertex normals, and the faces that make each polygon defined as a list
|
||||
* of vertices, and texture vertices.
|
||||
*
|
||||
* ```js
|
||||
* const loader = new OBJLoader();
|
||||
* const object = await loader.loadAsync( 'models/monster.obj' );
|
||||
* scene.add( object );
|
||||
* ```
|
||||
*
|
||||
* @augments Loader
|
||||
* @three_import import { OBJLoader } from 'three/addons/loaders/OBJLoader.js';
|
||||
*/
|
||||
class OBJLoader extends Loader {
|
||||
|
||||
/**
|
||||
* Constructs a new OBJ loader.
|
||||
*
|
||||
* @param {LoadingManager} [manager] - The loading manager.
|
||||
*/
|
||||
constructor( manager ) {
|
||||
|
||||
super( manager );
|
||||
|
||||
/**
|
||||
* A reference to a material creator.
|
||||
*
|
||||
* @type {?MaterialCreator}
|
||||
* @default null
|
||||
*/
|
||||
this.materials = null;
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Starts loading from the given URL and passes the loaded OBJ asset
|
||||
* to the `onLoad()` callback.
|
||||
*
|
||||
* @param {string} url - The path/URL of the file to be loaded. This can also be a data URI.
|
||||
* @param {function(Group)} onLoad - Executed when the loading process has been finished.
|
||||
* @param {onProgressCallback} onProgress - Executed while the loading is in progress.
|
||||
* @param {onErrorCallback} onError - Executed when errors occur.
|
||||
*/
|
||||
load( url, onLoad, onProgress, onError ) {
|
||||
|
||||
const scope = this;
|
||||
|
||||
const loader = new FileLoader( this.manager );
|
||||
loader.setPath( this.path );
|
||||
loader.setRequestHeader( this.requestHeader );
|
||||
loader.setWithCredentials( this.withCredentials );
|
||||
loader.load( url, function ( text ) {
|
||||
|
||||
try {
|
||||
|
||||
onLoad( scope.parse( text ) );
|
||||
|
||||
} catch ( e ) {
|
||||
|
||||
if ( onError ) {
|
||||
|
||||
onError( e );
|
||||
|
||||
} else {
|
||||
|
||||
console.error( e );
|
||||
|
||||
}
|
||||
|
||||
scope.manager.itemError( url );
|
||||
|
||||
}
|
||||
|
||||
}, onProgress, onError );
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the material creator for this OBJ. This object is loaded via {@link MTLLoader}.
|
||||
*
|
||||
* @param {MaterialCreator} materials - An object that creates the materials for this OBJ.
|
||||
* @return {OBJLoader} A reference to this loader.
|
||||
*/
|
||||
setMaterials( materials ) {
|
||||
|
||||
this.materials = materials;
|
||||
|
||||
return this;
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the given OBJ data and returns the resulting group.
|
||||
*
|
||||
* @param {string} text - The raw OBJ data as a string.
|
||||
* @return {Group} The parsed OBJ.
|
||||
*/
|
||||
parse( text ) {
|
||||
|
||||
const state = new ParserState();
|
||||
|
||||
if ( text.indexOf( '\r\n' ) !== - 1 ) {
|
||||
|
||||
// This is faster than String.split with regex that splits on both
|
||||
text = text.replace( /\r\n/g, '\n' );
|
||||
|
||||
}
|
||||
|
||||
if ( text.indexOf( '\\\n' ) !== - 1 ) {
|
||||
|
||||
// join lines separated by a line continuation character (\)
|
||||
text = text.replace( /\\\n/g, '' );
|
||||
|
||||
}
|
||||
|
||||
const lines = text.split( '\n' );
|
||||
let result = [];
|
||||
|
||||
for ( let i = 0, l = lines.length; i < l; i ++ ) {
|
||||
|
||||
const line = lines[ i ].trimStart();
|
||||
|
||||
if ( line.length === 0 ) continue;
|
||||
|
||||
const lineFirstChar = line.charAt( 0 );
|
||||
|
||||
// @todo invoke passed in handler if any
|
||||
if ( lineFirstChar === '#' ) continue; // skip comments
|
||||
|
||||
if ( lineFirstChar === 'v' ) {
|
||||
|
||||
const data = line.split( _face_vertex_data_separator_pattern );
|
||||
|
||||
switch ( data[ 0 ] ) {
|
||||
|
||||
case 'v':
|
||||
state.vertices.push(
|
||||
parseFloat( data[ 1 ] ),
|
||||
parseFloat( data[ 2 ] ),
|
||||
parseFloat( data[ 3 ] )
|
||||
);
|
||||
if ( data.length >= 7 ) {
|
||||
|
||||
_color.setRGB(
|
||||
parseFloat( data[ 4 ] ),
|
||||
parseFloat( data[ 5 ] ),
|
||||
parseFloat( data[ 6 ] ),
|
||||
SRGBColorSpace
|
||||
);
|
||||
|
||||
state.colors.push( _color.r, _color.g, _color.b );
|
||||
|
||||
} else {
|
||||
|
||||
// if no colors are defined, add placeholders so color and vertex indices match
|
||||
|
||||
state.colors.push( undefined, undefined, undefined );
|
||||
|
||||
}
|
||||
|
||||
break;
|
||||
case 'vn':
|
||||
state.normals.push(
|
||||
parseFloat( data[ 1 ] ),
|
||||
parseFloat( data[ 2 ] ),
|
||||
parseFloat( data[ 3 ] )
|
||||
);
|
||||
break;
|
||||
case 'vt':
|
||||
state.uvs.push(
|
||||
parseFloat( data[ 1 ] ),
|
||||
parseFloat( data[ 2 ] )
|
||||
);
|
||||
break;
|
||||
|
||||
}
|
||||
|
||||
} else if ( lineFirstChar === 'f' ) {
|
||||
|
||||
const lineData = line.slice( 1 ).trim();
|
||||
const vertexData = lineData.split( _face_vertex_data_separator_pattern );
|
||||
const faceVertices = [];
|
||||
|
||||
// Parse the face vertex data into an easy to work with format
|
||||
|
||||
for ( let j = 0, jl = vertexData.length; j < jl; j ++ ) {
|
||||
|
||||
const vertex = vertexData[ j ];
|
||||
|
||||
if ( vertex.length > 0 ) {
|
||||
|
||||
const vertexParts = vertex.split( '/' );
|
||||
faceVertices.push( vertexParts );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// Draw an edge between the first vertex and all subsequent vertices to form an n-gon
|
||||
|
||||
const v1 = faceVertices[ 0 ];
|
||||
|
||||
for ( let j = 1, jl = faceVertices.length - 1; j < jl; j ++ ) {
|
||||
|
||||
const v2 = faceVertices[ j ];
|
||||
const v3 = faceVertices[ j + 1 ];
|
||||
|
||||
state.addFace(
|
||||
v1[ 0 ], v2[ 0 ], v3[ 0 ],
|
||||
v1[ 1 ], v2[ 1 ], v3[ 1 ],
|
||||
v1[ 2 ], v2[ 2 ], v3[ 2 ]
|
||||
);
|
||||
|
||||
}
|
||||
|
||||
} else if ( lineFirstChar === 'l' ) {
|
||||
|
||||
const lineParts = line.substring( 1 ).trim().split( ' ' );
|
||||
let lineVertices = [];
|
||||
const lineUVs = [];
|
||||
|
||||
if ( line.indexOf( '/' ) === - 1 ) {
|
||||
|
||||
lineVertices = lineParts;
|
||||
|
||||
} else {
|
||||
|
||||
for ( let li = 0, llen = lineParts.length; li < llen; li ++ ) {
|
||||
|
||||
const parts = lineParts[ li ].split( '/' );
|
||||
|
||||
if ( parts[ 0 ] !== '' ) lineVertices.push( parts[ 0 ] );
|
||||
if ( parts[ 1 ] !== '' ) lineUVs.push( parts[ 1 ] );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
state.addLineGeometry( lineVertices, lineUVs );
|
||||
|
||||
} else if ( lineFirstChar === 'p' ) {
|
||||
|
||||
const lineData = line.slice( 1 ).trim();
|
||||
const pointData = lineData.split( ' ' );
|
||||
|
||||
state.addPointGeometry( pointData );
|
||||
|
||||
} else if ( ( result = _object_pattern.exec( line ) ) !== null ) {
|
||||
|
||||
// o object_name
|
||||
// or
|
||||
// g group_name
|
||||
|
||||
// WORKAROUND: https://bugs.chromium.org/p/v8/issues/detail?id=2869
|
||||
// let name = result[ 0 ].slice( 1 ).trim();
|
||||
const name = ( ' ' + result[ 0 ].slice( 1 ).trim() ).slice( 1 );
|
||||
|
||||
state.startObject( name );
|
||||
|
||||
} else if ( _material_use_pattern.test( line ) ) {
|
||||
|
||||
// material
|
||||
|
||||
state.object.startMaterial( line.substring( 7 ).trim(), state.materialLibraries );
|
||||
|
||||
} else if ( _material_library_pattern.test( line ) ) {
|
||||
|
||||
// mtl file
|
||||
|
||||
state.materialLibraries.push( line.substring( 7 ).trim() );
|
||||
|
||||
} else if ( _map_use_pattern.test( line ) ) {
|
||||
|
||||
// the line is parsed but ignored since the loader assumes textures are defined MTL files
|
||||
// (according to https://www.okino.com/conv/imp_wave.htm, 'usemap' is the old-style Wavefront texture reference method)
|
||||
|
||||
console.warn( 'THREE.OBJLoader: Rendering identifier "usemap" not supported. Textures must be defined in MTL files.' );
|
||||
|
||||
} else if ( lineFirstChar === 's' ) {
|
||||
|
||||
result = line.split( ' ' );
|
||||
|
||||
// smooth shading
|
||||
|
||||
// @todo Handle files that have varying smooth values for a set of faces inside one geometry,
|
||||
// but does not define a usemtl for each face set.
|
||||
// This should be detected and a dummy material created (later MultiMaterial and geometry groups).
|
||||
// This requires some care to not create extra material on each smooth value for "normal" obj files.
|
||||
// where explicit usemtl defines geometry groups.
|
||||
// Example asset: examples/models/obj/cerberus/Cerberus.obj
|
||||
|
||||
/*
|
||||
* http://paulbourke.net/dataformats/obj/
|
||||
*
|
||||
* From chapter "Grouping" Syntax explanation "s group_number":
|
||||
* "group_number is the smoothing group number. To turn off smoothing groups, use a value of 0 or off.
|
||||
* Polygonal elements use group numbers to put elements in different smoothing groups. For free-form
|
||||
* surfaces, smoothing groups are either turned on or off; there is no difference between values greater
|
||||
* than 0."
|
||||
*/
|
||||
if ( result.length > 1 ) {
|
||||
|
||||
const value = result[ 1 ].trim().toLowerCase();
|
||||
state.object.smooth = ( value !== '0' && value !== 'off' );
|
||||
|
||||
} else {
|
||||
|
||||
// ZBrush can produce "s" lines #11707
|
||||
state.object.smooth = true;
|
||||
|
||||
}
|
||||
|
||||
const material = state.object.currentMaterial();
|
||||
if ( material ) material.smooth = state.object.smooth;
|
||||
|
||||
} else {
|
||||
|
||||
// Handle null terminated files without exception
|
||||
if ( line === '\0' ) continue;
|
||||
|
||||
console.warn( 'THREE.OBJLoader: Unexpected line: "' + line + '"' );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
state.finalize();
|
||||
|
||||
const container = new Group();
|
||||
container.materialLibraries = [].concat( state.materialLibraries );
|
||||
|
||||
const hasPrimitives = ! ( state.objects.length === 1 && state.objects[ 0 ].geometry.vertices.length === 0 );
|
||||
|
||||
if ( hasPrimitives === true ) {
|
||||
|
||||
for ( let i = 0, l = state.objects.length; i < l; i ++ ) {
|
||||
|
||||
const object = state.objects[ i ];
|
||||
const geometry = object.geometry;
|
||||
const materials = object.materials;
|
||||
const isLine = ( geometry.type === 'Line' );
|
||||
const isPoints = ( geometry.type === 'Points' );
|
||||
let hasVertexColors = false;
|
||||
|
||||
// Skip o/g line declarations that did not follow with any faces
|
||||
if ( geometry.vertices.length === 0 ) continue;
|
||||
|
||||
const buffergeometry = new BufferGeometry();
|
||||
|
||||
buffergeometry.setAttribute( 'position', new Float32BufferAttribute( geometry.vertices, 3 ) );
|
||||
|
||||
if ( geometry.normals.length > 0 ) {
|
||||
|
||||
buffergeometry.setAttribute( 'normal', new Float32BufferAttribute( geometry.normals, 3 ) );
|
||||
|
||||
}
|
||||
|
||||
if ( geometry.colors.length > 0 ) {
|
||||
|
||||
hasVertexColors = true;
|
||||
buffergeometry.setAttribute( 'color', new Float32BufferAttribute( geometry.colors, 3 ) );
|
||||
|
||||
}
|
||||
|
||||
if ( geometry.hasUVIndices === true ) {
|
||||
|
||||
buffergeometry.setAttribute( 'uv', new Float32BufferAttribute( geometry.uvs, 2 ) );
|
||||
|
||||
}
|
||||
|
||||
// Create materials
|
||||
|
||||
const createdMaterials = [];
|
||||
|
||||
for ( let mi = 0, miLen = materials.length; mi < miLen; mi ++ ) {
|
||||
|
||||
const sourceMaterial = materials[ mi ];
|
||||
const materialHash = sourceMaterial.name + '_' + sourceMaterial.smooth + '_' + hasVertexColors;
|
||||
let material = state.materials[ materialHash ];
|
||||
|
||||
if ( this.materials !== null ) {
|
||||
|
||||
material = this.materials.create( sourceMaterial.name );
|
||||
|
||||
// mtl etc. loaders probably can't create line materials correctly, copy properties to a line material.
|
||||
if ( isLine && material && ! ( material instanceof LineBasicMaterial ) ) {
|
||||
|
||||
const materialLine = new LineBasicMaterial();
|
||||
Material.prototype.copy.call( materialLine, material );
|
||||
materialLine.color.copy( material.color );
|
||||
material = materialLine;
|
||||
|
||||
} else if ( isPoints && material && ! ( material instanceof PointsMaterial ) ) {
|
||||
|
||||
const materialPoints = new PointsMaterial( { size: 10, sizeAttenuation: false } );
|
||||
Material.prototype.copy.call( materialPoints, material );
|
||||
materialPoints.color.copy( material.color );
|
||||
materialPoints.map = material.map;
|
||||
material = materialPoints;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if ( material === undefined ) {
|
||||
|
||||
if ( isLine ) {
|
||||
|
||||
material = new LineBasicMaterial();
|
||||
|
||||
} else if ( isPoints ) {
|
||||
|
||||
material = new PointsMaterial( { size: 1, sizeAttenuation: false } );
|
||||
|
||||
} else {
|
||||
|
||||
material = new MeshPhongMaterial();
|
||||
|
||||
}
|
||||
|
||||
material.name = sourceMaterial.name;
|
||||
material.flatShading = sourceMaterial.smooth ? false : true;
|
||||
material.vertexColors = hasVertexColors;
|
||||
|
||||
state.materials[ materialHash ] = material;
|
||||
|
||||
}
|
||||
|
||||
createdMaterials.push( material );
|
||||
|
||||
}
|
||||
|
||||
// Create mesh
|
||||
|
||||
let mesh;
|
||||
|
||||
if ( createdMaterials.length > 1 ) {
|
||||
|
||||
for ( let mi = 0, miLen = materials.length; mi < miLen; mi ++ ) {
|
||||
|
||||
const sourceMaterial = materials[ mi ];
|
||||
buffergeometry.addGroup( sourceMaterial.groupStart, sourceMaterial.groupCount, mi );
|
||||
|
||||
}
|
||||
|
||||
if ( isLine ) {
|
||||
|
||||
mesh = new LineSegments( buffergeometry, createdMaterials );
|
||||
|
||||
} else if ( isPoints ) {
|
||||
|
||||
mesh = new Points( buffergeometry, createdMaterials );
|
||||
|
||||
} else {
|
||||
|
||||
mesh = new Mesh( buffergeometry, createdMaterials );
|
||||
|
||||
}
|
||||
|
||||
} else {
|
||||
|
||||
if ( isLine ) {
|
||||
|
||||
mesh = new LineSegments( buffergeometry, createdMaterials[ 0 ] );
|
||||
|
||||
} else if ( isPoints ) {
|
||||
|
||||
mesh = new Points( buffergeometry, createdMaterials[ 0 ] );
|
||||
|
||||
} else {
|
||||
|
||||
mesh = new Mesh( buffergeometry, createdMaterials[ 0 ] );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
mesh.name = object.name;
|
||||
|
||||
container.add( mesh );
|
||||
|
||||
}
|
||||
|
||||
} else {
|
||||
|
||||
// if there is only the default parser state object with no geometry data, interpret data as point cloud
|
||||
|
||||
if ( state.vertices.length > 0 ) {
|
||||
|
||||
const material = new PointsMaterial( { size: 1, sizeAttenuation: false } );
|
||||
|
||||
const buffergeometry = new BufferGeometry();
|
||||
|
||||
buffergeometry.setAttribute( 'position', new Float32BufferAttribute( state.vertices, 3 ) );
|
||||
|
||||
if ( state.colors.length > 0 && state.colors[ 0 ] !== undefined ) {
|
||||
|
||||
buffergeometry.setAttribute( 'color', new Float32BufferAttribute( state.colors, 3 ) );
|
||||
material.vertexColors = true;
|
||||
|
||||
}
|
||||
|
||||
const points = new Points( buffergeometry, material );
|
||||
container.add( points );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return container;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export { OBJLoader };
|
||||
968
src/real2sim_platform/static/vendor/three/addons/loaders/PLYLoader.js
vendored
Normal file
968
src/real2sim_platform/static/vendor/three/addons/loaders/PLYLoader.js
vendored
Normal file
@ -0,0 +1,968 @@
|
||||
import {
|
||||
BufferGeometry,
|
||||
FileLoader,
|
||||
BufferAttribute,
|
||||
Float32BufferAttribute,
|
||||
Int8BufferAttribute,
|
||||
Int16BufferAttribute,
|
||||
Int32BufferAttribute,
|
||||
Loader,
|
||||
Uint8BufferAttribute,
|
||||
Uint16BufferAttribute,
|
||||
Uint32BufferAttribute,
|
||||
Color,
|
||||
SRGBColorSpace
|
||||
} from 'three';
|
||||
|
||||
const _color = new Color();
|
||||
|
||||
/**
|
||||
* A loader for PLY the PLY format (known as the Polygon
|
||||
* File Format or the Stanford Triangle Format).
|
||||
*
|
||||
* Limitations:
|
||||
* - ASCII decoding assumes file is UTF-8.
|
||||
*
|
||||
* ```js
|
||||
* const loader = new PLYLoader();
|
||||
* const geometry = await loader.loadAsync( './models/ply/ascii/dolphins.ply' );
|
||||
* scene.add( new THREE.Mesh( geometry ) );
|
||||
* ```
|
||||
*
|
||||
* @augments Loader
|
||||
* @three_import import { PLYLoader } from 'three/addons/loaders/PLYLoader.js';
|
||||
*/
|
||||
class PLYLoader extends Loader {
|
||||
|
||||
/**
|
||||
* Constructs a new PLY loader.
|
||||
*
|
||||
* @param {LoadingManager} [manager] - The loading manager.
|
||||
*/
|
||||
constructor( manager ) {
|
||||
|
||||
super( manager );
|
||||
|
||||
// internals
|
||||
|
||||
this.propertyNameMapping = {};
|
||||
this.customPropertyMapping = {};
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Starts loading from the given URL and passes the loaded PLY asset
|
||||
* to the `onLoad()` callback.
|
||||
*
|
||||
* @param {string} url - The path/URL of the file to be loaded. This can also be a data URI.
|
||||
* @param {function(BufferGeometry)} onLoad - Executed when the loading process has been finished.
|
||||
* @param {onProgressCallback} onProgress - Executed while the loading is in progress.
|
||||
* @param {onErrorCallback} onError - Executed when errors occur.
|
||||
*/
|
||||
load( url, onLoad, onProgress, onError ) {
|
||||
|
||||
const scope = this;
|
||||
|
||||
const loader = new FileLoader( this.manager );
|
||||
loader.setPath( this.path );
|
||||
loader.setResponseType( 'arraybuffer' );
|
||||
loader.setRequestHeader( this.requestHeader );
|
||||
loader.setWithCredentials( this.withCredentials );
|
||||
loader.load( url, function ( text ) {
|
||||
|
||||
try {
|
||||
|
||||
onLoad( scope.parse( text ) );
|
||||
|
||||
} catch ( e ) {
|
||||
|
||||
if ( onError ) {
|
||||
|
||||
onError( e );
|
||||
|
||||
} else {
|
||||
|
||||
console.error( e );
|
||||
|
||||
}
|
||||
|
||||
scope.manager.itemError( url );
|
||||
|
||||
}
|
||||
|
||||
}, onProgress, onError );
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets a property name mapping that maps default property names
|
||||
* to custom ones. For example, the following maps the properties
|
||||
* “diffuse_(red|green|blue)” in the file to standard color names.
|
||||
*
|
||||
* ```js
|
||||
* loader.setPropertyNameMapping( {
|
||||
* diffuse_red: 'red',
|
||||
* diffuse_green: 'green',
|
||||
* diffuse_blue: 'blue'
|
||||
* } );
|
||||
* ```
|
||||
*
|
||||
* @param {Object} mapping - The mapping dictionary.
|
||||
*/
|
||||
setPropertyNameMapping( mapping ) {
|
||||
|
||||
this.propertyNameMapping = mapping;
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Custom properties outside of the defaults for position, uv, normal
|
||||
* and color attributes can be added using the setCustomPropertyNameMapping method.
|
||||
* For example, the following maps the element properties “custom_property_a”
|
||||
* and “custom_property_b” to an attribute “customAttribute” with an item size of 2.
|
||||
* Attribute item sizes are set from the number of element properties in the property array.
|
||||
*
|
||||
* ```js
|
||||
* loader.setCustomPropertyNameMapping( {
|
||||
* customAttribute: ['custom_property_a', 'custom_property_b'],
|
||||
* } );
|
||||
* ```
|
||||
* @param {Object} mapping - The mapping dictionary.
|
||||
*/
|
||||
setCustomPropertyNameMapping( mapping ) {
|
||||
|
||||
this.customPropertyMapping = mapping;
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the given PLY data and returns the resulting geometry.
|
||||
*
|
||||
* @param {ArrayBuffer} data - The raw PLY data as an array buffer.
|
||||
* @return {BufferGeometry} The parsed geometry.
|
||||
*/
|
||||
parse( data ) {
|
||||
|
||||
function parseHeader( data, headerLength = 0 ) {
|
||||
|
||||
const patternHeader = /^ply([\s\S]*)end_header(\r\n|\r|\n)/;
|
||||
let headerText = '';
|
||||
const result = patternHeader.exec( data );
|
||||
|
||||
if ( result !== null ) {
|
||||
|
||||
headerText = result[ 1 ];
|
||||
|
||||
}
|
||||
|
||||
const header = {
|
||||
comments: [],
|
||||
elements: [],
|
||||
headerLength: headerLength,
|
||||
objInfo: ''
|
||||
};
|
||||
|
||||
const lines = headerText.split( /\r\n|\r|\n/ );
|
||||
let currentElement;
|
||||
|
||||
function make_ply_element_property( propertyValues, propertyNameMapping ) {
|
||||
|
||||
const property = { type: propertyValues[ 0 ] };
|
||||
|
||||
if ( property.type === 'list' ) {
|
||||
|
||||
property.name = propertyValues[ 3 ];
|
||||
property.countType = propertyValues[ 1 ];
|
||||
property.itemType = propertyValues[ 2 ];
|
||||
|
||||
} else {
|
||||
|
||||
property.name = propertyValues[ 1 ];
|
||||
|
||||
}
|
||||
|
||||
if ( property.name in propertyNameMapping ) {
|
||||
|
||||
property.name = propertyNameMapping[ property.name ];
|
||||
|
||||
}
|
||||
|
||||
return property;
|
||||
|
||||
}
|
||||
|
||||
for ( let i = 0; i < lines.length; i ++ ) {
|
||||
|
||||
let line = lines[ i ];
|
||||
line = line.trim();
|
||||
|
||||
if ( line === '' ) continue;
|
||||
|
||||
const lineValues = line.split( /\s+/ );
|
||||
const lineType = lineValues.shift();
|
||||
line = lineValues.join( ' ' );
|
||||
|
||||
switch ( lineType ) {
|
||||
|
||||
case 'format':
|
||||
|
||||
header.format = lineValues[ 0 ];
|
||||
header.version = lineValues[ 1 ];
|
||||
|
||||
break;
|
||||
|
||||
case 'comment':
|
||||
|
||||
header.comments.push( line );
|
||||
|
||||
break;
|
||||
|
||||
case 'element':
|
||||
|
||||
if ( currentElement !== undefined ) {
|
||||
|
||||
header.elements.push( currentElement );
|
||||
|
||||
}
|
||||
|
||||
currentElement = {};
|
||||
currentElement.name = lineValues[ 0 ];
|
||||
currentElement.count = parseInt( lineValues[ 1 ] );
|
||||
currentElement.properties = [];
|
||||
|
||||
break;
|
||||
|
||||
case 'property':
|
||||
|
||||
currentElement.properties.push( make_ply_element_property( lineValues, scope.propertyNameMapping ) );
|
||||
|
||||
break;
|
||||
|
||||
case 'obj_info':
|
||||
|
||||
header.objInfo = line;
|
||||
|
||||
break;
|
||||
|
||||
|
||||
default:
|
||||
|
||||
console.log( 'unhandled', lineType, lineValues );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if ( currentElement !== undefined ) {
|
||||
|
||||
header.elements.push( currentElement );
|
||||
|
||||
}
|
||||
|
||||
return header;
|
||||
|
||||
}
|
||||
|
||||
function parseASCIINumber( n, type ) {
|
||||
|
||||
switch ( type ) {
|
||||
|
||||
case 'char': case 'uchar': case 'short': case 'ushort': case 'int': case 'uint':
|
||||
case 'int8': case 'uint8': case 'int16': case 'uint16': case 'int32': case 'uint32':
|
||||
|
||||
return parseInt( n );
|
||||
|
||||
case 'float': case 'double': case 'float32': case 'float64':
|
||||
|
||||
return parseFloat( n );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
function parseASCIIElement( properties, tokens ) {
|
||||
|
||||
const element = {};
|
||||
|
||||
for ( let i = 0; i < properties.length; i ++ ) {
|
||||
|
||||
if ( tokens.empty() ) return null;
|
||||
|
||||
if ( properties[ i ].type === 'list' ) {
|
||||
|
||||
const list = [];
|
||||
const n = parseASCIINumber( tokens.next(), properties[ i ].countType );
|
||||
|
||||
for ( let j = 0; j < n; j ++ ) {
|
||||
|
||||
if ( tokens.empty() ) return null;
|
||||
|
||||
list.push( parseASCIINumber( tokens.next(), properties[ i ].itemType ) );
|
||||
|
||||
}
|
||||
|
||||
element[ properties[ i ].name ] = list;
|
||||
|
||||
} else {
|
||||
|
||||
element[ properties[ i ].name ] = parseASCIINumber( tokens.next(), properties[ i ].type );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return element;
|
||||
|
||||
}
|
||||
|
||||
function createBuffer() {
|
||||
|
||||
const buffer = {
|
||||
indices: [],
|
||||
vertices: [],
|
||||
normals: [],
|
||||
uvs: [],
|
||||
faceVertexUvs: [],
|
||||
colors: [],
|
||||
faceVertexColors: [],
|
||||
descriptors: {}
|
||||
};
|
||||
|
||||
for ( const customProperty of Object.keys( scope.customPropertyMapping ) ) {
|
||||
|
||||
buffer[ customProperty ] = [];
|
||||
|
||||
}
|
||||
|
||||
return buffer;
|
||||
|
||||
}
|
||||
|
||||
|
||||
function getBufferAttributeClass( type ) {
|
||||
|
||||
switch ( type ) {
|
||||
|
||||
case 'int8': case 'char': return Int8BufferAttribute;
|
||||
case 'uint8': case 'uchar': return Uint8BufferAttribute;
|
||||
case 'int16': case 'short': return Int16BufferAttribute;
|
||||
case 'uint16': case 'ushort': return Uint16BufferAttribute;
|
||||
case 'int32': case 'int': return Int32BufferAttribute;
|
||||
case 'uint32': case 'uint': return Uint32BufferAttribute;
|
||||
case 'float32': case 'float': return Float32BufferAttribute;
|
||||
case 'float64': case 'double': return Float64BufferAttribute;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
function getColorScale( type ) {
|
||||
|
||||
switch ( type ) {
|
||||
|
||||
case 'uchar': case 'uint8': return 1 / 255;
|
||||
case 'ushort': case 'uint16': return 1 / 65535;
|
||||
case 'float': case 'float32':
|
||||
case 'double': case 'float64': return 1;
|
||||
default: return 1 / 255;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
function isFloatType( type ) {
|
||||
|
||||
return type === 'float' || type === 'float32' || type === 'double' || type === 'float64';
|
||||
|
||||
}
|
||||
|
||||
function getAttributeDescriptor( properties ) {
|
||||
|
||||
function findProperty( names ) {
|
||||
|
||||
for ( const name of names ) {
|
||||
|
||||
const property = properties.find( p => p.name === name );
|
||||
if ( property ) return property;
|
||||
|
||||
}
|
||||
|
||||
return null;
|
||||
|
||||
}
|
||||
|
||||
// property lookup
|
||||
|
||||
const x = findProperty( [ 'x', 'px', 'posx' ] );
|
||||
const y = findProperty( [ 'y', 'py', 'posy' ] );
|
||||
const z = findProperty( [ 'z', 'pz', 'posz' ] );
|
||||
const nx = findProperty( [ 'nx', 'normalx' ] );
|
||||
const ny = findProperty( [ 'ny', 'normaly' ] );
|
||||
const nz = findProperty( [ 'nz', 'normalz' ] );
|
||||
const s = findProperty( [ 's', 'u', 'texture_u', 'tx' ] );
|
||||
const t = findProperty( [ 't', 'v', 'texture_v', 'ty' ] );
|
||||
const r = findProperty( [ 'red', 'diffuse_red', 'r', 'diffuse_r' ] );
|
||||
const g = findProperty( [ 'green', 'diffuse_green', 'g', 'diffuse_g' ] );
|
||||
const b = findProperty( [ 'blue', 'diffuse_blue', 'b', 'diffuse_b' ] );
|
||||
const texcoord = findProperty( [ 'texcoord' ] );
|
||||
|
||||
// custom property lookup
|
||||
|
||||
const custom = {};
|
||||
|
||||
for ( const customAttr of Object.keys( scope.customPropertyMapping ) ) {
|
||||
|
||||
const propNames = scope.customPropertyMapping[ customAttr ];
|
||||
const matched = propNames.map( name => properties.find( p => p.name === name ) );
|
||||
const types = matched.filter( p => p ).map( p => p.type );
|
||||
const uniform = types.length > 0 && types.every( type => type === types[ 0 ] );
|
||||
|
||||
custom[ customAttr ] = {
|
||||
type: uniform ? types[ 0 ] : 'float32',
|
||||
usage: matched.every( p => p !== undefined ),
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
// build descriptor
|
||||
|
||||
return {
|
||||
position: {
|
||||
names: [ x ? x.name : 'x', y ? y.name : 'y', z ? z.name : 'z' ],
|
||||
type: x ? x.type : 'float32',
|
||||
usage: !! ( x && y && z ),
|
||||
},
|
||||
normal: {
|
||||
names: [ nx ? nx.name : 'nx', ny ? ny.name : 'ny', nz ? nz.name : 'nz' ],
|
||||
type: nx ? nx.type : 'float32',
|
||||
usage: !! ( nx && ny && nz ),
|
||||
},
|
||||
uv: {
|
||||
names: [ s ? s.name : 's', t ? t.name : 't' ],
|
||||
type: s ? s.type : 'float32',
|
||||
usage: !! ( s && t ),
|
||||
},
|
||||
texcoord: {
|
||||
type: texcoord ? texcoord.itemType : 'float32',
|
||||
usage: !! texcoord,
|
||||
},
|
||||
color: {
|
||||
names: [ r ? r.name : 'red', g ? g.name : 'green', b ? b.name : 'blue' ],
|
||||
type: r ? r.type : 'uchar',
|
||||
usage: !! ( r && g && b ),
|
||||
},
|
||||
custom: custom,
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
function parseASCII( data, header ) {
|
||||
|
||||
// PLY ascii format specification, as per http://en.wikipedia.org/wiki/PLY_(file_format)
|
||||
|
||||
const buffer = createBuffer();
|
||||
|
||||
const patternBody = /end_header\s+(\S[\s\S]*\S|\S)\s*$/;
|
||||
let body, matches;
|
||||
|
||||
if ( ( matches = patternBody.exec( data ) ) !== null ) {
|
||||
|
||||
body = matches[ 1 ].split( /\s+/ );
|
||||
|
||||
} else {
|
||||
|
||||
body = [ ];
|
||||
|
||||
}
|
||||
|
||||
const tokens = new ArrayStream( body );
|
||||
|
||||
loop: for ( let i = 0; i < header.elements.length; i ++ ) {
|
||||
|
||||
const elementDesc = header.elements[ i ];
|
||||
const attributeDescriptor = getAttributeDescriptor( elementDesc.properties );
|
||||
buffer.descriptors[ elementDesc.name ] = attributeDescriptor;
|
||||
|
||||
for ( let j = 0; j < elementDesc.count; j ++ ) {
|
||||
|
||||
const element = parseASCIIElement( elementDesc.properties, tokens );
|
||||
|
||||
if ( ! element ) break loop;
|
||||
|
||||
handleElement( buffer, elementDesc.name, element, attributeDescriptor );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return postProcess( buffer );
|
||||
|
||||
}
|
||||
|
||||
function postProcess( buffer ) {
|
||||
|
||||
let geometry = new BufferGeometry();
|
||||
|
||||
const vertexDescriptor = buffer.descriptors.vertex;
|
||||
|
||||
// mandatory buffer data
|
||||
|
||||
if ( buffer.indices.length > 0 ) {
|
||||
|
||||
geometry.setIndex( buffer.indices );
|
||||
|
||||
}
|
||||
|
||||
const PositionClass = getBufferAttributeClass( vertexDescriptor ? vertexDescriptor.position.type : 'float32' );
|
||||
geometry.setAttribute( 'position', new PositionClass( buffer.vertices, 3 ) );
|
||||
|
||||
// optional buffer data
|
||||
|
||||
if ( buffer.normals.length > 0 ) {
|
||||
|
||||
const NormalClass = getBufferAttributeClass( vertexDescriptor.normal.type );
|
||||
geometry.setAttribute( 'normal', new NormalClass( buffer.normals, 3 ) );
|
||||
|
||||
}
|
||||
|
||||
if ( buffer.uvs.length > 0 ) {
|
||||
|
||||
const UvClass = getBufferAttributeClass( vertexDescriptor.uv.type );
|
||||
geometry.setAttribute( 'uv', new UvClass( buffer.uvs, 2 ) );
|
||||
|
||||
}
|
||||
|
||||
if ( buffer.colors.length > 0 ) {
|
||||
|
||||
const colorType = vertexDescriptor.color.type;
|
||||
const normalized = ! isFloatType( colorType );
|
||||
|
||||
const ColorClass = getBufferAttributeClass( colorType );
|
||||
geometry.setAttribute( 'color', new ColorClass( buffer.colors, 3, normalized ) );
|
||||
|
||||
}
|
||||
|
||||
if ( buffer.faceVertexUvs.length > 0 || buffer.faceVertexColors.length > 0 ) {
|
||||
|
||||
geometry = geometry.toNonIndexed();
|
||||
|
||||
if ( buffer.faceVertexUvs.length > 0 ) {
|
||||
|
||||
const UvClass = getBufferAttributeClass( buffer.descriptors.face.texcoord.type );
|
||||
geometry.setAttribute( 'uv', new UvClass( buffer.faceVertexUvs, 2 ) );
|
||||
|
||||
}
|
||||
|
||||
if ( buffer.faceVertexColors.length > 0 ) {
|
||||
|
||||
const colorType = buffer.descriptors.face.color.type;
|
||||
const normalized = ! isFloatType( colorType );
|
||||
|
||||
const ColorClass = getBufferAttributeClass( colorType );
|
||||
geometry.setAttribute( 'color', new ColorClass( buffer.faceVertexColors, 3, normalized ) );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// custom buffer data
|
||||
|
||||
for ( const customProperty of Object.keys( scope.customPropertyMapping ) ) {
|
||||
|
||||
if ( buffer[ customProperty ].length > 0 ) {
|
||||
|
||||
const CustomClass = getBufferAttributeClass( vertexDescriptor.custom[ customProperty ].type );
|
||||
geometry.setAttribute( customProperty, new CustomClass( buffer[ customProperty ], scope.customPropertyMapping[ customProperty ].length ) );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
geometry.computeBoundingSphere();
|
||||
|
||||
return geometry;
|
||||
|
||||
}
|
||||
|
||||
function handleElement( buffer, elementName, element, attributeDescriptor ) {
|
||||
|
||||
if ( elementName === 'vertex' ) {
|
||||
|
||||
const { position, normal, uv, color } = attributeDescriptor;
|
||||
|
||||
if ( position.usage ) {
|
||||
|
||||
buffer.vertices.push(
|
||||
element[ position.names[ 0 ] ],
|
||||
element[ position.names[ 1 ] ],
|
||||
element[ position.names[ 2 ] ]
|
||||
);
|
||||
|
||||
}
|
||||
|
||||
if ( normal.usage ) {
|
||||
|
||||
buffer.normals.push(
|
||||
element[ normal.names[ 0 ] ],
|
||||
element[ normal.names[ 1 ] ],
|
||||
element[ normal.names[ 2 ] ]
|
||||
);
|
||||
|
||||
}
|
||||
|
||||
if ( uv.usage ) {
|
||||
|
||||
buffer.uvs.push(
|
||||
element[ uv.names[ 0 ] ],
|
||||
element[ uv.names[ 1 ] ]
|
||||
);
|
||||
|
||||
}
|
||||
|
||||
if ( color.usage ) {
|
||||
|
||||
const scale = getColorScale( color.type );
|
||||
const isFloat = isFloatType( color.type );
|
||||
|
||||
// convert to float for color space conversion
|
||||
|
||||
_color.setRGB(
|
||||
element[ color.names[ 0 ] ] * scale,
|
||||
element[ color.names[ 1 ] ] * scale,
|
||||
element[ color.names[ 2 ] ] * scale,
|
||||
SRGBColorSpace
|
||||
);
|
||||
|
||||
// convert back to original type
|
||||
|
||||
const invScale = 1 / scale;
|
||||
|
||||
buffer.colors.push(
|
||||
isFloat ? _color.r : Math.round( _color.r * invScale ),
|
||||
isFloat ? _color.g : Math.round( _color.g * invScale ),
|
||||
isFloat ? _color.b : Math.round( _color.b * invScale )
|
||||
);
|
||||
|
||||
}
|
||||
|
||||
for ( const customProperty of Object.keys( scope.customPropertyMapping ) ) {
|
||||
|
||||
for ( const elementProperty of scope.customPropertyMapping[ customProperty ] ) {
|
||||
|
||||
buffer[ customProperty ].push( element[ elementProperty ] );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
} else if ( elementName === 'face' ) {
|
||||
|
||||
const vertex_indices = element.vertex_indices || element.vertex_index; // issue #9338
|
||||
const texcoord = element.texcoord;
|
||||
|
||||
if ( vertex_indices.length === 3 ) {
|
||||
|
||||
buffer.indices.push( vertex_indices[ 0 ], vertex_indices[ 1 ], vertex_indices[ 2 ] );
|
||||
|
||||
if ( texcoord && texcoord.length === 6 ) {
|
||||
|
||||
buffer.faceVertexUvs.push( texcoord[ 0 ], texcoord[ 1 ] );
|
||||
buffer.faceVertexUvs.push( texcoord[ 2 ], texcoord[ 3 ] );
|
||||
buffer.faceVertexUvs.push( texcoord[ 4 ], texcoord[ 5 ] );
|
||||
|
||||
}
|
||||
|
||||
} else if ( vertex_indices.length === 4 ) {
|
||||
|
||||
buffer.indices.push( vertex_indices[ 0 ], vertex_indices[ 1 ], vertex_indices[ 3 ] );
|
||||
buffer.indices.push( vertex_indices[ 1 ], vertex_indices[ 2 ], vertex_indices[ 3 ] );
|
||||
|
||||
}
|
||||
|
||||
// face colors
|
||||
|
||||
const { color } = attributeDescriptor;
|
||||
|
||||
if ( color.usage ) {
|
||||
|
||||
// convert to float for color space conversion
|
||||
|
||||
const scale = getColorScale( color.type );
|
||||
|
||||
_color.setRGB(
|
||||
element[ color.names[ 0 ] ] * scale,
|
||||
element[ color.names[ 1 ] ] * scale,
|
||||
element[ color.names[ 2 ] ] * scale,
|
||||
SRGBColorSpace
|
||||
);
|
||||
|
||||
// convert back to original type
|
||||
|
||||
const invScale = 1 / scale;
|
||||
|
||||
const r = _color.r * invScale;
|
||||
const g = _color.g * invScale;
|
||||
const b = _color.b * invScale;
|
||||
|
||||
buffer.faceVertexColors.push( r, g, b );
|
||||
buffer.faceVertexColors.push( r, g, b );
|
||||
buffer.faceVertexColors.push( r, g, b );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
function binaryReadElement( at, properties ) {
|
||||
|
||||
const element = {};
|
||||
let read = 0;
|
||||
|
||||
for ( let i = 0; i < properties.length; i ++ ) {
|
||||
|
||||
const property = properties[ i ];
|
||||
const valueReader = property.valueReader;
|
||||
|
||||
if ( property.type === 'list' ) {
|
||||
|
||||
const list = [];
|
||||
|
||||
const n = property.countReader.read( at + read );
|
||||
read += property.countReader.size;
|
||||
|
||||
for ( let j = 0; j < n; j ++ ) {
|
||||
|
||||
list.push( valueReader.read( at + read ) );
|
||||
read += valueReader.size;
|
||||
|
||||
}
|
||||
|
||||
element[ property.name ] = list;
|
||||
|
||||
} else {
|
||||
|
||||
element[ property.name ] = valueReader.read( at + read );
|
||||
read += valueReader.size;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return [ element, read ];
|
||||
|
||||
}
|
||||
|
||||
function setPropertyBinaryReaders( properties, body, little_endian ) {
|
||||
|
||||
function getBinaryReader( dataview, type, little_endian ) {
|
||||
|
||||
switch ( type ) {
|
||||
|
||||
// correspondences for non-specific length types here match rply:
|
||||
case 'int8': case 'char': return { read: ( at ) => {
|
||||
|
||||
return dataview.getInt8( at );
|
||||
|
||||
}, size: 1 };
|
||||
case 'uint8': case 'uchar': return { read: ( at ) => {
|
||||
|
||||
return dataview.getUint8( at );
|
||||
|
||||
}, size: 1 };
|
||||
case 'int16': case 'short': return { read: ( at ) => {
|
||||
|
||||
return dataview.getInt16( at, little_endian );
|
||||
|
||||
}, size: 2 };
|
||||
case 'uint16': case 'ushort': return { read: ( at ) => {
|
||||
|
||||
return dataview.getUint16( at, little_endian );
|
||||
|
||||
}, size: 2 };
|
||||
case 'int32': case 'int': return { read: ( at ) => {
|
||||
|
||||
return dataview.getInt32( at, little_endian );
|
||||
|
||||
}, size: 4 };
|
||||
case 'uint32': case 'uint': return { read: ( at ) => {
|
||||
|
||||
return dataview.getUint32( at, little_endian );
|
||||
|
||||
}, size: 4 };
|
||||
case 'float32': case 'float': return { read: ( at ) => {
|
||||
|
||||
return dataview.getFloat32( at, little_endian );
|
||||
|
||||
}, size: 4 };
|
||||
case 'float64': case 'double': return { read: ( at ) => {
|
||||
|
||||
return dataview.getFloat64( at, little_endian );
|
||||
|
||||
}, size: 8 };
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
for ( let i = 0, l = properties.length; i < l; i ++ ) {
|
||||
|
||||
const property = properties[ i ];
|
||||
|
||||
if ( property.type === 'list' ) {
|
||||
|
||||
property.countReader = getBinaryReader( body, property.countType, little_endian );
|
||||
property.valueReader = getBinaryReader( body, property.itemType, little_endian );
|
||||
|
||||
} else {
|
||||
|
||||
property.valueReader = getBinaryReader( body, property.type, little_endian );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
function parseBinary( data, header ) {
|
||||
|
||||
const buffer = createBuffer();
|
||||
|
||||
const little_endian = ( header.format === 'binary_little_endian' );
|
||||
const body = new DataView( data, header.headerLength );
|
||||
let result, loc = 0;
|
||||
|
||||
for ( let currentElement = 0; currentElement < header.elements.length; currentElement ++ ) {
|
||||
|
||||
const elementDesc = header.elements[ currentElement ];
|
||||
const properties = elementDesc.properties;
|
||||
const attributeDescriptor = getAttributeDescriptor( properties );
|
||||
buffer.descriptors[ elementDesc.name ] = attributeDescriptor;
|
||||
|
||||
setPropertyBinaryReaders( properties, body, little_endian );
|
||||
|
||||
for ( let currentElementCount = 0; currentElementCount < elementDesc.count; currentElementCount ++ ) {
|
||||
|
||||
result = binaryReadElement( loc, properties );
|
||||
loc += result[ 1 ];
|
||||
const element = result[ 0 ];
|
||||
|
||||
handleElement( buffer, elementDesc.name, element, attributeDescriptor );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return postProcess( buffer );
|
||||
|
||||
}
|
||||
|
||||
function extractHeaderText( bytes ) {
|
||||
|
||||
let i = 0;
|
||||
let cont = true;
|
||||
|
||||
let line = '';
|
||||
const lines = [];
|
||||
|
||||
const startLine = new TextDecoder().decode( bytes.subarray( 0, 5 ) );
|
||||
const hasCRNL = /^ply\r\n/.test( startLine );
|
||||
|
||||
do {
|
||||
|
||||
const c = String.fromCharCode( bytes[ i ++ ] );
|
||||
|
||||
if ( c !== '\n' && c !== '\r' ) {
|
||||
|
||||
line += c;
|
||||
|
||||
} else {
|
||||
|
||||
if ( line === 'end_header' ) cont = false;
|
||||
if ( line !== '' ) {
|
||||
|
||||
lines.push( line );
|
||||
line = '';
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
} while ( cont && i < bytes.length );
|
||||
|
||||
// ascii section using \r\n as line endings
|
||||
if ( hasCRNL === true ) i ++;
|
||||
|
||||
return { headerText: lines.join( '\r' ) + '\r', headerLength: i };
|
||||
|
||||
}
|
||||
|
||||
//
|
||||
|
||||
let geometry;
|
||||
const scope = this;
|
||||
|
||||
if ( data instanceof ArrayBuffer ) {
|
||||
|
||||
const bytes = new Uint8Array( data );
|
||||
const { headerText, headerLength } = extractHeaderText( bytes );
|
||||
const header = parseHeader( headerText, headerLength );
|
||||
|
||||
if ( header.format === 'ascii' ) {
|
||||
|
||||
const text = new TextDecoder().decode( bytes );
|
||||
|
||||
geometry = parseASCII( text, header );
|
||||
|
||||
} else {
|
||||
|
||||
geometry = parseBinary( data, header );
|
||||
|
||||
}
|
||||
|
||||
} else {
|
||||
|
||||
geometry = parseASCII( data, parseHeader( data ) );
|
||||
|
||||
}
|
||||
|
||||
return geometry;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
class Float64BufferAttribute extends BufferAttribute {
|
||||
|
||||
constructor( array, itemSize, normalized ) {
|
||||
|
||||
super( new Float64Array( array ), itemSize, normalized );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
class ArrayStream {
|
||||
|
||||
constructor( arr ) {
|
||||
|
||||
this.arr = arr;
|
||||
this.i = 0;
|
||||
|
||||
}
|
||||
|
||||
empty() {
|
||||
|
||||
return this.i >= this.arr.length;
|
||||
|
||||
}
|
||||
|
||||
next() {
|
||||
|
||||
return this.arr[ this.i ++ ];
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export { PLYLoader };
|
||||
191
src/real2sim_platform/static/vendor/three/addons/postprocessing/Pass.js
vendored
Normal file
191
src/real2sim_platform/static/vendor/three/addons/postprocessing/Pass.js
vendored
Normal file
@ -0,0 +1,191 @@
|
||||
import {
|
||||
BufferGeometry,
|
||||
Float32BufferAttribute,
|
||||
OrthographicCamera,
|
||||
Mesh
|
||||
} from 'three';
|
||||
|
||||
/**
|
||||
* Abstract base class for all post processing passes.
|
||||
*
|
||||
* This module is only relevant for post processing with {@link WebGLRenderer}.
|
||||
*
|
||||
* @abstract
|
||||
* @three_import import { Pass } from 'three/addons/postprocessing/Pass.js';
|
||||
*/
|
||||
class Pass {
|
||||
|
||||
/**
|
||||
* Constructs a new pass.
|
||||
*/
|
||||
constructor() {
|
||||
|
||||
/**
|
||||
* This flag can be used for type testing.
|
||||
*
|
||||
* @type {boolean}
|
||||
* @readonly
|
||||
* @default true
|
||||
*/
|
||||
this.isPass = true;
|
||||
|
||||
/**
|
||||
* If set to `true`, the pass is processed by the composer.
|
||||
*
|
||||
* @type {boolean}
|
||||
* @default true
|
||||
*/
|
||||
this.enabled = true;
|
||||
|
||||
/**
|
||||
* If set to `true`, the pass indicates to swap read and write buffer after rendering.
|
||||
*
|
||||
* @type {boolean}
|
||||
* @default true
|
||||
*/
|
||||
this.needsSwap = true;
|
||||
|
||||
/**
|
||||
* If set to `true`, the pass clears its buffer before rendering
|
||||
*
|
||||
* @type {boolean}
|
||||
* @default false
|
||||
*/
|
||||
this.clear = false;
|
||||
|
||||
/**
|
||||
* If set to `true`, the result of the pass is rendered to screen. The last pass in the composers
|
||||
* pass chain gets automatically rendered to screen, no matter how this property is configured.
|
||||
*
|
||||
* @type {boolean}
|
||||
* @default false
|
||||
*/
|
||||
this.renderToScreen = false;
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the size of the pass.
|
||||
*
|
||||
* @abstract
|
||||
* @param {number} width - The width to set.
|
||||
* @param {number} height - The height to set.
|
||||
*/
|
||||
setSize( /* width, height */ ) {}
|
||||
|
||||
/**
|
||||
* This method holds the render logic of a pass. It must be implemented in all derived classes.
|
||||
*
|
||||
* @abstract
|
||||
* @param {WebGLRenderer} renderer - The renderer.
|
||||
* @param {WebGLRenderTarget} writeBuffer - The write buffer. This buffer is intended as the rendering
|
||||
* destination for the pass.
|
||||
* @param {WebGLRenderTarget} readBuffer - The read buffer. The pass can access the result from the
|
||||
* previous pass from this buffer.
|
||||
* @param {number} deltaTime - The delta time in seconds.
|
||||
* @param {boolean} maskActive - Whether masking is active or not.
|
||||
*/
|
||||
render( /* renderer, writeBuffer, readBuffer, deltaTime, maskActive */ ) {
|
||||
|
||||
console.error( 'THREE.Pass: .render() must be implemented in derived pass.' );
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Frees the GPU-related resources allocated by this instance. Call this
|
||||
* method whenever the pass is no longer used in your app.
|
||||
*
|
||||
* @abstract
|
||||
*/
|
||||
dispose() {}
|
||||
|
||||
}
|
||||
|
||||
// Helper for passes that need to fill the viewport with a single quad.
|
||||
|
||||
const _camera = new OrthographicCamera( - 1, 1, 1, - 1, 0, 1 );
|
||||
|
||||
// https://github.com/mrdoob/three.js/pull/21358
|
||||
|
||||
class FullscreenTriangleGeometry extends BufferGeometry {
|
||||
|
||||
constructor() {
|
||||
|
||||
super();
|
||||
|
||||
this.setAttribute( 'position', new Float32BufferAttribute( [ - 1, 3, 0, - 1, - 1, 0, 3, - 1, 0 ], 3 ) );
|
||||
this.setAttribute( 'uv', new Float32BufferAttribute( [ 0, 2, 0, 0, 2, 0 ], 2 ) );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
const _geometry = new FullscreenTriangleGeometry();
|
||||
|
||||
|
||||
/**
|
||||
* This module is a helper for passes which need to render a full
|
||||
* screen effect which is quite common in context of post processing.
|
||||
*
|
||||
* The intended usage is to reuse a single full screen quad for rendering
|
||||
* subsequent passes by just reassigning the `material` reference.
|
||||
*
|
||||
* This module can only be used with {@link WebGLRenderer}.
|
||||
*
|
||||
* @augments Mesh
|
||||
* @three_import import { FullScreenQuad } from 'three/addons/postprocessing/Pass.js';
|
||||
*/
|
||||
class FullScreenQuad {
|
||||
|
||||
/**
|
||||
* Constructs a new full screen quad.
|
||||
*
|
||||
* @param {?Material} material - The material to render te full screen quad with.
|
||||
*/
|
||||
constructor( material ) {
|
||||
|
||||
this._mesh = new Mesh( _geometry, material );
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Frees the GPU-related resources allocated by this instance. Call this
|
||||
* method whenever the instance is no longer used in your app.
|
||||
*/
|
||||
dispose() {
|
||||
|
||||
this._mesh.geometry.dispose();
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Renders the full screen quad.
|
||||
*
|
||||
* @param {WebGLRenderer} renderer - The renderer.
|
||||
*/
|
||||
render( renderer ) {
|
||||
|
||||
renderer.render( this._mesh, _camera );
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* The quad's material.
|
||||
*
|
||||
* @type {?Material}
|
||||
*/
|
||||
get material() {
|
||||
|
||||
return this._mesh.material;
|
||||
|
||||
}
|
||||
|
||||
set material( value ) {
|
||||
|
||||
this._mesh.material = value;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export { Pass, FullScreenQuad };
|
||||
1501
src/real2sim_platform/static/vendor/three/addons/utils/BufferGeometryUtils.js
vendored
Normal file
1501
src/real2sim_platform/static/vendor/three/addons/utils/BufferGeometryUtils.js
vendored
Normal file
File diff suppressed because it is too large
Load Diff
496
src/real2sim_platform/static/vendor/three/addons/utils/SkeletonUtils.js
vendored
Normal file
496
src/real2sim_platform/static/vendor/three/addons/utils/SkeletonUtils.js
vendored
Normal file
@ -0,0 +1,496 @@
|
||||
import {
|
||||
AnimationClip,
|
||||
AnimationMixer,
|
||||
Matrix4,
|
||||
Quaternion,
|
||||
QuaternionKeyframeTrack,
|
||||
SkeletonHelper,
|
||||
Vector3,
|
||||
VectorKeyframeTrack
|
||||
} from 'three';
|
||||
|
||||
/**
|
||||
* @module SkeletonUtils
|
||||
* @three_import import * as SkeletonUtils from 'three/addons/utils/SkeletonUtils.js';
|
||||
*/
|
||||
|
||||
function getBoneName( bone, options ) {
|
||||
|
||||
if ( options.getBoneName !== undefined ) {
|
||||
|
||||
return options.getBoneName( bone );
|
||||
|
||||
}
|
||||
|
||||
return options.names[ bone.name ];
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Retargets the skeleton from the given source to the target.
|
||||
*
|
||||
* Both `target` and `source` can be a 3D object with a skeleton property (e.g. a skinned mesh)
|
||||
* or a {@link Skeleton} directly.
|
||||
*
|
||||
* @param {Object3D|Skeleton} target - The target object.
|
||||
* @param {Object3D|Skeleton} source - The source object.
|
||||
* @param {module:SkeletonUtils~RetargetOptions} options - The options.
|
||||
*/
|
||||
function retarget( target, source, options = {} ) {
|
||||
|
||||
const quat = new Quaternion(),
|
||||
scale = new Vector3(),
|
||||
relativeMatrix = new Matrix4(),
|
||||
globalMatrix = new Matrix4();
|
||||
|
||||
options.preserveBoneMatrix = options.preserveBoneMatrix !== undefined ? options.preserveBoneMatrix : true;
|
||||
options.preserveBonePositions = options.preserveBonePositions !== undefined ? options.preserveBonePositions : true;
|
||||
options.useTargetMatrix = options.useTargetMatrix !== undefined ? options.useTargetMatrix : false;
|
||||
options.hip = options.hip !== undefined ? options.hip : 'hip';
|
||||
options.hipInfluence = options.hipInfluence !== undefined ? options.hipInfluence : new Vector3( 1, 1, 1 );
|
||||
options.scale = options.scale !== undefined ? options.scale : 1;
|
||||
options.names = options.names || {};
|
||||
|
||||
const sourceBones = source.isObject3D ? source.skeleton.bones : getBones( source ),
|
||||
bones = target.isObject3D ? target.skeleton.bones : getBones( target );
|
||||
|
||||
let bone, name, boneTo,
|
||||
bonesPosition;
|
||||
|
||||
// reset bones
|
||||
|
||||
if ( target.isObject3D ) {
|
||||
|
||||
target.skeleton.pose();
|
||||
|
||||
} else {
|
||||
|
||||
options.useTargetMatrix = true;
|
||||
options.preserveBoneMatrix = false;
|
||||
|
||||
}
|
||||
|
||||
if ( options.preserveBonePositions ) {
|
||||
|
||||
bonesPosition = [];
|
||||
|
||||
for ( let i = 0; i < bones.length; i ++ ) {
|
||||
|
||||
bonesPosition.push( bones[ i ].position.clone() );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if ( options.preserveBoneMatrix ) {
|
||||
|
||||
// reset matrix
|
||||
|
||||
target.updateMatrixWorld();
|
||||
|
||||
target.matrixWorld.identity();
|
||||
|
||||
// reset children matrix
|
||||
|
||||
for ( let i = 0; i < target.children.length; ++ i ) {
|
||||
|
||||
target.children[ i ].updateMatrixWorld( true );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
for ( let i = 0; i < bones.length; ++ i ) {
|
||||
|
||||
bone = bones[ i ];
|
||||
name = getBoneName( bone, options );
|
||||
|
||||
boneTo = getBoneByName( name, sourceBones );
|
||||
|
||||
globalMatrix.copy( bone.matrixWorld );
|
||||
|
||||
if ( boneTo ) {
|
||||
|
||||
boneTo.updateMatrixWorld();
|
||||
|
||||
if ( options.useTargetMatrix ) {
|
||||
|
||||
relativeMatrix.copy( boneTo.matrixWorld );
|
||||
|
||||
} else {
|
||||
|
||||
relativeMatrix.copy( target.matrixWorld ).invert();
|
||||
relativeMatrix.multiply( boneTo.matrixWorld );
|
||||
|
||||
}
|
||||
|
||||
// ignore scale to extract rotation
|
||||
|
||||
scale.setFromMatrixScale( relativeMatrix );
|
||||
relativeMatrix.scale( scale.set( 1 / scale.x, 1 / scale.y, 1 / scale.z ) );
|
||||
|
||||
// apply to global matrix
|
||||
|
||||
globalMatrix.makeRotationFromQuaternion( quat.setFromRotationMatrix( relativeMatrix ) );
|
||||
|
||||
if ( target.isObject3D ) {
|
||||
|
||||
if ( options.localOffsets ) {
|
||||
|
||||
if ( options.localOffsets[ bone.name ] ) {
|
||||
|
||||
globalMatrix.multiply( options.localOffsets[ bone.name ] );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
globalMatrix.copyPosition( relativeMatrix );
|
||||
|
||||
}
|
||||
|
||||
if ( name === options.hip ) {
|
||||
|
||||
globalMatrix.elements[ 12 ] *= options.scale * options.hipInfluence.x;
|
||||
globalMatrix.elements[ 13 ] *= options.scale * options.hipInfluence.y;
|
||||
globalMatrix.elements[ 14 ] *= options.scale * options.hipInfluence.z;
|
||||
|
||||
if ( options.hipPosition !== undefined ) {
|
||||
|
||||
globalMatrix.elements[ 12 ] += options.hipPosition.x * options.scale;
|
||||
globalMatrix.elements[ 13 ] += options.hipPosition.y * options.scale;
|
||||
globalMatrix.elements[ 14 ] += options.hipPosition.z * options.scale;
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if ( bone.parent ) {
|
||||
|
||||
bone.matrix.copy( bone.parent.matrixWorld ).invert();
|
||||
bone.matrix.multiply( globalMatrix );
|
||||
|
||||
} else {
|
||||
|
||||
bone.matrix.copy( globalMatrix );
|
||||
|
||||
}
|
||||
|
||||
bone.matrix.decompose( bone.position, bone.quaternion, bone.scale );
|
||||
|
||||
bone.updateMatrixWorld();
|
||||
|
||||
}
|
||||
|
||||
if ( options.preserveBonePositions ) {
|
||||
|
||||
for ( let i = 0; i < bones.length; ++ i ) {
|
||||
|
||||
bone = bones[ i ];
|
||||
name = getBoneName( bone, options ) || bone.name;
|
||||
|
||||
if ( name !== options.hip ) {
|
||||
|
||||
bone.position.copy( bonesPosition[ i ] );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if ( options.preserveBoneMatrix ) {
|
||||
|
||||
// restore matrix
|
||||
|
||||
target.updateMatrixWorld( true );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Retargets the animation clip of the source to the target 3D object.
|
||||
*
|
||||
* The `source` can be a 3D object with a skeleton property (e.g. a skinned mesh)
|
||||
* or a {@link Skeleton} directly.
|
||||
*
|
||||
* @param {Object3D} target - The target 3D object. Must have a `skeleton` property.
|
||||
* @param {Object3D|Skeleton} source - The source object.
|
||||
* @param {AnimationClip} clip - The animation clip.
|
||||
* @param {module:SkeletonUtils~RetargetOptions} options - The options.
|
||||
* @return {AnimationClip} The retargeted animation clip.
|
||||
*/
|
||||
function retargetClip( target, source, clip, options = {} ) {
|
||||
|
||||
options.useFirstFramePosition = options.useFirstFramePosition !== undefined ? options.useFirstFramePosition : false;
|
||||
|
||||
// Calculate the fps from the source clip based on the track with the most frames, unless fps is already provided.
|
||||
options.fps = options.fps !== undefined ? options.fps : ( Math.max( ...clip.tracks.map( track => track.times.length ) ) / clip.duration );
|
||||
options.names = options.names || [];
|
||||
|
||||
if ( ! source.isObject3D ) {
|
||||
|
||||
source = getHelperFromSkeleton( source );
|
||||
|
||||
}
|
||||
|
||||
const numFrames = Math.round( clip.duration * ( options.fps / 1000 ) * 1000 ),
|
||||
delta = clip.duration / ( numFrames - 1 ),
|
||||
convertedTracks = [],
|
||||
mixer = new AnimationMixer( source ),
|
||||
bones = getBones( target.skeleton ),
|
||||
boneDatas = [];
|
||||
|
||||
let positionOffset,
|
||||
bone, boneTo, boneData,
|
||||
name;
|
||||
|
||||
mixer.clipAction( clip ).play();
|
||||
|
||||
// trim
|
||||
|
||||
let start = 0, end = numFrames;
|
||||
|
||||
if ( options.trim !== undefined ) {
|
||||
|
||||
start = Math.round( options.trim[ 0 ] * options.fps );
|
||||
end = Math.min( Math.round( options.trim[ 1 ] * options.fps ), numFrames ) - start;
|
||||
|
||||
mixer.update( options.trim[ 0 ] );
|
||||
|
||||
} else {
|
||||
|
||||
mixer.update( 0 );
|
||||
|
||||
}
|
||||
|
||||
source.updateMatrixWorld();
|
||||
|
||||
//
|
||||
|
||||
for ( let frame = 0; frame < end; ++ frame ) {
|
||||
|
||||
const time = frame * delta;
|
||||
|
||||
retarget( target, source, options );
|
||||
|
||||
for ( let j = 0; j < bones.length; ++ j ) {
|
||||
|
||||
bone = bones[ j ];
|
||||
name = getBoneName( bone, options ) || bone.name;
|
||||
boneTo = getBoneByName( name, source.skeleton );
|
||||
|
||||
if ( boneTo ) {
|
||||
|
||||
boneData = boneDatas[ j ] = boneDatas[ j ] || { bone: bone };
|
||||
|
||||
if ( options.hip === name ) {
|
||||
|
||||
if ( ! boneData.pos ) {
|
||||
|
||||
boneData.pos = {
|
||||
times: new Float32Array( end ),
|
||||
values: new Float32Array( end * 3 )
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
if ( options.useFirstFramePosition ) {
|
||||
|
||||
if ( frame === 0 ) {
|
||||
|
||||
positionOffset = bone.position.clone();
|
||||
|
||||
}
|
||||
|
||||
bone.position.sub( positionOffset );
|
||||
|
||||
}
|
||||
|
||||
boneData.pos.times[ frame ] = time;
|
||||
|
||||
bone.position.toArray( boneData.pos.values, frame * 3 );
|
||||
|
||||
}
|
||||
|
||||
if ( ! boneData.quat ) {
|
||||
|
||||
boneData.quat = {
|
||||
times: new Float32Array( end ),
|
||||
values: new Float32Array( end * 4 )
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
boneData.quat.times[ frame ] = time;
|
||||
|
||||
bone.quaternion.toArray( boneData.quat.values, frame * 4 );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if ( frame === end - 2 ) {
|
||||
|
||||
// last mixer update before final loop iteration
|
||||
// make sure we do not go over or equal to clip duration
|
||||
mixer.update( delta - 0.0000001 );
|
||||
|
||||
} else {
|
||||
|
||||
mixer.update( delta );
|
||||
|
||||
}
|
||||
|
||||
source.updateMatrixWorld();
|
||||
|
||||
}
|
||||
|
||||
for ( let i = 0; i < boneDatas.length; ++ i ) {
|
||||
|
||||
boneData = boneDatas[ i ];
|
||||
|
||||
if ( boneData ) {
|
||||
|
||||
if ( boneData.pos ) {
|
||||
|
||||
convertedTracks.push( new VectorKeyframeTrack(
|
||||
'.bones[' + boneData.bone.name + '].position',
|
||||
boneData.pos.times,
|
||||
boneData.pos.values
|
||||
) );
|
||||
|
||||
}
|
||||
|
||||
convertedTracks.push( new QuaternionKeyframeTrack(
|
||||
'.bones[' + boneData.bone.name + '].quaternion',
|
||||
boneData.quat.times,
|
||||
boneData.quat.values
|
||||
) );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
mixer.uncacheAction( clip );
|
||||
|
||||
return new AnimationClip( clip.name, - 1, convertedTracks );
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Clones the given 3D object and its descendants, ensuring that any `SkinnedMesh` instances are
|
||||
* correctly associated with their bones. Bones are also cloned, and must be descendants of the
|
||||
* object passed to this method. Other data, like geometries and materials, are reused by reference.
|
||||
*
|
||||
* @param {Object3D} source - The 3D object to clone.
|
||||
* @return {Object3D} The cloned 3D object.
|
||||
*/
|
||||
function clone( source ) {
|
||||
|
||||
const sourceLookup = new Map();
|
||||
const cloneLookup = new Map();
|
||||
|
||||
const clone = source.clone();
|
||||
|
||||
parallelTraverse( source, clone, function ( sourceNode, clonedNode ) {
|
||||
|
||||
sourceLookup.set( clonedNode, sourceNode );
|
||||
cloneLookup.set( sourceNode, clonedNode );
|
||||
|
||||
} );
|
||||
|
||||
clone.traverse( function ( node ) {
|
||||
|
||||
if ( ! node.isSkinnedMesh ) return;
|
||||
|
||||
const clonedMesh = node;
|
||||
const sourceMesh = sourceLookup.get( node );
|
||||
const sourceBones = sourceMesh.skeleton.bones;
|
||||
|
||||
clonedMesh.skeleton = sourceMesh.skeleton.clone();
|
||||
clonedMesh.bindMatrix.copy( sourceMesh.bindMatrix );
|
||||
|
||||
clonedMesh.skeleton.bones = sourceBones.map( function ( bone ) {
|
||||
|
||||
return cloneLookup.get( bone );
|
||||
|
||||
} );
|
||||
|
||||
clonedMesh.bind( clonedMesh.skeleton, clonedMesh.bindMatrix );
|
||||
|
||||
} );
|
||||
|
||||
return clone;
|
||||
|
||||
}
|
||||
|
||||
// internal helper
|
||||
|
||||
function getBoneByName( name, skeleton ) {
|
||||
|
||||
for ( let i = 0, bones = getBones( skeleton ); i < bones.length; i ++ ) {
|
||||
|
||||
if ( name === bones[ i ].name )
|
||||
|
||||
return bones[ i ];
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
function getBones( skeleton ) {
|
||||
|
||||
return Array.isArray( skeleton ) ? skeleton : skeleton.bones;
|
||||
|
||||
}
|
||||
|
||||
|
||||
function getHelperFromSkeleton( skeleton ) {
|
||||
|
||||
const source = new SkeletonHelper( skeleton.bones[ 0 ] );
|
||||
source.skeleton = skeleton;
|
||||
|
||||
return source;
|
||||
|
||||
}
|
||||
|
||||
function parallelTraverse( a, b, callback ) {
|
||||
|
||||
callback( a, b );
|
||||
|
||||
for ( let i = 0; i < a.children.length; i ++ ) {
|
||||
|
||||
parallelTraverse( a.children[ i ], b.children[ i ], callback );
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Retarget options of `SkeletonUtils`.
|
||||
*
|
||||
* @typedef {Object} module:SkeletonUtils~RetargetOptions
|
||||
* @property {boolean} [useFirstFramePosition=false] - Whether to use the position of the first frame or not.
|
||||
* @property {number} [fps] - The FPS of the clip.
|
||||
* @property {Object<string,string>} [names] - A dictionary for mapping target to source bone names.
|
||||
* @property {function(string):string} [getBoneName] - A function for mapping bone names. Alternative to `names`.
|
||||
* @property {Array<number>} [trim] - Whether to trim the clip or not. If set the array should hold two values for the start and end.
|
||||
* @property {boolean} [preserveBoneMatrix=true] - Whether to preserve bone matrices or not.
|
||||
* @property {boolean} [preserveBonePositions=true] - Whether to preserve bone positions or not.
|
||||
* @property {boolean} [useTargetMatrix=false] - Whether to use the target matrix or not.
|
||||
* @property {string} [hip='hip'] - The name of the source's hip bone.
|
||||
* @property {Vector3} [hipInfluence=(1,1,1)] - The hip influence.
|
||||
* @property {number} [scale=1] - The scale.
|
||||
* @property {Object<string,Matrix4>} [localOffsets] - Per-bone local offset matrices, keyed by bone name.
|
||||
* @property {Vector3} [hipPosition] - An additional position offset applied to the hip bone.
|
||||
**/
|
||||
|
||||
export {
|
||||
retarget,
|
||||
retargetClip,
|
||||
clone,
|
||||
};
|
||||
60007
src/real2sim_platform/static/vendor/three/three.core.js
vendored
Normal file
60007
src/real2sim_platform/static/vendor/three/three.core.js
vendored
Normal file
File diff suppressed because one or more lines are too long
19610
src/real2sim_platform/static/vendor/three/three.module.js
vendored
Normal file
19610
src/real2sim_platform/static/vendor/three/three.module.js
vendored
Normal file
File diff suppressed because one or more lines are too long
393
src/real2sim_platform/store.py
Normal file
393
src/real2sim_platform/store.py
Normal file
@ -0,0 +1,393 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
class PlatformStore:
|
||||
def __init__(self, db_path: Path) -> None:
|
||||
self.db_path = db_path
|
||||
self.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._init()
|
||||
|
||||
def create_dataset(
|
||||
self,
|
||||
*,
|
||||
name: str,
|
||||
path: Path,
|
||||
scan: dict[str, Any],
|
||||
item_id: str | None = None,
|
||||
source_type: str = "local",
|
||||
source_uri: str = "",
|
||||
version: str = "",
|
||||
parent_task_id: str | None = None,
|
||||
manifest_path: Path | None = None,
|
||||
metrics: dict[str, Any] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
item_id = item_id or uuid.uuid4().hex
|
||||
now = _now()
|
||||
with self._connect() as db:
|
||||
db.execute(
|
||||
"insert into datasets(id,name,path,scan_json,source_type,source_uri,version,parent_task_id,manifest_path,metrics_json,tags_json,created_at,updated_at) values(?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
||||
(item_id, name, str(path), _json(scan), source_type, source_uri, version, parent_task_id, str(manifest_path or ""), _json(metrics or {}), _json(tags or []), now, now),
|
||||
)
|
||||
return self.get_dataset(item_id)
|
||||
|
||||
def list_datasets(self) -> list[dict[str, Any]]:
|
||||
return self._list("datasets", self._dataset_row)
|
||||
|
||||
def get_dataset(self, item_id: str) -> dict[str, Any]:
|
||||
return self._get("datasets", item_id, self._dataset_row)
|
||||
|
||||
def update_dataset(self, item_id: str, *, name: str | None = None, tags: list[str] | None = None, version: str | None = None) -> dict[str, Any]:
|
||||
self._update_entity("datasets", item_id, name=name, tags=tags, version=version)
|
||||
return self.get_dataset(item_id)
|
||||
|
||||
def delete_dataset(self, item_id: str) -> None:
|
||||
self._delete("datasets", item_id)
|
||||
|
||||
def create_scene(
|
||||
self,
|
||||
*,
|
||||
name: str,
|
||||
path: Path,
|
||||
metadata: dict[str, Any],
|
||||
item_id: str | None = None,
|
||||
source_type: str = "generated",
|
||||
source_uri: str = "",
|
||||
version: str = "",
|
||||
parent_task_id: str | None = None,
|
||||
manifest_path: Path | None = None,
|
||||
metrics: dict[str, Any] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
item_id = item_id or uuid.uuid4().hex
|
||||
now = _now()
|
||||
with self._connect() as db:
|
||||
db.execute(
|
||||
"insert into scenes(id,name,path,metadata_json,source_type,source_uri,version,parent_task_id,manifest_path,metrics_json,tags_json,created_at,updated_at) values(?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
||||
(item_id, name, str(path), _json(metadata), source_type, source_uri, version, parent_task_id, str(manifest_path or ""), _json(metrics or {}), _json(tags or []), now, now),
|
||||
)
|
||||
return self.get_scene(item_id)
|
||||
|
||||
def list_scenes(self) -> list[dict[str, Any]]:
|
||||
return self._list("scenes", self._scene_row)
|
||||
|
||||
def get_scene(self, item_id: str) -> dict[str, Any]:
|
||||
return self._get("scenes", item_id, self._scene_row)
|
||||
|
||||
def update_scene(
|
||||
self,
|
||||
item_id: str,
|
||||
*,
|
||||
name: str | None = None,
|
||||
tags: list[str] | None = None,
|
||||
version: str | None = None,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
metrics: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
self._update_entity("scenes", item_id, name=name, tags=tags, version=version)
|
||||
if metadata is not None or metrics is not None:
|
||||
assignments: list[str] = []
|
||||
values: list[Any] = []
|
||||
if metadata is not None:
|
||||
assignments.append("metadata_json=?")
|
||||
values.append(_json(metadata))
|
||||
if metrics is not None:
|
||||
assignments.append("metrics_json=?")
|
||||
values.append(_json(metrics))
|
||||
assignments.append("updated_at=?")
|
||||
values.append(_now())
|
||||
values.append(item_id)
|
||||
with self._connect() as db:
|
||||
db.execute(f"update scenes set {','.join(assignments)} where id=?", values)
|
||||
return self.get_scene(item_id)
|
||||
|
||||
def delete_scene(self, item_id: str) -> None:
|
||||
self._delete("scenes", item_id)
|
||||
|
||||
def create_model(
|
||||
self,
|
||||
*,
|
||||
name: str,
|
||||
path: Path,
|
||||
metadata: dict[str, Any],
|
||||
item_id: str | None = None,
|
||||
source_type: str = "training",
|
||||
source_uri: str = "",
|
||||
version: str = "",
|
||||
parent_task_id: str | None = None,
|
||||
manifest_path: Path | None = None,
|
||||
metrics: dict[str, Any] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
item_id = item_id or uuid.uuid4().hex
|
||||
now = _now()
|
||||
with self._connect() as db:
|
||||
db.execute(
|
||||
"insert into models(id,name,path,metadata_json,source_type,source_uri,version,parent_task_id,manifest_path,metrics_json,tags_json,created_at,updated_at) values(?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
||||
(item_id, name, str(path), _json(metadata), source_type, source_uri, version, parent_task_id, str(manifest_path or ""), _json(metrics or {}), _json(tags or []), now, now),
|
||||
)
|
||||
return self.get_model(item_id)
|
||||
|
||||
def list_models(self) -> list[dict[str, Any]]:
|
||||
return self._list("models", self._model_row)
|
||||
|
||||
def get_model(self, item_id: str) -> dict[str, Any]:
|
||||
return self._get("models", item_id, self._model_row)
|
||||
|
||||
def update_model(self, item_id: str, *, name: str | None = None, tags: list[str] | None = None, version: str | None = None) -> dict[str, Any]:
|
||||
self._update_entity("models", item_id, name=name, tags=tags, version=version)
|
||||
return self.get_model(item_id)
|
||||
|
||||
def delete_model(self, item_id: str) -> None:
|
||||
self._delete("models", item_id)
|
||||
|
||||
def create_task(self, *, task_type: str, params: dict[str, Any], run_dir: Path) -> dict[str, Any]:
|
||||
item_id = uuid.uuid4().hex
|
||||
now = _now()
|
||||
with self._connect() as db:
|
||||
db.execute(
|
||||
"insert into tasks(id,type,status,params_json,result_json,run_dir,created_at,updated_at) values(?,?,?,?,?,?,?,?)",
|
||||
(item_id, task_type, "queued", _json(params), "{}", str(run_dir), now, now),
|
||||
)
|
||||
return self.get_task(item_id)
|
||||
|
||||
def list_tasks(self) -> list[dict[str, Any]]:
|
||||
return self._list("tasks", self._task_row)
|
||||
|
||||
def get_task(self, item_id: str) -> dict[str, Any]:
|
||||
return self._get("tasks", item_id, self._task_row)
|
||||
|
||||
def next_queued_task(self) -> dict[str, Any] | None:
|
||||
with self._connect() as db:
|
||||
row = db.execute("select * from tasks where status='queued' order by created_at limit 1").fetchone()
|
||||
return self._task_row(row) if row else None
|
||||
|
||||
def update_task(
|
||||
self,
|
||||
task_id: str,
|
||||
*,
|
||||
status: str | None = None,
|
||||
result: dict[str, Any] | None = None,
|
||||
error: str | None = None,
|
||||
progress: int | None = None,
|
||||
stage: str | None = None,
|
||||
message: str | None = None,
|
||||
) -> None:
|
||||
task = self.get_task(task_id)
|
||||
next_progress = task["progress"] if progress is None else max(0, min(100, int(progress)))
|
||||
with self._connect() as db:
|
||||
db.execute(
|
||||
"update tasks set status=?, result_json=?, error=?, progress=?, stage=?, message=?, updated_at=? where id=?",
|
||||
(
|
||||
status or task["status"],
|
||||
_json(result if result is not None else task["result"]),
|
||||
error,
|
||||
next_progress,
|
||||
task["stage"] if stage is None else stage,
|
||||
task["message"] if message is None else message,
|
||||
_now(),
|
||||
task_id,
|
||||
),
|
||||
)
|
||||
|
||||
def set_task_pid(self, task_id: str, pid: int | None) -> None:
|
||||
with self._connect() as db:
|
||||
db.execute("update tasks set pid=?, updated_at=? where id=?", (pid, _now(), task_id))
|
||||
|
||||
def cancel_task(self, task_id: str) -> dict[str, Any]:
|
||||
task = self.get_task(task_id)
|
||||
with self._connect() as db:
|
||||
if task["status"] == "queued":
|
||||
db.execute(
|
||||
"update tasks set status='canceled', cancel_requested=1, progress=0, stage='已取消', message='任务启动前已取消', updated_at=? where id=?",
|
||||
(_now(), task_id),
|
||||
)
|
||||
elif task["status"] == "running":
|
||||
db.execute(
|
||||
"update tasks set status='canceling', cancel_requested=1, stage='取消中', message='正在终止运行进程', updated_at=? where id=?",
|
||||
(_now(), task_id),
|
||||
)
|
||||
return self.get_task(task_id)
|
||||
|
||||
def recover_interrupted_tasks(self) -> int:
|
||||
with self._connect() as db:
|
||||
cursor = db.execute(
|
||||
"update tasks set status='failed_recovered', error='service restarted while task was running', pid=null, stage='恢复失败', message='服务重启时任务仍在运行,已标记为可重试', updated_at=? where status in ('running','canceling')",
|
||||
(_now(),),
|
||||
)
|
||||
return cursor.rowcount
|
||||
|
||||
def retry_task(self, task_id: str) -> dict[str, Any]:
|
||||
task = self.get_task(task_id)
|
||||
base = Path(task["run_dir"])
|
||||
index = 1
|
||||
retry_dir = base.with_name(f"{base.name}_retry{index}")
|
||||
while retry_dir.exists():
|
||||
index += 1
|
||||
retry_dir = base.with_name(f"{base.name}_retry{index}")
|
||||
return self.create_task(task_type=task["type"], params=task["params"], run_dir=retry_dir)
|
||||
|
||||
def update_task_params(self, task_id: str, *, params: dict[str, Any]) -> dict[str, Any]:
|
||||
task = self.get_task(task_id)
|
||||
if task["status"] not in {"queued", "failed", "failed_recovered", "canceled"}:
|
||||
raise ValueError("only queued or terminal tasks can be edited")
|
||||
with self._connect() as db:
|
||||
db.execute("update tasks set params_json=?, updated_at=? where id=?", (_json(params), _now(), task_id))
|
||||
return self.get_task(task_id)
|
||||
|
||||
def delete_task(self, task_id: str) -> None:
|
||||
task = self.get_task(task_id)
|
||||
if task["status"] in {"running", "canceling"}:
|
||||
raise ValueError("running tasks cannot be deleted")
|
||||
self._delete("tasks", task_id)
|
||||
|
||||
def add_artifact(self, *, task_id: str | None, name: str, path: Path, kind: str) -> dict[str, Any]:
|
||||
item_id = uuid.uuid4().hex
|
||||
now = _now()
|
||||
with self._connect() as db:
|
||||
db.execute(
|
||||
"insert into artifacts(id,task_id,name,path,kind,created_at) values(?,?,?,?,?,?)",
|
||||
(item_id, task_id, name, str(path), kind, now),
|
||||
)
|
||||
return self.get_artifact(item_id)
|
||||
|
||||
def list_task_artifacts(self, task_id: str) -> list[dict[str, Any]]:
|
||||
with self._connect() as db:
|
||||
rows = db.execute("select * from artifacts where task_id=? order by created_at", (task_id,)).fetchall()
|
||||
return [self._artifact_row(row) for row in rows]
|
||||
|
||||
def get_artifact(self, item_id: str) -> dict[str, Any]:
|
||||
return self._get("artifacts", item_id, self._artifact_row)
|
||||
|
||||
def _init(self) -> None:
|
||||
with self._connect() as db:
|
||||
db.execute("create table if not exists datasets(id text primary key,name text,path text,scan_json text,created_at real,updated_at real)")
|
||||
db.execute("create table if not exists scenes(id text primary key,name text,path text,metadata_json text,created_at real,updated_at real)")
|
||||
db.execute("create table if not exists models(id text primary key,name text,path text,metadata_json text,created_at real,updated_at real)")
|
||||
db.execute(
|
||||
"create table if not exists tasks("
|
||||
"id text primary key,type text,status text,params_json text,result_json text not null default '{}',"
|
||||
"run_dir text,error text,cancel_requested integer not null default 0,created_at real,updated_at real)"
|
||||
)
|
||||
db.execute("create table if not exists artifacts(id text primary key,task_id text,name text,path text,kind text,created_at real)")
|
||||
for table in ["datasets", "scenes", "models"]:
|
||||
self._ensure_column(db, table, "source_type", "text not null default 'local'")
|
||||
self._ensure_column(db, table, "source_uri", "text not null default ''")
|
||||
self._ensure_column(db, table, "version", "text not null default ''")
|
||||
self._ensure_column(db, table, "parent_task_id", "text")
|
||||
self._ensure_column(db, table, "manifest_path", "text not null default ''")
|
||||
self._ensure_column(db, table, "metrics_json", "text not null default '{}'")
|
||||
self._ensure_column(db, table, "tags_json", "text not null default '[]'")
|
||||
self._ensure_column(db, "tasks", "pid", "integer")
|
||||
self._ensure_column(db, "tasks", "progress", "integer not null default 0")
|
||||
self._ensure_column(db, "tasks", "stage", "text not null default ''")
|
||||
self._ensure_column(db, "tasks", "message", "text not null default ''")
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
db = sqlite3.connect(self.db_path)
|
||||
db.row_factory = sqlite3.Row
|
||||
return db
|
||||
|
||||
def _ensure_column(self, db: sqlite3.Connection, table: str, column: str, spec: str) -> None:
|
||||
columns = {row["name"] for row in db.execute(f"pragma table_info({table})").fetchall()}
|
||||
if column not in columns:
|
||||
db.execute(f"alter table {table} add column {column} {spec}")
|
||||
|
||||
def _list(self, table: str, mapper) -> list[dict[str, Any]]:
|
||||
with self._connect() as db:
|
||||
rows = db.execute(f"select * from {table} order by created_at desc").fetchall()
|
||||
return [mapper(row) for row in rows]
|
||||
|
||||
def _get(self, table: str, item_id: str, mapper) -> dict[str, Any]:
|
||||
with self._connect() as db:
|
||||
row = db.execute(f"select * from {table} where id=?", (item_id,)).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(item_id)
|
||||
return mapper(row)
|
||||
|
||||
def _update_entity(self, table: str, item_id: str, *, name: str | None, tags: list[str] | None, version: str | None) -> None:
|
||||
self._get(table, item_id, lambda row: row)
|
||||
fields: list[str] = []
|
||||
values: list[Any] = []
|
||||
if name is not None:
|
||||
fields.append("name=?")
|
||||
values.append(name)
|
||||
if tags is not None:
|
||||
fields.append("tags_json=?")
|
||||
values.append(_json(tags))
|
||||
if version is not None:
|
||||
fields.append("version=?")
|
||||
values.append(version)
|
||||
if not fields:
|
||||
return
|
||||
fields.append("updated_at=?")
|
||||
values.append(_now())
|
||||
values.append(item_id)
|
||||
with self._connect() as db:
|
||||
db.execute(f"update {table} set {', '.join(fields)} where id=?", values)
|
||||
|
||||
def _delete(self, table: str, item_id: str) -> None:
|
||||
self._get(table, item_id, lambda row: row)
|
||||
with self._connect() as db:
|
||||
db.execute(f"delete from {table} where id=?", (item_id,))
|
||||
|
||||
def _dataset_row(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
return {**self._entity_common(row), "scan": json.loads(row["scan_json"])}
|
||||
|
||||
def _scene_row(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
return {**self._entity_common(row), "metadata": json.loads(row["metadata_json"])}
|
||||
|
||||
def _model_row(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
return {**self._entity_common(row), "metadata": json.loads(row["metadata_json"])}
|
||||
|
||||
def _entity_common(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
return {
|
||||
"id": row["id"],
|
||||
"name": row["name"],
|
||||
"path": row["path"],
|
||||
"source_type": row["source_type"],
|
||||
"source_uri": row["source_uri"],
|
||||
"version": row["version"],
|
||||
"parent_task_id": row["parent_task_id"],
|
||||
"manifest_path": row["manifest_path"],
|
||||
"metrics": json.loads(row["metrics_json"] or "{}"),
|
||||
"tags": json.loads(row["tags_json"] or "[]"),
|
||||
"created_at": row["created_at"],
|
||||
"updated_at": row["updated_at"],
|
||||
}
|
||||
|
||||
def _task_row(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
return {
|
||||
"id": row["id"],
|
||||
"type": row["type"],
|
||||
"status": row["status"],
|
||||
"params": json.loads(row["params_json"]),
|
||||
"result": json.loads(row["result_json"] or "{}"),
|
||||
"run_dir": row["run_dir"],
|
||||
"error": row["error"],
|
||||
"progress": int(row["progress"] or 0),
|
||||
"stage": row["stage"] or "",
|
||||
"message": row["message"] or "",
|
||||
"cancel_requested": bool(row["cancel_requested"]),
|
||||
"pid": row["pid"],
|
||||
"created_at": row["created_at"],
|
||||
"updated_at": row["updated_at"],
|
||||
}
|
||||
|
||||
def _artifact_row(self, row: sqlite3.Row) -> dict[str, Any]:
|
||||
return {"id": row["id"], "task_id": row["task_id"], "name": row["name"], "path": row["path"], "kind": row["kind"], "created_at": row["created_at"]}
|
||||
|
||||
|
||||
def _json(value: Any) -> str:
|
||||
return json.dumps(value, ensure_ascii=False)
|
||||
|
||||
|
||||
def _now() -> float:
|
||||
return time.time()
|
||||
67
src/real2sim_platform/worker.py
Normal file
67
src/real2sim_platform/worker.py
Normal file
@ -0,0 +1,67 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
import traceback
|
||||
from typing import Any
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .adapters import TaskCanceled, execute_task, index_artifacts_for_task
|
||||
from .config import PlatformConfig
|
||||
from .store import PlatformStore
|
||||
|
||||
|
||||
class TaskWorker:
|
||||
def __init__(self, store: PlatformStore, config: PlatformConfig) -> None:
|
||||
self.store = store
|
||||
self.config = config
|
||||
self._stop = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
def start(self) -> None:
|
||||
if self._thread and self._thread.is_alive():
|
||||
return
|
||||
self.store.recover_interrupted_tasks()
|
||||
self._thread = threading.Thread(target=self._loop, name="real2sim-platform-worker", daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop.set()
|
||||
|
||||
def _loop(self) -> None:
|
||||
while not self._stop.is_set():
|
||||
task = self.store.next_queued_task()
|
||||
if task is None:
|
||||
time.sleep(0.5)
|
||||
continue
|
||||
self._run(task)
|
||||
|
||||
def _run(self, task: dict[str, Any]) -> None:
|
||||
if self.store.get_task(task["id"])["cancel_requested"]:
|
||||
self.store.update_task(task["id"], status="canceled", error="task canceled before start", progress=0, stage="已取消", message="任务启动前已取消")
|
||||
return
|
||||
self.store.update_task(task["id"], status="running", progress=5, stage="启动中", message="任务已进入执行队列")
|
||||
try:
|
||||
result = execute_task(self.config, self.store, task)
|
||||
except TaskCanceled as exc:
|
||||
self._write_trace(task, exc)
|
||||
index_artifacts_for_task(self.store, task["id"], Path(task["run_dir"]))
|
||||
self.store.update_task(task["id"], status="canceled", error=str(exc), progress=0, stage="已取消", message=str(exc))
|
||||
return
|
||||
except Exception as exc:
|
||||
self._write_trace(task, exc)
|
||||
index_artifacts_for_task(self.store, task["id"], Path(task["run_dir"]))
|
||||
self.store.update_task(task["id"], status="failed", error=str(exc), stage="执行失败", message=str(exc))
|
||||
return
|
||||
status = "canceled" if self.store.get_task(task["id"])["cancel_requested"] else "succeeded"
|
||||
self.store.update_task(task["id"], status=status, result=result, progress=100 if status == "succeeded" else 0, stage="已完成" if status == "succeeded" else "已取消", message="任务完成" if status == "succeeded" else "任务已取消")
|
||||
|
||||
def _write_trace(self, task: dict[str, Any], exc: Exception) -> None:
|
||||
log_path = self.config.runs_dir / task["id"] / "logs" / "task.log"
|
||||
if "run_dir" in task:
|
||||
log_path = __import__("pathlib").Path(task["run_dir"]) / "logs" / "task.log"
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with log_path.open("a", encoding="utf-8") as handle:
|
||||
handle.write("\nERROR: " + str(exc) + "\n")
|
||||
handle.write(traceback.format_exc() + "\n")
|
||||
499
tests/test_platform.py
Normal file
499
tests/test_platform.py
Normal file
@ -0,0 +1,499 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from PIL import Image
|
||||
|
||||
from real2sim_platform.adapters import _export_3dgs_rgb_fallback, collect_task_reports
|
||||
from real2sim_platform.config import PlatformConfig
|
||||
from real2sim_platform.server import create_app
|
||||
from real2sim_platform.store import PlatformStore
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def test_health_and_doctor(tmp_path: Path) -> None:
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
client = TestClient(create_app(cfg))
|
||||
assert client.get("/api/health").json()["ok"] is True
|
||||
version = client.get("/api/version").json()
|
||||
assert version["features"]["media_prepare"] is True
|
||||
assert version["features"]["reconstruction_preflight"] is True
|
||||
assert "runtime_paths" in version
|
||||
assert "config_digest" in version
|
||||
doctor = client.get("/api/doctor").json()
|
||||
assert "checks" in doctor
|
||||
assert "media_prepare_deps" in doctor["checks"]
|
||||
assert "anysplat_root" in doctor["checks"]
|
||||
assert "three_bundle" in doctor["checks"]
|
||||
|
||||
|
||||
def test_dry_run_scene_task(tmp_path: Path) -> None:
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
response = client.post(
|
||||
"/api/tasks",
|
||||
json={
|
||||
"type": "scene_reconstruction",
|
||||
"params": {
|
||||
"input_path": str(ROOT / "real2sim-expert-policy/examples"),
|
||||
"dry_run": True,
|
||||
"renderer": "spark",
|
||||
"simulators": ["mujoco", "motrixsim"],
|
||||
},
|
||||
},
|
||||
)
|
||||
task = response.json()
|
||||
for _ in range(40):
|
||||
current = client.get(f"/api/tasks/{task['id']}").json()
|
||||
if current["status"] in {"succeeded", "failed"}:
|
||||
break
|
||||
time.sleep(0.1)
|
||||
assert current["status"] == "succeeded"
|
||||
artifacts = client.get(f"/api/tasks/{task['id']}/artifacts").json()
|
||||
assert any(item["name"] == "scene.json" for item in artifacts)
|
||||
scene_id = current["result"]["scene_id"]
|
||||
manifest = client.get(f"/api/scenes/{scene_id}/manifest").json()
|
||||
assert manifest["schema"] == "real2sim.scene_manifest.v1"
|
||||
assert manifest["renderer"] == "spark"
|
||||
assert all("size_bytes" in item and "autoload" in item for item in manifest["assets"])
|
||||
scene_artifact = next(item for item in artifacts if item["name"] == "scene.json")
|
||||
assert client.get(scene_artifact["url"]).status_code == 200
|
||||
|
||||
|
||||
def test_task_payload_validation(tmp_path: Path) -> None:
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
client = TestClient(create_app(cfg))
|
||||
response = client.post("/api/tasks", json={"type": "hf_import", "params": {"repo_id": "owner/name", "repo_type": "model"}})
|
||||
assert response.status_code == 400
|
||||
response = client.post("/api/tasks", json={"type": "media_prepare", "params": {}})
|
||||
assert response.status_code == 400
|
||||
response = client.post("/api/tasks", json={"type": "reconstruction_preflight", "params": {}})
|
||||
assert response.status_code == 400
|
||||
response = client.post("/api/tasks", json={"type": "scene_reconstruction", "params": {"input_path": str(ROOT), "renderer": "bad"}})
|
||||
assert response.status_code == 400
|
||||
response = client.post("/api/tasks", json={"type": "simulator_validation", "params": {"scene_path": str(ROOT), "level": "bad"}})
|
||||
assert response.status_code == 400
|
||||
response = client.post(
|
||||
"/api/tasks",
|
||||
json={
|
||||
"type": "expert_collection",
|
||||
"params": {"scene_path": str(ROOT), "policy_preset": "navigation_v1"},
|
||||
},
|
||||
)
|
||||
assert response.status_code == 400
|
||||
response = client.post(
|
||||
"/api/tasks",
|
||||
json={
|
||||
"type": "expert_collection",
|
||||
"params": {"scene_path": str(ROOT), "policy_preset": "pick_and_place_v1", "workers": 2},
|
||||
},
|
||||
)
|
||||
assert response.status_code == 400
|
||||
response = client.post(
|
||||
"/api/tasks",
|
||||
json={"type": "training", "params": {"dataset_id": "missing", "recipe": "diffusion_policy"}},
|
||||
)
|
||||
assert response.status_code == 400
|
||||
response = client.post(
|
||||
"/api/tasks",
|
||||
json={"type": "inference", "params": {"model_id": "missing", "scene_id": "missing", "workers": 2}},
|
||||
)
|
||||
assert response.status_code == 400
|
||||
|
||||
|
||||
def test_crud_metadata_and_task_params(tmp_path: Path) -> None:
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(ROOT,))
|
||||
client = TestClient(create_app(cfg))
|
||||
dataset = client.post("/api/datasets/import-path", json={"path": str(ROOT / "real2sim-expert-policy/examples"), "name": "examples"}).json()
|
||||
updated = client.patch(f"/api/datasets/{dataset['id']}", json={"name": "examples edited", "tags": ["demo", "crud"], "version": "v2"}).json()
|
||||
assert updated["name"] == "examples edited"
|
||||
assert updated["tags"] == ["demo", "crud"]
|
||||
assert updated["version"] == "v2"
|
||||
task = client.post("/api/tasks", json={"type": "renderer_benchmark", "params": {}}).json()
|
||||
edited = client.patch(f"/api/tasks/{task['id']}", json={"params": {"note": "edited before run"}}).json()
|
||||
assert edited["params"]["note"] == "edited before run"
|
||||
assert client.delete(f"/api/tasks/{task['id']}").json()["ok"] is True
|
||||
assert client.delete(f"/api/datasets/{dataset['id']}").json()["ok"] is True
|
||||
|
||||
|
||||
def test_task_progress_fields_and_bulk_actions(tmp_path: Path) -> None:
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
task = client.post("/api/tasks", json={"type": "renderer_benchmark", "params": {}}).json()
|
||||
current = _wait(client, task["id"])
|
||||
assert current["status"] == "succeeded"
|
||||
assert current["progress"] == 100
|
||||
assert current["stage"] == "已完成"
|
||||
retry = client.post("/api/tasks/bulk", json={"action": "retry", "ids": [task["id"]]}).json()
|
||||
assert retry["ok"] is True
|
||||
retry_id = retry["results"][0]["result"]["id"]
|
||||
cancel = client.post("/api/tasks/bulk", json={"action": "cancel", "ids": [retry_id]}).json()
|
||||
assert cancel["ok"] is True
|
||||
delete = client.post("/api/tasks/bulk", json={"action": "delete", "ids": [task["id"]]}).json()
|
||||
assert delete["ok"] is True
|
||||
|
||||
|
||||
def test_dataset_scene_model_bulk_interfaces(tmp_path: Path) -> None:
|
||||
package = tmp_path / "scene_package"
|
||||
package.mkdir()
|
||||
scene_file = package / "scene.json"
|
||||
scene_file.write_text('{"schema":"motrixsim.scene.v1"}\n', encoding="utf-8")
|
||||
card_dir = tmp_path / "model"
|
||||
card_dir.mkdir()
|
||||
(card_dir / "model_card.json").write_text('{"metrics":{"success_rate":0.91}}\n', encoding="utf-8")
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(ROOT, tmp_path))
|
||||
client = TestClient(create_app(cfg))
|
||||
dataset = client.post("/api/datasets/import-path", json={"path": str(ROOT / "real2sim-expert-policy/examples"), "name": "examples"}).json()
|
||||
dataset_bulk = client.post("/api/datasets/bulk", json={"action": "preflight", "ids": [dataset["id"]], "params": {"min_frames": 1}}).json()
|
||||
assert dataset_bulk["ok"] is True
|
||||
assert dataset_bulk["results"][0]["result"]["type"] == "reconstruction_preflight"
|
||||
scene = client.post("/api/scenes/import", json={"path": str(scene_file), "name": "fixture scene"}).json()
|
||||
scene_bulk = client.post("/api/scenes/bulk", json={"action": "validate", "ids": [scene["id"]], "params": {"level": "smoke"}}).json()
|
||||
assert scene_bulk["ok"] is True
|
||||
assert scene_bulk["results"][0]["result"]["type"] == "simulator_validation"
|
||||
export_bulk = client.post("/api/scenes/bulk", json={"action": "export", "ids": [scene["id"]]}).json()
|
||||
assert export_bulk["results"][0]["result"]["scene_path"].endswith("scene.json")
|
||||
rejected_training = client.post(
|
||||
"/api/tasks",
|
||||
json={"type": "training", "params": {"dataset_id": dataset["id"], "epochs": 1}},
|
||||
)
|
||||
assert rejected_training.status_code == 400
|
||||
assert "专家采集" in rejected_training.json()["detail"]
|
||||
# Directly index a model fixture for bulk model API coverage without running a trainer.
|
||||
store = PlatformStore(cfg.db_path)
|
||||
expert_dir = tmp_path / "expert_collection"
|
||||
expert_dir.mkdir()
|
||||
(expert_dir / "rollouts.jsonl").write_text(
|
||||
'{"episode":0,"step":0,"observation":[0,0,0],"action":[0,0,0]}\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
expert_dataset = store.create_dataset(
|
||||
name="expert rollouts",
|
||||
path=expert_dir,
|
||||
scan={"input_kind": "expert_rollouts", "files": 1},
|
||||
source_type="expert_collection",
|
||||
)
|
||||
model = client.post(
|
||||
"/api/tasks",
|
||||
json={"type": "training", "params": {"dataset_id": expert_dataset["id"], "epochs": 1, "recipe": "bc_fast"}},
|
||||
)
|
||||
assert model.status_code == 200, model.text
|
||||
model_entity = store.create_model(name="fixture model", path=card_dir, metadata={"metrics": {"success_rate": 0.91}}, manifest_path=card_dir / "model_card.json", metrics={"success_rate": 0.91})
|
||||
model_bulk = client.post("/api/models/bulk", json={"action": "compare", "ids": [model_entity["id"]]}).json()
|
||||
assert model_bulk["ok"] is True
|
||||
assert model_bulk["results"][0]["result"]["metrics"]["success_rate"] == 0.91
|
||||
assert model.json()["type"] == "training"
|
||||
|
||||
|
||||
def test_scene_asset_append_updates_manifest_and_artifact(tmp_path: Path) -> None:
|
||||
package = tmp_path / "scene_package"
|
||||
package.mkdir()
|
||||
scene_file = package / "scene.json"
|
||||
scene_file.write_text('{"schema":"motrixsim.scene.v1"}\n', encoding="utf-8")
|
||||
asset = package / "added_asset.json"
|
||||
asset.write_text('{"name":"fixture asset"}\n', encoding="utf-8")
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(tmp_path,))
|
||||
client = TestClient(create_app(cfg))
|
||||
scene = client.post("/api/scenes/import", json={"path": str(scene_file), "name": "editable scene"}).json()
|
||||
response = client.post(
|
||||
f"/api/scenes/{scene['id']}/assets",
|
||||
json={
|
||||
"path": str(asset),
|
||||
"name": "fixture metadata",
|
||||
"role": "simulator_import",
|
||||
"transform": {"translation": [1, 2, 3], "scale": 1.5},
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200, response.text
|
||||
payload = response.json()
|
||||
assert payload["asset"]["name"] == "fixture metadata"
|
||||
manifest = client.get(f"/api/scenes/{scene['id']}/manifest").json()
|
||||
added = [item for item in manifest["assets"] if item["name"] == "fixture metadata"]
|
||||
assert added
|
||||
assert added[0]["role"] == "simulator_import"
|
||||
assert added[0]["transform"]["translation"] == [1, 2, 3]
|
||||
assert client.get(added[0]["url"]).status_code == 200
|
||||
|
||||
|
||||
def test_scene_runtime_metadata_update(tmp_path: Path) -> None:
|
||||
store = PlatformStore(tmp_path / "state.db")
|
||||
scene_file = tmp_path / "scene.xml"
|
||||
scene_file.write_text("<mujoco/>", encoding="utf-8")
|
||||
scene = store.create_scene(
|
||||
name="runtime scene",
|
||||
path=scene_file,
|
||||
metadata={"status": "adapter_pending", "adapter_pending": True},
|
||||
tags=["motrixsim", "adapter-pending"],
|
||||
)
|
||||
updated = store.update_scene(
|
||||
scene["id"],
|
||||
metadata={
|
||||
"status": "runtime_validated",
|
||||
"adapter_pending": False,
|
||||
"runtime_validation": {"mujoco": True, "motrixsim": True},
|
||||
},
|
||||
metrics={"runtime_count": 2},
|
||||
tags=["motrixsim", "runtime-validated", "motrixsim-ready"],
|
||||
)
|
||||
assert updated["metadata"]["runtime_validation"] == {"mujoco": True, "motrixsim": True}
|
||||
assert updated["metadata"]["adapter_pending"] is False
|
||||
assert updated["metrics"]["runtime_count"] == 2
|
||||
assert "adapter-pending" not in updated["tags"]
|
||||
|
||||
|
||||
def test_media_prepare_video_task(tmp_path: Path) -> None:
|
||||
if shutil.which("ffmpeg") is None:
|
||||
pytest.skip("ffmpeg is required for media_prepare video fixture")
|
||||
video = tmp_path / "input.mp4"
|
||||
_write_fixture_video(video, tmp_path / "frames")
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
task = client.post(
|
||||
"/api/tasks",
|
||||
json={
|
||||
"type": "media_prepare",
|
||||
"params": {"input_path": str(video), "sample_fps": 2, "max_frames_per_video": 5, "max_videos": 1},
|
||||
},
|
||||
).json()
|
||||
current = _wait(client, task["id"])
|
||||
assert current["status"] == "succeeded", current
|
||||
manifest = client.get(f"/api/datasets/{current['result']['dataset_id']}/manifest").json()
|
||||
assert manifest["schema"] == "real2sim.prepared_manifest.v1"
|
||||
assert manifest["frame_count"] > 0
|
||||
assert manifest["video_frame_counts"][0]["frame_count"] > 0
|
||||
assert Path(manifest["frames_dir"]).is_dir()
|
||||
|
||||
|
||||
def test_scene_reconstruction_uses_prepared_dataset(tmp_path: Path) -> None:
|
||||
gs_root = tmp_path / "gs"
|
||||
script = gs_root / "scripts" / "real2sim_video_motrix_scene.py"
|
||||
script.parent.mkdir(parents=True)
|
||||
script.write_text(
|
||||
"import argparse, json\n"
|
||||
"from pathlib import Path\n"
|
||||
"p=argparse.ArgumentParser(); p.add_argument('--output'); p.add_argument('--work-dir'); p.add_argument('--frames-dir'); p.add_argument('--quality-preset'); p.add_argument('--sfm-report', action='store_true'); p.add_argument('--capture-acceptance', action='store_true'); p.add_argument('--pose-init-train', action='store_true'); p.add_argument('--anysplat-root'); p.add_argument('--anysplat-python'); args,_=p.parse_known_args()\n"
|
||||
"out=Path(args.output); out.mkdir(parents=True, exist_ok=True)\n"
|
||||
"(out/'scene.json').write_text(json.dumps({'schema':'fake.scene','coordinate_frame':'gs_metric_zup'}), encoding='utf-8')\n"
|
||||
"(out/'scene_mujoco.xml').write_text('<mujoco model=\"fake\"><worldbody /></mujoco>\\n', encoding='utf-8')\n"
|
||||
"(out/'motrix_import.json').write_text(json.dumps({'schema':'motrixsim.scene.v1'}), encoding='utf-8')\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
frames = tmp_path / "prepared" / "frames"
|
||||
frames.mkdir(parents=True)
|
||||
for index in range(30):
|
||||
Image.new("RGB", (640, 480), (index * 3, 48, 120)).save(frames / f"frame_{index:06d}.jpg")
|
||||
manifest = {
|
||||
"schema": "real2sim.prepared_manifest.v1",
|
||||
"frames_dir": str(frames),
|
||||
"frame_count": 30,
|
||||
"source_uri": str(tmp_path),
|
||||
}
|
||||
prepared_root = frames.parent
|
||||
(prepared_root / "prepared_manifest.json").write_text(__import__("json").dumps(manifest), encoding="utf-8")
|
||||
anysplat_root = tmp_path / "AnySplat"
|
||||
anysplat_root.mkdir()
|
||||
cfg = PlatformConfig(
|
||||
app_root=ROOT,
|
||||
workspace_dir=tmp_path / "workspace",
|
||||
gs_playground_root=gs_root,
|
||||
gsplat_python=Path(__import__("sys").executable),
|
||||
anysplat_root=anysplat_root,
|
||||
anysplat_python=Path(__import__("sys").executable),
|
||||
allowed_path_roots=(tmp_path,),
|
||||
)
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
dataset = client.post("/api/datasets/import-path", json={"path": str(prepared_root), "name": "prepared"}).json()
|
||||
task = client.post(
|
||||
"/api/tasks",
|
||||
json={"type": "scene_reconstruction", "params": {"dataset_id": dataset["id"], "dry_run": False, "renderer": "spark", "simulators": ["mujoco", "motrixsim"], "preflight_required": False}},
|
||||
).json()
|
||||
current = _wait(client, task["id"])
|
||||
assert current["status"] == "succeeded", current
|
||||
artifacts = client.get(f"/api/tasks/{task['id']}/artifacts").json()
|
||||
assert any(item["name"] == "reconstruction_report.json" for item in artifacts)
|
||||
reports = client.get(f"/api/tasks/{task['id']}/reports").json()
|
||||
assert any(item["name"] == "capture_preflight_report.json" for item in reports["reports"])
|
||||
|
||||
|
||||
def test_reconstruction_preflight_blocks_short_dataset(tmp_path: Path) -> None:
|
||||
frames = tmp_path / "prepared" / "frames"
|
||||
frames.mkdir(parents=True)
|
||||
Image.new("RGB", (320, 240), "red").save(frames / "frame_000001.jpg")
|
||||
prepared_root = frames.parent
|
||||
(prepared_root / "prepared_manifest.json").write_text(
|
||||
__import__("json").dumps({"schema": "real2sim.prepared_manifest.v1", "frames_dir": str(frames), "frame_count": 1}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", anysplat_root=tmp_path / "missing-anysplat", allowed_path_roots=(tmp_path,))
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
dataset = client.post("/api/datasets/import-path", json={"path": str(prepared_root), "name": "short"}).json()
|
||||
task = client.post("/api/tasks", json={"type": "reconstruction_preflight", "params": {"dataset_id": dataset["id"]}}).json()
|
||||
current = _wait(client, task["id"])
|
||||
assert current["status"] == "succeeded"
|
||||
assert current["result"]["ok"] is False
|
||||
assert "enough_frames" in current["result"]["report"]["failed_gates"]
|
||||
assert current["result"]["report"]["readiness_score"] < 100
|
||||
assert current["result"]["report"]["next_action"] == "run_media_prepare_or_recapture_more_frames"
|
||||
|
||||
|
||||
def test_scene_reconstruction_unprepared_media_error(tmp_path: Path) -> None:
|
||||
source = tmp_path / "hf_video_manifest"
|
||||
source.mkdir()
|
||||
(source / "0000.parquet").write_bytes(b"not a real parquet")
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
task = client.post("/api/tasks", json={"type": "scene_reconstruction", "params": {"input_path": str(source), "dry_run": False}}).json()
|
||||
current = _wait(client, task["id"])
|
||||
assert current["status"] == "failed"
|
||||
assert "media_prepare" in current["error"]
|
||||
|
||||
|
||||
def test_simulator_validation_task(tmp_path: Path) -> None:
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
scene_task = client.post(
|
||||
"/api/tasks",
|
||||
json={
|
||||
"type": "scene_reconstruction",
|
||||
"params": {
|
||||
"input_path": str(ROOT / "real2sim-expert-policy/examples"),
|
||||
"dry_run": True,
|
||||
"renderer": "auto",
|
||||
"simulators": ["mujoco", "motrixsim"],
|
||||
},
|
||||
},
|
||||
).json()
|
||||
scene_task = _wait(client, scene_task["id"])
|
||||
assert scene_task["status"] == "succeeded"
|
||||
validation = client.post(
|
||||
"/api/tasks",
|
||||
json={"type": "simulator_validation", "params": {"scene_id": scene_task["result"]["scene_id"], "simulators": ["mujoco", "motrixsim"], "level": "smoke"}},
|
||||
).json()
|
||||
validation = _wait(client, validation["id"])
|
||||
assert validation["status"] == "succeeded"
|
||||
assert validation["result"]["ok"] is True
|
||||
assert validation["result"]["report"]["level"] == "smoke"
|
||||
|
||||
|
||||
def test_simulator_validation_accepts_scene_package_layout(tmp_path: Path) -> None:
|
||||
package = tmp_path / "scene_package"
|
||||
(package / "mjcf").mkdir(parents=True)
|
||||
(package / "mjcf" / "scene.xml").write_text('<mujoco model="fixture"><worldbody /></mujoco>\n', encoding="utf-8")
|
||||
(package / "motrix_import.json").write_text('{"schema":"motrixsim.scene.v1"}\n', encoding="utf-8")
|
||||
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(tmp_path,))
|
||||
with TestClient(create_app(cfg)) as client:
|
||||
task = client.post(
|
||||
"/api/tasks",
|
||||
json={
|
||||
"type": "simulator_validation",
|
||||
"params": {"scene_path": str(package), "simulators": ["mujoco", "motrixsim"], "level": "smoke"},
|
||||
},
|
||||
).json()
|
||||
current = _wait(client, task["id"])
|
||||
assert current["status"] == "succeeded"
|
||||
report = current["result"]["report"]
|
||||
assert report["ok"] is True
|
||||
assert report["simulators"]["mujoco"]["path"].endswith("mjcf/scene.xml")
|
||||
assert report["simulators"]["motrixsim"]["path"].endswith("motrix_import.json")
|
||||
|
||||
|
||||
def test_task_reports_prefer_capture_acceptance_failure(tmp_path: Path) -> None:
|
||||
store = PlatformStore(tmp_path / "state.db")
|
||||
run_dir = tmp_path / "run"
|
||||
work_dir = run_dir / "work"
|
||||
work_dir.mkdir(parents=True)
|
||||
task = store.create_task(task_type="scene_reconstruction", params={}, run_dir=run_dir)
|
||||
store.update_task(task["id"], status="failed", error="command failed with exit code 1; see logs")
|
||||
(work_dir / "capture_acceptance_report.json").write_text(
|
||||
__import__("json").dumps(
|
||||
{
|
||||
"schema": "real2sim.capture_acceptance.v1",
|
||||
"accepted": False,
|
||||
"decision": "recapture_required_before_high_fidelity_reconstruction",
|
||||
"blockers": ["enough_registered_images", "registered_ratio"],
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
reports = collect_task_reports(store, task["id"])
|
||||
|
||||
assert reports["readable_error"].startswith("capture acceptance failed")
|
||||
assert "registered_ratio" in reports["readable_error"]
|
||||
|
||||
|
||||
def test_recover_interrupted_tasks(tmp_path: Path) -> None:
|
||||
store = PlatformStore(tmp_path / "state.db")
|
||||
task = store.create_task(task_type="renderer_benchmark", params={}, run_dir=tmp_path / "run")
|
||||
store.update_task(task["id"], status="running")
|
||||
assert store.recover_interrupted_tasks() == 1
|
||||
recovered = store.get_task(task["id"])
|
||||
assert recovered["status"] == "failed_recovered"
|
||||
|
||||
|
||||
def test_discoverse_3dgs_rgb_fallback_export(tmp_path: Path) -> None:
|
||||
source = tmp_path / "scene.ply"
|
||||
target = tmp_path / "fallback" / "scene_threejs_rgb.ply"
|
||||
report_path = tmp_path / "fallback" / "scene_threejs_rgb_report.json"
|
||||
fields = ["x", "y", "z", "f_dc_0", "f_dc_1", "f_dc_2", "opacity", "scale_0", "scale_1", "scale_2", "rot_0", "rot_1", "rot_2", "rot_3"]
|
||||
source.write_bytes(
|
||||
(
|
||||
"ply\n"
|
||||
"format binary_little_endian 1.0\n"
|
||||
"element vertex 2\n"
|
||||
+ "".join(f"property float {name}\n" for name in fields)
|
||||
+ "end_header\n"
|
||||
).encode("ascii")
|
||||
+ struct.pack("<" + "f" * len(fields), 0, 0, 0, 0.5, 0.0, -0.5, 4.0, 0.01, 0.01, 0.01, 1, 0, 0, 0)
|
||||
+ struct.pack("<" + "f" * len(fields), 1, 2, 3, -0.5, 0.2, 0.5, 4.0, 0.01, 0.01, 0.01, 1, 0, 0, 0)
|
||||
)
|
||||
|
||||
report = _export_3dgs_rgb_fallback(source, target, report_path)
|
||||
|
||||
assert report["schema"] == "real2sim.discoverse_exact_3dgs_rgb_fallback.v1"
|
||||
assert report["vertices"] == 2
|
||||
assert target.is_file()
|
||||
assert report_path.is_file()
|
||||
header = target.read_bytes().split(b"end_header\n", 1)[0].decode("ascii")
|
||||
assert "property uchar red" in header
|
||||
|
||||
|
||||
def _wait(client: TestClient, task_id: str) -> dict:
|
||||
current = {}
|
||||
for _ in range(60):
|
||||
current = client.get(f"/api/tasks/{task_id}").json()
|
||||
if current["status"] in {"succeeded", "failed", "canceled", "failed_recovered"}:
|
||||
break
|
||||
time.sleep(0.1)
|
||||
return current
|
||||
|
||||
|
||||
def _write_fixture_video(video_path: Path, frame_dir: Path) -> None:
|
||||
frame_dir.mkdir(parents=True, exist_ok=True)
|
||||
for index in range(1, 9):
|
||||
Image.new("RGB", (96, 64), (index * 20, 80, 120)).save(frame_dir / f"frame_{index:03d}.jpg")
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-y",
|
||||
"-framerate",
|
||||
"8",
|
||||
"-i",
|
||||
str(frame_dir / "frame_%03d.jpg"),
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
str(video_path),
|
||||
],
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
Loading…
x
Reference in New Issue
Block a user