chore: establish Real2Sim platform baseline

This commit is contained in:
eust-w 2026-07-10 16:32:59 +08:00
commit ad41f0fe72
46 changed files with 119363 additions and 0 deletions

21
.gitignore vendored Normal file
View File

@ -0,0 +1,21 @@
.DS_Store
.env
.env.*
!.env.example
.pytest_cache/
.ruff_cache/
.venv/
__pycache__/
*.py[cod]
*.log
config.toml
workspace/
acceptance_runs/
dist/
build/
*.egg-info/
certs/
secrets/
models/
checkpoints/

89
README.md Normal file
View File

@ -0,0 +1,89 @@
# Real2Sim Platform
管理视频/图片到 3D 场景重建、Scene Builder 场景、专家策略采集、数据集、训练和推理任务的单机平台。
## 本机启动
```bash
cd /Users/d-robotics/workSpace/real2sim2model/real2sim-platform
PYTHONPATH=src python -m real2sim_platform.server --host 127.0.0.1 --port 18126
```
打开:
```text
http://127.0.0.1:18126/
```
## 任务类型
- `hf_import`: 从 Hugging Face dataset 下载指定 revision / allow patterns,落到 `workspace/datasets/{dataset_id}` 并生成 `dataset_manifest.json`。
- `media_prepare`: 把 HuggingFace parquet/video URL、本地 video、图片目录统一准备成 `prepared_media` dataset,默认按 `sample_fps=2`、`max_frames_per_video=80` 抽帧并生成 `prepared_manifest.json`。
- `reconstruction_preflight`: 对 prepared frames / 图片目录做重建前检查,默认要求 `min_frames=30`,输出 `capture_preflight_report.json`。
- `scene_reconstruction`: 视频/图片/RGBD 到 Real2Sim 场景包,默认走 `gs_playground/scripts/real2sim_video_motrix_scene.py`,支持 `mujoco` 和 `motrixsim` 产物。
- `simulator_validation`: 按 `smoke`、`runtime_if_available` 或 `runtime_required` 校验场景包中的 MuJoCo / MotrixSim 产物,输出 `validation_report.json`。
- `expert_collection`: 调用 `real2sim_expert.collect`,输出 `rollouts.jsonl`、`manifest.json` 和可视化帧。
- `training`: 调用 `real2sim_train.train`,输出 `policy_head.json`、`model_card.json`。
- `inference`: 调用 `real2sim_infer.run`,输出 `prediction_rollout.json` 和 `viewer.html`。
- `renderer_benchmark`: 记录 Aholo Viewer 与 Spark 的集成选择,默认 Spark,Aholo 用作大规模 LOD 备选。
## 关键接口
- `GET /api/version`
- `GET /api/datasets/{id}/manifest`
- `GET /api/scenes/{id}/manifest`
- `GET /api/models/{id}/card`
- `GET /api/tasks/{id}/reports`
- `GET /api/artifacts/{id}/content`
前端固定页面:总览、验收、数据、场景、3D 预览、采集、训练、推理、任务。
3D 预览页读取 scene manifest,优先用 vendored `@sparkjsdev/spark` 加载 PLY/SPZ/SPLAT/KSPLAT/SOG,失败或不可用时明确标为 Three.js fallback;Spark 和 Three.js runtime 都随静态文件部署,不依赖浏览器访问外部 CDN。Manifest 会给资产补充 `size_bytes`、`autoload`、`viewer_hint`,超过默认阈值的大 PLY/点云不会自动加载,页面会提示手动点击加载。
## 重建运行时
无影默认运行时配置:
```text
runtime_root = /opt/real2sim_paper_runtime
anysplat_root = /opt/real2sim_paper_runtime/AnySplat
anysplat_python = /opt/real2sim_paper_runtime/envs/anysplat/bin/python
cuda_home = /usr/local/cuda-12.1
```
`scene_reconstruction` 会向上游脚本显式传入 `--anysplat-root` 和
`--anysplat-python`,并导出 `REAL2SIM_ANYSPLAT_ROOT`、
`REAL2SIM_ANY_SPLAT_PYTHON`、`REAL2SIM_COLMAP_BIN` 等环境变量。
`reconstruction_preflight` 报告包含 `readiness_score` 和 `next_action`,用于 UI 向导区分“可以重建”“需要补帧/重采”“运行时配置缺失”等状态。
## 无影部署
默认通过 SSH 隧道访问,不把服务直接暴露公网。部署脚本会同步整个
`real2sim2model` 工作区到 `/opt/real2sim2model`,并保留远端
`real2sim-platform/workspace/` 中的历史任务、数据集和模型。
```bash
bash scripts/deploy_wuying.sh --restart
ssh -i /Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519 \
-L 18126:127.0.0.1:18126 root@39.106.101.95
```
另开一个本地终端建立隧道:
```bash
ssh -i /Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519 \
-L 18126:127.0.0.1:18126 root@39.106.101.95
```
无影侧服务地址:
```text
http://127.0.0.1:18126/
```
部署脚本会安装 `huggingface_hub`、`pyarrow`、`pillow`、`imageio`、`opencv-python-headless`、`mujoco` 等依赖,写入 systemd unit,并检查 `ffmpeg`、Spark bundle、Three.js bundle 和 AnySplat runtime。`images23dgs` 缺失只作为 `/api/doctor` warning。
远端验收:
```bash
bash scripts/acceptance_wuying.sh
```

View File

@ -0,0 +1,46 @@
# Real2Sim Platform 实现细案
## 目标闭环
1. 场景:上传或导入视频、图片、RGBD 数据,创建 `scene_reconstruction` 任务,产出 3DGS/点云/mesh/collider、`scene.json`、MuJoCo/MotrixSim 可用文件和预览。
2. 数据:创建 `hf_import` 任务,从 Hugging Face dataset 下载小批量验收数据,生成 `dataset_manifest.json`。
3. 准备:创建 `media_prepare` 任务,把 HF parquet/video URL、本地 video、图片目录统一抽帧成 `prepared_media` 数据集,输出 `prepared_manifest.json` 和每个视频的帧贡献。
4. 预检:创建 `reconstruction_preflight` 任务,检查帧数、分辨率、帧间变化和 AnySplat/COLMAP/ffmpeg runtime,输出 `capture_preflight_report.json`。
5. 校验:创建 `simulator_validation` 任务,按 `smoke` / `runtime_if_available` / `runtime_required` 生成 MuJoCo/MotrixSim `validation_report.json`。
6. 采集:选择场景、机器人配置、任务配置和 episode/step 参数,创建 `expert_collection` 任务,产出专家数据集。
7. 训练:选择采集数据集,创建 `training` 任务,产出 `policy_head.json`、`model_card.json`。
8. 推理:选择场景和模型,创建 `inference` 任务,产出 `prediction_rollout.json` 和 `viewer.html`。
## 渲染器选择
- 默认:Spark。理由是它直接集成 Three.js,适合把 3DGS、Mesh、MuJoCo/MotrixSim proxy、轨迹和 UI overlay 放进同一个前端视图。平台随包部署 `@sparkjsdev/spark` 静态 ESM bundle。
- 回退:Three.js fallback。用于 Spark bundle 不存在、Spark runtime 加载失败或只是普通点云预览时,界面会明确显示 `Three.js fallback`,不再冒充 Spark;Three.js ESM、controls 和 loaders 随静态文件部署。
- 大资产策略:scene manifest 会在读取时补齐 `size_bytes`、`autoload` 和 `viewer_hint`。超过默认阈值的大 PLY/点云只显示手动加载入口,避免页面长时间停在 loading。
- 备选:Aholo Viewer。理由是它强调高性能 3DGS 和 chunked streaming LOD,更适合大场景或移动端流式加载,但需要把资产转成 Aholo 最适合的 packed/LOD 格式。
- 平台 API 保留 `renderer` 参数:`spark | aholo | auto`。首版 `auto` 在没有 Aholo LOD package 时等价于 `spark`,当 scene package 内存在 Aholo LOD 资产时切到 `aholo`。
## 仿真器支持
- `mujoco`:保留 `scene_mujoco.xml`、V-HACD/convex collision、MJCF smoke/validation artifact。
- `motrixsim`:保留 `motrix_scene.json`、`motrix_import.json`、预览 mesh/pointcloud/3DGS manifest。
- UI 中每个场景记录 `simulators: ["mujoco", "motrixsim"]`,任务不声称两者物理完全一致,分别展示各自产物和验证结果。
## API
- `POST /api/datasets/import-path`:导入服务器已有路径。
- `POST /api/datasets/upload`:上传 zip/video/RGBD。
- `POST /api/scenes/import`:导入已有 `scene.json`、`motrix_import.json` 或 `project_manifest.json`。
- `POST /api/tasks`:统一创建任务,`type` 为 `hf_import`、`media_prepare`、`reconstruction_preflight`、`scene_reconstruction`、`simulator_validation`、`expert_collection`、`training`、`inference`、`renderer_benchmark`。
- `GET /api/tasks/{id}/logs`、`GET /api/tasks/{id}/artifacts`、`GET /api/artifacts/{id}/content`:统一验收与下载。
- `GET /api/tasks/{id}/reports`:读取结构化报告索引和可读失败原因。
- `reconstruction_preflight` report 包含 `readiness_score` 和 `next_action`,任务详情页会直接展示失败门槛、关键指标和重采建议。
- `GET /api/datasets/{id}/manifest`、`GET /api/scenes/{id}/manifest`、`GET /api/models/{id}/card`:数据、场景、模型治理信息。
- `GET /api/version`:返回平台版本、构建时间、部署路径、功能开关和当前 renderer backend。
## 无影策略
- 无影作为重建、训练和部署主环境:`39.106.101.95`,root 登录,GPU、COLMAP、Real2Sim 和 gsplat Python 已验证。
- 服务默认绑定 `127.0.0.1:18126`,通过 SSH 隧道访问,避免公网暴露上传和模型产物。
- 缺失 Node 不影响后端运行;Spark bundle 在本地通过 `scripts/vendor_spark.sh` 生成,Three.js bundle 通过 `scripts/vendor_three.sh` 生成,二者都随静态文件部署。
- 无影 `.venv` 显式安装 `huggingface_hub`、`pyarrow`、`pillow`、`imageio`、`opencv-python-headless`、`mujoco`,`/api/doctor` 检查 `ffmpeg`、Spark bundle、Three.js bundle、媒体依赖、MuJoCo Python、AnySplat root/python、CUDA 和核心路径。
- `deploy_wuying.sh --restart` 写入 systemd unit 并重启服务;`acceptance_wuying.sh` 通过远端 API 跑 preflight、非 dry-run 重建和仿真校验。

View File

@ -0,0 +1,8 @@
# Real2Sim product prototypes
The prototype board in this directory is the visual contract for the POC
productization branch. It covers login, the seven product pages, the task
drawer, and the key mobile layouts.
Implementation screenshots are stored under acceptance output and are not
committed to Git.

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 MiB

38
pyproject.toml Normal file
View File

@ -0,0 +1,38 @@
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "real2sim-platform"
version = "0.1.0"
description = "Real2Sim scene, collection, training, and inference management platform"
requires-python = ">=3.10"
dependencies = [
"fastapi>=0.110",
"huggingface_hub>=0.24",
"imageio>=2.34",
"opencv-python-headless>=4.9",
"pillow>=10",
"pyarrow>=15",
"mujoco>=3.2",
"pydantic>=2",
"python-multipart>=0.0.9",
"uvicorn[standard]>=0.27",
]
[project.scripts]
real2sim-platform = "real2sim_platform.server:main"
[tool.setuptools.packages.find]
where = ["src"]
[tool.setuptools.package-data]
"real2sim_platform" = [
"static/*",
"static/vendor/*",
"static/vendor/three/*",
"static/vendor/three/addons/controls/*",
"static/vendor/three/addons/loaders/*",
"static/vendor/three/addons/postprocessing/*",
"static/vendor/three/addons/utils/*",
]

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

74
scripts/acceptance_wuying.sh Executable file
View File

@ -0,0 +1,74 @@
#!/usr/bin/env bash
set -euo pipefail
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
FIXTURE="${REAL2SIM_ACCEPTANCE_FIXTURE:-/opt/real2sim_paper_runtime/sam-3d-objects/notebook/images/shutterstock_modern_colorful_Interior_2620125197}"
MIN_FRAMES="${REAL2SIM_ACCEPTANCE_MIN_FRAMES:-30}"
TIMEOUT_SECONDS="${REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS:-3600}"
ssh -i "$KEY" -o BatchMode=yes -o StrictHostKeyChecking=accept-new "$HOST" \
"REAL2SIM_ACCEPTANCE_FIXTURE='$FIXTURE' REAL2SIM_ACCEPTANCE_MIN_FRAMES='$MIN_FRAMES' REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS='$TIMEOUT_SECONDS' python3 - <<'PY'
import json
import os
import time
import urllib.error
import urllib.request
BASE = 'http://127.0.0.1:18126'
FIXTURE = os.environ['REAL2SIM_ACCEPTANCE_FIXTURE']
MIN_FRAMES = int(os.environ['REAL2SIM_ACCEPTANCE_MIN_FRAMES'])
TIMEOUT = int(os.environ['REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS'])
def api(path, payload=None):
data = None
headers = {}
if payload is not None:
data = json.dumps(payload).encode('utf-8')
headers['Content-Type'] = 'application/json'
req = urllib.request.Request(BASE + path, data=data, headers=headers)
with urllib.request.urlopen(req, timeout=30) as response:
return json.loads(response.read().decode('utf-8'))
def create_task(task_type, params):
task = api('/api/tasks', {'type': task_type, 'params': params})
print(f'{task_type}: {task[\"id\"]}', flush=True)
return wait_task(task['id'])
def wait_task(task_id):
deadline = time.time() + TIMEOUT
while time.time() < deadline:
task = api(f'/api/tasks/{task_id}')
if task['status'] in {'succeeded', 'failed', 'canceled', 'failed_recovered'}:
print(json.dumps({'task_id': task_id, 'type': task['type'], 'status': task['status'], 'result': task.get('result'), 'error': task.get('error')}, ensure_ascii=False), flush=True)
return task
time.sleep(5)
raise SystemExit(f'timeout waiting for {task_id}')
doctor = api('/api/doctor')
version = api('/api/version')
print(json.dumps({'doctor_ok': doctor['ok'], 'warnings': doctor.get('warnings'), 'version': version.get('version'), 'build_id': version.get('build_id')}, ensure_ascii=False), flush=True)
preflight = create_task('reconstruction_preflight', {'input_path': FIXTURE, 'min_frames': MIN_FRAMES})
if not preflight.get('result', {}).get('ok'):
raise SystemExit('preflight failed; inspect reports for task ' + preflight['id'])
reconstruction = create_task('scene_reconstruction', {
'input_path': FIXTURE,
'dry_run': False,
'renderer': 'spark',
'simulators': ['mujoco', 'motrixsim'],
'min_frames': MIN_FRAMES,
'quality_preset': 'capture-high',
})
if reconstruction['status'] != 'succeeded':
raise SystemExit('scene_reconstruction failed; inspect reports for task ' + reconstruction['id'])
scene_id = reconstruction['result']['scene_id']
validation = create_task('simulator_validation', {'scene_id': scene_id, 'simulators': ['mujoco', 'motrixsim'], 'level': 'runtime_if_available'})
print(json.dumps({'scene_id': scene_id, 'validation_task': validation['id'], 'validation_ok': validation.get('result', {}).get('ok')}, ensure_ascii=False), flush=True)
PY"

View File

@ -0,0 +1,89 @@
import fs from "node:fs/promises";
import path from "node:path";
import { pathToFileURL } from "node:url";
function parseArgs(argv) {
const args = {};
for (let index = 2; index < argv.length; index += 1) {
const item = argv[index];
if (!item.startsWith("--")) continue;
const key = item.slice(2);
const next = argv[index + 1];
if (!next || next.startsWith("--")) args[key] = true;
else {
args[key] = next;
index += 1;
}
}
return args;
}
async function ensureSparkImport(vendorRoot) {
const tmpRoot = path.join("/tmp", `real2sim-spark-node-${process.pid}`);
const threeRoot = path.join(tmpRoot, "node_modules", "three");
await fs.mkdir(path.join(threeRoot, "addons", "postprocessing"), { recursive: true });
await fs.writeFile(
path.join(threeRoot, "package.json"),
JSON.stringify({ name: "three", type: "module", main: "./three.module.js", exports: { ".": "./three.module.js", "./addons/*": "./addons/*" } }),
);
await fs.copyFile(path.join(vendorRoot, "three", "three.module.js"), path.join(threeRoot, "three.module.js"));
await fs.copyFile(path.join(vendorRoot, "three", "three.core.js"), path.join(threeRoot, "three.core.js"));
await fs.copyFile(path.join(vendorRoot, "three", "addons", "postprocessing", "Pass.js"), path.join(threeRoot, "addons", "postprocessing", "Pass.js"));
await fs.copyFile(path.join(vendorRoot, "spark.module.min.js"), path.join(tmpRoot, "spark.module.min.js"));
return import(pathToFileURL(path.join(tmpRoot, "spark.module.min.js")).href);
}
async function main() {
const args = parseArgs(process.argv);
const input = args.input;
const output = args.output;
const vendorRoot = args["vendor-root"];
const reportPath = args.report;
if (!input || !output || !vendorRoot || !reportPath) {
throw new Error("usage: node convert_3dgs_ply_to_spz.mjs --input scene.ply --output scene.spz --vendor-root static/vendor --report report.json");
}
const started = Date.now();
const spark = await ensureSparkImport(vendorRoot);
const fileBytes = new Uint8Array(await fs.readFile(input));
const reader = new spark.PlyReader({ fileBytes });
await reader.parseHeader();
await reader.parseData(() => {});
const numSplats = Number(reader.numSplats || 0);
if (!numSplats) throw new Error("Spark PlyReader returned zero splats");
const writer = new spark.SpzWriter({ numSplats, shDegree: 0 });
let written = 0;
reader.parseSplats((index, x, y, z, scaleX, scaleY, scaleZ, quatX, quatY, quatZ, quatW, opacity, red, green, blue) => {
writer.setCenter(index, x, y, z);
writer.setScale(index, Math.max(scaleX, 1e-6), Math.max(scaleY, 1e-6), Math.max(scaleZ, 1e-6));
writer.setQuat(index, quatX, quatY, quatZ, quatW);
writer.setAlpha(index, Math.max(0, Math.min(1, Number(opacity ?? 1))));
writer.setRgb(index, Math.max(0, Math.min(1, red)), Math.max(0, Math.min(1, green)), Math.max(0, Math.min(1, blue)));
written += 1;
});
const encoded = await writer.finalize();
await fs.mkdir(path.dirname(output), { recursive: true });
await fs.writeFile(output, encoded);
const inputStat = await fs.stat(input);
const outputStat = await fs.stat(output);
const report = {
schema: "real2sim.spz_export_report.v1",
input,
output,
splats: numSplats,
written_splats: written,
input_size_bytes: inputStat.size,
output_size_bytes: outputStat.size,
compression_ratio: Number((outputStat.size / Math.max(inputStat.size, 1)).toFixed(4)),
elapsed_ms: Date.now() - started,
clipped_count: writer.clippedCount || 0,
};
await fs.writeFile(reportPath, `${JSON.stringify(report, null, 2)}\n`);
console.log(JSON.stringify(report));
}
main().catch((error) => {
console.error(error?.stack || error?.message || String(error));
process.exit(1);
});

127
scripts/deploy_wuying.sh Executable file
View File

@ -0,0 +1,127 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
PROJECT_ROOT="$(cd "$ROOT/.." && pwd)"
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
SSH_CONNECT_TIMEOUT="${REAL2SIM_SSH_CONNECT_TIMEOUT:-20}"
REMOTE_BASE="${REAL2SIM_REMOTE_BASE:-/opt/real2sim2model}"
REMOTE="$REMOTE_BASE/real2sim-platform"
SERVICE="real2sim-platform.service"
BUILD_TIME="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
BUILD_ID="${REAL2SIM_PLATFORM_BUILD_ID:-$BUILD_TIME}"
RUNTIME_ROOT="${REAL2SIM_RUNTIME_ROOT:-/opt/real2sim_paper_runtime}"
ANYSPLAT_ROOT="${REAL2SIM_ANYSPLAT_ROOT:-$RUNTIME_ROOT/AnySplat}"
ANYSPLAT_PYTHON="${REAL2SIM_ANY_SPLAT_PYTHON:-$RUNTIME_ROOT/envs/anysplat/bin/python}"
CUDA_HOME_REMOTE="${CUDA_HOME_REMOTE:-/usr/local/cuda-12.1}"
RESTART=0
STATUS_ONLY=0
SETUP_DESKTOP=0
for arg in "$@"; do
case "$arg" in
--restart) RESTART=1 ;;
--status) STATUS_ONLY=1 ;;
--desktop-launcher) SETUP_DESKTOP=1 ;;
*) echo "unknown argument: $arg" >&2; exit 2 ;;
esac
done
ssh_opts=(-i "$KEY" -o BatchMode=yes -o ConnectTimeout="$SSH_CONNECT_TIMEOUT" -o ServerAliveInterval=10 -o ServerAliveCountMax=2 -o StrictHostKeyChecking=accept-new)
ssh_cmd=(ssh "${ssh_opts[@]}" "$HOST")
if [[ "$STATUS_ONLY" == "1" ]]; then
"${ssh_cmd[@]}" "systemctl status '$SERVICE' --no-pager || true; curl -fsS http://127.0.0.1:18126/api/health || true; echo; curl -fsS http://127.0.0.1:18126/api/doctor || true"
exit 0
fi
if [[ ! -f "$ROOT/src/real2sim_platform/static/vendor/spark.module.min.js" ]]; then
bash "$ROOT/scripts/vendor_spark.sh"
fi
if [[ ! -f "$ROOT/src/real2sim_platform/static/vendor/three/three.module.js" ]]; then
bash "$ROOT/scripts/vendor_three.sh"
fi
"${ssh_cmd[@]}" "mkdir -p '$REMOTE_BASE'"
rsync -az --delete \
--exclude '.git/' \
--exclude 'real2sim-platform/workspace/' \
--exclude 'real2sim-platform/.venv/' \
--exclude '.pytest_cache/' \
--exclude '__pycache__/' \
-e "ssh -i $KEY -o BatchMode=yes -o ConnectTimeout=$SSH_CONNECT_TIMEOUT -o ServerAliveInterval=10 -o ServerAliveCountMax=2 -o StrictHostKeyChecking=accept-new" \
"$PROJECT_ROOT/" "$HOST:$REMOTE_BASE/"
"${ssh_cmd[@]}" "cd '$REMOTE' && python3 -m venv .venv && .venv/bin/python -m pip install -q --upgrade pip && .venv/bin/python -m pip install -q -e ."
"${ssh_cmd[@]}" "cd '$REMOTE' && .venv/bin/python - <<'PY'
import importlib.util
required = ['huggingface_hub', 'pyarrow', 'PIL', 'imageio', 'cv2', 'mujoco']
missing = [name for name in required if importlib.util.find_spec(name) is None]
if missing:
raise SystemExit('missing runtime deps: ' + ', '.join(missing))
PY
command -v ffmpeg >/dev/null
test -f '$REMOTE/src/real2sim_platform/static/vendor/spark.module.min.js'
test -f '$REMOTE/src/real2sim_platform/static/vendor/three/three.module.js'
test -d '$ANYSPLAT_ROOT'
test -x '$ANYSPLAT_PYTHON'"
"${ssh_cmd[@]}" "cat > '$REMOTE/config.toml' <<'EOF'
app_root = \"$REMOTE\"
workspace_dir = \"$REMOTE/workspace\"
host = \"127.0.0.1\"
port = 18126
default_renderer = \"spark\"
real2sim2model_root = \"$REMOTE_BASE\"
expert_policy_root = \"$REMOTE_BASE/real2sim-expert-policy\"
training_root = \"$REMOTE_BASE/real2sim-policy-training\"
inference_root = \"$REMOTE_BASE/real2sim-policy-inference\"
discoverse_root = \"/opt/gs_playground_real2sim_48q/DISCOVERSE\"
gs_playground_root = \"/opt/gs_playground_real2sim_48q\"
images23dgs_root = \"/opt/images23dgs_app/src\"
colmap_binary = \"/usr/local/bin/colmap\"
gsplat_python = \"$ANYSPLAT_PYTHON\"
runtime_root = \"$RUNTIME_ROOT\"
anysplat_root = \"$ANYSPLAT_ROOT\"
anysplat_python = \"$ANYSPLAT_PYTHON\"
cuda_home = \"$CUDA_HOME_REMOTE\"
runtime_env_script = \"/opt/gs_playground_real2sim_48q/scripts/real2sim_wuying_paper_env.sh\"
allowed_path_roots = \"$REMOTE_BASE:$REMOTE/workspace:/opt/gs_playground_real2sim_48q:$RUNTIME_ROOT:$ANYSPLAT_ROOT:/tmp\"
EOF"
"${ssh_cmd[@]}" "cat > /etc/systemd/system/$SERVICE <<'EOF'
[Unit]
Description=Real2Sim Platform
After=network.target
[Service]
Type=simple
WorkingDirectory=$REMOTE
Environment=REAL2SIM_PLATFORM_CONFIG=$REMOTE/config.toml
Environment=REAL2SIM_PLATFORM_BUILD_TIME=$BUILD_TIME
Environment=REAL2SIM_PLATFORM_BUILD_ID=$BUILD_ID
Environment=MUJOCO_GL=egl
ExecStart=$REMOTE/.venv/bin/real2sim-platform
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable '$SERVICE' >/dev/null"
if [[ "$RESTART" == "1" ]]; then
"${ssh_cmd[@]}" "systemctl stop '$SERVICE' || true; port_pid=\$(ss -ltnp 2>/dev/null | sed -n 's/.*127\\.0\\.0\\.1:18126.*pid=\\([0-9][0-9]*\\).*/\\1/p' | head -1); if [ -n \"\$port_pid\" ]; then kill \"\$port_pid\" || true; fi; sleep 2; systemctl restart '$SERVICE'; sleep 2; systemctl status '$SERVICE' --no-pager --lines=20"
fi
if [[ "$SETUP_DESKTOP" == "1" ]]; then
"${ssh_cmd[@]}" "bash '$REMOTE/scripts/setup_wuying_desktop_launcher.sh' --local --project-root '$REMOTE' --service '$SERVICE' --port 18126"
fi
echo "deployed to $HOST:$REMOTE"
echo "service: $SERVICE"
echo "health: ssh -i $KEY -L 18126:127.0.0.1:18126 $HOST"
echo "restart: bash scripts/deploy_wuying.sh --restart"
echo "desktop launcher: bash scripts/deploy_wuying.sh --restart --desktop-launcher"

View File

@ -0,0 +1,303 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { createRequire } from "node:module";
import fs from "node:fs";
import path from "node:path";
const key = process.env.REAL2SIM_WUYING_KEY || "/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519";
const host = process.env.REAL2SIM_WUYING_HOST || "root@39.106.101.95";
const remotePort = Number(process.env.REAL2SIM_REMOTE_PORT || "18126");
const requestedBase = process.env.REAL2SIM_FRONTEND_BASE_URL || "";
const outRoot = process.env.REAL2SIM_ACCEPTANCE_OUT || path.resolve("real2sim-platform", "workspace", "acceptance_runs", "frontend-local");
const timestamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..*/, "Z");
const outDir = path.join(outRoot, timestamp);
const reportPath = path.join(outDir, "frontend_acceptance_report.json");
const screenshots = [];
const failures = [];
const warnings = [];
const consoleMessages = [];
const pageChecks = [];
let tunnel = null;
fs.mkdirSync(outDir, { recursive: true });
function fail(message, context = {}) {
failures.push({ message, ...context });
console.error("FAIL:", message, context);
}
function saveReport(extra = {}) {
const report = {
schema: "real2sim.frontend_acceptance_report.v1",
started_at: timestamp,
finished_at: new Date().toISOString(),
base_url: extra.baseUrl || requestedBase,
screenshots,
console_messages: consoleMessages,
warnings,
page_checks: pageChecks,
failures,
pass: failures.length === 0,
...extra,
};
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
return report;
}
async function main() {
const { chromium } = await importPlaywright();
const baseUrl = requestedBase || await startTunnel();
const browser = await chromium.launch({ headless: true });
try {
const page = await browser.newPage({ viewport: { width: 1440, height: 900 }, deviceScaleFactor: 1 });
page.on("console", (message) => {
const entry = { type: message.type(), text: message.text() };
consoleMessages.push(entry);
if (message.type() === "error") fail("browser console error", entry);
});
page.on("pageerror", (error) => fail("browser page error", { error: error.message }));
await page.goto(baseUrl, { waitUntil: "domcontentloaded", timeout: 60000 });
await page.waitForSelector(".app-shell", { timeout: 30000 });
await neutralizeEmbeddedViewers(page);
await expectText(page, "Real2Sim", "brand");
await expectAbsentText(page, "验收", "old acceptance nav");
await expectAbsentText(page, "3D预览", "old viewer nav");
await expectAbsentText(page, "workspace", "old workspace footer");
await expectAbsentText(page, "打开云桌面", "old cloud desktop button");
const pages = [
{ tab: "overview", title: "总览", shot: "desktop-overview.png", selectors: ["#journeySteps", "#recentTaskList"] },
{ tab: "data", title: "数据管理", shot: "desktop-data.png", selectors: ["#datasetList", "#dataTaskList"] },
{ tab: "reconstruct", title: "场景重建", shot: "desktop-reconstruct.png", selectors: ["#reconstructInput", "#sceneTaskList"] },
{ tab: "scenes", title: "场景管理", shot: "", selectors: ["#sceneList", "#scenePreview"] },
{ tab: "collect", title: "数据采集", shot: "desktop-collect.png", selectors: ["#collectScene", "#collectTaskList"] },
{ tab: "train", title: "模型训练", shot: "desktop-train.png", selectors: ["#trainDataset", "#trainTaskList", "#modelList"] },
{ tab: "infer", title: "在线推理", shot: "desktop-infer.png", selectors: ["#inferModel", "#inferScene", "#inferTaskList"] },
];
for (const item of pages) {
await openTab(page, item.tab);
await expectText(page, item.title, `${item.tab} title`);
for (const selector of item.selectors) await expectSelector(page, selector, `${item.tab} ${selector}`);
await assertNoOverflow(page, `desktop ${item.tab}`);
if (item.shot) await screenshot(page, item.shot, item.shotOptions || {});
else warnings.push(`${item.tab} screenshot skipped; page contains embedded WebGL iframe that can hang headless screenshot capture`);
pageChecks.push({ tab: item.tab, viewport: "desktop", ok: true });
}
await openTab(page, "reconstruct");
const taskCount = await page.locator("#sceneTaskList [data-task]").count();
if (taskCount === 0) {
warnings.push("no scene tasks visible for drawer/report validation");
} else {
await page.locator("#sceneTaskList [data-task]").first().click({ timeout: 10000 });
await page.waitForTimeout(1500);
await expectSelector(page, "#taskDrawer[aria-hidden=\"false\"]", "task drawer opened");
await expectSelector(page, "#drawerBody", "task drawer body");
await screenshot(page, "desktop-task-drawer.png");
}
await validateSceneViewerManifestGate(baseUrl);
warnings.push("live scene viewer probe is excluded from the main frontend gate; SPZ manifest/autoload is validated here and live WebGL should be checked with the dedicated viewer probe.");
await page.setViewportSize({ width: 390, height: 900 });
await neutralizeEmbeddedViewers(page);
for (const item of [pages[0], pages[1], pages[2], pages[3]]) {
await openTab(page, item.tab);
await expectText(page, item.title, `mobile ${item.tab} title`);
await assertNoOverflow(page, `mobile ${item.tab}`);
if (item.shot) await screenshot(page, item.shot.replace("desktop", "mobile"), item.shotOptions || {});
else warnings.push(`mobile ${item.tab} screenshot skipped; page contains embedded WebGL iframe that can hang headless screenshot capture`);
pageChecks.push({ tab: item.tab, viewport: "mobile", ok: true });
}
const report = saveReport({ baseUrl });
console.log(JSON.stringify({ pass: report.pass, report: reportPath, screenshots }, null, 2));
process.exitCode = report.pass ? 0 : 1;
} finally {
await browser.close();
if (tunnel) tunnel.kill("SIGTERM");
}
}
async function importPlaywright() {
const moduleDir = process.env.REAL2SIM_PLAYWRIGHT_NODE_MODULES || "";
if (moduleDir) {
const require = createRequire(path.join(moduleDir, "real2sim-playwright-loader.cjs"));
return require("playwright");
}
try {
return await import("playwright");
} catch (error) {
throw new Error(`playwright is required. Install it locally with: npm install playwright, or set REAL2SIM_PLAYWRIGHT_NODE_MODULES=/path/to/node_modules. Original error: ${error.message}`);
}
}
async function startTunnel() {
const port = Number(process.env.REAL2SIM_LOCAL_TUNNEL_PORT || await findPort(18130));
tunnel = spawn("ssh", [
"-i", key,
"-o", "BatchMode=yes",
"-o", "StrictHostKeyChecking=accept-new",
"-N",
"-L", `${port}:127.0.0.1:${remotePort}`,
host,
], { stdio: ["ignore", "pipe", "pipe"] });
tunnel.stderr.on("data", (chunk) => process.stderr.write(chunk));
tunnel.stdout.on("data", (chunk) => process.stdout.write(chunk));
const baseUrl = `http://127.0.0.1:${port}`;
const deadline = Date.now() + 30000;
while (Date.now() < deadline) {
try {
await fetchJson(baseUrl, "/api/health");
return baseUrl;
} catch {
await sleep(500);
}
}
throw new Error(`SSH tunnel did not become ready at ${baseUrl}`);
}
async function findPort(start) {
const net = await import("node:net");
for (let port = start; port < start + 100; port += 1) {
const ok = await new Promise((resolve) => {
const server = net.createServer();
server.once("error", () => resolve(false));
server.once("listening", () => server.close(() => resolve(true)));
server.listen(port, "127.0.0.1");
});
if (ok) return port;
}
throw new Error("no local tunnel port available");
}
async function fetchJson(baseUrl, apiPath) {
const response = await fetch(`${baseUrl}${apiPath}`);
if (!response.ok) throw new Error(`${apiPath} failed with ${response.status}: ${await response.text()}`);
return response.json();
}
async function openTab(page, tab) {
await page.evaluate((targetTab) => {
const button = document.querySelector(`[data-tab="${targetTab}"]`);
if (!button) throw new Error(`tab not found: ${targetTab}`);
button.click();
}, tab);
await page.waitForFunction((targetTab) => document.getElementById(targetTab)?.classList.contains("active"), tab, { timeout: 10000 });
}
async function neutralizeEmbeddedViewers(page) {
await page.evaluate(() => {
document.querySelectorAll("iframe").forEach((iframe) => {
iframe.dataset.acceptanceSrc = iframe.getAttribute("src") || "";
iframe.setAttribute("src", "about:blank");
});
}).catch(() => {});
}
async function assertNoOverflow(page, label) {
const metrics = await page.evaluate(() => ({
scrollWidth: document.documentElement.scrollWidth,
clientWidth: document.documentElement.clientWidth,
bodyScrollWidth: document.body.scrollWidth,
bodyClientWidth: document.body.clientWidth,
}));
if (metrics.scrollWidth > metrics.clientWidth + 2 || metrics.bodyScrollWidth > metrics.bodyClientWidth + 2) {
fail("horizontal overflow", { label, metrics });
}
}
async function expectText(page, text, label) {
const visible = await page.getByText(text, { exact: false }).first().isVisible().catch(() => false);
if (!visible) fail("expected text not visible", { label, text });
}
async function expectAbsentText(page, text, label) {
const visible = await page.getByText(text, { exact: false }).first().isVisible().catch(() => false);
if (visible) fail("legacy/debug text should not be visible", { label, text });
}
async function expectSelector(page, selector, label) {
const count = await page.locator(selector).count().catch(() => 0);
if (count <= 0) fail("expected selector not found", { label, selector });
}
async function pickPreviewableScene(baseUrl) {
const scenes = await fetchJson(baseUrl, "/api/scenes");
if (!Array.isArray(scenes) || scenes.length === 0) return null;
const scored = [];
for (const scene of scenes) {
try {
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(scene.id)}/manifest`);
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
let score = 0;
if (assets.some((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")))) score += 100;
if (assets.some((asset) => /pose_init_trained_scene\.ply$/i.test(String(asset.name || asset.path || "")))) score += 50;
if (assets.some((asset) => /\.(ply|glb|gltf|obj)$/i.test(String(asset.name || asset.path || "")))) score += 20;
scored.push({ scene, score });
} catch (error) {
warnings.push(`scene manifest unavailable for ${scene.id}: ${error.message}`);
}
}
scored.sort((left, right) => right.score - left.score);
return scored[0]?.scene || scenes[0];
}
async function validateSceneViewerManifestGate(baseUrl) {
const scene = await pickPreviewableScene(baseUrl);
if (!scene) {
fail("no scenes available for 3D viewer manifest validation");
return;
}
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(scene.id)}/manifest`);
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
const optimized = assets.find((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")));
if (!optimized) {
fail("scene manifest has no optimized splat asset for browser preview", { scene_id: scene.id });
return;
}
if (!optimized.url) fail("optimized splat asset has no URL", { scene_id: scene.id, asset: optimized });
if (optimized.autoload === false) fail("optimized splat asset is not marked autoload", { scene_id: scene.id, asset: optimized });
pageChecks.push({ tab: "scene-viewer", viewport: "desktop", ok: true, asset: optimized.name || optimized.path, size: optimized.size_label || optimized.size_bytes || "" });
}
async function screenshot(page, name, options = {}) {
const target = path.join(outDir, name);
const timeoutMs = Number(options.timeoutMs || 20000);
let styleHandle = null;
try {
if (options.hideIframes) {
styleHandle = await page.evaluateHandle(() => {
const style = document.createElement("style");
style.dataset.acceptanceScreenshot = "true";
style.textContent = "iframe, canvas { visibility: hidden !important; }";
document.head.appendChild(style);
return style;
});
await page.waitForTimeout(150);
}
await page.screenshot({ path: target, fullPage: options.fullPage ?? true, timeout: timeoutMs });
screenshots.push(target);
} catch (error) {
const message = `screenshot skipped for ${name}: ${error.message}`;
warnings.push(message);
} finally {
if (styleHandle) {
await page.evaluate(() => document.querySelector("[data-acceptance-screenshot='true']")?.remove()).catch(() => {});
await styleHandle.dispose().catch(() => {});
}
}
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
main().catch((error) => {
fail("frontend acceptance runner crashed", { error: error.message, stack: error.stack });
const report = saveReport();
console.error(JSON.stringify({ pass: false, report: reportPath, error: error.message }, null, 2));
if (tunnel) tunnel.kill("SIGTERM");
process.exit(1);
});

View File

@ -0,0 +1,254 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import math
import sys
import time
import xml.etree.ElementTree as ET
from pathlib import Path
from typing import Any
import numpy as np
import trimesh
SCRIPT_DIR = Path(__file__).resolve().parent
APP_ROOT = SCRIPT_DIR.parent
SRC_ROOT = APP_ROOT / "src"
if str(SRC_ROOT) not in sys.path:
sys.path.insert(0, str(SRC_ROOT))
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
from real2sim_platform.store import PlatformStore # noqa: E402
SCENE_ID = "1f2618bdfcac4e45456569980716405a"
PACKAGE_SLUG = "parlour_rm65_pick_place"
def main() -> int:
parser = argparse.ArgumentParser(description="Generate a browser GLB preview from the MotrixSim parlour + RM65 scaffold package.")
parser.add_argument("--scene-id", default=SCENE_ID)
parser.add_argument("--max-parlour-meshes", type=int, default=240, help="Limit parlour meshes to keep the browser preview responsive.")
parser.add_argument("--skip-rm65", action="store_true")
args = parser.parse_args()
config = load_config()
ensure_workspace(config)
store = PlatformStore(config.db_path)
scene = store.get_scene(args.scene_id)
package_dir = Path(str((scene.get("metadata") or {}).get("package") or config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG))
manifest_path = Path(str(scene.get("manifest_path") or package_dir / "motrixsim_scene_manifest.json"))
if not manifest_path.is_file():
raise SystemExit(f"manifest not found: {manifest_path}")
output_dir = package_dir / "viewer_assets"
output_dir.mkdir(parents=True, exist_ok=True)
output_path = output_dir / "motrixsim_parlour_rm65_preview.glb"
report_path = output_dir / "motrixsim_preview_report.json"
scene_glb = trimesh.Scene()
report: dict[str, Any] = {
"schema": "real2sim.motrixsim_preview_report.v1",
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"inputs": [],
"warnings": [],
}
parlour_xml = package_dir / "assets/parlour/parlour.xml"
parlour_stats = add_mjcf_visuals(scene_glb, parlour_xml, max_meshes=args.max_parlour_meshes, name_prefix="parlour")
report["inputs"].append({"role": "parlour", "xml": str(parlour_xml), **parlour_stats})
if not args.skip_rm65:
rm65_xml = package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml"
rm65_base = translation_matrix(np.array([0.8, 0.0, 0.55]))
rm65_stats = add_mjcf_visuals(scene_glb, rm65_xml, root_transform=rm65_base, max_meshes=80, name_prefix="rm65")
report["inputs"].append({"role": "rm65", "xml": str(rm65_xml), **rm65_stats})
if not scene_glb.geometry:
raise SystemExit("no mesh geometry was loaded from MotrixSim package")
data = scene_glb.export(file_type="glb")
output_path.write_bytes(data)
report["output"] = {"path": str(output_path), "size_bytes": output_path.stat().st_size, "geometry_count": len(scene_glb.geometry)}
report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
artifact = store.add_artifact(task_id=None, name=output_path.name, path=output_path, kind="mesh")
report_artifact = store.add_artifact(task_id=None, name=report_path.name, path=report_path, kind="json")
update_manifest(manifest_path, output_path, artifact["id"], report_path, report_artifact["id"], report)
print(json.dumps({"ok": True, "scene_id": args.scene_id, "glb": str(output_path), "report": str(report_path), "geometry_count": len(scene_glb.geometry), "size_bytes": output_path.stat().st_size}, ensure_ascii=False, indent=2))
return 0
def add_mjcf_visuals(
scene: trimesh.Scene,
xml_path: Path,
*,
root_transform: np.ndarray | None = None,
max_meshes: int,
name_prefix: str,
) -> dict[str, Any]:
root = ET.parse(xml_path).getroot()
asset_dir = xml_path.parent / (root.find("compiler").attrib.get("assetdir", "") if root.find("compiler") is not None else "")
mesh_defs = {
item.attrib["name"]: asset_dir / item.attrib["file"]
for item in root.findall("./asset/mesh")
if item.attrib.get("name") and item.attrib.get("file")
}
stats = {"mesh_defs": len(mesh_defs), "loaded": 0, "missing": 0, "skipped": 0}
world = root.find("worldbody")
if world is None:
return stats
root_matrix = root_transform if root_transform is not None else np.eye(4)
for node in list(world):
traverse_mjcf_node(scene, node, mesh_defs, root_matrix, stats, max_meshes=max_meshes, name_prefix=name_prefix)
return stats
def traverse_mjcf_node(
scene: trimesh.Scene,
node: ET.Element,
mesh_defs: dict[str, Path],
parent_transform: np.ndarray,
stats: dict[str, int],
*,
max_meshes: int,
name_prefix: str,
) -> None:
current = parent_transform
if node.tag == "body":
current = parent_transform @ node_transform(node.attrib)
elif node.tag == "geom":
mesh_name = node.attrib.get("mesh")
geom_class = node.attrib.get("class", "")
if mesh_name and mesh_name in mesh_defs and (geom_class != "collision" or node.attrib.get("group") == "2"):
if stats["loaded"] >= max_meshes:
stats["skipped"] += 1
else:
mesh_path = mesh_defs[mesh_name]
if mesh_path.is_file():
try:
mesh = load_mesh(mesh_path)
mesh.apply_transform(parent_transform @ node_transform(node.attrib))
scene.add_geometry(mesh, geom_name=f"{name_prefix}_{mesh_name}_{stats['loaded']}")
stats["loaded"] += 1
except Exception:
stats["skipped"] += 1
else:
stats["missing"] += 1
for child in list(node):
traverse_mjcf_node(scene, child, mesh_defs, current, stats, max_meshes=max_meshes, name_prefix=name_prefix)
def load_mesh(path: Path) -> trimesh.Trimesh:
loaded = trimesh.load(path, force="mesh", process=False)
if isinstance(loaded, trimesh.Scene):
return trimesh.util.concatenate(tuple(loaded.geometry.values()))
return loaded
def node_transform(attrs: dict[str, str]) -> np.ndarray:
transform = np.eye(4)
if "pos" in attrs:
transform = transform @ translation_matrix(parse_vec(attrs["pos"], 3))
if "quat" in attrs:
transform = transform @ quat_matrix(parse_vec(attrs["quat"], 4))
elif "euler" in attrs:
transform = transform @ euler_matrix(parse_vec(attrs["euler"], 3))
if "scale" in attrs:
scale = parse_vec(attrs["scale"], 3)
transform = transform @ np.diag([scale[0], scale[1], scale[2], 1.0])
return transform
def parse_vec(value: str, count: int) -> np.ndarray:
parts = [float(item) for item in value.split()]
if len(parts) == 1 and count == 3:
parts = parts * 3
while len(parts) < count:
parts.append(0.0)
return np.array(parts[:count], dtype=float)
def translation_matrix(vec: np.ndarray) -> np.ndarray:
matrix = np.eye(4)
matrix[:3, 3] = vec[:3]
return matrix
def quat_matrix(q: np.ndarray) -> np.ndarray:
w, x, y, z = q
norm = math.sqrt(w * w + x * x + y * y + z * z)
if norm <= 0:
return np.eye(4)
w, x, y, z = w / norm, x / norm, y / norm, z / norm
matrix = np.eye(4)
matrix[:3, :3] = np.array(
[
[1 - 2 * y * y - 2 * z * z, 2 * x * y - 2 * z * w, 2 * x * z + 2 * y * w],
[2 * x * y + 2 * z * w, 1 - 2 * x * x - 2 * z * z, 2 * y * z - 2 * x * w],
[2 * x * z - 2 * y * w, 2 * y * z + 2 * x * w, 1 - 2 * x * x - 2 * y * y],
]
)
return matrix
def euler_matrix(euler: np.ndarray) -> np.ndarray:
x, y, z = euler
cx, cy, cz = math.cos(x), math.cos(y), math.cos(z)
sx, sy, sz = math.sin(x), math.sin(y), math.sin(z)
rx = np.array([[1, 0, 0, 0], [0, cx, -sx, 0], [0, sx, cx, 0], [0, 0, 0, 1]])
ry = np.array([[cy, 0, sy, 0], [0, 1, 0, 0], [-sy, 0, cy, 0], [0, 0, 0, 1]])
rz = np.array([[cz, -sz, 0, 0], [sz, cz, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]])
return rz @ ry @ rx
def update_manifest(manifest_path: Path, glb_path: Path, artifact_id: str, report_path: Path, report_artifact_id: str, report: dict[str, Any]) -> None:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and item.get("name") == glb_path.name)]
glb_asset = {
"name": glb_path.name,
"role": "viewer_preview",
"kind": "mesh",
"path": str(glb_path),
"relative_path": str(glb_path.relative_to(manifest_path.parent)),
"url": f"/api/artifacts/{artifact_id}/content",
"exists": True,
"size_bytes": glb_path.stat().st_size,
"size_label": format_bytes(glb_path.stat().st_size),
"autoload": True,
"generated": True,
}
report_asset = {
"name": report_path.name,
"role": "preview_report",
"kind": "metadata",
"path": str(report_path),
"relative_path": str(report_path.relative_to(manifest_path.parent)),
"url": f"/api/artifacts/{report_artifact_id}/content",
"exists": True,
"size_bytes": report_path.stat().st_size,
"size_label": format_bytes(report_path.stat().st_size),
"generated": True,
}
manifest["assets"] = [glb_asset, report_asset, *assets]
manifest["viewer_assets"] = {"primary": glb_asset, "report": report_asset, "generator": "generate_motrixsim_preview.py"}
manifest["preview_generation"] = report
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
def format_bytes(value: int) -> str:
units = ["B", "KB", "MB", "GB"]
size = float(value)
for unit in units:
if size < 1024 or unit == units[-1]:
return f"{size:.1f} {unit}" if unit != "B" else f"{int(size)} B"
size /= 1024
return f"{value} B"
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -0,0 +1,287 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import os
import subprocess
import sys
import time
from pathlib import Path
from typing import Any
SCRIPT_DIR = Path(__file__).resolve().parent
APP_ROOT = SCRIPT_DIR.parent
SRC_ROOT = APP_ROOT / "src"
if str(SRC_ROOT) not in sys.path:
sys.path.insert(0, str(SRC_ROOT))
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
from real2sim_platform.store import PlatformStore # noqa: E402
SCENE_ID = "1f2618bdfcac4e45456569980716405a"
PACKAGE_SLUG = "parlour_rm65_pick_place"
def main() -> int:
parser = argparse.ArgumentParser(description="Generate a runnable RM65 + coffee-table pick-place smoke MJCF for the MotrixSim asset package.")
parser.add_argument("--scene-id", default=SCENE_ID)
parser.add_argument("--steps", type=int, default=120)
parser.add_argument("--motrixsim-python", default="")
args = parser.parse_args()
config = load_config()
ensure_workspace(config)
store = PlatformStore(config.db_path)
scene = store.get_scene(args.scene_id)
package_dir = Path(str((scene.get("metadata") or {}).get("package") or config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG))
manifest_path = Path(str(scene.get("manifest_path") or package_dir / "motrixsim_scene_manifest.json"))
sim_dir = package_dir / "simulator_assets"
sim_dir.mkdir(parents=True, exist_ok=True)
runtime_rm65 = sim_dir / "rmjx_rm65_runtime.xml"
source_rm65 = package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml"
runtime_xml = source_rm65.read_text(encoding="utf-8").replace(
'assetdir="assets"',
'assetdir="../assets/rm65_open_cabinet/assets"',
)
# The upstream pad capsules are visual-only diagnostics. Enable collision on
# the runtime copy so pick-place validation measures real bilateral contact.
runtime_xml = runtime_xml.replace(
'contype="0" conaffinity="0" density="0" group="2"/>',
'contype="1" conaffinity="1" density="0" friction="1.2 0.01 0.001" group="3"/>',
)
runtime_rm65.write_text(runtime_xml, encoding="utf-8")
combined = sim_dir / "rm65_table_pick_place.xml"
combined.write_text(runtime_scene_xml(), encoding="utf-8")
mujoco_report = validate_mujoco(combined, steps=args.steps)
motrixsim_python = find_motrixsim_python(args.motrixsim_python)
motrixsim_report = validate_motrixsim(combined, steps=args.steps, python=motrixsim_python)
report = {
"schema": "real2sim.motrixsim_runtime_scene_validation.v2",
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"combined_mjcf": str(combined),
"steps_requested": args.steps,
"ok": bool(mujoco_report.get("ok") and motrixsim_report.get("ok")),
"runtimes": {"mujoco": mujoco_report, "motrixsim": motrixsim_report},
}
report_path = sim_dir / "rm65_table_pick_place_validation.json"
report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
scene_artifact = store.add_artifact(task_id=None, name=combined.name, path=combined, kind="simulator")
rm65_artifact = store.add_artifact(task_id=None, name=runtime_rm65.name, path=runtime_rm65, kind="simulator")
report_artifact = store.add_artifact(task_id=None, name=report_path.name, path=report_path, kind="json")
update_manifest(manifest_path, combined, scene_artifact["id"], runtime_rm65, rm65_artifact["id"], report_path, report_artifact["id"], report)
runtime_ready = bool(report["ok"])
metadata = dict(scene.get("metadata") or {})
metadata.update(
{
"status": "runtime_validated" if runtime_ready else "runtime_validation_failed",
"adapter_pending": not runtime_ready,
"runtime_validation": {"mujoco": bool(mujoco_report.get("ok")), "motrixsim": bool(motrixsim_report.get("ok"))},
}
)
tags = [tag for tag in scene.get("tags", []) if tag not in {"adapter-pending", "runtime-validated", "motrixsim-ready"}]
if runtime_ready:
tags.extend(["runtime-validated", "motrixsim-ready"])
else:
tags.append("adapter-pending")
store.update_scene(args.scene_id, metadata=metadata, tags=tags)
print(json.dumps({"ok": report["ok"], "scene_id": args.scene_id, "combined_mjcf": str(combined), "report": str(report_path), "validation": report}, ensure_ascii=False, indent=2))
return 0 if report["ok"] else 1
def runtime_scene_xml() -> str:
return """<mujoco model="rm65 table pick place smoke">
<include file="rmjx_rm65_runtime.xml"/>
<visual>
<global offwidth="960" offheight="540"/>
</visual>
<worldbody>
<body name="coffee_table" pos="0.65 0 0.25">
<geom name="coffee_table_top" type="box" size="0.45 0.32 0.03" rgba="0.45 0.30 0.18 1" friction="0.8 0.01 0.001"/>
<geom name="coffee_table_leg_1" type="box" pos="0.36 0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
<geom name="coffee_table_leg_2" type="box" pos="-0.36 0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
<geom name="coffee_table_leg_3" type="box" pos="0.36 -0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
<geom name="coffee_table_leg_4" type="box" pos="-0.36 -0.25 -0.14" size="0.025 0.025 0.14" rgba="0.20 0.13 0.08 1"/>
</body>
<body name="pick_cube" pos="0.65 0 0.33">
<freejoint name="pick_cube_free"/>
<geom name="pick_cube_geom" type="box" size="0.03 0.03 0.03" rgba="0.9 0.2 0.12 1" mass="0.05" friction="0.9 0.01 0.001"/>
</body>
<body name="place_target" pos="0.43 0.18 0.335">
<geom name="place_target_marker" type="cylinder" size="0.055 0.004" rgba="0.1 0.8 0.6 0.35" contype="0" conaffinity="0"/>
</body>
<camera name="overview" pos="1.7 -1.6 1.25" xyaxes="0.68 0.73 0 -0.37 0.34 0.86"/>
</worldbody>
<keyframe>
<key name="home" qpos="0 0 0 -1.57079 0 1.57079 -0.7853 0 0 0 0 0 0.65 0 0.33 1 0 0 0" ctrl="0 0 0 -1.57079 0 1.57079 -0.7853"/>
</keyframe>
</mujoco>
"""
def validate_mujoco(path: Path, *, steps: int) -> dict[str, Any]:
started = time.time()
report: dict[str, Any] = {
"schema": "real2sim.motrixsim_runtime_scene_validation.v1",
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"combined_mjcf": str(path),
"steps_requested": steps,
"ok": False,
}
try:
import mujoco
model = mujoco.MjModel.from_xml_path(str(path))
data = mujoco.MjData(model)
for _ in range(max(0, steps)):
mujoco.mj_step(model, data)
report.update(
{
"ok": True,
"runtime": "mujoco",
"nq": int(model.nq),
"nv": int(model.nv),
"nbody": int(model.nbody),
"ngeom": int(model.ngeom),
"nkey": int(model.nkey),
"sim_time": float(data.time),
"elapsed_ms": round((time.time() - started) * 1000),
}
)
except Exception as exc:
report.update({"ok": False, "error": f"{type(exc).__name__}: {exc}", "elapsed_ms": round((time.time() - started) * 1000)})
return report
def find_motrixsim_python(explicit: str) -> Path | None:
candidates = [
explicit,
os.environ.get("REAL2SIM_MOTRIXSIM_PYTHON", ""),
"/Users/d-robotics/workSpace/motrixsim-docs/.venv/bin/python",
"/opt/Motphys/motrixsim-docs/.venv/bin/python",
]
for candidate in candidates:
if candidate and Path(candidate).is_file():
return Path(candidate)
return None
def validate_motrixsim(path: Path, *, steps: int, python: Path | None) -> dict[str, Any]:
started = time.time()
report: dict[str, Any] = {"ok": False, "runtime": "motrixsim", "python": str(python or "")}
if python is None:
report.update({"status": "runtime_unavailable", "error": "motrixsim python environment was not found"})
return report
probe = """
import json, sys
from motrixsim import SceneData, load_model, step
model = load_model(sys.argv[1])
data = SceneData(model)
for _ in range(int(sys.argv[2])):
step(model, data)
print("REAL2SIM_RESULT=" + json.dumps({
"timestep": float(model.options.timestep),
"nq": len(data.low.dof_positions),
"nv": len(data.low.dof_velocities),
}))
"""
try:
completed = subprocess.run(
[str(python), "-c", probe, str(path), str(steps)],
check=True,
capture_output=True,
text=True,
timeout=120,
)
result_line = next(line for line in completed.stdout.splitlines() if line.startswith("REAL2SIM_RESULT="))
result = json.loads(result_line.split("=", 1)[1])
report.update(
{
"ok": True,
"status": "loaded_and_stepped",
"steps": steps,
**result,
"elapsed_ms": round((time.time() - started) * 1000),
}
)
except Exception as exc:
stderr = getattr(exc, "stderr", "") or ""
report.update(
{
"ok": False,
"status": "runtime_failed",
"error": f"{type(exc).__name__}: {exc}",
"stderr": stderr[-2000:],
"elapsed_ms": round((time.time() - started) * 1000),
}
)
return report
def update_manifest(
manifest_path: Path,
combined: Path,
combined_artifact_id: str,
runtime_rm65: Path,
runtime_rm65_artifact_id: str,
report_path: Path,
report_artifact_id: str,
report: dict[str, Any],
) -> None:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
generated_names = {combined.name, runtime_rm65.name, report_path.name}
assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and item.get("name") in generated_names)]
generated = [
asset_entry(combined, combined_artifact_id, "runtime_scene", "simulator", "可运行 RM65 茶几 pick-place smoke MJCF", manifest_path),
asset_entry(runtime_rm65, runtime_rm65_artifact_id, "runtime_robot", "simulator", "RM65 runtime assetdir 修正版", manifest_path),
asset_entry(report_path, report_artifact_id, "runtime_validation_report", "metadata", "运行时加载校验报告", manifest_path),
]
manifest["assets"] = [*generated, *assets]
manifest["runtime_scene"] = {
"combined_mjcf": generated[0],
"runtime_rm65": generated[1],
"validation_report": generated[2],
"validation": report,
"status": "mujoco_and_motrixsim_smoke_passed" if report.get("ok") else "failed",
"scope": "combined_rm65_table_scene_runtime_load_and_step",
}
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
def asset_entry(path: Path, artifact_id: str, role: str, kind: str, name: str, manifest_path: Path) -> dict[str, Any]:
size = path.stat().st_size
return {
"name": path.name,
"display_name": name,
"role": role,
"kind": kind,
"path": str(path),
"relative_path": str(path.relative_to(manifest_path.parent)),
"url": f"/api/artifacts/{artifact_id}/content",
"exists": True,
"size_bytes": size,
"size_label": format_bytes(size),
"generated": True,
}
def format_bytes(value: int) -> str:
units = ["B", "KB", "MB", "GB"]
size = float(value)
for unit in units:
if size < 1024 or unit == units[-1]:
return f"{size:.1f} {unit}" if unit != "B" else f"{int(size)} B"
size /= 1024
return f"{value} B"
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -0,0 +1,328 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import hashlib
import json
import shutil
import subprocess
import sys
import time
from pathlib import Path
from typing import Any
SCRIPT_DIR = Path(__file__).resolve().parent
APP_ROOT = SCRIPT_DIR.parent
SRC_ROOT = APP_ROOT / "src"
if str(SRC_ROOT) not in sys.path:
sys.path.insert(0, str(SRC_ROOT))
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
from real2sim_platform.store import PlatformStore # noqa: E402
DEFAULT_SOURCE = Path("/Users/d-robotics/workSpace/motrixsim-docs")
PACKAGE_SLUG = "parlour_rm65_pick_place"
SCENE_ID = hashlib.sha1(f"motrixsim:{PACKAGE_SLUG}".encode("utf-8")).hexdigest()[:32]
REQUIRED_FILES = {
"parlour_scene": Path("examples/assets/parlour/parlour.xml"),
"rm65_robot": Path("examples/assets/rm65_open_cabinet/mjx_rm65.xml"),
"rm65_reference_scene": Path("examples/assets/rm65_open_cabinet/scene.xml"),
"rm65_reference_controller": Path("examples/control/rm65_open_cabinet.py"),
"pick_place_template": Path("examples/assets/franka_emika_panda/scene_pick_cube.xml"),
"parlour_locomotion_example": Path("examples/control/robot_locomotion.py"),
}
DIRECTORIES_TO_COPY = {
"parlour": Path("examples/assets/parlour"),
"rm65_open_cabinet": Path("examples/assets/rm65_open_cabinet"),
"franka_emika_panda": Path("examples/assets/franka_emika_panda"),
}
def main() -> int:
parser = argparse.ArgumentParser(description="Import MotrixSim parlour + RM65 pick-place scaffold into Real2Sim Platform.")
parser.add_argument("--source", type=Path, default=DEFAULT_SOURCE, help="Local MotrixSim docs repository path.")
parser.add_argument("--name", default="MotrixSim 客厅 RM65 Pick-Place", help="Scene name shown in Real2Sim.")
parser.add_argument("--replace-index", action="store_true", help="Replace the existing deterministic scene index row.")
parser.add_argument("--skip-copy", action="store_true", help="Only refresh manifest/index using existing package files.")
parser.add_argument("--allow-missing-source", action="store_true", help="Allow registering an already-copied package when the MotrixSim repo is not present.")
args = parser.parse_args()
source = args.source.expanduser().resolve()
config = load_config()
ensure_workspace(config)
store = PlatformStore(config.db_path)
package_dir = config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG
if source.is_dir():
_assert_source(source)
source_commit = _git_commit(source)
elif args.skip_copy and args.allow_missing_source:
source_commit = _existing_source_commit(package_dir) or ""
else:
raise SystemExit(f"MotrixSim docs repo not found: {source}")
if not args.skip_copy:
_copy_package(source, package_dir)
_assert_package(package_dir)
scene_descriptor = package_dir / f"{PACKAGE_SLUG}.scene.json"
manifest_path = package_dir / "motrixsim_scene_manifest.json"
scene_descriptor.write_text(
json.dumps(
{
"schema": "real2sim.scene_descriptor.v1",
"name": args.name,
"package": str(package_dir),
"manifest_path": str(manifest_path),
"simulator": "motrixsim",
"status": "asset_imported_adapter_pending",
},
ensure_ascii=False,
indent=2,
)
+ "\n",
encoding="utf-8",
)
artifacts = _register_file_artifacts(store, package_dir)
manifest = _build_manifest(
name=args.name,
source=source,
source_commit=source_commit,
package_dir=package_dir,
scene_descriptor=scene_descriptor,
artifact_urls=artifacts,
)
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
scene = _upsert_scene(
store=store,
name=args.name,
package_dir=package_dir,
scene_descriptor=scene_descriptor,
manifest_path=manifest_path,
manifest=manifest,
replace_index=args.replace_index,
)
print(json.dumps({"ok": True, "scene_id": scene["id"], "scene_name": scene["name"], "package_dir": str(package_dir), "manifest_path": str(manifest_path)}, ensure_ascii=False, indent=2))
return 0
def _assert_source(source: Path) -> None:
if not source.is_dir():
raise SystemExit(f"MotrixSim docs repo not found: {source}")
missing = [str(source / rel) for rel in REQUIRED_FILES.values() if not (source / rel).is_file()]
missing.extend(str(source / rel) for rel in DIRECTORIES_TO_COPY.values() if not (source / rel).is_dir())
if missing:
raise SystemExit("Missing required MotrixSim assets:\n" + "\n".join(missing))
def _copy_package(source: Path, package_dir: Path) -> None:
if package_dir.exists():
shutil.rmtree(package_dir)
(package_dir / "assets").mkdir(parents=True, exist_ok=True)
(package_dir / "examples" / "control").mkdir(parents=True, exist_ok=True)
for name, rel in DIRECTORIES_TO_COPY.items():
shutil.copytree(source / rel, package_dir / "assets" / name)
for rel in [Path("examples/control/rm65_open_cabinet.py"), Path("examples/control/robot_locomotion.py")]:
shutil.copy2(source / rel, package_dir / "examples" / "control" / rel.name)
def _assert_package(package_dir: Path) -> None:
required = [
package_dir / "assets/parlour/parlour.xml",
package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml",
package_dir / "assets/rm65_open_cabinet/scene.xml",
package_dir / "assets/franka_emika_panda/scene_pick_cube.xml",
package_dir / "examples/control/rm65_open_cabinet.py",
package_dir / "examples/control/robot_locomotion.py",
]
missing = [str(path) for path in required if not path.is_file()]
if missing:
raise SystemExit("Package is incomplete:\n" + "\n".join(missing))
def _register_file_artifacts(store: PlatformStore, package_dir: Path) -> dict[str, str]:
files = {
"scene_descriptor": package_dir / f"{PACKAGE_SLUG}.scene.json",
"parlour_scene": package_dir / "assets/parlour/parlour.xml",
"rm65_robot": package_dir / "assets/rm65_open_cabinet/mjx_rm65.xml",
"rm65_reference_scene": package_dir / "assets/rm65_open_cabinet/scene.xml",
"rm65_controller": package_dir / "examples/control/rm65_open_cabinet.py",
"pick_place_template": package_dir / "assets/franka_emika_panda/scene_pick_cube.xml",
"franka_preview": package_dir / "assets/franka_emika_panda/panda.png",
}
urls: dict[str, str] = {}
for role, path in files.items():
if not path.is_file():
continue
artifact = store.add_artifact(task_id=None, name=path.name, path=path, kind=_artifact_kind(path))
urls[role] = f"/api/artifacts/{artifact['id']}/content"
return urls
def _build_manifest(
*,
name: str,
source: Path,
source_commit: str,
package_dir: Path,
scene_descriptor: Path,
artifact_urls: dict[str, str],
) -> dict[str, Any]:
ready_assets = [
_asset(package_dir, "assets/parlour/parlour.xml", "parlour_scene", "simulator", "MotrixSim parlour 客厅场景", artifact_urls.get("parlour_scene")),
_asset(package_dir, "assets/rm65_open_cabinet/mjx_rm65.xml", "rm65_robot", "robot", "RM65 机械臂 MJCF", artifact_urls.get("rm65_robot")),
_asset(package_dir, "assets/rm65_open_cabinet/scene.xml", "rm65_reference_scene", "simulator", "RM65 开柜门参考场景", artifact_urls.get("rm65_reference_scene")),
_asset(package_dir, "examples/control/rm65_open_cabinet.py", "rm65_controller", "controller", "RM65 参考控制脚本", artifact_urls.get("rm65_controller")),
_asset(package_dir, "assets/franka_emika_panda/scene_pick_cube.xml", "pick_place_template", "simulator", "Franka pick-cube 任务模板", artifact_urls.get("pick_place_template")),
]
preview = _asset(package_dir, "assets/franka_emika_panda/panda.png", "reference_preview", "image", "机械臂参考预览图", artifact_urls.get("franka_preview"))
assets = [preview, *ready_assets] if preview["exists"] else ready_assets
total_size = _directory_size(package_dir)
return {
"schema": "real2sim.motrixsim_asset_package.v1",
"name": name,
"status": "asset_imported_adapter_pending",
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"package_dir": str(package_dir),
"scene_descriptor": str(scene_descriptor),
"source": {
"repo": "https://github.com/Motphys/motrixsim-docs",
"docs": "https://motrixsim.readthedocs.io/zh-cn/latest/user_guide/overview/examples.html",
"local_path": str(source),
"commit": source_commit,
},
"ready_assets": ready_assets,
"assets": assets,
"adapter_plan": {
"goal": "将 parlour 客厅场景中的人形机器人示例替换为 RM65 机械臂,并在茶几区域完成 pick-place。",
"base_scene": "assets/parlour/parlour.xml",
"robot_asset": "assets/rm65_open_cabinet/mjx_rm65.xml",
"reference_controller": "examples/control/rm65_open_cabinet.py",
"pick_place_template": "assets/franka_emika_panda/scene_pick_cube.xml",
"recommended_steps": [
"在 parlour 中定位茶几 mesh/collision,并确定表面高度与可达工作区。",
"把 RM65 base 固定到茶几旁或茶几边缘,完成 z-up 坐标、尺度和碰撞组检查。",
"把 Franka pick-cube 的目标物、home/grasp/lift 阶段迁移到 RM65 关节空间。",
"将 RM65 开柜门示例中的 7 维动作和 gripper 控制改成 cube pick-place controller。",
"生成 combined MJCF 后再进入 simulator_validation runtime_required。",
],
},
"runbook": {
"source_parlour_demo": "uv run examples/control/robot_locomotion.py --robot g1 --scene parlour",
"source_rm65_demo": "uv run examples/control/rm65_open_cabinet.py",
"source_pick_template": "uv run examples/bench/grasp/shake_test_mx.py --object cube",
},
"known_gaps": [
"当前包是资产接入 scaffold,不是可直接运行的 combined MJCF。",
"尚未完成 parlour 茶几 mesh 的语义定位和 RM65 base 位姿标定。",
"尚未把 Franka pick-cube keyframe/controller 迁移到 RM65。",
],
"metrics": {
"asset_count": sum(1 for path in package_dir.rglob("*") if path.is_file()),
"total_size_bytes": total_size,
"copied_directories": sorted(DIRECTORIES_TO_COPY),
},
}
def _asset(package_dir: Path, rel: str, role: str, kind: str, name: str, url: str | None = None) -> dict[str, Any]:
path = package_dir / rel
return {
"name": name,
"role": role,
"kind": kind,
"path": str(path),
"relative_path": rel,
"url": url or "",
"exists": path.is_file(),
"size_bytes": path.stat().st_size if path.is_file() else 0,
}
def _upsert_scene(
*,
store: PlatformStore,
name: str,
package_dir: Path,
scene_descriptor: Path,
manifest_path: Path,
manifest: dict[str, Any],
replace_index: bool,
) -> dict[str, Any]:
try:
existing = store.get_scene(SCENE_ID)
except KeyError:
existing = None
if existing and not replace_index:
store.update_scene(SCENE_ID, name=name, tags=_tags(), version=str(manifest["source"]["commit"]))
return store.get_scene(SCENE_ID)
if existing and replace_index:
store.delete_scene(SCENE_ID)
return store.create_scene(
item_id=SCENE_ID,
name=name,
path=scene_descriptor,
metadata={
"package": str(package_dir),
"manifest_path": str(manifest_path),
"simulator": "motrixsim",
"status": "asset_imported_adapter_pending",
"adapter_pending": True,
},
source_type="motrixsim_docs",
source_uri="https://github.com/Motphys/motrixsim-docs",
version=str(manifest["source"]["commit"]),
manifest_path=manifest_path,
metrics=manifest["metrics"],
tags=_tags(),
)
def _tags() -> list[str]:
return ["motrixsim", "parlour", "rm65", "pick-place", "adapter-pending"]
def _git_commit(source: Path) -> str:
try:
return subprocess.check_output(["git", "-C", str(source), "rev-parse", "HEAD"], text=True).strip()
except (OSError, subprocess.CalledProcessError):
return ""
def _existing_source_commit(package_dir: Path) -> str:
manifest_path = package_dir / "motrixsim_scene_manifest.json"
if not manifest_path.is_file():
return ""
try:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
except json.JSONDecodeError:
return ""
return str((manifest.get("source") or {}).get("commit") or "")
def _directory_size(path: Path) -> int:
return sum(item.stat().st_size for item in path.rglob("*") if item.is_file())
def _artifact_kind(path: Path) -> str:
suffix = path.suffix.lower()
if suffix in {".png", ".jpg", ".jpeg", ".webp"}:
return "image"
if suffix == ".xml":
return "simulator"
if suffix == ".py":
return "controller"
if suffix == ".json":
return "metadata"
return suffix.lstrip(".") or "file"
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -0,0 +1,441 @@
#!/usr/bin/env bash
set -euo pipefail
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
BASE_URL="${REAL2SIM_ACCEPTANCE_BASE_URL:-http://127.0.0.1:18126}"
MODE="${REAL2SIM_ACCEPTANCE_MODE:-production}"
SCALE="${REAL2SIM_ACCEPTANCE_SCALE:-pressure}"
OUT="${REAL2SIM_ACCEPTANCE_OUT:-/opt/real2sim2model/real2sim-platform/workspace/acceptance_runs}"
HF_REPO="${REAL2SIM_ACCEPTANCE_HF_REPO:-shashuo0104/xarm7_pusht}"
HF_PATTERNS="${REAL2SIM_ACCEPTANCE_HF_PATTERNS:-videos/chunk-000/observation.images.front/*.mp4,data/chunk-000/*.parquet,meta/**,README.md}"
RECON_FIXTURE="${REAL2SIM_ACCEPTANCE_RECON_FIXTURE:-/opt/real2sim_paper_runtime/sam-3d-objects/notebook/images/shutterstock_modern_colorful_Interior_2620125197}"
RECON_FALLBACKS="${REAL2SIM_ACCEPTANCE_RECON_FALLBACKS:-/opt/real2sim_paper_runtime/sam2/notebooks/videos/bedroom}"
MIN_RECON_FRAMES="${REAL2SIM_ACCEPTANCE_MIN_RECON_FRAMES:-80}"
TARGET_HF_FRAMES="${REAL2SIM_ACCEPTANCE_TARGET_HF_FRAMES:-240}"
MIN_HF_FRAMES="${REAL2SIM_ACCEPTANCE_MIN_HF_FRAMES:-120}"
TIMEOUT_SECONDS="${REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS:-7200}"
REQUIRE_MOTRIXSIM="${REAL2SIM_REQUIRE_MOTRIXSIM:-0}"
REPEAT_RECONSTRUCTIONS="${REAL2SIM_ACCEPTANCE_REPEAT_RECONSTRUCTIONS:-2}"
PRESSURE_TASKS="${REAL2SIM_ACCEPTANCE_PRESSURE_TASKS:-12}"
CONTINUE_AFTER_RECON_QUALITY_FAIL="${REAL2SIM_ACCEPTANCE_CONTINUE_AFTER_RECON_QUALITY_FAIL:-1}"
ssh -i "$KEY" -o BatchMode=yes -o StrictHostKeyChecking=accept-new "$HOST" \
env \
REAL2SIM_ACCEPTANCE_BASE_URL="$BASE_URL" \
REAL2SIM_ACCEPTANCE_MODE="$MODE" \
REAL2SIM_ACCEPTANCE_SCALE="$SCALE" \
REAL2SIM_ACCEPTANCE_OUT="$OUT" \
REAL2SIM_ACCEPTANCE_HF_REPO="$HF_REPO" \
REAL2SIM_ACCEPTANCE_HF_PATTERNS="$HF_PATTERNS" \
REAL2SIM_ACCEPTANCE_RECON_FIXTURE="$RECON_FIXTURE" \
REAL2SIM_ACCEPTANCE_RECON_FALLBACKS="$RECON_FALLBACKS" \
REAL2SIM_ACCEPTANCE_MIN_RECON_FRAMES="$MIN_RECON_FRAMES" \
REAL2SIM_ACCEPTANCE_TARGET_HF_FRAMES="$TARGET_HF_FRAMES" \
REAL2SIM_ACCEPTANCE_MIN_HF_FRAMES="$MIN_HF_FRAMES" \
REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS="$TIMEOUT_SECONDS" \
REAL2SIM_REQUIRE_MOTRIXSIM="$REQUIRE_MOTRIXSIM" \
REAL2SIM_ACCEPTANCE_REPEAT_RECONSTRUCTIONS="$REPEAT_RECONSTRUCTIONS" \
REAL2SIM_ACCEPTANCE_PRESSURE_TASKS="$PRESSURE_TASKS" \
REAL2SIM_ACCEPTANCE_CONTINUE_AFTER_RECON_QUALITY_FAIL="$CONTINUE_AFTER_RECON_QUALITY_FAIL" \
python3 -s <<'PY'
from __future__ import annotations
import json
import os
import sys
import time
import traceback
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any
BASE = os.environ["REAL2SIM_ACCEPTANCE_BASE_URL"].rstrip("/")
MODE = os.environ["REAL2SIM_ACCEPTANCE_MODE"]
SCALE = os.environ["REAL2SIM_ACCEPTANCE_SCALE"]
OUT_ROOT = Path(os.environ["REAL2SIM_ACCEPTANCE_OUT"])
HF_REPO = os.environ["REAL2SIM_ACCEPTANCE_HF_REPO"]
HF_PATTERNS = [item.strip() for item in os.environ["REAL2SIM_ACCEPTANCE_HF_PATTERNS"].split(",") if item.strip()]
RECON_FIXTURE = os.environ["REAL2SIM_ACCEPTANCE_RECON_FIXTURE"]
RECON_FALLBACKS = [item.strip() for item in os.environ["REAL2SIM_ACCEPTANCE_RECON_FALLBACKS"].split(":") if item.strip()]
MIN_RECON_FRAMES = int(os.environ["REAL2SIM_ACCEPTANCE_MIN_RECON_FRAMES"])
TARGET_HF_FRAMES = int(os.environ["REAL2SIM_ACCEPTANCE_TARGET_HF_FRAMES"])
MIN_HF_FRAMES = int(os.environ["REAL2SIM_ACCEPTANCE_MIN_HF_FRAMES"])
TIMEOUT = int(os.environ["REAL2SIM_ACCEPTANCE_TIMEOUT_SECONDS"])
REQUIRE_MOTRIXSIM = os.environ["REAL2SIM_REQUIRE_MOTRIXSIM"] == "1"
REPEAT_RECONSTRUCTIONS = int(os.environ["REAL2SIM_ACCEPTANCE_REPEAT_RECONSTRUCTIONS"])
PRESSURE_TASKS = int(os.environ["REAL2SIM_ACCEPTANCE_PRESSURE_TASKS"])
CONTINUE_AFTER_RECON_QUALITY_FAIL = os.environ["REAL2SIM_ACCEPTANCE_CONTINUE_AFTER_RECON_QUALITY_FAIL"] == "1"
TERMINAL_STATUSES = {"succeeded", "failed", "canceled", "failed_recovered"}
started_at = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
run_dir = OUT_ROOT / time.strftime("%Y%m%dT%H%M%SZ", time.gmtime())
run_dir.mkdir(parents=True, exist_ok=True)
report: dict[str, Any] = {
"schema": "real2sim.acceptance_report.v1",
"mode": MODE,
"scale": SCALE,
"started_at": started_at,
"finished_at": "",
"base_url": BASE,
"inputs": {
"hf_repo": HF_REPO,
"hf_patterns": HF_PATTERNS,
"reconstruction_fixture": RECON_FIXTURE,
"reconstruction_fallbacks": RECON_FALLBACKS,
"min_reconstruction_frames": MIN_RECON_FRAMES,
"target_hf_frames": TARGET_HF_FRAMES,
"min_hf_frames": MIN_HF_FRAMES,
"repeat_reconstructions": REPEAT_RECONSTRUCTIONS,
"pressure_tasks": PRESSURE_TASKS,
"require_motrixsim": REQUIRE_MOTRIXSIM,
"continue_after_reconstruction_quality_fail": CONTINUE_AFTER_RECON_QUALITY_FAIL,
},
"version": {},
"doctor": {},
"tasks": [],
"datasets": [],
"scenes": [],
"models": [],
"browser": {},
"viewer_probe": {
"status": "not_run",
"reason": "Run scripts/scene_viewer_probe_local.mjs separately; production API acceptance does not launch local Playwright.",
},
"screenshots": [],
"artifacts": [],
"warnings": [],
"failures": [],
"pass": False,
}
def api(path: str, payload: Any | None = None, timeout: int = 60) -> Any:
data = None
headers = {}
if payload is not None:
data = json.dumps(payload).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(BASE + path, data=data, headers=headers)
try:
with urllib.request.urlopen(req, timeout=timeout) as response:
body = response.read().decode("utf-8")
if not body:
return None
ctype = response.headers.get("content-type", "")
return json.loads(body) if "application/json" in ctype else body
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", errors="replace")
raise RuntimeError(f"{path} failed with HTTP {exc.code}: {detail}") from exc
def record_failure(message: str, **context: Any) -> None:
failure = {"message": message, **context}
print("FAIL:", json.dumps(failure, ensure_ascii=False), flush=True)
report["failures"].append(failure)
def save_report() -> None:
report["finished_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
(run_dir / "acceptance_report.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
lines = [
"# Real2Sim Production Acceptance",
"",
f"- pass: {str(report['pass']).lower()}",
f"- started_at: {report['started_at']}",
f"- finished_at: {report['finished_at']}",
f"- build_id: {report.get('version', {}).get('build_id', '')}",
f"- tasks: {len(report['tasks'])}",
f"- failures: {len(report['failures'])}",
f"- warnings: {len(report['warnings'])}",
f"- viewer_probe: {report.get('viewer_probe', {}).get('status', 'not_run')}",
"",
"## Task IDs",
]
for task in report["tasks"]:
lines.append(f"- {task.get('type')} {task.get('id')} {task.get('status')}")
if report["failures"]:
lines.extend(["", "## Failures"])
for failure in report["failures"]:
lines.append(f"- {failure.get('message')} {json.dumps(failure, ensure_ascii=False)[:600]}")
if report["warnings"]:
lines.extend(["", "## Warnings"])
for warning in report["warnings"]:
lines.append(f"- {warning}")
(run_dir / "summary.md").write_text("\n".join(lines) + "\n", encoding="utf-8")
def task_snapshot(task_id: str) -> dict[str, Any]:
task = api(f"/api/tasks/{task_id}")
try:
reports = api(f"/api/tasks/{task_id}/reports")
except Exception as exc:
reports = {"error": str(exc), "reports": []}
try:
artifacts = api(f"/api/tasks/{task_id}/artifacts")
except Exception as exc:
artifacts = [{"error": str(exc)}]
try:
logs = api(f"/api/tasks/{task_id}/logs")
except Exception as exc:
logs = f"log unavailable: {exc}"
snapshot = {
"id": task["id"],
"type": task["type"],
"status": task["status"],
"params": task.get("params"),
"result": task.get("result"),
"error": task.get("error"),
"run_dir": task.get("run_dir"),
"reports": reports,
"artifacts": artifacts,
"log_tail": str(logs)[-5000:],
}
(run_dir / f"task_{task_id}.json").write_text(json.dumps(snapshot, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
return snapshot
def wait_task(task_id: str, *, required: bool = True) -> dict[str, Any]:
deadline = time.time() + TIMEOUT
while time.time() < deadline:
task = api(f"/api/tasks/{task_id}")
if task["status"] in TERMINAL_STATUSES:
snapshot = task_snapshot(task_id)
report["tasks"].append(snapshot)
if required and task["status"] != "succeeded":
record_failure(f"required task {task['type']} did not succeed", task_id=task_id, status=task["status"], error=task.get("error"))
print(json.dumps({"task_id": task_id, "type": task["type"], "status": task["status"]}, ensure_ascii=False), flush=True)
return snapshot
time.sleep(5)
snapshot = task_snapshot(task_id)
report["tasks"].append(snapshot)
record_failure("task timed out", task_id=task_id)
if required:
raise TimeoutError(task_id)
return snapshot
def create_task(task_type: str, params: dict[str, Any], *, required: bool = True) -> dict[str, Any]:
task = api("/api/tasks", {"type": task_type, "params": params})
print(f"{task_type}: {task['id']}", flush=True)
return wait_task(task["id"], required=required)
def assert_artifacts(task: dict[str, Any], required_names: list[str] | None = None) -> None:
artifacts = task.get("artifacts") or []
names = {item.get("name") for item in artifacts}
if not artifacts:
record_failure("succeeded task has no artifacts", task_id=task.get("id"), task_type=task.get("type"))
for name in required_names or []:
if name not in names:
record_failure("required artifact missing", task_id=task.get("id"), task_type=task.get("type"), artifact=name, available=sorted(n for n in names if n))
for item in artifacts:
if item.get("name"):
report["artifacts"].append({"task_id": task.get("id"), "name": item.get("name"), "path": item.get("path"), "kind": item.get("kind")})
def refresh_entities() -> None:
report["datasets"] = api("/api/datasets")
report["scenes"] = api("/api/scenes")
report["models"] = api("/api/models")
def check_doctor() -> None:
report["version"] = api("/api/version")
doctor = api("/api/doctor")
report["doctor"] = doctor
checks = doctor.get("checks", {})
required = ["anysplat_root", "anysplat_python", "spark_bundle", "mujoco_python", "media_prepare_deps", "service_process"]
for name in required:
if not checks.get(name):
record_failure("doctor required check failed", check=name, value=checks.get(name))
for warning in doctor.get("warnings", []) or []:
report["warnings"].append(f"doctor: {warning}")
if not doctor.get("ok"):
record_failure("doctor returned ok=false", doctor=doctor)
def latest_scene_id(task: dict[str, Any]) -> str:
scene_id = (((task.get("result") or {}).get("scene_id")) or "")
if not scene_id:
record_failure("scene task did not return scene_id", task_id=task.get("id"))
return scene_id
def run_pressure_tasks(scene_id: str, expert_dataset_id: str, model_id: str, reconstruction_input: str) -> None:
queued = []
templates = [
("simulator_validation", {"scene_id": scene_id, "simulators": ["mujoco", "motrixsim"], "level": "runtime_if_available"}),
("expert_collection", {"scene_id": scene_id, "episodes": 3, "steps": 24}),
("training", {"dataset_id": expert_dataset_id, "epochs": 1}),
("inference", {"scene_id": scene_id, "model_id": model_id, "steps": 40}),
("reconstruction_preflight", {"input_path": reconstruction_input, "min_frames": MIN_RECON_FRAMES}),
]
for index in range(max(PRESSURE_TASKS, 12)):
task_type, params = templates[index % len(templates)]
task = api("/api/tasks", {"type": task_type, "params": params})
queued.append(task)
print(f"pressure {index + 1}: {task_type} {task['id']}", flush=True)
for task in queued:
wait_task(task["id"], required=True)
active = [task for task in api("/api/tasks") if task.get("status") in {"queued", "running", "canceling"}]
if active:
record_failure("active tasks remain after pressure batch", active=active[:20])
def select_reconstruction_input() -> tuple[str, dict[str, Any]]:
candidates = [RECON_FIXTURE, *RECON_FALLBACKS]
snapshots = []
for index, candidate in enumerate(candidates):
preflight = create_task("reconstruction_preflight", {
"input_path": candidate,
"min_frames": MIN_RECON_FRAMES,
})
assert_artifacts(preflight, ["capture_preflight_report.json"])
snapshots.append(preflight)
result = preflight.get("result") or {}
if result.get("ok"):
if index > 0:
report["warnings"].append(f"primary reconstruction fixture failed preflight; using fallback: {candidate}")
return candidate, preflight
report["warnings"].append(f"reconstruction fixture candidate failed preflight: {candidate}")
record_failure("no reconstruction fixture passed preflight", candidates=candidates)
return candidates[0], snapshots[-1] if snapshots else {}
def reconstruction_failed_on_quality_gate(task: dict[str, Any]) -> bool:
reports = task.get("reports") or {}
readable = str(reports.get("readable_error") or task.get("error") or "")
for report_item in reports.get("reports", []) or []:
payload = report_item.get("payload") or {}
readable += " " + str(payload.get("readable_error") or payload.get("decision") or "")
readable += " " + " ".join(str(item) for item in payload.get("blockers", []) or [])
return any(marker in readable for marker in ["allow-low-quality", "capture acceptance failed", "recapture_required", "registered_ratio", "sparse_points"])
def run_reconstruction_with_optional_continuation(reconstruction_input: str, repeat_index: int) -> dict[str, Any]:
params = {
"input_path": reconstruction_input,
"dry_run": False,
"renderer": "spark",
"simulators": ["mujoco", "motrixsim"],
"min_frames": MIN_RECON_FRAMES,
"quality_preset": "capture-high",
}
strict = create_task("scene_reconstruction", params)
if strict.get("status") == "succeeded":
assert_artifacts(strict, ["environment.json", "reconstruction_command.json", "capture_preflight_report.json", "reconstruction_report.json", "scene_manifest.json"])
return strict
assert_artifacts(strict, ["capture_preflight_report.json", "reconstruction_report.json"])
if not (CONTINUE_AFTER_RECON_QUALITY_FAIL and reconstruction_failed_on_quality_gate(strict)):
raise RuntimeError(f"scene_reconstruction failed: {strict.get('id')}")
report["warnings"].append(f"strict reconstruction {strict.get('id')} failed quality gate; continuing with allow_low_quality for downstream coverage")
continuation_params = {**params, "allow_low_quality": True, "name": f"acceptance-continuation-{repeat_index}"}
continuation = create_task("scene_reconstruction", continuation_params)
if continuation.get("status") != "succeeded":
assert_artifacts(continuation, ["capture_preflight_report.json", "reconstruction_report.json"])
raise RuntimeError(f"allow_low_quality scene_reconstruction failed: {continuation.get('id')}")
assert_artifacts(continuation, ["environment.json", "reconstruction_command.json", "capture_preflight_report.json", "reconstruction_report.json", "scene_manifest.json"])
return continuation
def main() -> int:
try:
check_doctor()
hf_import = create_task("hf_import", {
"repo_id": HF_REPO,
"repo_type": "dataset",
"allow_patterns": HF_PATTERNS,
"local_name": "acceptance-xarm7-pusht",
})
assert_artifacts(hf_import)
hf_dataset_id = (hf_import.get("result") or {}).get("dataset_id")
if not hf_dataset_id:
record_failure("hf_import did not return dataset_id", task_id=hf_import.get("id"))
media = create_task("media_prepare", {
"dataset_id": hf_dataset_id,
"camera_label": "observation.images.front",
"sample_fps": 2,
"max_frames_per_video": 80,
"max_videos": 16,
"target_frame_count": TARGET_HF_FRAMES,
"min_total_frames": MIN_HF_FRAMES,
"output_name": "acceptance-xarm7-pusht-prepared",
})
assert_artifacts(media)
prepared_id = (media.get("result") or {}).get("dataset_id")
frame_count = int((media.get("result") or {}).get("frame_count") or 0)
if frame_count < MIN_HF_FRAMES:
record_failure("media_prepare produced too few frames", frame_count=frame_count, minimum=MIN_HF_FRAMES)
hf_preflight = create_task("reconstruction_preflight", {
"dataset_id": prepared_id,
"min_frames": MIN_RECON_FRAMES,
"allow_low_quality": True,
})
assert_artifacts(hf_preflight, ["capture_preflight_report.json"])
reconstruction_input, fixture_preflight = select_reconstruction_input()
if not ((fixture_preflight.get("result") or {}).get("ok")):
raise RuntimeError("no reconstruction input passed preflight")
reconstruction_tasks = []
for index in range(max(REPEAT_RECONSTRUCTIONS, 2)):
reconstruction_tasks.append(run_reconstruction_with_optional_continuation(reconstruction_input, index + 1))
scene_id = latest_scene_id(reconstruction_tasks[-1])
if not scene_id:
raise RuntimeError("scene_reconstruction did not produce scene_id")
validation = create_task("simulator_validation", {
"scene_id": scene_id,
"simulators": ["mujoco", "motrixsim"],
"level": "runtime_required" if REQUIRE_MOTRIXSIM else "runtime_if_available",
})
assert_artifacts(validation, ["validation_report.json"])
validation_report = (validation.get("result") or {}).get("report") or {}
motrix = (validation_report.get("simulators") or {}).get("motrixsim") or {}
if REQUIRE_MOTRIXSIM and not motrix.get("ok"):
record_failure("MotrixSim runtime is required but validation failed", motrix=motrix)
elif not motrix.get("ok"):
report["warnings"].append(f"motrixsim validation not fully ok: {motrix.get('reason') or motrix.get('status')}")
expert = create_task("expert_collection", {"scene_id": scene_id, "episodes": 12, "steps": 48, "name": "acceptance-expert-rollouts"})
assert_artifacts(expert, ["manifest.json", "rollouts.jsonl"])
expert_dataset_id = (expert.get("result") or {}).get("dataset_id")
if not expert_dataset_id:
record_failure("expert_collection did not return dataset_id", task_id=expert.get("id"))
training = create_task("training", {"dataset_id": expert_dataset_id, "epochs": 3, "name": "acceptance-policy"})
assert_artifacts(training, ["model_card.json", "policy_head.json"])
model_id = (training.get("result") or {}).get("model_id")
if not model_id:
record_failure("training did not return model_id", task_id=training.get("id"))
inference = create_task("inference", {"scene_id": scene_id, "model_id": model_id, "steps": 80})
assert_artifacts(inference, ["prediction_rollout.json", "viewer.html"])
if int((inference.get("result") or {}).get("rollout_steps") or 0) <= 0:
record_failure("inference produced empty rollout", task_id=inference.get("id"), result=inference.get("result"))
run_pressure_tasks(scene_id, expert_dataset_id, model_id, reconstruction_input)
refresh_entities()
report["pass"] = len(report["failures"]) == 0
save_report()
print(json.dumps({"pass": report["pass"], "run_dir": str(run_dir), "report": str(run_dir / "acceptance_report.json")}, ensure_ascii=False), flush=True)
return 0 if report["pass"] else 1
except Exception as exc:
record_failure("acceptance runner crashed", error=str(exc), traceback=traceback.format_exc())
try:
refresh_entities()
except Exception:
pass
report["pass"] = False
save_report()
print(json.dumps({"pass": False, "run_dir": str(run_dir), "report": str(run_dir / "acceptance_report.json"), "error": str(exc)}, ensure_ascii=False), flush=True)
return 1
sys.exit(main())
PY

View File

@ -0,0 +1,356 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import math
import sys
import time
from pathlib import Path
from typing import Any
import imageio.v2 as imageio
import numpy as np
SCRIPT_DIR = Path(__file__).resolve().parent
APP_ROOT = SCRIPT_DIR.parent
SRC_ROOT = APP_ROOT / "src"
if str(SRC_ROOT) not in sys.path:
sys.path.insert(0, str(SRC_ROOT))
from real2sim_platform.config import ensure_workspace, load_config # noqa: E402
from real2sim_platform.store import PlatformStore # noqa: E402
SCENE_ID = "1f2618bdfcac4e45456569980716405a"
PACKAGE_SLUG = "parlour_rm65_pick_place"
def main() -> int:
parser = argparse.ArgumentParser(description="Run a scripted RM65 pick-place rollout on the generated smoke MJCF.")
parser.add_argument("--scene-id", default=SCENE_ID)
parser.add_argument("--fps", type=int, default=20)
parser.add_argument("--width", type=int, default=960)
parser.add_argument("--height", type=int, default=540)
parser.add_argument("--no-render", action="store_true")
parser.add_argument(
"--grasp-mode",
choices=("physical", "kinematic_attachment"),
default="physical",
help="Use contact physics by default; kinematic attachment is retained only for diagnostics.",
)
args = parser.parse_args()
config = load_config()
ensure_workspace(config)
store = PlatformStore(config.db_path)
scene = store.get_scene(args.scene_id)
package_dir = Path(str((scene.get("metadata") or {}).get("package") or config.workspace_dir / "motrixsim_assets" / PACKAGE_SLUG))
manifest_path = Path(str(scene.get("manifest_path") or package_dir / "motrixsim_scene_manifest.json"))
sim_path = package_dir / "simulator_assets/rm65_table_pick_place.xml"
if not sim_path.is_file():
raise SystemExit(f"runtime scene not found: {sim_path}")
rollout_dir = package_dir / "rollouts" / time.strftime("%Y%m%d_%H%M%S")
rollout_dir.mkdir(parents=True, exist_ok=True)
result = run_rollout(
sim_path,
rollout_dir,
fps=args.fps,
width=args.width,
height=args.height,
render=not args.no_render,
grasp_mode=args.grasp_mode,
)
artifacts = {}
for key, path in result["artifact_paths"].items():
if Path(path).is_file():
artifacts[key] = store.add_artifact(task_id=None, name=Path(path).name, path=Path(path), kind=artifact_kind(Path(path)))
update_manifest(manifest_path, rollout_dir, result, artifacts)
print(json.dumps({"ok": result["ok"], "scene_id": args.scene_id, "rollout_dir": str(rollout_dir), "summary": result["summary"], "artifacts": result["artifact_paths"]}, ensure_ascii=False, indent=2))
return 0 if result["ok"] else 1
def run_rollout(
sim_path: Path,
out_dir: Path,
*,
fps: int,
width: int,
height: int,
render: bool,
grasp_mode: str = "physical",
) -> dict[str, Any]:
import mujoco
model = mujoco.MjModel.from_xml_path(str(sim_path))
data = mujoco.MjData(model)
mujoco.mj_resetDataKeyframe(model, data, 0)
mujoco.mj_forward(model, data)
site_id = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_SITE, "gripper")
cube_joint = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "pick_cube_free")
cube_qadr = int(model.jnt_qposadr[cube_joint])
cube_geom = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_GEOM, "pick_cube_geom")
finger_geoms = {
mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_GEOM, "left_finger_pad_z_axis"),
mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_GEOM, "right_finger_pad_z_axis"),
}
ctrl_ranges = model.actuator_ctrlrange.copy()
gripper_joint = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "gripper_Left_1_Joint")
gripper_qadr = int(model.jnt_qposadr[gripper_joint])
joint_ids = [mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, f"joint_{i}") for i in range(1, 7)]
qpos_adrs = [int(model.jnt_qposadr[jid]) for jid in joint_ids]
dof_adrs = [int(model.jnt_dofadr[jid]) for jid in joint_ids]
duration_scale = 1 if grasp_mode == "kinematic_attachment" else 8
phases = [
("home", np.array([0.42, 0.00, 0.72]), 0.0, 35 * duration_scale, False),
("approach", np.array([0.71, 0.015, 0.42]), 0.0, 55 * duration_scale, False),
("descend", np.array([0.71, 0.015, 0.24]), 0.0, 40 * duration_scale, False),
("grasp", np.array([0.71, 0.015, 0.24]), -0.91, 45 * duration_scale, True),
("lift", np.array([0.71, 0.015, 0.60]), -0.91, 200 * duration_scale, True),
("place", np.array([0.43, 0.18, 0.40]), -0.91, 65 * duration_scale, True),
("release", np.array([0.43, 0.18, 0.40]), 0.0, 35 * duration_scale, False),
("retreat", np.array([0.42, 0.00, 0.72]), 0.0, 45 * duration_scale, False),
]
trajectory = []
frames = []
renderer = None
if render:
renderer = mujoco.Renderer(model, height=height, width=width)
attached = False
contact_steps = 0
bilateral_contact_steps = 0
phase_contacts = {phase: {"contact_steps": 0, "bilateral_contact_steps": 0} for phase, *_ in phases}
contact_force_metrics = {"max_normal": 0.0, "max_tangent": 0.0, "max_normal_z": 0.0, "samples": 0}
gripper_qpos_range = [float(data.qpos[gripper_qadr]), float(data.qpos[gripper_qadr])]
max_cube_z = float(data.qpos[cube_qadr + 2])
initial_cube = data.qpos[cube_qadr : cube_qadr + 3].copy()
settled_cube_z = float(initial_cube[2])
started = time.time()
frame_stride = max(1, round((1.0 / model.opt.timestep) / fps))
step_index = 0
for phase, target, gripper_ctrl, count, attach_cube in phases:
if phase == "grasp":
settled_cube_z = float(data.qpos[cube_qadr + 2])
max_cube_z = settled_cube_z
q_target = solve_ik(model, data, site_id, target, qpos_adrs, dof_adrs, ctrl_ranges)
for local in range(count):
alpha = (local + 1) / count
current = data.ctrl[:6].copy()
data.ctrl[:6] = current * (1 - alpha) + q_target * alpha
data.ctrl[6] = gripper_ctrl
if grasp_mode == "kinematic_attachment" and (attach_cube or attached):
attached = attach_cube
if attached:
set_cube_pose(data, cube_qadr, data.site_xpos[site_id] + np.array([0.0, 0.0, -0.095]))
mujoco.mj_step(model, data)
if grasp_mode == "kinematic_attachment" and attached:
set_cube_pose(data, cube_qadr, data.site_xpos[site_id] + np.array([0.0, 0.0, -0.095]))
mujoco.mj_forward(model, data)
touching_fingers = cube_finger_contacts(data, cube_geom, finger_geoms)
accumulate_contact_metrics(model, data, cube_geom, finger_geoms, contact_force_metrics)
if touching_fingers:
contact_steps += 1
phase_contacts[phase]["contact_steps"] += 1
if len(touching_fingers) == 2:
bilateral_contact_steps += 1
phase_contacts[phase]["bilateral_contact_steps"] += 1
gripper_qpos_range[0] = min(gripper_qpos_range[0], float(data.qpos[gripper_qadr]))
gripper_qpos_range[1] = max(gripper_qpos_range[1], float(data.qpos[gripper_qadr]))
max_cube_z = max(max_cube_z, float(data.qpos[cube_qadr + 2]))
if step_index % frame_stride == 0:
trajectory.append(sample_state(model, data, phase, step_index, site_id, cube_qadr))
if renderer is not None:
renderer.update_scene(data, camera="overview")
frames.append(renderer.render().copy())
step_index += 1
if renderer is not None:
renderer.close()
trajectory_path = out_dir / "rm65_pick_place_trajectory.json"
report_path = out_dir / "rm65_pick_place_rollout_report.json"
gif_path = out_dir / "rm65_pick_place_rollout.gif"
keyframe_path = out_dir / "rm65_pick_place_final.png"
trajectory_path.write_text(json.dumps({"schema": "real2sim.rm65_pick_place_trajectory.v1", "samples": trajectory}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
artifact_paths = {"trajectory": str(trajectory_path)}
if frames:
imageio.mimsave(gif_path, frames, fps=fps)
imageio.imwrite(keyframe_path, frames[-1])
artifact_paths["gif"] = str(gif_path)
artifact_paths["final_frame"] = str(keyframe_path)
final = trajectory[-1] if trajectory else {}
final_cube = np.asarray(final.get("cube_pos") or initial_cube, dtype=float)
target_cube = np.array([0.43, 0.18, 0.31])
lift_height = max_cube_z - settled_cube_z
place_error = float(np.linalg.norm(final_cube - target_cube))
physical_success = bilateral_contact_steps > 0 and lift_height >= 0.10 and place_error <= 0.12
summary = {
"mode": "scripted_ik_physical_contact" if grasp_mode == "physical" else "scripted_ik_with_cube_attachment",
"steps": step_index,
"samples": len(trajectory),
"sim_time": float(data.time),
"final_gripper": final.get("gripper_pos"),
"final_cube": final.get("cube_pos"),
"initial_cube": [float(x) for x in initial_cube],
"settled_cube_z": settled_cube_z,
"max_cube_z": max_cube_z,
"lift_height": lift_height,
"place_target": [float(x) for x in target_cube],
"place_error": place_error,
"contact_steps": contact_steps,
"bilateral_contact_steps": bilateral_contact_steps,
"phase_contacts": phase_contacts,
"gripper_qpos_range": gripper_qpos_range,
"contact_force_metrics": contact_force_metrics,
"physical_success": physical_success if grasp_mode == "physical" else None,
"elapsed_ms": round((time.time() - started) * 1000),
}
report = {
"schema": "real2sim.rm65_pick_place_rollout_report.v1",
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"ok": physical_success if grasp_mode == "physical" else True,
"runtime": "mujoco",
"scene": str(sim_path),
"summary": summary,
"phases": [phase for phase, *_ in phases],
"note": (
"Scripted IK with contact-physics grasp; the cube pose is never overwritten. This is not a learned policy."
if grasp_mode == "physical"
else "Diagnostic-only rollout. Cube follows the gripper during grasp/lift/place; this is not a learned policy."
),
}
report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
artifact_paths["report"] = str(report_path)
return {"ok": bool(report["ok"]), "summary": summary, "artifact_paths": artifact_paths}
def cube_finger_contacts(data: Any, cube_geom: int, finger_geoms: set[int]) -> set[int]:
touching: set[int] = set()
for index in range(data.ncon):
contact = data.contact[index]
pair = {int(contact.geom1), int(contact.geom2)}
if cube_geom not in pair:
continue
touching.update(pair & finger_geoms)
return touching
def accumulate_contact_metrics(model: Any, data: Any, cube_geom: int, finger_geoms: set[int], metrics: dict[str, Any]) -> None:
import mujoco
for index in range(data.ncon):
contact = data.contact[index]
pair = {int(contact.geom1), int(contact.geom2)}
if cube_geom not in pair or not (pair & finger_geoms):
continue
force = np.zeros(6)
mujoco.mj_contactForce(model, data, index, force)
metrics["max_normal"] = max(float(metrics["max_normal"]), abs(float(force[0])))
metrics["max_tangent"] = max(float(metrics["max_tangent"]), float(np.linalg.norm(force[1:3])))
metrics["max_normal_z"] = max(float(metrics["max_normal_z"]), abs(float(contact.frame[2])))
metrics["samples"] = int(metrics["samples"]) + 1
def solve_ik(model: Any, data: Any, site_id: int, target: np.ndarray, qpos_adrs: list[int], dof_adrs: list[int], ctrl_ranges: np.ndarray) -> np.ndarray:
import mujoco
q_backup = data.qpos.copy()
q = data.qpos[qpos_adrs].copy()
damping = 1e-3
for _ in range(80):
data.qpos[qpos_adrs] = q
mujoco.mj_forward(model, data)
err = target - data.site_xpos[site_id]
if np.linalg.norm(err) < 0.012:
break
jacp = np.zeros((3, model.nv))
jacr = np.zeros((3, model.nv))
mujoco.mj_jacSite(model, data, jacp, jacr, site_id)
j = jacp[:, dof_adrs]
dq = j.T @ np.linalg.solve(j @ j.T + damping * np.eye(3), err)
q = q + np.clip(dq, -0.08, 0.08)
for i in range(6):
q[i] = float(np.clip(q[i], ctrl_ranges[i, 0], ctrl_ranges[i, 1]))
data.qpos[:] = q_backup
mujoco.mj_forward(model, data)
return q
def set_cube_pose(data: Any, qadr: int, pos: np.ndarray) -> None:
data.qpos[qadr : qadr + 3] = pos
data.qpos[qadr + 3 : qadr + 7] = np.array([1.0, 0.0, 0.0, 0.0])
data.qvel[qadr : qadr + 6] = 0
def sample_state(model: Any, data: Any, phase: str, step: int, site_id: int, cube_qadr: int) -> dict[str, Any]:
return {
"step": step,
"time": float(data.time),
"phase": phase,
"qpos_arm": [float(x) for x in data.qpos[:6]],
"gripper_ctrl": float(data.ctrl[6]),
"gripper_pos": [float(x) for x in data.site_xpos[site_id]],
"cube_pos": [float(x) for x in data.qpos[cube_qadr : cube_qadr + 3]],
}
def update_manifest(manifest_path: Path, rollout_dir: Path, result: dict[str, Any], artifacts: dict[str, dict[str, Any]]) -> None:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
new_assets = []
for key, artifact in artifacts.items():
path = Path(artifact["path"])
new_assets.append(
{
"name": path.name,
"role": f"rollout_{key}",
"kind": artifact_kind(path),
"path": str(path),
"relative_path": str(path.relative_to(manifest_path.parent)),
"url": f"/api/artifacts/{artifact['id']}/content",
"exists": True,
"size_bytes": path.stat().st_size,
"size_label": format_bytes(path.stat().st_size),
"generated": True,
}
)
old_assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and str(item.get("role", "")).startswith("rollout_"))]
manifest["assets"] = [*new_assets, *old_assets]
manifest["latest_rollout"] = {
"dir": str(rollout_dir),
"summary": result["summary"],
"artifacts": new_assets,
"status": "scripted_rollout_succeeded",
}
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
def artifact_kind(path: Path) -> str:
suffix = path.suffix.lower()
if suffix in {".png", ".jpg", ".jpeg", ".webp", ".gif"}:
return "image"
if suffix == ".json":
return "metadata"
return suffix.lstrip(".") or "file"
def format_bytes(value: int) -> str:
units = ["B", "KB", "MB", "GB"]
size = float(value)
for unit in units:
if size < 1024 or unit == units[-1]:
return f"{size:.1f} {unit}" if unit != "B" else f"{int(size)} B"
size /= 1024
return f"{value} B"
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -0,0 +1,244 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { createRequire } from "node:module";
import fs from "node:fs";
import path from "node:path";
const key = process.env.REAL2SIM_WUYING_KEY || "/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519";
const host = process.env.REAL2SIM_WUYING_HOST || "root@39.106.101.95";
const remotePort = Number(process.env.REAL2SIM_REMOTE_PORT || "18126");
const requestedBase = process.env.REAL2SIM_FRONTEND_BASE_URL || "";
const outRoot = process.env.REAL2SIM_ACCEPTANCE_OUT || path.resolve("real2sim-platform", "workspace", "acceptance_runs", "viewer-probe");
const liveRequired = process.env.REAL2SIM_VIEWER_LIVE_REQUIRED === "1";
const liveTimeoutMs = Number(process.env.REAL2SIM_VIEWER_LIVE_TIMEOUT_MS || "150000");
const sceneIdOverride = process.env.REAL2SIM_VIEWER_SCENE_ID || "";
const timestamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..*/, "Z");
const outDir = path.join(outRoot, timestamp);
const reportPath = path.join(outDir, "scene_viewer_probe_report.json");
const failures = [];
const warnings = [];
let tunnel = null;
fs.mkdirSync(outDir, { recursive: true });
function fail(message, context = {}) {
failures.push({ message, ...context });
console.error("FAIL:", message, context);
}
function warn(message, context = {}) {
warnings.push({ message, ...context });
console.warn("WARN:", message, context);
}
function writeReport(report) {
fs.writeFileSync(reportPath, `${JSON.stringify(report, null, 2)}\n`);
return report;
}
async function main() {
const startedAt = new Date().toISOString();
const baseUrl = requestedBase || await startTunnel();
const scene = sceneIdOverride ? { id: sceneIdOverride } : await pickPreviewableScene(baseUrl);
let manifestGate = null;
let liveProbe = null;
if (!scene) {
fail("no scene available for viewer probe");
} else {
manifestGate = await validateManifestGate(baseUrl, scene.id);
liveProbe = await runLiveProbe(baseUrl, scene.id).catch((error) => {
const payload = { ok: false, error: error.message, stack: error.stack };
if (liveRequired) fail("live viewer probe failed", payload);
else warn("live viewer probe failed in non-required mode", payload);
return payload;
});
}
const report = writeReport({
schema: "real2sim.scene_viewer_probe_report.v1",
started_at: startedAt,
finished_at: new Date().toISOString(),
base_url: baseUrl,
scene_id: scene?.id || "",
live_required: liveRequired,
manifest_gate: manifestGate,
live_probe: liveProbe,
failures,
warnings,
pass: failures.length === 0,
});
console.log(JSON.stringify({ pass: report.pass, report: reportPath, scene_id: report.scene_id, live_probe: report.live_probe }, null, 2));
if (tunnel) tunnel.kill("SIGTERM");
process.exitCode = report.pass ? 0 : 1;
}
async function runLiveProbe(baseUrl, sceneId) {
const { chromium } = await importPlaywright();
const browser = await chromium.launch({ headless: true });
const page = await browser.newPage({ viewport: { width: 1440, height: 900 }, deviceScaleFactor: 1 });
const consoleErrors = [];
page.on("console", (message) => {
if (message.type() === "error") consoleErrors.push(message.text());
});
try {
const viewerUrl = `${baseUrl}/static/scene-viewer.html?scene_id=${encodeURIComponent(sceneId)}`;
await withTimeout((async () => {
await page.goto(viewerUrl, { waitUntil: "domcontentloaded", timeout: 30000 });
await page.waitForSelector("canvas", { timeout: 20000 });
await page.waitForFunction(() => {
const debug = window.real2simViewerDebug;
return debug && debug.canvas && debug.canvas.width > 0 && debug.canvas.height > 0 && (debug.loaded || debug.backend === "placeholder" || debug.errors?.length);
}, null, { timeout: Math.max(30000, liveTimeoutMs - 30000) });
await page.mouse.move(720, 450);
await page.mouse.down();
await page.mouse.move(900, 530, { steps: 8 });
await page.mouse.up();
})(), liveTimeoutMs, `live viewer probe timed out after ${liveTimeoutMs}ms`);
const debug = await page.evaluate(() => window.real2simViewerDebug.snapshot?.() || window.real2simViewerDebug);
const ok = !!debug && !debug.errors?.length && (debug.loaded || debug.backend === "placeholder");
if (liveRequired && !ok) fail("live viewer debug is not acceptable", { debug, console_errors: consoleErrors });
return { ok, url: viewerUrl, debug, console_errors: consoleErrors };
} finally {
await page.close().catch(() => {});
await browser.close().catch(() => {});
}
}
async function validateManifestGate(baseUrl, sceneId) {
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(sceneId)}/manifest`);
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
const optimized = assets.find((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")));
if (!optimized) {
fail("scene manifest has no optimized splat asset", { scene_id: sceneId });
return { ok: false, scene_id: sceneId };
}
if (!optimized.url) fail("optimized splat asset has no URL", { scene_id: sceneId, asset: optimized });
if (optimized.autoload === false) fail("optimized splat asset is not autoload", { scene_id: sceneId, asset: optimized });
return {
ok: failures.length === 0,
scene_id: sceneId,
asset: {
name: optimized.name || "",
url: optimized.url || "",
size_label: optimized.size_label || "",
size_bytes: optimized.size_bytes || 0,
autoload: optimized.autoload !== false,
},
};
}
async function pickPreviewableScene(baseUrl) {
const scenes = await fetchJson(baseUrl, "/api/scenes");
if (!Array.isArray(scenes) || scenes.length === 0) return null;
const scored = [];
for (const scene of scenes) {
try {
const manifest = await fetchJson(baseUrl, `/api/scenes/${encodeURIComponent(scene.id)}/manifest`);
const assets = Array.isArray(manifest.assets) ? manifest.assets : [];
let score = 0;
if (assets.some((asset) => /\.(spz|splat|ksplat|sog)$/i.test(String(asset.name || asset.path || "")))) score += 100;
if (assets.some((asset) => /pose_init_trained_scene\.ply$/i.test(String(asset.name || asset.path || "")))) score += 50;
if (assets.some((asset) => /\.(ply|glb|gltf|obj)$/i.test(String(asset.name || asset.path || "")))) score += 20;
scored.push({ scene, score });
} catch (error) {
warn("scene manifest unavailable", { scene_id: scene.id, error: error.message });
}
}
scored.sort((left, right) => right.score - left.score);
return scored[0]?.scene || scenes[0];
}
async function importPlaywright() {
const moduleDir = process.env.REAL2SIM_PLAYWRIGHT_NODE_MODULES || "";
if (moduleDir) {
const require = createRequire(path.join(moduleDir, "real2sim-playwright-loader.cjs"));
return require("playwright");
}
try {
return await import("playwright");
} catch (error) {
throw new Error(`playwright is required. Set REAL2SIM_PLAYWRIGHT_NODE_MODULES=/path/to/node_modules. Original error: ${error.message}`);
}
}
async function startTunnel() {
const port = Number(process.env.REAL2SIM_LOCAL_TUNNEL_PORT || await findPort(18140));
tunnel = spawn("ssh", [
"-i", key,
"-o", "BatchMode=yes",
"-o", "StrictHostKeyChecking=accept-new",
"-N",
"-L", `${port}:127.0.0.1:${remotePort}`,
host,
], { stdio: ["ignore", "pipe", "pipe"] });
tunnel.stderr.on("data", (chunk) => process.stderr.write(chunk));
tunnel.stdout.on("data", (chunk) => process.stdout.write(chunk));
const baseUrl = `http://127.0.0.1:${port}`;
const deadline = Date.now() + 30000;
while (Date.now() < deadline) {
try {
await fetchJson(baseUrl, "/api/health");
return baseUrl;
} catch {
await sleep(500);
}
}
throw new Error(`SSH tunnel did not become ready at ${baseUrl}`);
}
async function findPort(start) {
const net = await import("node:net");
for (let port = start; port < start + 100; port += 1) {
const ok = await new Promise((resolve) => {
const server = net.createServer();
server.once("error", () => resolve(false));
server.once("listening", () => server.close(() => resolve(true)));
server.listen(port, "127.0.0.1");
});
if (ok) return port;
}
throw new Error("no local tunnel port available");
}
async function fetchJson(baseUrl, apiPath) {
const response = await fetch(`${baseUrl}${apiPath}`);
if (!response.ok) throw new Error(`${apiPath} failed with ${response.status}: ${await response.text()}`);
return response.json();
}
async function withTimeout(promise, timeoutMs, message) {
let timer;
try {
return await Promise.race([
promise,
new Promise((_, reject) => {
timer = setTimeout(() => reject(new Error(message)), timeoutMs);
}),
]);
} finally {
clearTimeout(timer);
}
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
main().catch((error) => {
fail("scene viewer probe runner crashed", { error: error.message, stack: error.stack });
const report = writeReport({
schema: "real2sim.scene_viewer_probe_report.v1",
started_at: timestamp,
finished_at: new Date().toISOString(),
base_url: requestedBase,
scene_id: sceneIdOverride,
live_required: liveRequired,
failures,
warnings,
pass: false,
});
console.error(JSON.stringify({ pass: false, report: reportPath, error: error.message }, null, 2));
if (tunnel) tunnel.kill("SIGTERM");
process.exit(report.pass ? 0 : 1);
});

View File

@ -0,0 +1,331 @@
#!/usr/bin/env bash
set -euo pipefail
KEY="${REAL2SIM_WUYING_KEY:-/Users/d-robotics/workSpace/document/80-private/orgs/drobotics/keys/wuying-39.106.101.95_ed25519}"
HOST="${REAL2SIM_WUYING_HOST:-root@39.106.101.95}"
PROJECT_ROOT="${REAL2SIM_REMOTE_PLATFORM_ROOT:-/opt/real2sim2model/real2sim-platform}"
SERVICE="${REAL2SIM_PLATFORM_SERVICE:-real2sim-platform.service}"
PORT="${REAL2SIM_PLATFORM_PORT:-18126}"
DESKTOP_NAME="${REAL2SIM_DESKTOP_NAME:-Real2Sim 平台}"
DESKTOP_FILE_NAME="${REAL2SIM_DESKTOP_FILE_NAME:-Real2Sim Platform.desktop}"
MODE="local"
CLEAN_DESKTOP=1
CREATE_LAUNCHER=1
usage() {
cat <<'EOF'
Usage:
scripts/setup_wuying_desktop_launcher.sh --remote
scripts/setup_wuying_desktop_launcher.sh --local
Options:
--remote Run the setup on the Wuying host over SSH.
--local Run on the current Linux desktop host.
--no-clean Do not move existing desktop icons into backup.
--no-launcher Do not create/update the Real2Sim launcher.
--project-root PATH Remote Real2Sim platform root. Default: /opt/real2sim2model/real2sim-platform
--service NAME systemd service name. Default: real2sim-platform.service
--port PORT Local service port. Default: 18126
The cleanup is reversible: existing desktop entries are moved into a hidden
.real2sim_desktop_backup_<timestamp> directory on the Desktop, not deleted.
EOF
}
while [[ $# -gt 0 ]]; do
case "$1" in
--remote) MODE="remote"; shift ;;
--local) MODE="local"; shift ;;
--no-clean) CLEAN_DESKTOP=0; shift ;;
--no-launcher) CREATE_LAUNCHER=0; shift ;;
--project-root) PROJECT_ROOT="$2"; shift 2 ;;
--service) SERVICE="$2"; shift 2 ;;
--port) PORT="$2"; shift 2 ;;
-h|--help) usage; exit 0 ;;
*) echo "unknown argument: $1" >&2; usage >&2; exit 2 ;;
esac
done
if [[ "$MODE" == "remote" ]]; then
ssh -i "$KEY" \
-o BatchMode=yes \
-o ConnectTimeout=20 \
-o StrictHostKeyChecking=accept-new \
"$HOST" \
"REAL2SIM_REMOTE_PLATFORM_ROOT='$PROJECT_ROOT' REAL2SIM_PLATFORM_SERVICE='$SERVICE' REAL2SIM_PLATFORM_PORT='$PORT' REAL2SIM_DESKTOP_NAME='$DESKTOP_NAME' REAL2SIM_DESKTOP_FILE_NAME='$DESKTOP_FILE_NAME' bash -s -- --local $([[ "$CLEAN_DESKTOP" == "0" ]] && printf '%s' '--no-clean') $([[ "$CREATE_LAUNCHER" == "0" ]] && printf '%s' '--no-launcher')" < "$0"
exit 0
fi
if [[ "$(uname -s)" != "Linux" ]]; then
echo "This setup must run on the Wuying Linux desktop host." >&2
exit 1
fi
timestamp="$(date +%Y%m%d-%H%M%S)"
add_desktop_dir() {
local dir="$1"
[[ -n "$dir" ]] || return 0
if [[ ! -d "$dir" ]]; then
mkdir -p "$dir"
fi
printf '%s\n' "$dir"
}
discover_desktop_dirs() {
local dirs=()
if [[ -n "${XDG_DESKTOP_DIR:-}" ]]; then
dirs+=("$XDG_DESKTOP_DIR")
fi
dirs+=("$HOME/Desktop" "$HOME/桌面")
if [[ "$(id -u)" == "0" ]]; then
dirs+=("/root/Desktop" "/root/桌面")
local home_dir
for home_dir in /home/*; do
[[ -d "$home_dir" ]] || continue
dirs+=("$home_dir/Desktop" "$home_dir/桌面")
done
fi
local seen="" dir
for dir in "${dirs[@]}"; do
[[ -n "$dir" ]] || continue
case ":$seen:" in
*":$dir:"*) continue ;;
esac
seen="$seen:$dir"
add_desktop_dir "$dir"
done
}
owner_for_path() {
local path="$1"
stat -c '%U:%G' "$path" 2>/dev/null || printf 'root:root\n'
}
write_launcher_script() {
local home_dir="$1"
local owner_group="$2"
local bin_dir="$home_dir/.local/bin"
local state_dir="$home_dir/.local/state"
local launcher="$bin_dir/real2sim-platform-launch"
mkdir -p "$bin_dir" "$state_dir"
cat > "$launcher" <<EOF
#!/usr/bin/env bash
set -u
PROJECT_ROOT="$PROJECT_ROOT"
SERVICE="$SERVICE"
PORT="$PORT"
URL="http://127.0.0.1:$PORT/"
LOG="\${XDG_STATE_HOME:-\$HOME/.local/state}/real2sim-launcher.log"
mkdir -p "\$(dirname "\$LOG")"
{
echo "==== \$(date '+%Y-%m-%d %H:%M:%S') Real2Sim launcher ===="
run_with_timeout() {
if command -v timeout >/dev/null 2>&1; then
timeout 25 "\$@"
else
"\$@"
fi
}
health_ok() {
if command -v curl >/dev/null 2>&1; then
curl -fsS "http://127.0.0.1:$PORT/api/health" >/dev/null 2>&1
else
python3 - <<'PY' >/dev/null 2>&1
import urllib.request
urllib.request.urlopen('http://127.0.0.1:$PORT/api/health', timeout=2).read()
PY
fi
}
wait_for_health() {
local i
for i in \$(seq 1 40); do
health_ok && return 0
sleep 1
done
return 1
}
if command -v systemctl >/dev/null 2>&1 && systemctl list-unit-files "$SERVICE" >/dev/null 2>&1; then
if systemctl is-failed --quiet "$SERVICE"; then
run_with_timeout systemctl restart "$SERVICE" || true
else
run_with_timeout systemctl start "$SERVICE" || true
fi
fi
if ! wait_for_health; then
if [[ -x "\$PROJECT_ROOT/.venv/bin/real2sim-platform" ]]; then
if ! pgrep -f "\$PROJECT_ROOT/.venv/bin/real2sim-platform" >/dev/null 2>&1; then
cd "\$PROJECT_ROOT" || exit 1
REAL2SIM_PLATFORM_CONFIG="\$PROJECT_ROOT/config.toml" nohup "\$PROJECT_ROOT/.venv/bin/real2sim-platform" >> "\$LOG" 2>&1 &
fi
wait_for_health || true
fi
fi
launch_chrome_app() {
local browser="\$1"
mkdir -p "\$HOME/.config/real2sim-platform-chrome"
nohup "\$browser" \\
--app="\$URL" \\
--new-window \\
--user-data-dir="\$HOME/.config/real2sim-platform-chrome" \\
--no-first-run \\
--disable-sync \\
--disable-translate \\
--disable-background-networking \\
--disable-component-update \\
--disable-domain-reliability \\
--disable-client-side-phishing-detection \\
--no-default-browser-check \\
--no-service-autorun \\
--disable-features=Translate,TranslateUI,AutofillServerCommunication \\
--window-size=1440,900 \\
--force-device-scale-factor=1 \\
>/dev/null 2>&1 &
}
if command -v google-chrome >/dev/null 2>&1; then
launch_chrome_app google-chrome
elif command -v chromium-browser >/dev/null 2>&1; then
launch_chrome_app chromium-browser
elif command -v xdg-open >/dev/null 2>&1; then
nohup xdg-open "\$URL" >/dev/null 2>&1 &
elif command -v firefox >/dev/null 2>&1; then
nohup firefox "\$URL" >/dev/null 2>&1 &
else
echo "No browser opener found. Open \$URL manually."
fi
if ! health_ok; then
echo "Real2Sim did not become healthy. Check: \$LOG"
if command -v x-terminal-emulator >/dev/null 2>&1; then
x-terminal-emulator -e "bash -lc 'tail -n 120 \"\$LOG\"; echo; read -p \"Press Enter to close\"'" &
elif command -v gnome-terminal >/dev/null 2>&1; then
gnome-terminal -- bash -lc "tail -n 120 \"\$LOG\"; echo; read -p 'Press Enter to close'" &
fi
fi
} >> "\$LOG" 2>&1
EOF
chmod 0755 "$launcher"
chown "$owner_group" "$launcher" "$bin_dir" "$state_dir" 2>/dev/null || true
printf '%s\n' "$launcher"
}
write_icon() {
local home_dir="$1"
local owner_group="$2"
local icon_dir="$home_dir/.local/share/icons"
local icon_file="$icon_dir/real2sim-platform.svg"
mkdir -p "$icon_dir"
cat > "$icon_file" <<'EOF'
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<defs>
<linearGradient id="g" x1="16" y1="16" x2="112" y2="112" gradientUnits="userSpaceOnUse">
<stop stop-color="#66ead4"/>
<stop offset="0.55" stop-color="#23bca4"/>
<stop offset="1" stop-color="#6aa8ff"/>
</linearGradient>
</defs>
<rect width="128" height="128" rx="28" fill="#071119"/>
<path d="M64 16 104 39v50L64 112 24 89V39L64 16Z" fill="none" stroke="url(#g)" stroke-width="8" stroke-linejoin="round"/>
<path d="M64 36 84 47.5v25L64 84 44 72.5v-25L64 36Z" fill="none" stroke="#9ff7ea" stroke-width="6" stroke-linejoin="round"/>
<path d="M64 36v48M44 48l40 24M84 48 44 72" stroke="#2fdbc4" stroke-width="4" stroke-linecap="round"/>
</svg>
EOF
chmod 0644 "$icon_file"
chown "$owner_group" "$icon_file" "$icon_dir" 2>/dev/null || true
printf '%s\n' "$icon_file"
}
clean_desktop_dir() {
local desktop_dir="$1"
local keep_file="$desktop_dir/$DESKTOP_FILE_NAME"
local backup_dir="$desktop_dir/.real2sim_desktop_backup_$timestamp"
local moved=0
shopt -s nullglob dotglob
local item base
for item in "$desktop_dir"/* "$desktop_dir"/.[!.]* "$desktop_dir"/..?*; do
[[ -e "$item" ]] || continue
base="$(basename "$item")"
case "$base" in
"$DESKTOP_FILE_NAME"|".directory"|".real2sim_desktop_backup_"*) continue ;;
esac
if [[ "$item" == "$keep_file" ]]; then
continue
fi
mkdir -p "$backup_dir"
mv "$item" "$backup_dir/"
moved=$((moved + 1))
done
shopt -u nullglob dotglob
if [[ "$moved" == "0" && -d "$backup_dir" ]]; then
rmdir "$backup_dir" 2>/dev/null || true
fi
printf '%s\n' "$moved"
}
write_desktop_file() {
local desktop_dir="$1"
local owner_group="$2"
local launcher="$3"
local icon_file="$4"
local desktop_file="$desktop_dir/$DESKTOP_FILE_NAME"
cat > "$desktop_file" <<EOF
[Desktop Entry]
Type=Application
Name=$DESKTOP_NAME
Comment=启动 Real2Sim 服务并打开平台
Exec=$launcher
Icon=$icon_file
Terminal=false
Categories=Development;Science;
StartupNotify=true
EOF
chmod 0755 "$desktop_file"
chown "$owner_group" "$desktop_file" 2>/dev/null || true
if command -v gio >/dev/null 2>&1; then
gio set "$desktop_file" metadata::trusted true >/dev/null 2>&1 || true
fi
printf '%s\n' "$desktop_file"
}
main() {
if [[ ! -d "$PROJECT_ROOT" ]]; then
echo "warning: project root does not exist yet: $PROJECT_ROOT" >&2
fi
local desktop_dir owner_group home_dir launcher icon_file moved desktop_file
local count=0
while IFS= read -r desktop_dir; do
[[ -d "$desktop_dir" ]] || continue
owner_group="$(owner_for_path "$desktop_dir")"
home_dir="$(dirname "$desktop_dir")"
if [[ "$CREATE_LAUNCHER" == "1" ]]; then
launcher="$(write_launcher_script "$home_dir" "$owner_group")"
icon_file="$(write_icon "$home_dir" "$owner_group")"
desktop_file="$(write_desktop_file "$desktop_dir" "$owner_group" "$launcher" "$icon_file")"
else
desktop_file="$desktop_dir/$DESKTOP_FILE_NAME"
fi
if [[ "$CLEAN_DESKTOP" == "1" ]]; then
moved="$(clean_desktop_dir "$desktop_dir")"
else
moved=0
fi
echo "desktop=$desktop_dir launcher=$desktop_file moved_to_backup=$moved"
count=$((count + 1))
done < <(discover_desktop_dirs)
echo "Real2Sim desktop launcher setup complete. desktop_dirs=$count"
}
main

4
scripts/start_local.sh Normal file
View File

@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
PYTHONPATH=src python -m real2sim_platform.server --host 127.0.0.1 --port 18126

16
scripts/vendor_spark.sh Executable file
View File

@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
VERSION="${SPARK_VERSION:-2.1.0}"
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
cd "$TMP"
npm pack "@sparkjsdev/spark@$VERSION" >/dev/null
tar -xzf "sparkjsdev-spark-$VERSION.tgz"
mkdir -p "$ROOT/src/real2sim_platform/static/vendor"
cp package/dist/spark.module.min.js "$ROOT/src/real2sim_platform/static/vendor/spark.module.min.js"
cp package/LICENSE "$ROOT/src/real2sim_platform/static/vendor/spark.LICENSE"
printf '%s\n' "$VERSION" > "$ROOT/src/real2sim_platform/static/vendor/spark.VERSION"
echo "vendored @sparkjsdev/spark@$VERSION"

20
scripts/vendor_three.sh Executable file
View File

@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
VERSION="${THREE_VERSION:-0.185.1}"
VENDOR="$ROOT/src/real2sim_platform/static/vendor/three"
BASE="https://unpkg.com/three@$VERSION"
mkdir -p "$VENDOR/addons/controls" "$VENDOR/addons/loaders" "$VENDOR/addons/postprocessing" "$VENDOR/addons/utils"
curl -fsSL "$BASE/build/three.module.js" -o "$VENDOR/three.module.js"
curl -fsSL "$BASE/build/three.core.js" -o "$VENDOR/three.core.js"
curl -fsSL "$BASE/examples/jsm/controls/OrbitControls.js" -o "$VENDOR/addons/controls/OrbitControls.js"
curl -fsSL "$BASE/examples/jsm/loaders/OBJLoader.js" -o "$VENDOR/addons/loaders/OBJLoader.js"
curl -fsSL "$BASE/examples/jsm/loaders/PLYLoader.js" -o "$VENDOR/addons/loaders/PLYLoader.js"
curl -fsSL "$BASE/examples/jsm/loaders/GLTFLoader.js" -o "$VENDOR/addons/loaders/GLTFLoader.js"
curl -fsSL "$BASE/examples/jsm/postprocessing/Pass.js" -o "$VENDOR/addons/postprocessing/Pass.js"
curl -fsSL "$BASE/examples/jsm/utils/BufferGeometryUtils.js" -o "$VENDOR/addons/utils/BufferGeometryUtils.js"
curl -fsSL "$BASE/examples/jsm/utils/SkeletonUtils.js" -o "$VENDOR/addons/utils/SkeletonUtils.js"
printf '%s\n' "$VERSION" > "$ROOT/src/real2sim_platform/static/vendor/three.VERSION"
echo "vendored three@$VERSION"

View File

@ -0,0 +1,3 @@
__all__ = ["__version__"]
__version__ = "0.1.0"

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,186 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
from typing import Any
APP_ROOT = Path(__file__).resolve().parents[2]
REAL2SIM2MODEL_ROOT = APP_ROOT.parent
WORKSPACE_ROOT = Path(os.environ.get("REAL2SIM_PLATFORM_WORKSPACE", APP_ROOT / "workspace"))
@dataclass(frozen=True)
class PlatformConfig:
app_root: Path = APP_ROOT
workspace_dir: Path = WORKSPACE_ROOT
host: str = "127.0.0.1"
port: int = 18126
default_renderer: str = "spark"
real2sim2model_root: Path = REAL2SIM2MODEL_ROOT
expert_policy_root: Path = REAL2SIM2MODEL_ROOT / "real2sim-expert-policy"
training_root: Path = REAL2SIM2MODEL_ROOT / "real2sim-policy-training"
inference_root: Path = REAL2SIM2MODEL_ROOT / "real2sim-policy-inference"
discoverse_root: Path = Path("/Users/d-robotics/workSpace/DISCOVERSE")
gs_playground_root: Path = Path("/Users/d-robotics/workSpace/gs_playground")
images23dgs_root: Path = Path("/Users/d-robotics/workSpace/images23dgs")
colmap_binary: Path = Path("/usr/local/bin/colmap")
gsplat_python: Path = Path("/opt/real2sim_paper_runtime/envs/anysplat/bin/python")
runtime_root: Path = Path("/opt/real2sim_paper_runtime")
anysplat_root: Path = Path("/opt/real2sim_paper_runtime/AnySplat")
anysplat_python: Path = Path("/opt/real2sim_paper_runtime/envs/anysplat/bin/python")
cuda_home: Path = Path("/usr/local/cuda-12.1")
runtime_env_script: Path = Path("/opt/gs_playground_real2sim_48q/scripts/real2sim_wuying_paper_env.sh")
allowed_path_roots: tuple[Path, ...] = ()
@property
def db_path(self) -> Path:
return self.workspace_dir / "state.db"
@property
def uploads_dir(self) -> Path:
return self.workspace_dir / "uploads"
@property
def datasets_dir(self) -> Path:
return self.workspace_dir / "datasets"
@property
def scenes_dir(self) -> Path:
return self.workspace_dir / "scenes"
@property
def runs_dir(self) -> Path:
return self.workspace_dir / "runs"
@property
def models_dir(self) -> Path:
return self.workspace_dir / "models"
@property
def manifests_dir(self) -> Path:
return self.workspace_dir / "manifests"
@property
def effective_allowed_path_roots(self) -> tuple[Path, ...]:
if self.allowed_path_roots:
return self.allowed_path_roots
return (
self.workspace_dir,
self.real2sim2model_root,
self.gs_playground_root,
self.discoverse_root,
self.runtime_root,
self.anysplat_root,
Path("/opt/gs_playground_real2sim_48q"),
Path("/tmp"),
)
def load_config(path: Path | None = None) -> PlatformConfig:
config_path = path or Path(os.environ.get("REAL2SIM_PLATFORM_CONFIG", ""))
if not config_path or not config_path.is_file():
return PlatformConfig()
values = _parse_simple_toml(config_path.read_text(encoding="utf-8"))
app_root = Path(str(values.get("app_root", APP_ROOT)))
workspace_dir = Path(str(values.get("workspace_dir", WORKSPACE_ROOT)))
return PlatformConfig(
app_root=app_root,
workspace_dir=workspace_dir,
host=str(values.get("host", "127.0.0.1")),
port=int(values.get("port", 18126)),
default_renderer=str(values.get("default_renderer", "spark")),
real2sim2model_root=Path(str(values.get("real2sim2model_root", REAL2SIM2MODEL_ROOT))),
expert_policy_root=Path(str(values.get("expert_policy_root", REAL2SIM2MODEL_ROOT / "real2sim-expert-policy"))),
training_root=Path(str(values.get("training_root", REAL2SIM2MODEL_ROOT / "real2sim-policy-training"))),
inference_root=Path(str(values.get("inference_root", REAL2SIM2MODEL_ROOT / "real2sim-policy-inference"))),
discoverse_root=Path(str(values.get("discoverse_root", "/Users/d-robotics/workSpace/DISCOVERSE"))),
gs_playground_root=Path(str(values.get("gs_playground_root", "/Users/d-robotics/workSpace/gs_playground"))),
images23dgs_root=Path(str(values.get("images23dgs_root", "/Users/d-robotics/workSpace/images23dgs"))),
colmap_binary=Path(str(values.get("colmap_binary", "/usr/local/bin/colmap"))),
gsplat_python=Path(str(values.get("gsplat_python", "/opt/real2sim_paper_runtime/envs/anysplat/bin/python"))),
runtime_root=Path(str(values.get("runtime_root", "/opt/real2sim_paper_runtime"))),
anysplat_root=Path(str(values.get("anysplat_root", "/opt/real2sim_paper_runtime/AnySplat"))),
anysplat_python=Path(str(values.get("anysplat_python", values.get("gsplat_python", "/opt/real2sim_paper_runtime/envs/anysplat/bin/python")))),
cuda_home=Path(str(values.get("cuda_home", "/usr/local/cuda-12.1"))),
runtime_env_script=Path(str(values.get("runtime_env_script", "/opt/gs_playground_real2sim_48q/scripts/real2sim_wuying_paper_env.sh"))),
allowed_path_roots=_parse_path_tuple(str(values.get("allowed_path_roots", ""))),
)
def ensure_workspace(config: PlatformConfig) -> None:
for path in [config.workspace_dir, config.uploads_dir, config.datasets_dir, config.scenes_dir, config.runs_dir, config.models_dir, config.manifests_dir]:
path.mkdir(parents=True, exist_ok=True)
def write_default_config(path: Path, config: PlatformConfig | None = None) -> None:
cfg = config or PlatformConfig(app_root=path.parent, workspace_dir=path.parent / "workspace")
path.parent.mkdir(parents=True, exist_ok=True)
lines = [
f'app_root = "{cfg.app_root.as_posix()}"',
f'workspace_dir = "{cfg.workspace_dir.as_posix()}"',
f'host = "{cfg.host}"',
f"port = {cfg.port}",
f'default_renderer = "{cfg.default_renderer}"',
f'real2sim2model_root = "{cfg.real2sim2model_root.as_posix()}"',
f'expert_policy_root = "{cfg.expert_policy_root.as_posix()}"',
f'training_root = "{cfg.training_root.as_posix()}"',
f'inference_root = "{cfg.inference_root.as_posix()}"',
f'discoverse_root = "{cfg.discoverse_root.as_posix()}"',
f'gs_playground_root = "{cfg.gs_playground_root.as_posix()}"',
f'images23dgs_root = "{cfg.images23dgs_root.as_posix()}"',
f'colmap_binary = "{cfg.colmap_binary.as_posix()}"',
f'gsplat_python = "{cfg.gsplat_python.as_posix()}"',
f'runtime_root = "{cfg.runtime_root.as_posix()}"',
f'anysplat_root = "{cfg.anysplat_root.as_posix()}"',
f'anysplat_python = "{cfg.anysplat_python.as_posix()}"',
f'cuda_home = "{cfg.cuda_home.as_posix()}"',
f'runtime_env_script = "{cfg.runtime_env_script.as_posix()}"',
f'allowed_path_roots = "{_format_path_tuple(cfg.effective_allowed_path_roots)}"',
"",
]
path.write_text("\n".join(lines), encoding="utf-8")
def is_path_allowed(path: Path, config: PlatformConfig) -> bool:
try:
resolved = path.expanduser().resolve()
except OSError:
resolved = path.expanduser().absolute()
for root in config.effective_allowed_path_roots:
try:
resolved.relative_to(root.expanduser().resolve())
return True
except (OSError, ValueError):
continue
return False
def _parse_simple_toml(text: str) -> dict[str, Any]:
values: dict[str, Any] = {}
for raw in text.splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, value = line.split("=", 1)
key = key.strip()
value = value.strip()
if value.startswith('"') and value.endswith('"'):
values[key] = value[1:-1]
else:
try:
values[key] = int(value)
except ValueError:
values[key] = value
return values
def _parse_path_tuple(value: str) -> tuple[Path, ...]:
if not value:
return ()
return tuple(Path(part) for part in value.split(":") if part)
def _format_path_tuple(paths: tuple[Path, ...]) -> str:
return ":".join(path.as_posix() for path in paths)

View File

@ -0,0 +1,901 @@
from __future__ import annotations
import argparse
import hashlib
import json
import os
import shutil
import subprocess
import time
import zipfile
from pathlib import Path
from typing import Any
from fastapi import FastAPI, File, HTTPException, UploadFile
from fastapi.responses import FileResponse, PlainTextResponse
from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel
from . import __version__
from .adapters import SPARK_VENDOR_PATH, collect_task_reports, enrich_scene_manifest_assets, run_doctor, scan_dataset
from .config import PlatformConfig, ensure_workspace, is_path_allowed, load_config, write_default_config
from .store import PlatformStore
from .worker import TaskWorker
class ImportPathRequest(BaseModel):
path: str
name: str | None = None
class ImportSceneRequest(BaseModel):
path: str
name: str | None = None
metadata: dict[str, Any] = {}
class SceneAssetRequest(BaseModel):
path: str
name: str | None = None
role: str = "added_asset"
kind: str | None = None
transform: dict[str, Any] = {}
class CreateTaskRequest(BaseModel):
type: str
params: dict[str, Any] = {}
class UpdateEntityRequest(BaseModel):
name: str | None = None
tags: list[str] | None = None
version: str | None = None
class UpdateTaskRequest(BaseModel):
params: dict[str, Any]
class BulkActionRequest(BaseModel):
action: str
ids: list[str]
params: dict[str, Any] = {}
def create_app(config: PlatformConfig | None = None) -> FastAPI:
cfg = config or load_config()
ensure_workspace(cfg)
store = PlatformStore(cfg.db_path)
worker = TaskWorker(store, cfg)
app = FastAPI(title="Real2Sim Platform", version="0.1.0")
static_dir = Path(__file__).parent / "static"
app.mount("/static", StaticFiles(directory=static_dir), name="static")
app.mount("/datasets", StaticFiles(directory=cfg.datasets_dir), name="datasets")
@app.on_event("startup")
def _startup() -> None:
store.recover_interrupted_tasks()
worker.start()
@app.on_event("shutdown")
def _shutdown() -> None:
worker.stop()
@app.get("/")
def index():
return FileResponse(static_dir / "index.html")
@app.get("/api/health")
def health() -> dict[str, Any]:
return {"ok": True, "workspace": str(cfg.workspace_dir)}
@app.get("/api/version")
def version() -> dict[str, Any]:
spark_available = SPARK_VENDOR_PATH.is_file()
return {
"version": __version__,
"build_time": os.environ.get("REAL2SIM_PLATFORM_BUILD_TIME", "dev"),
"app_root": str(cfg.app_root),
"workspace": str(cfg.workspace_dir),
"deploy_path": str(cfg.app_root),
"build_id": os.environ.get("REAL2SIM_PLATFORM_BUILD_ID", os.environ.get("REAL2SIM_PLATFORM_BUILD_TIME", "dev")),
"config_digest": _config_digest(cfg),
"git_commit": _git_commit(cfg.app_root),
"source_snapshot": _source_snapshot(cfg.app_root),
"runtime_paths": _runtime_paths(cfg),
"renderer_backend": "spark" if spark_available else "three_fallback",
"features": {
"hf_import": True,
"media_prepare": True,
"reconstruction_preflight": True,
"scene_reconstruction": True,
"spark_viewer": spark_available,
"three_fallback_viewer": True,
"aholo_optional": True,
"mujoco": True,
"motrixsim": True,
},
}
@app.get("/api/doctor")
def doctor() -> dict[str, Any]:
return run_doctor(cfg)
@app.get("/api/datasets")
def list_datasets() -> list[dict[str, Any]]:
return [_with_view_model(item, "dataset") for item in store.list_datasets()]
@app.post("/api/datasets/import-path")
def import_dataset(payload: ImportPathRequest) -> dict[str, Any]:
path = Path(payload.path)
if not path.exists():
raise HTTPException(status_code=404, detail="path not found")
if not is_path_allowed(path, cfg):
raise HTTPException(status_code=403, detail="path is outside configured import whitelist")
scan = scan_dataset(path)
return store.create_dataset(name=payload.name or path.name, path=path, scan=scan, source_type="local_path", source_uri=str(path))
@app.post("/api/datasets/upload")
def upload_dataset(file: UploadFile = File(...)) -> dict[str, Any]:
target = cfg.uploads_dir / Path(file.filename or "upload.bin").name
with target.open("wb") as handle:
shutil.copyfileobj(file.file, handle)
dataset_dir = cfg.datasets_dir / target.stem
if dataset_dir.exists():
shutil.rmtree(dataset_dir)
dataset_dir.mkdir(parents=True, exist_ok=True)
if target.suffix.lower() == ".zip":
with zipfile.ZipFile(target) as archive:
_safe_extract(archive, dataset_dir)
path = dataset_dir
else:
shutil.copy2(target, dataset_dir / target.name)
path = dataset_dir
scan = scan_dataset(path)
manifest_path = dataset_dir / "dataset_manifest.json"
manifest_path.write_text(json.dumps({"schema": "real2sim.dataset_manifest.v1", "name": dataset_dir.name, "source_type": "upload", "scan": scan}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
return store.create_dataset(name=dataset_dir.name, path=path, scan=scan, source_type="upload", source_uri=str(target), manifest_path=manifest_path)
@app.get("/api/scenes")
def list_scenes() -> list[dict[str, Any]]:
return [_with_view_model(item, "scene") for item in store.list_scenes()]
@app.post("/api/scenes/import")
def import_scene(payload: ImportSceneRequest) -> dict[str, Any]:
path = Path(payload.path)
if not path.is_file():
raise HTTPException(status_code=404, detail="scene file not found")
if not is_path_allowed(path, cfg):
raise HTTPException(status_code=403, detail="scene path is outside configured import whitelist")
return store.create_scene(name=payload.name or path.stem, path=path, metadata=payload.metadata, source_type="local_path", source_uri=str(path), manifest_path=Path(str(payload.metadata.get("manifest_path", ""))) if payload.metadata.get("manifest_path") else None)
@app.get("/api/datasets/{dataset_id}/manifest")
def dataset_manifest(dataset_id: str) -> dict[str, Any]:
try:
dataset = store.get_dataset(dataset_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="dataset not found") from exc
return _entity_manifest(dataset, fallback_kind="dataset")
@app.patch("/api/datasets/{dataset_id}")
def update_dataset(dataset_id: str, payload: UpdateEntityRequest) -> dict[str, Any]:
try:
return store.update_dataset(dataset_id, name=payload.name, tags=payload.tags, version=payload.version)
except KeyError as exc:
raise HTTPException(status_code=404, detail="dataset not found") from exc
@app.delete("/api/datasets/{dataset_id}")
def delete_dataset(dataset_id: str) -> dict[str, Any]:
try:
store.delete_dataset(dataset_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="dataset not found") from exc
return {"ok": True}
@app.get("/api/scenes/{scene_id}/manifest")
def scene_manifest(scene_id: str) -> dict[str, Any]:
try:
scene = store.get_scene(scene_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="scene not found") from exc
return _entity_manifest(scene, fallback_kind="scene")
@app.post("/api/scenes/{scene_id}/assets")
def add_scene_asset(scene_id: str, payload: SceneAssetRequest) -> dict[str, Any]:
try:
scene = store.get_scene(scene_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="scene not found") from exc
asset_path = Path(payload.path).expanduser()
if not asset_path.is_file():
raise HTTPException(status_code=404, detail="asset file not found")
if not is_path_allowed(asset_path, cfg):
raise HTTPException(status_code=403, detail="asset path is outside configured import whitelist")
manifest_path = _scene_manifest_path_for_write(scene)
manifest = _read_manifest_or_default(scene, manifest_path)
artifact = store.add_artifact(task_id=None, name=payload.name or asset_path.name, path=asset_path, kind=(payload.kind or _scene_asset_kind(asset_path)))
asset = _scene_asset_entry(
asset_path,
url=f"/api/artifacts/{artifact['id']}/content",
name=payload.name,
kind=payload.kind,
role=payload.role,
transform=payload.transform,
)
assets = [item for item in manifest.get("assets", []) if not (isinstance(item, dict) and str(item.get("path") or "") == str(asset_path))]
manifest["assets"] = [asset] + assets
manifest["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
manifest["asset_editing"] = {"supports_added_assets": True, "last_added_asset": asset["name"]}
manifest_path.parent.mkdir(parents=True, exist_ok=True)
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
tags = list(dict.fromkeys([*(scene.get("tags") or []), "added_asset"]))
store.update_scene(scene_id, tags=tags)
return {"ok": True, "scene": store.get_scene(scene_id), "asset": asset, "manifest_path": str(manifest_path)}
@app.patch("/api/scenes/{scene_id}")
def update_scene(scene_id: str, payload: UpdateEntityRequest) -> dict[str, Any]:
try:
return store.update_scene(scene_id, name=payload.name, tags=payload.tags, version=payload.version)
except KeyError as exc:
raise HTTPException(status_code=404, detail="scene not found") from exc
@app.delete("/api/scenes/{scene_id}")
def delete_scene(scene_id: str) -> dict[str, Any]:
try:
store.delete_scene(scene_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="scene not found") from exc
return {"ok": True}
@app.get("/api/models")
def list_models() -> list[dict[str, Any]]:
return [_with_view_model(item, "model") for item in store.list_models()]
@app.get("/api/models/{model_id}/card")
def model_card(model_id: str) -> dict[str, Any]:
try:
model = store.get_model(model_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="model not found") from exc
return _entity_manifest(model, fallback_kind="model")
@app.patch("/api/models/{model_id}")
def update_model(model_id: str, payload: UpdateEntityRequest) -> dict[str, Any]:
try:
return store.update_model(model_id, name=payload.name, tags=payload.tags, version=payload.version)
except KeyError as exc:
raise HTTPException(status_code=404, detail="model not found") from exc
@app.delete("/api/models/{model_id}")
def delete_model(model_id: str) -> dict[str, Any]:
try:
store.delete_model(model_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="model not found") from exc
return {"ok": True}
def _queue_task(task_type: str, params: dict[str, Any]) -> dict[str, Any]:
payload = CreateTaskRequest(type=task_type, params=params)
_validate_task_payload(payload)
_validate_task_resources(store, payload)
if payload.type not in {"scene_reconstruction", "reconstruction_preflight", "media_prepare", "expert_collection", "training", "inference", "renderer_benchmark", "scene_cover_render", "scene_splat_optimize", "hf_import", "simulator_validation"}:
raise ValueError(f"unknown task type: {payload.type}")
run_dir = cfg.runs_dir / payload.type / _next_run_name(cfg.runs_dir / payload.type, payload.type)
return store.create_task(task_type=payload.type, params=payload.params, run_dir=run_dir)
def _bulk_results(ids: list[str], handler) -> dict[str, Any]:
results: list[dict[str, Any]] = []
for item_id in ids:
try:
results.append({"id": item_id, "ok": True, "result": handler(item_id)})
except Exception as exc:
results.append({"id": item_id, "ok": False, "error": str(exc)})
return {"ok": all(item["ok"] for item in results), "results": results}
@app.post("/api/tasks/bulk")
def bulk_tasks(payload: BulkActionRequest) -> dict[str, Any]:
if payload.action not in {"retry", "cancel", "delete"}:
raise HTTPException(status_code=400, detail="tasks bulk action must be retry, cancel, or delete")
def handle(task_id: str) -> dict[str, Any]:
if payload.action == "retry":
return store.retry_task(task_id)
if payload.action == "cancel":
return store.cancel_task(task_id)
store.delete_task(task_id)
return {"deleted": True}
return _bulk_results(payload.ids, handle)
@app.post("/api/datasets/bulk")
def bulk_datasets(payload: BulkActionRequest) -> dict[str, Any]:
if payload.action not in {"prepare", "preflight", "delete"}:
raise HTTPException(status_code=400, detail="datasets bulk action must be prepare, preflight, or delete")
def handle(dataset_id: str) -> dict[str, Any]:
dataset = store.get_dataset(dataset_id)
if payload.action == "prepare":
params = {
"dataset_id": dataset_id,
"camera_label": payload.params.get("camera_label", "observation.images.front"),
"sample_fps": payload.params.get("sample_fps", 2),
"max_frames_per_video": payload.params.get("max_frames_per_video", 80),
"max_videos": payload.params.get("max_videos", 1),
"target_frame_count": payload.params.get("target_frame_count", 0),
"min_total_frames": payload.params.get("min_total_frames", 0),
}
return _queue_task("media_prepare", params)
if payload.action == "preflight":
return _queue_task("reconstruction_preflight", {"dataset_id": dataset_id, "min_frames": payload.params.get("min_frames", 30)})
store.delete_dataset(dataset["id"])
return {"deleted": True}
return _bulk_results(payload.ids, handle)
@app.post("/api/scenes/bulk")
def bulk_scenes(payload: BulkActionRequest) -> dict[str, Any]:
if payload.action not in {"validate", "export", "delete"}:
raise HTTPException(status_code=400, detail="scenes bulk action must be validate, export, or delete")
def handle(scene_id: str) -> dict[str, Any]:
scene = store.get_scene(scene_id)
if payload.action == "validate":
return _queue_task(
"simulator_validation",
{
"scene_id": scene_id,
"simulators": payload.params.get("simulators", ["mujoco", "motrixsim"]),
"level": payload.params.get("level", "runtime_if_available"),
},
)
if payload.action == "export":
return {"scene_path": scene["path"], "manifest_path": scene.get("manifest_path", ""), "package_path": scene.get("metadata", {}).get("package", str(Path(scene["path"]).parent))}
store.delete_scene(scene["id"])
return {"deleted": True}
return _bulk_results(payload.ids, handle)
@app.post("/api/models/bulk")
def bulk_models(payload: BulkActionRequest) -> dict[str, Any]:
if payload.action not in {"compare", "export", "delete"}:
raise HTTPException(status_code=400, detail="models bulk action must be compare, export, or delete")
def handle(model_id: str) -> dict[str, Any]:
model = store.get_model(model_id)
if payload.action == "compare":
return {"model_id": model_id, "name": model["name"], "metrics": model.get("metrics", {})}
if payload.action == "export":
return {"model_path": model["path"], "card_path": model.get("manifest_path", "")}
store.delete_model(model["id"])
return {"deleted": True}
return _bulk_results(payload.ids, handle)
@app.get("/api/tasks")
def list_tasks() -> list[dict[str, Any]]:
return [_with_task_view_model(item) for item in store.list_tasks()]
@app.post("/api/tasks")
def create_task(payload: CreateTaskRequest) -> dict[str, Any]:
try:
return _queue_task(payload.type, payload.params)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.get("/api/tasks/{task_id}")
def get_task(task_id: str) -> dict[str, Any]:
try:
return store.get_task(task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
@app.get("/api/tasks/{task_id}/logs")
def get_task_logs(task_id: str):
try:
task = store.get_task(task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
path = Path(task["run_dir"]) / "logs" / "task.log"
return PlainTextResponse(path.read_text(encoding="utf-8") if path.is_file() else "")
@app.get("/api/tasks/{task_id}/artifacts")
def get_task_artifacts(task_id: str) -> list[dict[str, Any]]:
try:
store.get_task(task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
return [{**item, "url": f"/api/artifacts/{item['id']}/content"} for item in store.list_task_artifacts(task_id)]
@app.get("/api/tasks/{task_id}/reports")
def get_task_reports(task_id: str) -> dict[str, Any]:
try:
return collect_task_reports(store, task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
@app.post("/api/tasks/{task_id}/cancel")
def cancel_task(task_id: str) -> dict[str, Any]:
try:
return store.cancel_task(task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
@app.post("/api/tasks/{task_id}/retry")
def retry_task(task_id: str) -> dict[str, Any]:
try:
return store.retry_task(task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
@app.patch("/api/tasks/{task_id}")
def update_task(task_id: str, payload: UpdateTaskRequest) -> dict[str, Any]:
try:
return store.update_task_params(task_id, params=payload.params)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
@app.delete("/api/tasks/{task_id}")
def delete_task(task_id: str) -> dict[str, Any]:
try:
store.delete_task(task_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="task not found") from exc
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"ok": True}
@app.get("/api/artifacts/{artifact_id}/content")
def artifact_content(artifact_id: str):
try:
artifact = store.get_artifact(artifact_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail="artifact not found") from exc
path = Path(artifact["path"])
if not path.is_file():
raise HTTPException(status_code=404, detail="artifact file not found")
return FileResponse(path, filename=path.name)
@app.get("/api/config/default")
def default_config() -> dict[str, Any]:
return {"default_renderer": cfg.default_renderer, "simulators": ["mujoco", "motrixsim"], "port": cfg.port}
app.mount("/runs", StaticFiles(directory=cfg.runs_dir), name="runs")
return app
def _entity_manifest(entity: dict[str, Any], *, fallback_kind: str) -> dict[str, Any]:
manifest_path = Path(entity.get("manifest_path") or "")
if manifest_path.is_file():
return _postprocess_manifest(json.loads(manifest_path.read_text(encoding="utf-8")), fallback_kind=fallback_kind)
path = Path(entity["path"])
candidates = []
if fallback_kind == "dataset":
candidates = [path / "dataset_manifest.json", path / "manifest.json"]
elif fallback_kind == "scene":
candidates = [path.parent / "scene_manifest.json", path.parent / "package_manifest.json", path]
elif fallback_kind == "model":
candidates = [path / "model_card.json", path / "manifest.json"]
for candidate in candidates:
if candidate.is_file():
return _postprocess_manifest(json.loads(candidate.read_text(encoding="utf-8")), fallback_kind=fallback_kind)
return _postprocess_manifest({"schema": f"real2sim.{fallback_kind}_manifest.synthetic.v1", "id": entity["id"], "name": entity["name"], "path": entity["path"], "source_type": entity.get("source_type"), "source_uri": entity.get("source_uri"), "metrics": entity.get("metrics", {})}, fallback_kind=fallback_kind)
def _scene_manifest_path_for_write(scene: dict[str, Any]) -> Path:
manifest_path = Path(scene.get("manifest_path") or "")
if manifest_path.is_file():
return manifest_path
scene_path = Path(scene["path"])
package_dir = scene_path if scene_path.is_dir() else scene_path.parent
return package_dir / "scene_manifest.json"
def _read_manifest_or_default(scene: dict[str, Any], manifest_path: Path) -> dict[str, Any]:
if manifest_path.is_file():
payload = json.loads(manifest_path.read_text(encoding="utf-8"))
if isinstance(payload, dict):
payload.setdefault("assets", [])
return payload
return {
"schema": "real2sim.scene_manifest.v1",
"name": scene.get("name") or scene.get("id"),
"input_path": scene.get("source_uri") or "",
"renderer": scene.get("metadata", {}).get("renderer", "auto"),
"renderer_status": {},
"simulators": scene.get("metadata", {}).get("simulators", []),
"dry_run": scene.get("metadata", {}).get("dry_run", False),
"assets": [],
"simulator_artifacts": {},
}
def _scene_asset_entry(path: Path, *, url: str, name: str | None, kind: str | None, role: str, transform: dict[str, Any]) -> dict[str, Any]:
size_bytes = path.stat().st_size
asset_kind = kind or _scene_asset_kind(path)
autoload = asset_kind in {"splat_or_pointcloud", "mesh"} and size_bytes <= 25 * 1024 * 1024
return {
"name": name or path.name,
"path": str(path),
"url": url,
"kind": asset_kind,
"role": role or "added_asset",
"format": path.suffix.lower().lstrip("."),
"size_bytes": size_bytes,
"autoload": autoload,
"viewer_hint": "autoload" if autoload else "manual_load",
"transform": transform or {},
"added_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
}
def _scene_asset_kind(path: Path) -> str:
suffix = path.suffix.lower()
if suffix in {".ply", ".splat", ".spz", ".ksplat", ".sog", ".zip", ".rad"}:
return "splat_or_pointcloud"
if suffix in {".glb", ".gltf", ".obj"}:
return "mesh"
if suffix in {".png", ".jpg", ".jpeg", ".webp", ".bmp"}:
return "texture"
if suffix in {".xml"}:
return "simulator"
if suffix in {".json", ".jsonl"}:
return "metadata"
return "file"
def _postprocess_manifest(manifest: dict[str, Any], *, fallback_kind: str) -> dict[str, Any]:
if fallback_kind == "scene":
return enrich_scene_manifest_assets(manifest)
return manifest
def _with_view_model(entity: dict[str, Any], kind: str) -> dict[str, Any]:
item = dict(entity)
manifest = _safe_entity_manifest(item, kind)
primary_asset = _primary_asset(manifest)
thumbnail = _thumbnail_from_manifest(item, manifest, primary_asset)
metrics = item.get("metrics") or {}
scan = item.get("scan") or {}
item["view"] = {
"display_name": item.get("name") or Path(str(item.get("path") or "")).stem or item.get("id", ""),
"thumbnail_url": thumbnail,
"preview_url": _preview_url(kind, item, primary_asset),
"primary_asset": primary_asset,
"metrics_summary": _metrics_summary(kind, metrics, scan, manifest),
"subtitle": _resource_subtitle(kind, item, manifest),
}
return item
def _with_task_view_model(task: dict[str, Any]) -> dict[str, Any]:
item = dict(task)
progress = int(float(item.get("progress") or 0))
if item.get("status") == "succeeded":
progress = 100
item["view"] = {
"display_name": _task_display_name(item),
"stage_label": item.get("stage") or _task_type_label(str(item.get("type") or "")),
"progress": max(0, min(100, progress)),
"message": item.get("message") or item.get("error") or "",
"updated_label": time.strftime("%m/%d %H:%M", time.localtime(float(item.get("updated_at") or item.get("created_at") or time.time()))),
}
return item
def _safe_entity_manifest(entity: dict[str, Any], kind: str) -> dict[str, Any]:
try:
return _entity_manifest(entity, fallback_kind=kind)
except Exception:
return {}
def _primary_asset(manifest: dict[str, Any]) -> dict[str, Any] | None:
assets = manifest.get("assets") if isinstance(manifest, dict) else []
if not isinstance(assets, list):
return None
candidates = [asset for asset in assets if isinstance(asset, dict)]
if not candidates:
return None
return sorted(candidates, key=_asset_view_rank)[0]
def _asset_view_rank(asset: dict[str, Any]) -> tuple[int, int, str]:
name = str(asset.get("name") or asset.get("path") or "").lower()
path_text = str(asset.get("path") or "").lower()
kind = str(asset.get("kind") or "").lower()
size = int(asset.get("size_bytes") or 0)
if asset.get("cover") or name == "scene_cover.png":
rank = 0
elif name.endswith("_threejs_rgb.ply"):
rank = 1
elif name.endswith((".splat", ".spz", ".ksplat", ".sog")):
rank = 2
elif name == "pose_init_trained_scene.ply":
rank = 3
elif name == "scene.ply" and "/3dgs/" in path_text:
rank = 4
elif name in {"pointcloud_projection_preview.png", "source_gs_preview.png"}:
rank = 5
elif name.endswith((".png", ".jpg", ".jpeg", ".webp")) and ("preview" in name or "render" in name):
rank = 6
elif name.startswith("thumb_") and name.endswith((".png", ".jpg", ".jpeg", ".webp")):
rank = 7
elif name == "scene.ply":
rank = 8
elif kind in {"mesh", "splat_or_pointcloud"} and "sparse" not in name:
rank = 9
elif name in {"background_image.png"}:
rank = 10
elif name.endswith((".png", ".jpg", ".jpeg", ".webp")):
rank = 11
elif "colmap_sparse" in name or "sparse_points" in name:
rank = 12
else:
rank = 13
return (rank, -size, name)
def _thumbnail_from_manifest(entity: dict[str, Any], manifest: dict[str, Any], primary_asset: dict[str, Any] | None) -> str:
assets = manifest.get("assets") if isinstance(manifest, dict) else []
if isinstance(assets, list):
image_assets = [asset for asset in assets if isinstance(asset, dict) and str(asset.get("name") or asset.get("path") or "").lower().endswith((".png", ".jpg", ".jpeg", ".webp"))]
image_assets.sort(key=_asset_view_rank)
for asset in image_assets:
if asset.get("url"):
return str(asset["url"])
dataset_thumb = _dataset_thumbnail_url(entity)
if dataset_thumb:
return dataset_thumb
return str(primary_asset.get("url") or "") if primary_asset and str(primary_asset.get("kind") or "") == "image" else ""
def _dataset_thumbnail_url(entity: dict[str, Any]) -> str:
root = Path(str(entity.get("path") or ""))
candidates = [
root / "thumbnails" / "thumb_01.jpg",
root / "thumbnails" / "thumb_01.png",
root / "thumbnail.jpg",
root / "preview.jpg",
root / "preview.png",
]
for path in candidates:
if path.is_file():
return _workspace_public_url(path)
return ""
def _workspace_public_url(path: Path) -> str:
parts = path.parts
for marker in ("runs", "datasets"):
if marker in parts:
index = parts.index(marker)
return "/" + "/".join(parts[index:])
return ""
def _preview_url(kind: str, entity: dict[str, Any], primary_asset: dict[str, Any] | None) -> str:
if kind == "scene":
return f"/static/scene-viewer.html?scene_id={entity['id']}"
return str(primary_asset.get("url") or "") if primary_asset else ""
def _metrics_summary(kind: str, metrics: dict[str, Any], scan: dict[str, Any], manifest: dict[str, Any]) -> str:
if kind == "dataset":
frames = metrics.get("frame_count") or manifest.get("frame_count") or scan.get("image_count")
videos = scan.get("video_count")
samples = metrics.get("samples") or scan.get("samples")
parts = []
if frames:
parts.append(f"{frames} 帧")
if videos:
parts.append(f"{videos} 视频")
if samples:
parts.append(f"{samples} 样本")
return " · ".join(parts) or "等待扫描"
if kind == "scene":
assets = manifest.get("assets") if isinstance(manifest.get("assets"), list) else []
return f"{len(assets)} 个资产" if assets else "暂无资产"
score = metrics.get("success_rate") or metrics.get("score") or metrics.get("accuracy")
loss = metrics.get("loss") or metrics.get("val_loss")
parts = []
if score is not None:
parts.append(f"成功率 {float(score) * 100:.1f}%" if float(score) <= 1 else f"成功率 {float(score):.1f}%")
if loss is not None:
parts.append(f"loss {float(loss):.3f}")
return " · ".join(parts) or "暂无指标"
def _resource_subtitle(kind: str, entity: dict[str, Any], manifest: dict[str, Any]) -> str:
source = str(entity.get("source_type") or manifest.get("source_type") or "")
version = str(entity.get("version") or "")
label = {"prepared_media": "已准备帧数据", "expert_collection": "专家采集数据", "scene_reconstruction": "场景重建", "huggingface": "HuggingFace", "upload": "上传文件", "local_path": "本地导入"}.get(source, source or kind)
return f"{label}{' · ' + version if version else ''}"
def _task_display_name(task: dict[str, Any]) -> str:
params = task.get("params") or {}
return str(params.get("scene_name") or params.get("output_name") or _task_type_label(str(task.get("type") or "")))
def _task_type_label(task_type: str) -> str:
return {
"hf_import": "数据导入",
"media_prepare": "媒体准备",
"reconstruction_preflight": "重建预检",
"scene_reconstruction": "场景重建",
"simulator_validation": "仿真校验",
"expert_collection": "专家采集",
"training": "模型训练",
"inference": "在线推理",
"renderer_benchmark": "渲染测试",
"scene_splat_optimize": "3DGS 优化",
}.get(task_type, task_type)
def _validate_task_payload(payload: CreateTaskRequest) -> None:
params = payload.params
if payload.type == "hf_import":
repo_type = params.get("repo_type", "dataset")
if repo_type != "dataset":
raise HTTPException(status_code=400, detail="hf_import only supports repo_type='dataset'")
if not params.get("repo_id"):
raise HTTPException(status_code=400, detail="hf_import requires repo_id")
if payload.type == "media_prepare":
if not (params.get("dataset_id") or params.get("input_path")):
raise HTTPException(status_code=400, detail="media_prepare requires dataset_id or input_path")
if float(params.get("sample_fps", 2.0)) <= 0:
raise HTTPException(status_code=400, detail="sample_fps must be > 0")
if int(params.get("max_frames_per_video", 80)) <= 0:
raise HTTPException(status_code=400, detail="max_frames_per_video must be > 0")
if int(params.get("target_frame_count", 0) or 0) < 0:
raise HTTPException(status_code=400, detail="target_frame_count must be >= 0")
if int(params.get("min_total_frames", 0) or 0) < 0:
raise HTTPException(status_code=400, detail="min_total_frames must be >= 0")
if payload.type == "reconstruction_preflight":
if not (params.get("dataset_id") or params.get("input_path")):
raise HTTPException(status_code=400, detail="reconstruction_preflight requires dataset_id or input_path")
if int(params.get("min_frames", 30)) <= 0:
raise HTTPException(status_code=400, detail="min_frames must be > 0")
if payload.type == "scene_reconstruction":
renderer = params.get("renderer", "auto")
if renderer not in {"auto", "spark", "aholo"}:
raise HTTPException(status_code=400, detail="renderer must be auto, spark, or aholo")
simulators = set(params.get("simulators", ["mujoco", "motrixsim"]))
if not simulators <= {"mujoco", "motrixsim"}:
raise HTTPException(status_code=400, detail="simulators must be mujoco and/or motrixsim")
if not (params.get("dataset_id") or params.get("input_path")):
raise HTTPException(status_code=400, detail="scene_reconstruction requires dataset_id or input_path")
if int(params.get("min_frames", 30)) <= 0:
raise HTTPException(status_code=400, detail="min_frames must be > 0")
if payload.type == "simulator_validation":
if not (params.get("scene_id") or params.get("scene_path")):
raise HTTPException(status_code=400, detail="simulator_validation requires scene_id or scene_path")
if params.get("level", "runtime_if_available") not in {"smoke", "runtime_if_available", "runtime_required"}:
raise HTTPException(status_code=400, detail="level must be smoke, runtime_if_available, or runtime_required")
if payload.type == "expert_collection":
if not (params.get("scene_id") or params.get("scene_path")):
raise HTTPException(status_code=400, detail="expert_collection requires scene_id or scene_path")
if params.get("policy_preset", "pick_and_place_v1") != "pick_and_place_v1":
raise HTTPException(status_code=400, detail="当前仅支持抓取放置策略;导航和精细操作策略尚未接入执行器")
if int(params.get("workers", 1)) != 1:
raise HTTPException(status_code=400, detail="当前专家采集执行器仅支持 workers=1")
if int(params.get("episodes", 4)) <= 0 or int(params.get("steps", 24)) <= 0:
raise HTTPException(status_code=400, detail="episodes and steps must be > 0")
if payload.type == "training":
if not (params.get("dataset_id") or params.get("dataset_path")):
raise HTTPException(status_code=400, detail="training requires dataset_id or dataset_path")
if params.get("recipe", "bc_v2") not in {"bc_v2", "bc_fast"}:
raise HTTPException(status_code=400, detail="当前仅支持行为克隆 v2 和快速验证;扩散策略尚未接入训练器")
if int(params.get("epochs", 100)) <= 0:
raise HTTPException(status_code=400, detail="epochs must be > 0")
if payload.type == "inference":
if not (params.get("model_id") or params.get("model_path")):
raise HTTPException(status_code=400, detail="inference requires model_id or model_path")
if not (params.get("scene_id") or params.get("scene_path")):
raise HTTPException(status_code=400, detail="inference requires scene_id or scene_path")
if int(params.get("workers", 1)) != 1:
raise HTTPException(status_code=400, detail="当前推理执行器仅支持 workers=1")
if int(params.get("steps", 80)) <= 0:
raise HTTPException(status_code=400, detail="steps must be > 0")
def _validate_task_resources(store: PlatformStore, payload: CreateTaskRequest) -> None:
params = payload.params
try:
if payload.type == "expert_collection" and params.get("scene_id"):
store.get_scene(str(params["scene_id"]))
if payload.type == "training" and params.get("dataset_id"):
dataset = store.get_dataset(str(params["dataset_id"]))
source = str(dataset.get("source_type") or "")
input_kind = str(dataset.get("scan", {}).get("input_kind") or "")
if source != "expert_collection" and input_kind != "expert_rollouts":
raise HTTPException(status_code=400, detail="训练只能使用专家采集产生的 rollout 数据集")
if payload.type == "inference":
if params.get("model_id"):
store.get_model(str(params["model_id"]))
if params.get("scene_id"):
store.get_scene(str(params["scene_id"]))
except KeyError as exc:
raise HTTPException(status_code=400, detail=f"任务引用的资源不存在: {exc.args[0]}") from exc
def _runtime_paths(cfg: PlatformConfig) -> dict[str, str]:
return {
"gs_playground_root": str(cfg.gs_playground_root),
"runtime_root": str(cfg.runtime_root),
"anysplat_root": str(cfg.anysplat_root),
"anysplat_python": str(cfg.anysplat_python),
"gsplat_python": str(cfg.gsplat_python),
"cuda_home": str(cfg.cuda_home),
"runtime_env_script": str(cfg.runtime_env_script),
}
def _config_digest(cfg: PlatformConfig) -> str:
payload = json.dumps(_runtime_paths(cfg) | {"workspace": str(cfg.workspace_dir), "renderer": cfg.default_renderer}, sort_keys=True)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:16]
def _git_commit(root: Path) -> str:
try:
return subprocess.check_output(["git", "-C", str(root), "rev-parse", "--short", "HEAD"], text=True, stderr=subprocess.DEVNULL).strip()
except Exception:
return ""
def _source_snapshot(root: Path) -> str:
files = [root / "pyproject.toml", root / "src/real2sim_platform/adapters.py", root / "src/real2sim_platform/server.py", root / "src/real2sim_platform/config.py"]
digest = hashlib.sha256()
for path in files:
if path.is_file():
digest.update(path.name.encode("utf-8"))
digest.update(path.read_bytes())
return digest.hexdigest()[:16]
def _safe_extract(archive: zipfile.ZipFile, target_dir: Path) -> None:
root = target_dir.resolve()
for member in archive.infolist():
destination = (target_dir / member.filename).resolve()
try:
destination.relative_to(root)
except ValueError as exc:
raise HTTPException(status_code=400, detail=f"unsafe zip member: {member.filename}") from exc
archive.extractall(target_dir)
def _next_run_name(root: Path, prefix: str) -> str:
root.mkdir(parents=True, exist_ok=True)
index = 1
while (root / f"{prefix}_{index:04d}").exists():
index += 1
return f"{prefix}_{index:04d}"
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--config", type=Path)
parser.add_argument("--write-config", type=Path)
parser.add_argument("--host")
parser.add_argument("--port", type=int)
args = parser.parse_args()
cfg = load_config(args.config)
if args.host:
cfg = PlatformConfig(**{**cfg.__dict__, "host": args.host})
if args.port:
cfg = PlatformConfig(**{**cfg.__dict__, "port": args.port})
if args.write_config:
write_default_config(args.write_config, cfg)
print(args.write_config)
return 0
import uvicorn
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port)
return 0
if __name__ == "__main__":
raise SystemExit(main())

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,586 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Real2Sim</title>
<link rel="icon" href="data:,">
<link rel="stylesheet" href="/static/styles.css?v=20260710-motrix-runtime-v7">
</head>
<body>
<div class="app-shell">
<aside class="sidebar">
<div class="brand">
<strong>Real2Sim</strong>
<span>真实数据到仿真智能体</span>
</div>
<nav class="side-nav" aria-label="主导航">
<button class="tab active" data-tab="overview">总览</button>
<button class="tab" data-tab="data">数据管理</button>
<button class="tab" data-tab="reconstruct">场景重建</button>
<button class="tab" data-tab="scenes">场景管理</button>
<button class="tab" data-tab="collect">数据采集</button>
<button class="tab" data-tab="train">模型训练</button>
<button class="tab" data-tab="infer">在线推理</button>
</nav>
</aside>
<main class="main-surface">
<section id="overview" class="page active page-dashboard">
<div class="page-head">
<div>
<h1>总览</h1>
<p>按数据准备、场景重建、专家采集、训练和推理组织当前项目资源。</p>
</div>
<button id="nextStepAction" class="primary-button">继续下一步</button>
</div>
<div class="overview-grid">
<section class="panel journey-panel">
<div class="panel-head">
<div>
<h2>我的流程</h2>
<p>从真实数据到在线推理的当前进度</p>
</div>
</div>
<div class="journey" id="journeySteps"></div>
</section>
<section class="panel next-step">
<div class="panel-head">
<div>
<h2>推荐下一步</h2>
<p id="nextStepText">正在读取当前资源状态</p>
</div>
</div>
<div id="overviewMetrics" class="metric-strip"></div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>最近任务</h2>
<p>显示真实阶段、进度和失败原因</p>
</div>
</div>
<div id="recentTaskList" class="list-stack compact-list"></div>
</section>
<section class="panel wide scene-hero-panel">
<div class="panel-head">
<div>
<h2>最近场景</h2>
<p>重建完成后可直接进入预览和资产增强</p>
</div>
</div>
<div id="overviewScenePreview" class="overview-scene"></div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>模型与推理</h2>
<p>训练结果、模型指标和最近推理轨迹</p>
</div>
</div>
<div id="latestModelCard" class="model-summary"></div>
</section>
</div>
</section>
<section id="data" class="page page-data">
<div class="page-head">
<div>
<h1>数据管理</h1>
<p>导入数据、抽帧准备、预检质量,并把可用数据集交给重建或训练流程。</p>
</div>
<button class="primary-button" data-open-import="hf">新增数据</button>
</div>
<div class="source-grid">
<button class="source-card" data-open-import="hf">
<strong>HuggingFace 样例</strong>
<span>xarm7、3DGS fixture 和公开数据集</span>
</button>
<button class="source-card" data-open-import="upload">
<strong>上传文件</strong>
<span>zip、视频或图片包</span>
</button>
<button class="source-card" data-open-import="local">
<strong>本地目录</strong>
<span>从配置白名单导入已有数据</span>
</button>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>数据集</h2>
<p>多选后可批量准备、预检或删除索引</p>
</div>
<div class="toolbar">
<select id="datasetFilter" aria-label="筛选数据集">
<option value="all">全部数据</option>
<option value="prepared_media">已准备帧数据</option>
<option value="expert_collection">采集数据</option>
<option value="huggingface">HuggingFace</option>
</select>
<input id="datasetSearch" placeholder="搜索数据集名称">
</div>
</div>
<div class="bulk-bar" data-bulk-resource-scope="datasetList">
<label><input type="checkbox" data-select-all-resources="datasetList"> 全选</label>
<span data-selected-count="datasetList">已选择 0 项</span>
<button data-bulk-resource-action="prepare">批量准备</button>
<button data-bulk-resource-action="preflight">批量预检</button>
<button class="danger-button" data-bulk-resource-action="delete">批量删除</button>
</div>
<div class="table-shell dataset-shell">
<div class="table-header dataset-header">
<span></span><span>缩略图</span><span>数据集名称</span><span>来源</span><span>规模</span><span>操作</span>
</div>
<div id="datasetList" class="resource-table"></div>
</div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>准备输入</h2>
<p>将视频或图片目录整理成可重建帧数据</p>
</div>
<button id="createMediaPrepare" class="primary-button">创建准备任务</button>
</div>
<div class="form-grid">
<label class="span-2">原始数据集<select id="prepareDataset"></select></label>
<label>相机视角<input id="prepareCamera" value="observation.images.front"></label>
<label>每秒抽帧<input id="prepareFps" type="number" min="0.1" step="0.1" value="2"></label>
<label>每段最多帧<input id="prepareMaxFrames" type="number" min="1" value="80"></label>
<label>最多视频数<input id="prepareMaxVideos" type="number" min="1" value="1"></label>
<label>目标总帧数<input id="prepareTargetFrames" type="number" min="0" value="0"></label>
<label>最低总帧数<input id="prepareMinFrames" type="number" min="0" value="0"></label>
</div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>数据任务</h2>
<p>导入、上传和准备任务会在这里反馈进度</p>
</div>
</div>
<div class="bulk-bar" data-bulk-scope="dataTaskList">
<label><input type="checkbox" data-select-all-tasks="dataTaskList"> 全选</label>
<span data-selected-count="dataTaskList">已选择 0 项</span>
<button data-bulk-task-action="retry">批量重试</button>
<button data-bulk-task-action="cancel">批量取消</button>
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
</div>
<div id="dataTaskList" class="list-stack"></div>
</section>
</section>
<section id="reconstruct" class="page page-reconstruct">
<div class="page-head">
<div>
<h1>场景重建</h1>
<p>场景重建负责生成新的点云、网格、场景包和仿真导入产物;完成后进入场景管理继续预览和增强。</p>
</div>
<button id="createSceneTask" class="primary-button">开始重建</button>
</div>
<div class="reconstruct-grid">
<section class="panel">
<div class="panel-head">
<div>
<h2>重建配置</h2>
<p>先选 prepared 数据,再选择质量、几何和仿真策略</p>
</div>
</div>
<div id="reconstructDatasetCards" class="selection-card-rail"></div>
<div class="form-grid">
<label class="span-2">场景名称<input id="sceneName" placeholder="例如 Playroom 高质量重建"></label>
<label class="span-2">已准备数据集<select id="reconstructInput"></select></label>
<label>重建预设<select id="qualityPreset"><option value="discoverse-high">DISCOVERSE 高质量</option><option value="motrix-high">仿真优先</option><option value="balanced">平衡</option></select></label>
<label>渲染方式<select id="rendererSelect"><option value="auto">自动选择</option><option value="spark">Spark 预览</option><option value="aholo">Aholo LOD</option></select></label>
<label>仿真器<select id="simulatorSelect"><option value="both">MuJoCo + MotrixSim</option><option value="mujoco">MuJoCo</option><option value="motrixsim">MotrixSim</option></select></label>
<label>最少帧数<input id="sceneMinFrames" type="number" min="1" value="30"></label>
<label>验收标准<select id="acceptanceLevel"><option value="balanced">平衡</option><option value="strict">严格</option><option value="preview">仅预览</option></select></label>
</div>
<details class="option-group">
<summary>高级重建参数</summary>
<div class="form-grid">
<label>相机模型<select id="cameraModel"><option value="auto">自动识别</option><option value="SIMPLE_RADIAL">SIMPLE_RADIAL</option><option value="PINHOLE">PINHOLE</option><option value="OPENCV">OPENCV</option></select></label>
<label>匹配策略<select id="matcherSelect"><option value="exhaustive">全量匹配</option><option value="sequential">顺序匹配</option></select></label>
<label>最大图像边<input id="maxImageSize" type="number" min="512" step="128" value="1600"></label>
<label>SIFT 特征数<input id="siftFeatures" type="number" min="1000" step="1000" value="8192"></label>
<label>顺序重叠帧<input id="sequentialOverlap" type="number" min="1" value="12"></label>
<label>AnySplat 视角数<input id="anysplatMaxViews" type="number" min="8" step="8" value="80"></label>
<label>训练步数<input id="poseTrainSteps" type="number" min="1000" step="1000" value="15000"></label>
<label>目标高斯数<input id="targetGaussians" type="number" min="100000" step="100000" value="1500000"></label>
<label>尺度参考<select id="scaleHint"><option value="auto">自动估计</option><option value="metric_room">房间尺度</option><option value="tabletop">桌面尺度</option><option value="robot_workspace">机器人工作区</option></select></label>
<label>坐标系<select id="coordinateFrame"><option value="gs_metric_zup">Z-up 米制</option><option value="mujoco_zup">MuJoCo Z-up</option><option value="motrixsim_world">MotrixSim World</option></select></label>
<label class="check"><input id="autoPrepareScene" type="checkbox"> 自动准备输入</label>
<label class="check"><input id="allowLowQualityScene" type="checkbox"> 允许低质量继续</label>
<label class="check"><input id="guidedMatching" type="checkbox" checked> 引导匹配</label>
<label class="check"><input id="poseInitTrain" type="checkbox" checked> 位姿初始化训练</label>
<label class="check"><input id="replaceSceneWithPoseTrain" type="checkbox" checked> 用训练结果替换主场景</label>
<label class="check"><input id="exportTextures" type="checkbox" checked> 生成纹理资产</label>
<label class="check"><input id="keepIntermediate" type="checkbox" checked> 保留中间产物</label>
<label class="check muted-check"><input id="dryRunScene" type="checkbox"> 仅生成样例</label>
</div>
</details>
<div class="button-row">
<button id="createPreflightTask">只做预检</button>
<button id="rendererBenchmark">渲染器对比</button>
</div>
</section>
<section class="panel preflight-panel">
<div class="panel-head">
<div>
<h2>质量预检</h2>
<p>帧数、输入类型、验收档位和运行建议</p>
</div>
</div>
<div id="preflightSummary" class="quality-grid"></div>
</section>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>重建任务与预览</h2>
<p>任务创建后立即出现在这里;成功后可直接预览</p>
</div>
</div>
<div class="bulk-bar" data-bulk-scope="sceneTaskList">
<label><input type="checkbox" data-select-all-tasks="sceneTaskList"> 全选</label>
<span data-selected-count="sceneTaskList">已选择 0 项</span>
<button data-bulk-task-action="retry">批量重试</button>
<button data-bulk-task-action="cancel">批量取消</button>
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
</div>
<div id="sceneTaskList" class="list-stack"></div>
</section>
</section>
<section id="scenes" class="page">
<div class="page-head">
<div>
<h1>场景管理</h1>
<p>场景管理用于查看重建结果、添加资产、仿真校验、导出,并绑定后续采集和推理。</p>
</div>
<button id="validateSelectedScenes" class="primary-button">校验已选场景</button>
</div>
<div class="scene-workbench">
<section class="panel scene-library">
<div class="panel-head">
<div>
<h2>场景库</h2>
<p>多选后可校验、导出或删除索引</p>
</div>
</div>
<div class="bulk-bar" data-bulk-resource-scope="sceneList">
<label><input type="checkbox" data-select-all-resources="sceneList"> 全选</label>
<span data-selected-count="sceneList">已选择 0 项</span>
<button data-bulk-resource-action="validate">批量校验</button>
<button data-bulk-resource-action="export">批量导出</button>
<button class="danger-button" data-bulk-resource-action="delete">批量删除</button>
</div>
<div id="sceneList" class="resource-table scene-list"></div>
</section>
<section class="panel viewer-panel">
<div class="panel-head">
<div>
<h2>场景预览</h2>
<p id="viewerStatus">选择场景后加载 Spark/Three 预览</p>
</div>
<button id="openSceneViewerFull" class="ghost-button" type="button">最大化预览</button>
</div>
<iframe id="scenePreview" title="场景预览"></iframe>
</section>
<section class="panel asset-editor-panel">
<div class="panel-head">
<div>
<h2>场景增强</h2>
<p>从当前场景包选择资产,或在高级入口导入白名单文件</p>
</div>
<button id="addSceneAsset" class="primary-button">添加资产</button>
</div>
<div class="form-grid">
<label class="span-2">目标场景<select id="assetScene"></select></label>
<label class="span-2">资产来源<select id="assetPreset"></select></label>
<details class="span-2 option-group">
<summary>高级:从白名单路径添加新资产</summary>
<label>资产文件<input id="assetPath" placeholder="仅支持配置白名单内文件"></label>
</details>
<label>显示名称<input id="assetName" placeholder="例如柜体碰撞模型"></label>
<label>资产类型<select id="assetKind"><option value="auto">自动识别</option><option value="mesh">网格 / 物体</option><option value="splat_or_pointcloud">点云 / Splat</option><option value="texture">纹理</option><option value="simulator">仿真文件</option><option value="metadata">配置 / 元数据</option></select></label>
<label>用途<select id="assetRole"><option value="added_asset">新增资产</option><option value="collision">碰撞体</option><option value="visual">可视化模型</option><option value="texture">纹理贴图</option><option value="simulator_import">仿真导入</option></select></label>
<label>尺度<input id="assetScale" type="number" min="0.001" step="0.001" value="1"></label>
<label>位置 X<input id="assetPosX" type="number" step="0.01" value="0"></label>
<label>位置 Y<input id="assetPosY" type="number" step="0.01" value="0"></label>
<label>位置 Z<input id="assetPosZ" type="number" step="0.01" value="0"></label>
<label>朝向 Yaw<input id="assetYaw" type="number" step="1" value="0"></label>
</div>
</section>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>资产清单</h2>
<p>点云、网格、纹理、相机和仿真产物</p>
</div>
</div>
<div id="assetRail" class="asset-grid"></div>
</section>
</section>
<section id="collect" class="page">
<div class="page-head">
<div>
<h1>数据采集</h1>
<p>选择已重建场景和专家策略,批量生成可训练的 rollout 数据。</p>
</div>
<button id="createCollectTask" class="primary-button">开始采集</button>
</div>
<div class="two-column">
<section class="panel">
<div class="panel-head">
<div>
<h2>采集配置</h2>
<p>场景、策略和 rollout 规模</p>
</div>
</div>
<div class="form-grid">
<label class="span-2">选择场景<select id="collectScene"></select></label>
<label>专家策略<select id="policyPreset"><option value="pick_and_place_v1">抓取放置策略</option><option value="navigation_v1" disabled>导航采集策略(待接入)</option><option value="manipulation_v2" disabled>精细操作策略(待接入)</option></select></label>
<label>执行并发<input id="collectWorkers" type="number" min="1" max="1" value="1" disabled title="当前专家采集执行器仅支持单 worker"></label>
<label>Episode 数<input id="episodes" type="number" value="4"></label>
<label>每个 Episode 步数<input id="steps" type="number" value="24"></label>
</div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>采集数据集</h2>
<p>生成后可直接用于模型训练</p>
</div>
</div>
<div id="collectionDatasetList" class="resource-table compact-table"></div>
</section>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>采集任务</h2>
<p>支持多选取消、重试、删除和查看产物</p>
</div>
</div>
<div class="bulk-bar" data-bulk-scope="collectTaskList">
<label><input type="checkbox" data-select-all-tasks="collectTaskList"> 全选</label>
<span data-selected-count="collectTaskList">已选择 0 项</span>
<button data-bulk-task-action="retry">批量重试</button>
<button data-bulk-task-action="cancel">批量取消</button>
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
</div>
<div id="collectTaskList" class="list-stack"></div>
</section>
</section>
<section id="train" class="page">
<div class="page-head">
<div>
<h1>模型训练</h1>
<p>基于专家采集数据训练策略模型,并追踪指标、模型卡和导出产物。</p>
</div>
<button id="createTrainTask" class="primary-button">开始训练</button>
</div>
<div class="two-column">
<section class="panel">
<div class="panel-head">
<div>
<h2>训练配置</h2>
<p>选择训练数据和配方</p>
</div>
</div>
<div class="form-grid">
<label class="span-2">训练数据集<select id="trainDataset"></select></label>
<label>训练配方<select id="trainRecipe"><option value="bc_v2">行为克隆 v2</option><option value="bc_fast">快速验证</option><option value="diffusion_policy" disabled>扩散策略(待接入)</option></select></label>
<label>Epoch 数<input id="epochs" type="number" value="100"></label>
</div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>训练指标</h2>
<p>成功率、损失和验证集表现</p>
</div>
</div>
<div id="trainingChart" class="chart-panel"></div>
</section>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>训练任务</h2>
<p>查看训练进度和模型产物</p>
</div>
</div>
<div class="bulk-bar" data-bulk-scope="trainTaskList">
<label><input type="checkbox" data-select-all-tasks="trainTaskList"> 全选</label>
<span data-selected-count="trainTaskList">已选择 0 项</span>
<button data-bulk-task-action="retry">批量重试</button>
<button data-bulk-task-action="cancel">批量取消</button>
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
</div>
<div id="trainTaskList" class="list-stack"></div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>模型列表</h2>
<p>多选后可对比、导出或删除索引</p>
</div>
</div>
<div class="bulk-bar" data-bulk-resource-scope="modelList">
<label><input type="checkbox" data-select-all-resources="modelList"> 全选</label>
<span data-selected-count="modelList">已选择 0 项</span>
<button data-bulk-resource-action="compare">批量对比</button>
<button data-bulk-resource-action="export">批量导出</button>
<button class="danger-button" data-bulk-resource-action="delete">批量删除</button>
</div>
<div id="modelList" class="model-grid"></div>
<div id="modelCompareResult" class="compare-result"></div>
</section>
</section>
<section id="infer" class="page">
<div class="page-head">
<div>
<h1>在线推理</h1>
<p>选择模型和场景运行在线推理,查看 rollout 播放、轨迹产物和对比结果。</p>
</div>
<button id="createInferTask" class="primary-button">开始推理</button>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>推理配置</h2>
<p>模型、场景和 rollout 参数</p>
</div>
</div>
<div class="form-grid">
<label>选择模型<select id="inferModel"></select></label>
<label>选择场景<select id="inferScene"></select></label>
<label>步数上限<input id="inferSteps" type="number" min="1" value="80"></label>
<label>执行并发<input id="inferWorkers" type="number" min="1" max="1" value="1" disabled title="当前推理执行器仅支持单 worker"></label>
</div>
</section>
<div class="two-column">
<section class="panel rollout-panel">
<div class="panel-head">
<div>
<h2>Rollout 回放</h2>
<p>推理完成后展示 viewer 或轨迹预览</p>
</div>
</div>
<div id="rolloutPreview" class="rollout-preview"></div>
</section>
<section class="panel">
<div class="panel-head">
<div>
<h2>轨迹产物</h2>
<p>rollout、viewer 和模型输出</p>
</div>
</div>
<div id="trajectoryList" class="list-stack"></div>
</section>
</div>
<section class="panel">
<div class="panel-head">
<div>
<h2>推理任务</h2>
<p>支持多选处理、重试和删除</p>
</div>
</div>
<div class="bulk-bar" data-bulk-scope="inferTaskList">
<label><input type="checkbox" data-select-all-tasks="inferTaskList"> 全选</label>
<span data-selected-count="inferTaskList">已选择 0 项</span>
<button data-bulk-task-action="retry">批量重试</button>
<button data-bulk-task-action="cancel">批量取消</button>
<button class="danger-button" data-bulk-task-action="delete">批量删除</button>
</div>
<div id="inferTaskList" class="list-stack"></div>
</section>
</section>
</main>
</div>
<aside id="taskDrawer" class="task-drawer" aria-hidden="true">
<div class="drawer-head">
<div>
<span>任务详情</span>
<h2 id="drawerTitle">未选择任务</h2>
</div>
<button id="closeDrawer" class="ghost-button">关闭</button>
</div>
<div id="drawerBody"></div>
</aside>
<dialog id="importDialog">
<form id="importForm" class="dialog-card">
<div class="panel-head">
<div>
<h2 id="importTitle">新增数据</h2>
<p id="importSubtitle">选择来源后提交导入任务</p>
</div>
<button type="button" data-close-dialog="importDialog" class="ghost-button">关闭</button>
</div>
<div id="importFields" class="form-grid"></div>
<menu>
<button type="button" data-close-dialog="importDialog">取消</button>
<button id="submitImport" type="submit" class="primary-button">提交</button>
</menu>
</form>
</dialog>
<dialog id="editDialog">
<form id="editForm" class="dialog-card">
<div class="panel-head">
<div>
<h2 id="editTitle">编辑</h2>
<p id="editSubtitle">更新名称、批次和标签</p>
</div>
<button type="button" data-close-dialog="editDialog" class="ghost-button">关闭</button>
</div>
<div id="editFields" class="form-grid"></div>
<menu>
<button type="button" data-close-dialog="editDialog">取消</button>
<button id="saveEdit" type="submit" class="primary-button">保存</button>
</menu>
</form>
</dialog>
<div id="toast" class="toast" role="status"></div>
<script src="/static/app.js?v=20260710-motrix-runtime-v7"></script>
</body>
</html>

View File

@ -0,0 +1,560 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Real2Sim Scene Viewer</title>
<link rel="icon" href="data:,">
<style>
* { box-sizing: border-box; }
body { margin: 0; overflow: hidden; background: #080b0d; color: #e7edf2; font: 13px/1.4 system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
#hud { position: fixed; z-index: 10; left: 12px; top: 12px; width: min(560px, calc(100vw - 24px)); min-width: 280px; min-height: 76px; max-width: calc(100vw - 24px); max-height: calc(100vh - 24px); padding: 10px 12px; border: 1px solid #2a343d; border-radius: 8px; background: rgba(12, 17, 22, 0.92); box-shadow: 0 12px 30px rgba(0, 0, 0, 0.25); overflow: hidden; }
#hudTitlebar { display: flex; align-items: center; justify-content: space-between; gap: 12px; margin: -10px -12px 8px; padding: 9px 10px 7px 12px; border-bottom: 1px solid rgba(75, 91, 104, 0.45); cursor: grab; user-select: none; touch-action: none; }
#hudTitlebar:active { cursor: grabbing; }
#hud strong { display: block; margin: 0; }
#hudToggle { flex: 0 0 auto; margin: 0; min-width: 28px; min-height: 26px; padding: 2px 7px; font-size: 15px; line-height: 1; }
#hudContent { overflow: auto; max-height: calc(100vh - 116px); }
#hud.is-collapsed { width: auto !important; min-width: 0; min-height: 0; height: auto !important; padding-bottom: 10px; resize: none; }
#hud.is-collapsed #hudTitlebar { margin-bottom: -1px; border-bottom: 0; }
#hud.is-collapsed #hudContent, #hud.is-collapsed #hudResizer { display: none; }
#hudResizer { position: absolute; right: 1px; bottom: 1px; width: 17px; height: 17px; cursor: nwse-resize; touch-action: none; background: linear-gradient(135deg, transparent 46%, #52717a 47%, #52717a 55%, transparent 56%, transparent 64%, #52717a 65%, #52717a 73%, transparent 74%); opacity: 0.8; }
#hud div { color: #9aa8b4; overflow-wrap: anywhere; }
#hint { margin-top: 6px; color: #c4d0dc; }
#assetList { margin-top: 8px; max-height: 120px; overflow: auto; }
#assetList button[data-autoload="false"] { border-color: #6b5a33; }
#assetList button[data-renderable="false"] { opacity: 0.72; border-color: #33404a; }
#assetList button[data-quality="recommended"] { border-color: #28d8bd; box-shadow: inset 0 0 0 1px rgba(40, 216, 189, 0.18); }
#assetList button[data-quality="full"] { border-color: #c9a85d; }
#assetList button[data-quality="diagnostic"] { opacity: 0.72; }
button { margin: 3px 4px 0 0; border: 1px solid #2a343d; border-radius: 6px; background: #17232b; color: #e7edf2; padding: 5px 8px; cursor: pointer; }
canvas { display: block; width: 100vw; height: 100vh; }
@media (max-width: 640px) {
#hud { left: 8px; top: 8px; width: calc(100vw - 16px); min-width: 0; max-height: min(46vh, 360px); }
#hudContent { max-height: calc(min(46vh, 360px) - 56px); }
#hudResizer { display: none; }
}
</style>
<script type="importmap">
{
"imports": {
"three": "/static/vendor/three/three.module.js",
"three/addons/": "/static/vendor/three/addons/"
}
}
</script>
</head>
<body>
<div id="hud" aria-label="场景预览资源面板">
<div id="hudTitlebar">
<strong>Real2Sim Scene Viewer</strong>
<button id="hudToggle" type="button" title="收起资源面板" aria-label="收起资源面板">−</button>
</div>
<div id="hudContent">
<div id="status">loading</div>
<div id="hint"></div>
<div id="assetList"></div>
</div>
<div id="hudResizer" aria-label="调整面板大小" title="拖动调整面板大小"></div>
</div>
<script type="module">
import * as THREE from "three";
import { OrbitControls } from "three/addons/controls/OrbitControls.js";
import { OBJLoader } from "three/addons/loaders/OBJLoader.js";
import { PLYLoader } from "three/addons/loaders/PLYLoader.js";
const status = document.getElementById("status");
const hint = document.getElementById("hint");
const assetList = document.getElementById("assetList");
const hud = document.getElementById("hud");
const hudTitlebar = document.getElementById("hudTitlebar");
const hudToggle = document.getElementById("hudToggle");
const hudResizer = document.getElementById("hudResizer");
const params = new URLSearchParams(location.search);
const sceneId = params.get("scene_id");
const embeddedMode = params.get("embedded") === "1";
const coverMode = params.get("cover") === "1";
const coverView = params.get("cover_view") || "hero";
const probeFull = params.get("probe_full") === "1";
if (coverMode) document.getElementById("hud").style.display = "none";
setupHudPanel();
if (embeddedMode || innerWidth <= 640) setHudCollapsed(true);
const nativeWarn = console.warn.bind(console);
console.warn = (...args) => {
const message = args.map((item) => item?.message || String(item)).join(" ");
if (message.includes("THREE.Clock: This module has been deprecated")) {
console.debug(...args);
return;
}
nativeWarn(...args);
};
const renderer = new THREE.WebGLRenderer({ antialias: true, preserveDrawingBuffer: coverMode });
const basePixelRatio = Math.min(devicePixelRatio, 2);
renderer.setPixelRatio(basePixelRatio);
renderer.setSize(innerWidth, innerHeight);
renderer.setClearColor(0x080b0d, 1);
document.body.appendChild(renderer.domElement);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(55, innerWidth / innerHeight, 0.01, 1000);
setCoverCamera();
const controls = new OrbitControls(camera, renderer.domElement);
controls.target.set(0, 0, 0);
controls.enableDamping = true;
scene.add(new THREE.GridHelper(4, 16, 0x2f6d68, 0x24313a));
scene.add(new THREE.AxesHelper(1.2));
const light = new THREE.DirectionalLight(0xffffff, 1.5);
light.position.set(3, 4, 2);
scene.add(light);
scene.add(new THREE.AmbientLight(0x9fb6c8, 0.8));
let loadedObject = null;
let sparkRenderer = null;
let sparkModule = null;
const debug = {
backend: "loading",
loaded: false,
assetName: "",
pointCount: 0,
errors: [],
warnings: [],
canvas: { width: 0, height: 0 },
};
window.real2simViewerDebug = debug;
window.real2simViewerDebug.snapshot = () => ({ ...debug, canvas: { ...debug.canvas } });
function setupHudPanel() {
let drag = null;
let resize = null;
const clamp = (value, min, max) => Math.max(min, Math.min(value, max));
const clearPointerState = () => {
drag = null;
resize = null;
};
hudToggle.addEventListener("click", (event) => {
event.stopPropagation();
setHudCollapsed(!hud.classList.contains("is-collapsed"));
});
hudTitlebar.addEventListener("pointerdown", (event) => {
if (event.target.closest("button") || hud.classList.contains("is-collapsed")) return;
const rect = hud.getBoundingClientRect();
drag = { offsetX: event.clientX - rect.left, offsetY: event.clientY - rect.top };
hudTitlebar.setPointerCapture(event.pointerId);
event.preventDefault();
});
hudResizer.addEventListener("pointerdown", (event) => {
if (hud.classList.contains("is-collapsed")) return;
const rect = hud.getBoundingClientRect();
resize = { startX: event.clientX, startY: event.clientY, width: rect.width, height: rect.height };
hudResizer.setPointerCapture(event.pointerId);
event.preventDefault();
});
addEventListener("pointermove", (event) => {
if (drag) {
const rect = hud.getBoundingClientRect();
const left = clamp(event.clientX - drag.offsetX, 8, innerWidth - rect.width - 8);
const top = clamp(event.clientY - drag.offsetY, 8, innerHeight - rect.height - 8);
hud.style.left = `${left}px`;
hud.style.top = `${top}px`;
hud.style.right = "auto";
hud.style.bottom = "auto";
}
if (resize) {
const width = clamp(resize.width + event.clientX - resize.startX, 280, innerWidth - hud.offsetLeft - 8);
const height = clamp(resize.height + event.clientY - resize.startY, 76, innerHeight - hud.offsetTop - 8);
hud.style.width = `${width}px`;
hud.style.height = `${height}px`;
document.getElementById("hudContent").style.maxHeight = `${Math.max(32, height - 56)}px`;
}
});
addEventListener("pointerup", clearPointerState);
addEventListener("pointercancel", clearPointerState);
}
function setHudCollapsed(collapsed) {
hud.classList.toggle("is-collapsed", collapsed);
hudToggle.textContent = collapsed ? "+" : "−";
hudToggle.title = collapsed ? "展开资源面板" : "收起资源面板";
hudToggle.setAttribute("aria-label", hudToggle.title);
}
async function main() {
if (!sceneId) {
addPlaceholder("missing scene_id");
return;
}
const manifest = await fetch(`/api/scenes/${encodeURIComponent(sceneId)}/manifest`).then((response) => {
if (!response.ok) throw new Error(`manifest request failed: ${response.status}`);
return response.json();
});
const assets = Array.isArray(manifest.assets) ? prioritizeAssets(manifest.assets) : [];
const renderable = assets.filter(isRenderableAsset);
assetList.innerHTML = assets.slice(0, 20).map((item, index) => {
const quality = assetQuality(item, index);
const renderable = isRenderableAsset(item);
const label = [assetQualityLabel(quality), assetTypeLabel(item), item.name || item.path || "asset", item.size_label || "", item.autoload === false ? "手动加载" : ""].filter(Boolean).join(" · ");
const title = renderable ? "加载此预览资产" : "查看资源说明,不会替换当前 3D 预览";
return `<button data-index="${index}" data-renderable="${renderable}" data-autoload="${item.autoload !== false}" data-quality="${quality}" title="${escapeHtml(title)}">${escapeHtml(label)}</button>`;
}).join("");
assetList.addEventListener("click", (event) => {
const button = event.target.closest("[data-index]");
if (!button) return;
if (button.dataset.renderable === "false") {
showAssetInfo(assets[Number(button.dataset.index)]);
return;
}
loadAsset(assets[Number(button.dataset.index)], { manual: true }).catch(showError);
});
status.textContent = rendererStatusText(manifest, assets.length);
hint.textContent = viewerHint(assets, renderable);
const preferred = renderable.find((item) => item.autoload !== false) || null;
if (preferred) await loadAsset(preferred);
else if (renderable[0]) addPlaceholder(`可渲染大文件未自动加载:${renderable[0].name || "asset"} · ${renderable[0].size_label || "unknown size"} · 点击按钮手动加载`);
else addPlaceholder("当前清单没有可直接渲染的 PLY/SPZ/SPLAT/GLB/OBJ 资产");
if (probeFull) {
const full = assets.find((item) => {
const name = String(item.name || item.path || "").toLowerCase();
return name.endsWith(".spz") || name.endsWith(".splat") || name.endsWith(".ksplat") || name.endsWith(".sog");
}) || assets.find((item) => assetQuality(item, 1) === "full" && String(item.name || item.path || "").toLowerCase().includes("scene.ply"));
if (full) {
const started = performance.now();
try {
await loadAsset(full, { manual: true });
writeProbeResult({ loaded: debug.loaded, backend: debug.backend, assetName: debug.assetName, pointCount: debug.pointCount, elapsedMs: Math.round(performance.now() - started), canvas: debug.canvas, warnings: debug.warnings, errors: debug.errors });
} catch (error) {
writeProbeResult({ loaded: false, error: error.message || String(error), elapsedMs: Math.round(performance.now() - started), warnings: debug.warnings, errors: debug.errors });
}
} else {
writeProbeResult({ loaded: false, error: "full 3DGS asset not found" });
}
}
}
function writeProbeResult(result) {
let node = document.getElementById("probeResult");
if (!node) {
node = document.createElement("pre");
node.id = "probeResult";
node.style.display = "none";
document.body.appendChild(node);
}
node.textContent = JSON.stringify(result);
document.body.dataset.probeResult = node.textContent;
}
async function loadAsset(asset, options = {}) {
if (!asset?.url) {
addPlaceholder("asset has no preview URL");
return;
}
if (asset.autoload === false && !options.manual) {
addPlaceholder(`large asset skipped: ${asset.name || "asset"} · ${asset.size_label || "unknown size"} · click to load`);
return;
}
if (loadedObject) scene.remove(loadedObject);
const suffix = (asset.name || asset.path || "").toLowerCase().split(".").pop();
debug.loaded = false;
debug.assetName = asset.name || asset.path || "asset";
debug.pointCount = 0;
setRenderQualityForAsset(asset);
const size = asset.size_label ? ` · ${asset.size_label}` : "";
if (suffix === "ply" && shouldUseSparkForPly(asset)) {
await withLoadingTimeout(loadSparkOrFallback(asset, suffix), `loading timed out: ${asset.name || "asset"}${size}`, 90000);
} else if (suffix === "ply") {
await loadPlyFallback(asset);
} else if (["spz", "splat", "ksplat", "sog", "zip", "rad"].includes(suffix)) {
await withLoadingTimeout(loadSparkOrFallback(asset, suffix), `loading timed out: ${asset.name || "asset"}${size}`);
} else if (suffix === "glb" || suffix === "gltf") {
status.textContent = `Three.js fallback · loading ${asset.name}${size}`;
const { GLTFLoader } = await import("three/addons/loaders/GLTFLoader.js");
const gltf = await withLoadingTimeout(new GLTFLoader().loadAsync(asset.url), `loading timed out: ${asset.name || "asset"}${size}`);
loadedObject = gltf.scene;
normalizeObjectFromBox(loadedObject);
scene.add(loadedObject);
debug.backend = "three_fallback";
debug.loaded = true;
status.textContent = `Three.js fallback · ${asset.name}`;
} else if (suffix === "obj") {
status.textContent = `Three.js fallback · loading ${asset.name}${size}`;
loadedObject = await withLoadingTimeout(new OBJLoader().loadAsync(asset.url), `loading timed out: ${asset.name || "asset"}${size}`);
normalizeObjectFromBox(loadedObject);
scene.add(loadedObject);
debug.backend = "three_fallback";
debug.loaded = true;
status.textContent = `Three.js fallback · ${asset.name}`;
} else if (["xml", "urdf", "mjcf"].includes(suffix)) {
status.textContent = "这是仿真描述文件,用于 MotrixSim/MuJoCo 加载,不会替换当前 3D 预览";
} else if (suffix === "py") {
status.textContent = "这是控制脚本,用于运行策略或示例,不会替换当前 3D 预览";
} else if (["png", "jpg", "jpeg", "webp"].includes(suffix)) {
addImagePlane(asset);
} else {
addPlaceholder(`当前浏览器预览暂不支持 .${suffix || "file"} 文件`);
}
}
function isRenderableAsset(asset) {
const suffix = String(asset?.name || asset?.path || "").toLowerCase().split(".").pop();
return ["ply", "spz", "splat", "ksplat", "sog", "glb", "gltf", "obj", "png", "jpg", "jpeg", "webp"].includes(suffix);
}
function assetTypeLabel(asset) {
const suffix = String(asset?.name || asset?.path || "").toLowerCase().split(".").pop();
const kind = String(asset?.kind || "");
if (["ply", "spz", "splat", "ksplat", "sog", "glb", "gltf", "obj"].includes(suffix)) return "可预览";
if (["png", "jpg", "jpeg", "webp"].includes(suffix)) return "图片";
if (["xml", "urdf", "mjcf"].includes(suffix) || kind === "simulator") return "仿真文件";
if (suffix === "py" || kind === "controller") return "控制脚本";
return "资源";
}
function viewerHint(assets, renderable) {
if (renderable.some((asset) => !["png", "jpg", "jpeg", "webp"].includes(String(asset.name || asset.path || "").toLowerCase().split(".").pop()))) {
return "可直接预览 PLY、SPZ/SPLAT、GLB/OBJ 等资产;XML 和 Python 只作为仿真/控制资源列出。";
}
if (assets.length) return "当前包主要是仿真 XML、控制脚本和参考图;需要生成 PLY/GLB/3DGS 后才能看到真实三维场景。";
return "当前场景清单为空。";
}
function showAssetInfo(asset) {
const suffix = String(asset?.name || asset?.path || "").toLowerCase().split(".").pop();
const name = asset?.name || asset?.path || "资源";
const size = asset?.size_label ? ` · ${asset.size_label}` : "";
const labels = {
xml: "仿真描述文件,用于 MuJoCo/MotrixSim 加载",
urdf: "机器人/场景描述文件,用于仿真器加载",
mjcf: "MuJoCo MJCF 文件,用于仿真器加载",
py: "控制脚本,用于运行策略或示例",
json: "结构化报告或轨迹数据,可在任务/清单中查看",
md: "说明文档",
txt: "文本资源",
gif: "rollout 回放资源,可在场景管理的资产区查看",
mp4: "视频资源,可在场景管理的资产区查看",
};
status.textContent = `${labels[suffix] || "资源文件"} · ${name}${size}`;
hint.textContent = "这个条目不会替换当前 3D 场景。真正可旋转预览的是 GLB/OBJ/PLY/3DGS 等几何资产。";
}
function addImagePlane(asset) {
if (loadedObject) scene.remove(loadedObject);
const texture = new THREE.TextureLoader().load(asset.url);
const geometry = new THREE.PlaneGeometry(1.6, 0.9);
const material = new THREE.MeshBasicMaterial({ map: texture, side: THREE.DoubleSide });
loadedObject = new THREE.Mesh(geometry, material);
scene.add(loadedObject);
debug.backend = "image_preview";
debug.loaded = true;
debug.pointCount = 0;
status.textContent = `图片预览 · ${asset.name || "asset"}`;
}
function setRenderQualityForAsset(asset) {
const sizeBytes = Number(asset?.size_bytes || 0);
const heavy = shouldUseSparkForPly(asset) || sizeBytes > 80 * 1024 * 1024;
renderer.setPixelRatio(heavy ? 1 : basePixelRatio);
renderer.setSize(innerWidth, innerHeight);
}
function shouldUseSparkForPly(asset) {
const name = String(asset?.name || "").toLowerCase();
const path = String(asset?.path || asset?.url || "").toLowerCase();
const sizeBytes = Number(asset?.size_bytes || 0);
if (name.endsWith("_threejs_rgb.ply")) return false;
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return false;
if (name === "pose_init_trained_scene.ply") return false;
return name === "scene.ply" && (path.includes("/3dgs/") || sizeBytes > 50 * 1024 * 1024);
}
async function loadSparkOrFallback(asset, suffix) {
try {
const spark = await loadSparkModule();
if (!sparkRenderer) {
sparkRenderer = new spark.SparkRenderer({ renderer });
scene.add(sparkRenderer);
}
status.textContent = `Spark · loading ${asset.name}${asset.size_label ? ` · ${asset.size_label}` : ""}`;
const splatMesh = new spark.SplatMesh({ url: asset.url });
loadedObject = splatMesh;
scene.add(splatMesh);
await splatMesh.initialized;
normalizeObjectFromBox(splatMesh);
const count = splatMesh.splats?.getNumSplats?.() || splatMesh.packedSplats?.numSplats || 0;
debug.backend = "spark";
debug.loaded = true;
debug.pointCount = count;
status.textContent = `Spark · ${asset.name}${count ? ` · ${count} splats` : ""}`;
} catch (error) {
console.info("Spark preview unavailable; using Three.js fallback", error);
debug.warnings.push(error.message || String(error));
if (loadedObject) {
scene.remove(loadedObject);
loadedObject = null;
}
if (suffix !== "ply") throw error;
await loadPlyFallback(asset);
}
}
async function loadPlyFallback(asset) {
status.textContent = `Three.js fallback · loading ${asset.name}${asset.size_label ? ` · ${asset.size_label}` : ""}`;
const geometry = await withLoadingTimeout(new PLYLoader().loadAsync(asset.url), `loading timed out: ${asset.name || "asset"}${asset.size_label ? ` · ${asset.size_label}` : ""}`);
geometry.computeBoundingSphere();
geometry.computeVertexNormals();
const count = geometry.getAttribute("position")?.count || 0;
const material = new THREE.PointsMaterial({
size: pointSizeFor(count, asset),
sizeAttenuation: true,
vertexColors: geometry.hasAttribute("color"),
color: 0x58c4a8,
});
loadedObject = new THREE.Points(geometry, material);
normalizeObject(loadedObject, geometry.boundingSphere);
scene.add(loadedObject);
debug.backend = "three_fallback";
debug.loaded = true;
debug.pointCount = count;
status.textContent = `Three.js fallback · ${asset.name} · ${count} points`;
}
function pointSizeFor(count, asset) {
const name = String(asset?.name || asset?.path || "").toLowerCase();
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return 0.022;
if (count <= 100000) return 0.018;
if (count <= 500000) return 0.011;
return 0.006;
}
function withLoadingTimeout(promise, message, timeoutMs = 45000) {
let timer;
return Promise.race([
promise.finally(() => clearTimeout(timer)),
new Promise((_, reject) => {
timer = setTimeout(() => reject(new Error(message)), timeoutMs);
}),
]);
}
async function loadSparkModule() {
if (sparkModule) return sparkModule;
sparkModule = await import("/static/vendor/spark.module.min.js");
if (!sparkModule.SparkRenderer || !sparkModule.SplatMesh) throw new Error("Spark bundle missing SparkRenderer/SplatMesh exports");
return sparkModule;
}
function normalizeObject(object, sphere) {
if (!sphere) return;
object.position.sub(sphere.center);
const scale = sphere.radius > 0 ? 1.6 / sphere.radius : 1;
object.scale.setScalar(scale);
controls.target.set(0, 0, 0);
setCoverCamera();
controls.update();
}
function setCoverCamera() {
if (!coverMode) {
camera.position.set(2.8, 2.2, 2.8);
return;
}
const views = {
hero: [2.6, 1.8, 2.4],
front: [0.0, 1.55, 3.2],
side: [3.2, 1.55, 0.2],
top: [0.9, 3.4, 1.1],
};
const pos = views[coverView] || views.hero;
camera.position.set(pos[0], pos[1], pos[2]);
}
function normalizeObjectFromBox(object) {
const box = new THREE.Box3().setFromObject(object);
if (!Number.isFinite(box.min.x) || !Number.isFinite(box.max.x)) return;
const sphere = new THREE.Sphere();
box.getBoundingSphere(sphere);
normalizeObject(object, sphere);
}
function addPlaceholder(message) {
if (loadedObject) scene.remove(loadedObject);
const geometry = new THREE.BoxGeometry(0.8, 0.8, 0.8);
const material = new THREE.MeshStandardMaterial({ color: 0x58c4a8, roughness: 0.65 });
loadedObject = new THREE.Mesh(geometry, material);
scene.add(loadedObject);
debug.backend = "placeholder";
debug.loaded = false;
status.textContent = message;
}
function showError(error) {
console.error(error);
debug.errors.push(error.message || String(error));
addPlaceholder(error.message || String(error));
}
function rendererStatusText(manifest, assetCount) {
const renderer = manifest.renderer || "auto";
const rendererStatus = manifest.renderer_status || {};
if (rendererStatus.aholo_available && rendererStatus.selected === "aholo") return `Aholo · ${assetCount} assets`;
if (rendererStatus.spark_available) return `Spark · ${assetCount} assets · ${rendererStatus.aholo_reason || ""}`;
return `Three.js fallback · ${assetCount} assets · ${rendererStatus.fallback_reason || "Spark bundle unavailable"}`;
}
function prioritizeAssets(assets) {
return assets.slice().sort((left, right) => assetPriority(right) - assetPriority(left));
}
function assetPriority(asset) {
const name = String(asset?.name || asset?.path || "").toLowerCase();
const role = String(asset?.role || "").toLowerCase();
if (role === "viewer_preview" && name.endsWith(".glb")) return 20;
if (role === "rollout_gif" || role === "rollout_final_frame") return 14;
if (name.endsWith("_threejs_rgb.ply")) return 12;
if (name.endsWith(".spz") || name.endsWith(".splat") || name.endsWith(".ksplat") || name.endsWith(".sog")) return 11;
if (name.endsWith("/pose_init_trained_scene.ply") || name === "pose_init_trained_scene.ply") return 10;
if (name.endsWith("/scene.ply") || name === "scene.ply") return asset?.autoload === false ? 7 : 9;
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return 2;
if (["added_asset", "collision", "visual", "simulator_import"].includes(role)) return 4;
const kind = String(asset?.kind || "");
if (["mesh", "splat_or_pointcloud"].includes(kind)) return 3;
if (kind === "simulator") return 2;
return 1;
}
function assetQuality(asset, index) {
const name = String(asset?.name || asset?.path || "").toLowerCase();
if (index === 0) return "recommended";
if (name === "scene.ply" || name.endsWith("/3dgs/scene.ply") || name.endsWith(".spz") || name.endsWith(".splat") || name.endsWith(".ksplat") || name.endsWith(".sog")) return "full";
if (name.includes("colmap_sparse") || name.includes("sparse_points")) return "diagnostic";
if (asset?.autoload === false) return "manual";
return "normal";
}
function assetQualityLabel(quality) {
return ({ recommended: "推荐", full: "完整3DGS", diagnostic: "诊断", manual: "大文件" })[quality] || "";
}
function escapeHtml(value) {
return String(value).replace(/[&<>"']/g, (ch) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[ch]));
}
function animate() {
controls.update();
debug.canvas = { width: renderer.domElement.width, height: renderer.domElement.height };
renderer.render(scene, camera);
requestAnimationFrame(animate);
}
addEventListener("resize", () => {
camera.aspect = innerWidth / innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(innerWidth, innerHeight);
});
main().catch(showError);
animate();
</script>
</body>
</html>

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,21 @@
The MIT License
Copyright © 2025 WORLD LABS TECHNOLOGIES, INC.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View File

@ -0,0 +1 @@
2.1.0

File diff suppressed because one or more lines are too long

View File

@ -0,0 +1 @@
0.185.1

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,955 @@
import {
BufferGeometry,
FileLoader,
Float32BufferAttribute,
Group,
LineBasicMaterial,
LineSegments,
Loader,
Material,
Mesh,
MeshPhongMaterial,
Points,
PointsMaterial,
Vector3,
Color,
SRGBColorSpace
} from 'three';
// o object_name | g group_name
const _object_pattern = /^[og]\s*(.+)?/;
// mtllib file_reference
const _material_library_pattern = /^mtllib /;
// usemtl material_name
const _material_use_pattern = /^usemtl /;
// usemap map_name
const _map_use_pattern = /^usemap /;
const _face_vertex_data_separator_pattern = /\s+/;
const _vA = new Vector3();
const _vB = new Vector3();
const _vC = new Vector3();
const _ab = new Vector3();
const _cb = new Vector3();
const _color = new Color();
function ParserState() {
const state = {
objects: [],
object: {},
vertices: [],
normals: [],
colors: [],
uvs: [],
materials: {},
materialLibraries: [],
startObject: function ( name, fromDeclaration ) {
// If the current object (initial from reset) is not from a g/o declaration in the parsed
// file. We need to use it for the first parsed g/o to keep things in sync.
if ( this.object && this.object.fromDeclaration === false ) {
this.object.name = name;
this.object.fromDeclaration = ( fromDeclaration !== false );
return;
}
const previousMaterial = ( this.object && typeof this.object.currentMaterial === 'function' ? this.object.currentMaterial() : undefined );
if ( this.object && typeof this.object._finalize === 'function' ) {
this.object._finalize( true );
}
this.object = {
name: name || '',
fromDeclaration: ( fromDeclaration !== false ),
geometry: {
vertices: [],
normals: [],
colors: [],
uvs: [],
hasUVIndices: false
},
materials: [],
smooth: true,
startMaterial: function ( name, libraries ) {
const previous = this._finalize( false );
// New usemtl declaration overwrites an inherited material, except if faces were declared
// after the material, then it must be preserved for proper MultiMaterial continuation.
if ( previous && ( previous.inherited || previous.groupCount <= 0 ) ) {
this.materials.splice( previous.index, 1 );
}
const material = {
index: this.materials.length,
name: name || '',
mtllib: ( Array.isArray( libraries ) && libraries.length > 0 ? libraries[ libraries.length - 1 ] : '' ),
smooth: ( previous !== undefined ? previous.smooth : this.smooth ),
groupStart: ( previous !== undefined ? previous.groupEnd : 0 ),
groupEnd: - 1,
groupCount: - 1,
inherited: false,
clone: function ( index ) {
const cloned = {
index: ( typeof index === 'number' ? index : this.index ),
name: this.name,
mtllib: this.mtllib,
smooth: this.smooth,
groupStart: 0,
groupEnd: - 1,
groupCount: - 1,
inherited: false
};
cloned.clone = this.clone.bind( cloned );
return cloned;
}
};
this.materials.push( material );
return material;
},
currentMaterial: function () {
if ( this.materials.length > 0 ) {
return this.materials[ this.materials.length - 1 ];
}
return undefined;
},
_finalize: function ( end ) {
const lastMultiMaterial = this.currentMaterial();
if ( lastMultiMaterial && lastMultiMaterial.groupEnd === - 1 ) {
lastMultiMaterial.groupEnd = this.geometry.vertices.length / 3;
lastMultiMaterial.groupCount = lastMultiMaterial.groupEnd - lastMultiMaterial.groupStart;
lastMultiMaterial.inherited = false;
}
// Ignore objects tail materials if no face declarations followed them before a new o/g started.
if ( end && this.materials.length > 1 ) {
for ( let mi = this.materials.length - 1; mi >= 0; mi -- ) {
if ( this.materials[ mi ].groupCount <= 0 ) {
this.materials.splice( mi, 1 );
}
}
}
// Guarantee at least one empty material, this makes the creation later more straight forward.
if ( end && this.materials.length === 0 ) {
this.materials.push( {
name: '',
smooth: this.smooth
} );
}
return lastMultiMaterial;
}
};
// Inherit previous objects material.
// Spec tells us that a declared material must be set to all objects until a new material is declared.
// If a usemtl declaration is encountered while this new object is being parsed, it will
// overwrite the inherited material. Exception being that there was already face declarations
// to the inherited material, then it will be preserved for proper MultiMaterial continuation.
if ( previousMaterial && previousMaterial.name && typeof previousMaterial.clone === 'function' ) {
const declared = previousMaterial.clone( 0 );
declared.inherited = true;
this.object.materials.push( declared );
}
this.objects.push( this.object );
},
finalize: function () {
if ( this.object && typeof this.object._finalize === 'function' ) {
this.object._finalize( true );
}
},
parseVertexIndex: function ( value, len ) {
const index = parseInt( value, 10 );
return ( index >= 0 ? index - 1 : index + len / 3 ) * 3;
},
parseNormalIndex: function ( value, len ) {
const index = parseInt( value, 10 );
return ( index >= 0 ? index - 1 : index + len / 3 ) * 3;
},
parseUVIndex: function ( value, len ) {
const index = parseInt( value, 10 );
return ( index >= 0 ? index - 1 : index + len / 2 ) * 2;
},
addVertex: function ( a, b, c ) {
const src = this.vertices;
const dst = this.object.geometry.vertices;
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
dst.push( src[ b + 0 ], src[ b + 1 ], src[ b + 2 ] );
dst.push( src[ c + 0 ], src[ c + 1 ], src[ c + 2 ] );
},
addVertexPoint: function ( a ) {
const src = this.vertices;
const dst = this.object.geometry.vertices;
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
},
addVertexLine: function ( a ) {
const src = this.vertices;
const dst = this.object.geometry.vertices;
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
},
addNormal: function ( a, b, c ) {
const src = this.normals;
const dst = this.object.geometry.normals;
dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
dst.push( src[ b + 0 ], src[ b + 1 ], src[ b + 2 ] );
dst.push( src[ c + 0 ], src[ c + 1 ], src[ c + 2 ] );
},
addFaceNormal: function ( a, b, c ) {
const src = this.vertices;
const dst = this.object.geometry.normals;
_vA.fromArray( src, a );
_vB.fromArray( src, b );
_vC.fromArray( src, c );
_cb.subVectors( _vC, _vB );
_ab.subVectors( _vA, _vB );
_cb.cross( _ab );
_cb.normalize();
dst.push( _cb.x, _cb.y, _cb.z );
dst.push( _cb.x, _cb.y, _cb.z );
dst.push( _cb.x, _cb.y, _cb.z );
},
addColor: function ( a, b, c ) {
const src = this.colors;
const dst = this.object.geometry.colors;
if ( src[ a ] !== undefined ) dst.push( src[ a + 0 ], src[ a + 1 ], src[ a + 2 ] );
if ( src[ b ] !== undefined ) dst.push( src[ b + 0 ], src[ b + 1 ], src[ b + 2 ] );
if ( src[ c ] !== undefined ) dst.push( src[ c + 0 ], src[ c + 1 ], src[ c + 2 ] );
},
addUV: function ( a, b, c ) {
const src = this.uvs;
const dst = this.object.geometry.uvs;
dst.push( src[ a + 0 ], src[ a + 1 ] );
dst.push( src[ b + 0 ], src[ b + 1 ] );
dst.push( src[ c + 0 ], src[ c + 1 ] );
},
addDefaultUV: function () {
const dst = this.object.geometry.uvs;
dst.push( 0, 0 );
dst.push( 0, 0 );
dst.push( 0, 0 );
},
addUVLine: function ( a ) {
const src = this.uvs;
const dst = this.object.geometry.uvs;
dst.push( src[ a + 0 ], src[ a + 1 ] );
},
addFace: function ( a, b, c, ua, ub, uc, na, nb, nc ) {
const vLen = this.vertices.length;
let ia = this.parseVertexIndex( a, vLen );
let ib = this.parseVertexIndex( b, vLen );
let ic = this.parseVertexIndex( c, vLen );
this.addVertex( ia, ib, ic );
this.addColor( ia, ib, ic );
// normals
if ( na !== undefined && na !== '' ) {
const nLen = this.normals.length;
ia = this.parseNormalIndex( na, nLen );
ib = this.parseNormalIndex( nb, nLen );
ic = this.parseNormalIndex( nc, nLen );
this.addNormal( ia, ib, ic );
} else {
this.addFaceNormal( ia, ib, ic );
}
// uvs
if ( ua !== undefined && ua !== '' ) {
const uvLen = this.uvs.length;
ia = this.parseUVIndex( ua, uvLen );
ib = this.parseUVIndex( ub, uvLen );
ic = this.parseUVIndex( uc, uvLen );
this.addUV( ia, ib, ic );
this.object.geometry.hasUVIndices = true;
} else {
// add placeholder values (for inconsistent face definitions)
this.addDefaultUV();
}
},
addPointGeometry: function ( vertices ) {
this.object.geometry.type = 'Points';
const vLen = this.vertices.length;
for ( let vi = 0, l = vertices.length; vi < l; vi ++ ) {
const index = this.parseVertexIndex( vertices[ vi ], vLen );
this.addVertexPoint( index );
this.addColor( index );
}
},
addLineGeometry: function ( vertices, uvs ) {
this.object.geometry.type = 'Line';
const vLen = this.vertices.length;
const uvLen = this.uvs.length;
for ( let vi = 0, l = vertices.length; vi < l; vi ++ ) {
this.addVertexLine( this.parseVertexIndex( vertices[ vi ], vLen ) );
}
for ( let uvi = 0, l = uvs.length; uvi < l; uvi ++ ) {
this.addUVLine( this.parseUVIndex( uvs[ uvi ], uvLen ) );
}
}
};
state.startObject( '', false );
return state;
}
/**
* A loader for the OBJ format.
*
* The [OBJ format](https://en.wikipedia.org/wiki/Wavefront_.obj_file) is a simple data-format that
* represents 3D geometry in a human readable format as the position of each vertex, the UV position of
* each texture coordinate vertex, vertex normals, and the faces that make each polygon defined as a list
* of vertices, and texture vertices.
*
* ```js
* const loader = new OBJLoader();
* const object = await loader.loadAsync( 'models/monster.obj' );
* scene.add( object );
* ```
*
* @augments Loader
* @three_import import { OBJLoader } from 'three/addons/loaders/OBJLoader.js';
*/
class OBJLoader extends Loader {
/**
* Constructs a new OBJ loader.
*
* @param {LoadingManager} [manager] - The loading manager.
*/
constructor( manager ) {
super( manager );
/**
* A reference to a material creator.
*
* @type {?MaterialCreator}
* @default null
*/
this.materials = null;
}
/**
* Starts loading from the given URL and passes the loaded OBJ asset
* to the `onLoad()` callback.
*
* @param {string} url - The path/URL of the file to be loaded. This can also be a data URI.
* @param {function(Group)} onLoad - Executed when the loading process has been finished.
* @param {onProgressCallback} onProgress - Executed while the loading is in progress.
* @param {onErrorCallback} onError - Executed when errors occur.
*/
load( url, onLoad, onProgress, onError ) {
const scope = this;
const loader = new FileLoader( this.manager );
loader.setPath( this.path );
loader.setRequestHeader( this.requestHeader );
loader.setWithCredentials( this.withCredentials );
loader.load( url, function ( text ) {
try {
onLoad( scope.parse( text ) );
} catch ( e ) {
if ( onError ) {
onError( e );
} else {
console.error( e );
}
scope.manager.itemError( url );
}
}, onProgress, onError );
}
/**
* Sets the material creator for this OBJ. This object is loaded via {@link MTLLoader}.
*
* @param {MaterialCreator} materials - An object that creates the materials for this OBJ.
* @return {OBJLoader} A reference to this loader.
*/
setMaterials( materials ) {
this.materials = materials;
return this;
}
/**
* Parses the given OBJ data and returns the resulting group.
*
* @param {string} text - The raw OBJ data as a string.
* @return {Group} The parsed OBJ.
*/
parse( text ) {
const state = new ParserState();
if ( text.indexOf( '\r\n' ) !== - 1 ) {
// This is faster than String.split with regex that splits on both
text = text.replace( /\r\n/g, '\n' );
}
if ( text.indexOf( '\\\n' ) !== - 1 ) {
// join lines separated by a line continuation character (\)
text = text.replace( /\\\n/g, '' );
}
const lines = text.split( '\n' );
let result = [];
for ( let i = 0, l = lines.length; i < l; i ++ ) {
const line = lines[ i ].trimStart();
if ( line.length === 0 ) continue;
const lineFirstChar = line.charAt( 0 );
// @todo invoke passed in handler if any
if ( lineFirstChar === '#' ) continue; // skip comments
if ( lineFirstChar === 'v' ) {
const data = line.split( _face_vertex_data_separator_pattern );
switch ( data[ 0 ] ) {
case 'v':
state.vertices.push(
parseFloat( data[ 1 ] ),
parseFloat( data[ 2 ] ),
parseFloat( data[ 3 ] )
);
if ( data.length >= 7 ) {
_color.setRGB(
parseFloat( data[ 4 ] ),
parseFloat( data[ 5 ] ),
parseFloat( data[ 6 ] ),
SRGBColorSpace
);
state.colors.push( _color.r, _color.g, _color.b );
} else {
// if no colors are defined, add placeholders so color and vertex indices match
state.colors.push( undefined, undefined, undefined );
}
break;
case 'vn':
state.normals.push(
parseFloat( data[ 1 ] ),
parseFloat( data[ 2 ] ),
parseFloat( data[ 3 ] )
);
break;
case 'vt':
state.uvs.push(
parseFloat( data[ 1 ] ),
parseFloat( data[ 2 ] )
);
break;
}
} else if ( lineFirstChar === 'f' ) {
const lineData = line.slice( 1 ).trim();
const vertexData = lineData.split( _face_vertex_data_separator_pattern );
const faceVertices = [];
// Parse the face vertex data into an easy to work with format
for ( let j = 0, jl = vertexData.length; j < jl; j ++ ) {
const vertex = vertexData[ j ];
if ( vertex.length > 0 ) {
const vertexParts = vertex.split( '/' );
faceVertices.push( vertexParts );
}
}
// Draw an edge between the first vertex and all subsequent vertices to form an n-gon
const v1 = faceVertices[ 0 ];
for ( let j = 1, jl = faceVertices.length - 1; j < jl; j ++ ) {
const v2 = faceVertices[ j ];
const v3 = faceVertices[ j + 1 ];
state.addFace(
v1[ 0 ], v2[ 0 ], v3[ 0 ],
v1[ 1 ], v2[ 1 ], v3[ 1 ],
v1[ 2 ], v2[ 2 ], v3[ 2 ]
);
}
} else if ( lineFirstChar === 'l' ) {
const lineParts = line.substring( 1 ).trim().split( ' ' );
let lineVertices = [];
const lineUVs = [];
if ( line.indexOf( '/' ) === - 1 ) {
lineVertices = lineParts;
} else {
for ( let li = 0, llen = lineParts.length; li < llen; li ++ ) {
const parts = lineParts[ li ].split( '/' );
if ( parts[ 0 ] !== '' ) lineVertices.push( parts[ 0 ] );
if ( parts[ 1 ] !== '' ) lineUVs.push( parts[ 1 ] );
}
}
state.addLineGeometry( lineVertices, lineUVs );
} else if ( lineFirstChar === 'p' ) {
const lineData = line.slice( 1 ).trim();
const pointData = lineData.split( ' ' );
state.addPointGeometry( pointData );
} else if ( ( result = _object_pattern.exec( line ) ) !== null ) {
// o object_name
// or
// g group_name
// WORKAROUND: https://bugs.chromium.org/p/v8/issues/detail?id=2869
// let name = result[ 0 ].slice( 1 ).trim();
const name = ( ' ' + result[ 0 ].slice( 1 ).trim() ).slice( 1 );
state.startObject( name );
} else if ( _material_use_pattern.test( line ) ) {
// material
state.object.startMaterial( line.substring( 7 ).trim(), state.materialLibraries );
} else if ( _material_library_pattern.test( line ) ) {
// mtl file
state.materialLibraries.push( line.substring( 7 ).trim() );
} else if ( _map_use_pattern.test( line ) ) {
// the line is parsed but ignored since the loader assumes textures are defined MTL files
// (according to https://www.okino.com/conv/imp_wave.htm, 'usemap' is the old-style Wavefront texture reference method)
console.warn( 'THREE.OBJLoader: Rendering identifier "usemap" not supported. Textures must be defined in MTL files.' );
} else if ( lineFirstChar === 's' ) {
result = line.split( ' ' );
// smooth shading
// @todo Handle files that have varying smooth values for a set of faces inside one geometry,
// but does not define a usemtl for each face set.
// This should be detected and a dummy material created (later MultiMaterial and geometry groups).
// This requires some care to not create extra material on each smooth value for "normal" obj files.
// where explicit usemtl defines geometry groups.
// Example asset: examples/models/obj/cerberus/Cerberus.obj
/*
* http://paulbourke.net/dataformats/obj/
*
* From chapter "Grouping" Syntax explanation "s group_number":
* "group_number is the smoothing group number. To turn off smoothing groups, use a value of 0 or off.
* Polygonal elements use group numbers to put elements in different smoothing groups. For free-form
* surfaces, smoothing groups are either turned on or off; there is no difference between values greater
* than 0."
*/
if ( result.length > 1 ) {
const value = result[ 1 ].trim().toLowerCase();
state.object.smooth = ( value !== '0' && value !== 'off' );
} else {
// ZBrush can produce "s" lines #11707
state.object.smooth = true;
}
const material = state.object.currentMaterial();
if ( material ) material.smooth = state.object.smooth;
} else {
// Handle null terminated files without exception
if ( line === '\0' ) continue;
console.warn( 'THREE.OBJLoader: Unexpected line: "' + line + '"' );
}
}
state.finalize();
const container = new Group();
container.materialLibraries = [].concat( state.materialLibraries );
const hasPrimitives = ! ( state.objects.length === 1 && state.objects[ 0 ].geometry.vertices.length === 0 );
if ( hasPrimitives === true ) {
for ( let i = 0, l = state.objects.length; i < l; i ++ ) {
const object = state.objects[ i ];
const geometry = object.geometry;
const materials = object.materials;
const isLine = ( geometry.type === 'Line' );
const isPoints = ( geometry.type === 'Points' );
let hasVertexColors = false;
// Skip o/g line declarations that did not follow with any faces
if ( geometry.vertices.length === 0 ) continue;
const buffergeometry = new BufferGeometry();
buffergeometry.setAttribute( 'position', new Float32BufferAttribute( geometry.vertices, 3 ) );
if ( geometry.normals.length > 0 ) {
buffergeometry.setAttribute( 'normal', new Float32BufferAttribute( geometry.normals, 3 ) );
}
if ( geometry.colors.length > 0 ) {
hasVertexColors = true;
buffergeometry.setAttribute( 'color', new Float32BufferAttribute( geometry.colors, 3 ) );
}
if ( geometry.hasUVIndices === true ) {
buffergeometry.setAttribute( 'uv', new Float32BufferAttribute( geometry.uvs, 2 ) );
}
// Create materials
const createdMaterials = [];
for ( let mi = 0, miLen = materials.length; mi < miLen; mi ++ ) {
const sourceMaterial = materials[ mi ];
const materialHash = sourceMaterial.name + '_' + sourceMaterial.smooth + '_' + hasVertexColors;
let material = state.materials[ materialHash ];
if ( this.materials !== null ) {
material = this.materials.create( sourceMaterial.name );
// mtl etc. loaders probably can't create line materials correctly, copy properties to a line material.
if ( isLine && material && ! ( material instanceof LineBasicMaterial ) ) {
const materialLine = new LineBasicMaterial();
Material.prototype.copy.call( materialLine, material );
materialLine.color.copy( material.color );
material = materialLine;
} else if ( isPoints && material && ! ( material instanceof PointsMaterial ) ) {
const materialPoints = new PointsMaterial( { size: 10, sizeAttenuation: false } );
Material.prototype.copy.call( materialPoints, material );
materialPoints.color.copy( material.color );
materialPoints.map = material.map;
material = materialPoints;
}
}
if ( material === undefined ) {
if ( isLine ) {
material = new LineBasicMaterial();
} else if ( isPoints ) {
material = new PointsMaterial( { size: 1, sizeAttenuation: false } );
} else {
material = new MeshPhongMaterial();
}
material.name = sourceMaterial.name;
material.flatShading = sourceMaterial.smooth ? false : true;
material.vertexColors = hasVertexColors;
state.materials[ materialHash ] = material;
}
createdMaterials.push( material );
}
// Create mesh
let mesh;
if ( createdMaterials.length > 1 ) {
for ( let mi = 0, miLen = materials.length; mi < miLen; mi ++ ) {
const sourceMaterial = materials[ mi ];
buffergeometry.addGroup( sourceMaterial.groupStart, sourceMaterial.groupCount, mi );
}
if ( isLine ) {
mesh = new LineSegments( buffergeometry, createdMaterials );
} else if ( isPoints ) {
mesh = new Points( buffergeometry, createdMaterials );
} else {
mesh = new Mesh( buffergeometry, createdMaterials );
}
} else {
if ( isLine ) {
mesh = new LineSegments( buffergeometry, createdMaterials[ 0 ] );
} else if ( isPoints ) {
mesh = new Points( buffergeometry, createdMaterials[ 0 ] );
} else {
mesh = new Mesh( buffergeometry, createdMaterials[ 0 ] );
}
}
mesh.name = object.name;
container.add( mesh );
}
} else {
// if there is only the default parser state object with no geometry data, interpret data as point cloud
if ( state.vertices.length > 0 ) {
const material = new PointsMaterial( { size: 1, sizeAttenuation: false } );
const buffergeometry = new BufferGeometry();
buffergeometry.setAttribute( 'position', new Float32BufferAttribute( state.vertices, 3 ) );
if ( state.colors.length > 0 && state.colors[ 0 ] !== undefined ) {
buffergeometry.setAttribute( 'color', new Float32BufferAttribute( state.colors, 3 ) );
material.vertexColors = true;
}
const points = new Points( buffergeometry, material );
container.add( points );
}
}
return container;
}
}
export { OBJLoader };

View File

@ -0,0 +1,968 @@
import {
BufferGeometry,
FileLoader,
BufferAttribute,
Float32BufferAttribute,
Int8BufferAttribute,
Int16BufferAttribute,
Int32BufferAttribute,
Loader,
Uint8BufferAttribute,
Uint16BufferAttribute,
Uint32BufferAttribute,
Color,
SRGBColorSpace
} from 'three';
const _color = new Color();
/**
* A loader for PLY the PLY format (known as the Polygon
* File Format or the Stanford Triangle Format).
*
* Limitations:
* - ASCII decoding assumes file is UTF-8.
*
* ```js
* const loader = new PLYLoader();
* const geometry = await loader.loadAsync( './models/ply/ascii/dolphins.ply' );
* scene.add( new THREE.Mesh( geometry ) );
* ```
*
* @augments Loader
* @three_import import { PLYLoader } from 'three/addons/loaders/PLYLoader.js';
*/
class PLYLoader extends Loader {
/**
* Constructs a new PLY loader.
*
* @param {LoadingManager} [manager] - The loading manager.
*/
constructor( manager ) {
super( manager );
// internals
this.propertyNameMapping = {};
this.customPropertyMapping = {};
}
/**
* Starts loading from the given URL and passes the loaded PLY asset
* to the `onLoad()` callback.
*
* @param {string} url - The path/URL of the file to be loaded. This can also be a data URI.
* @param {function(BufferGeometry)} onLoad - Executed when the loading process has been finished.
* @param {onProgressCallback} onProgress - Executed while the loading is in progress.
* @param {onErrorCallback} onError - Executed when errors occur.
*/
load( url, onLoad, onProgress, onError ) {
const scope = this;
const loader = new FileLoader( this.manager );
loader.setPath( this.path );
loader.setResponseType( 'arraybuffer' );
loader.setRequestHeader( this.requestHeader );
loader.setWithCredentials( this.withCredentials );
loader.load( url, function ( text ) {
try {
onLoad( scope.parse( text ) );
} catch ( e ) {
if ( onError ) {
onError( e );
} else {
console.error( e );
}
scope.manager.itemError( url );
}
}, onProgress, onError );
}
/**
* Sets a property name mapping that maps default property names
* to custom ones. For example, the following maps the properties
* “diffuse_(red|green|blue)” in the file to standard color names.
*
* ```js
* loader.setPropertyNameMapping( {
* diffuse_red: 'red',
* diffuse_green: 'green',
* diffuse_blue: 'blue'
* } );
* ```
*
* @param {Object} mapping - The mapping dictionary.
*/
setPropertyNameMapping( mapping ) {
this.propertyNameMapping = mapping;
}
/**
* Custom properties outside of the defaults for position, uv, normal
* and color attributes can be added using the setCustomPropertyNameMapping method.
* For example, the following maps the element properties “custom_property_a”
* and “custom_property_b” to an attribute “customAttribute” with an item size of 2.
* Attribute item sizes are set from the number of element properties in the property array.
*
* ```js
* loader.setCustomPropertyNameMapping( {
* customAttribute: ['custom_property_a', 'custom_property_b'],
* } );
* ```
* @param {Object} mapping - The mapping dictionary.
*/
setCustomPropertyNameMapping( mapping ) {
this.customPropertyMapping = mapping;
}
/**
* Parses the given PLY data and returns the resulting geometry.
*
* @param {ArrayBuffer} data - The raw PLY data as an array buffer.
* @return {BufferGeometry} The parsed geometry.
*/
parse( data ) {
function parseHeader( data, headerLength = 0 ) {
const patternHeader = /^ply([\s\S]*)end_header(\r\n|\r|\n)/;
let headerText = '';
const result = patternHeader.exec( data );
if ( result !== null ) {
headerText = result[ 1 ];
}
const header = {
comments: [],
elements: [],
headerLength: headerLength,
objInfo: ''
};
const lines = headerText.split( /\r\n|\r|\n/ );
let currentElement;
function make_ply_element_property( propertyValues, propertyNameMapping ) {
const property = { type: propertyValues[ 0 ] };
if ( property.type === 'list' ) {
property.name = propertyValues[ 3 ];
property.countType = propertyValues[ 1 ];
property.itemType = propertyValues[ 2 ];
} else {
property.name = propertyValues[ 1 ];
}
if ( property.name in propertyNameMapping ) {
property.name = propertyNameMapping[ property.name ];
}
return property;
}
for ( let i = 0; i < lines.length; i ++ ) {
let line = lines[ i ];
line = line.trim();
if ( line === '' ) continue;
const lineValues = line.split( /\s+/ );
const lineType = lineValues.shift();
line = lineValues.join( ' ' );
switch ( lineType ) {
case 'format':
header.format = lineValues[ 0 ];
header.version = lineValues[ 1 ];
break;
case 'comment':
header.comments.push( line );
break;
case 'element':
if ( currentElement !== undefined ) {
header.elements.push( currentElement );
}
currentElement = {};
currentElement.name = lineValues[ 0 ];
currentElement.count = parseInt( lineValues[ 1 ] );
currentElement.properties = [];
break;
case 'property':
currentElement.properties.push( make_ply_element_property( lineValues, scope.propertyNameMapping ) );
break;
case 'obj_info':
header.objInfo = line;
break;
default:
console.log( 'unhandled', lineType, lineValues );
}
}
if ( currentElement !== undefined ) {
header.elements.push( currentElement );
}
return header;
}
function parseASCIINumber( n, type ) {
switch ( type ) {
case 'char': case 'uchar': case 'short': case 'ushort': case 'int': case 'uint':
case 'int8': case 'uint8': case 'int16': case 'uint16': case 'int32': case 'uint32':
return parseInt( n );
case 'float': case 'double': case 'float32': case 'float64':
return parseFloat( n );
}
}
function parseASCIIElement( properties, tokens ) {
const element = {};
for ( let i = 0; i < properties.length; i ++ ) {
if ( tokens.empty() ) return null;
if ( properties[ i ].type === 'list' ) {
const list = [];
const n = parseASCIINumber( tokens.next(), properties[ i ].countType );
for ( let j = 0; j < n; j ++ ) {
if ( tokens.empty() ) return null;
list.push( parseASCIINumber( tokens.next(), properties[ i ].itemType ) );
}
element[ properties[ i ].name ] = list;
} else {
element[ properties[ i ].name ] = parseASCIINumber( tokens.next(), properties[ i ].type );
}
}
return element;
}
function createBuffer() {
const buffer = {
indices: [],
vertices: [],
normals: [],
uvs: [],
faceVertexUvs: [],
colors: [],
faceVertexColors: [],
descriptors: {}
};
for ( const customProperty of Object.keys( scope.customPropertyMapping ) ) {
buffer[ customProperty ] = [];
}
return buffer;
}
function getBufferAttributeClass( type ) {
switch ( type ) {
case 'int8': case 'char': return Int8BufferAttribute;
case 'uint8': case 'uchar': return Uint8BufferAttribute;
case 'int16': case 'short': return Int16BufferAttribute;
case 'uint16': case 'ushort': return Uint16BufferAttribute;
case 'int32': case 'int': return Int32BufferAttribute;
case 'uint32': case 'uint': return Uint32BufferAttribute;
case 'float32': case 'float': return Float32BufferAttribute;
case 'float64': case 'double': return Float64BufferAttribute;
}
}
function getColorScale( type ) {
switch ( type ) {
case 'uchar': case 'uint8': return 1 / 255;
case 'ushort': case 'uint16': return 1 / 65535;
case 'float': case 'float32':
case 'double': case 'float64': return 1;
default: return 1 / 255;
}
}
function isFloatType( type ) {
return type === 'float' || type === 'float32' || type === 'double' || type === 'float64';
}
function getAttributeDescriptor( properties ) {
function findProperty( names ) {
for ( const name of names ) {
const property = properties.find( p => p.name === name );
if ( property ) return property;
}
return null;
}
// property lookup
const x = findProperty( [ 'x', 'px', 'posx' ] );
const y = findProperty( [ 'y', 'py', 'posy' ] );
const z = findProperty( [ 'z', 'pz', 'posz' ] );
const nx = findProperty( [ 'nx', 'normalx' ] );
const ny = findProperty( [ 'ny', 'normaly' ] );
const nz = findProperty( [ 'nz', 'normalz' ] );
const s = findProperty( [ 's', 'u', 'texture_u', 'tx' ] );
const t = findProperty( [ 't', 'v', 'texture_v', 'ty' ] );
const r = findProperty( [ 'red', 'diffuse_red', 'r', 'diffuse_r' ] );
const g = findProperty( [ 'green', 'diffuse_green', 'g', 'diffuse_g' ] );
const b = findProperty( [ 'blue', 'diffuse_blue', 'b', 'diffuse_b' ] );
const texcoord = findProperty( [ 'texcoord' ] );
// custom property lookup
const custom = {};
for ( const customAttr of Object.keys( scope.customPropertyMapping ) ) {
const propNames = scope.customPropertyMapping[ customAttr ];
const matched = propNames.map( name => properties.find( p => p.name === name ) );
const types = matched.filter( p => p ).map( p => p.type );
const uniform = types.length > 0 && types.every( type => type === types[ 0 ] );
custom[ customAttr ] = {
type: uniform ? types[ 0 ] : 'float32',
usage: matched.every( p => p !== undefined ),
};
}
// build descriptor
return {
position: {
names: [ x ? x.name : 'x', y ? y.name : 'y', z ? z.name : 'z' ],
type: x ? x.type : 'float32',
usage: !! ( x && y && z ),
},
normal: {
names: [ nx ? nx.name : 'nx', ny ? ny.name : 'ny', nz ? nz.name : 'nz' ],
type: nx ? nx.type : 'float32',
usage: !! ( nx && ny && nz ),
},
uv: {
names: [ s ? s.name : 's', t ? t.name : 't' ],
type: s ? s.type : 'float32',
usage: !! ( s && t ),
},
texcoord: {
type: texcoord ? texcoord.itemType : 'float32',
usage: !! texcoord,
},
color: {
names: [ r ? r.name : 'red', g ? g.name : 'green', b ? b.name : 'blue' ],
type: r ? r.type : 'uchar',
usage: !! ( r && g && b ),
},
custom: custom,
};
}
function parseASCII( data, header ) {
// PLY ascii format specification, as per http://en.wikipedia.org/wiki/PLY_(file_format)
const buffer = createBuffer();
const patternBody = /end_header\s+(\S[\s\S]*\S|\S)\s*$/;
let body, matches;
if ( ( matches = patternBody.exec( data ) ) !== null ) {
body = matches[ 1 ].split( /\s+/ );
} else {
body = [ ];
}
const tokens = new ArrayStream( body );
loop: for ( let i = 0; i < header.elements.length; i ++ ) {
const elementDesc = header.elements[ i ];
const attributeDescriptor = getAttributeDescriptor( elementDesc.properties );
buffer.descriptors[ elementDesc.name ] = attributeDescriptor;
for ( let j = 0; j < elementDesc.count; j ++ ) {
const element = parseASCIIElement( elementDesc.properties, tokens );
if ( ! element ) break loop;
handleElement( buffer, elementDesc.name, element, attributeDescriptor );
}
}
return postProcess( buffer );
}
function postProcess( buffer ) {
let geometry = new BufferGeometry();
const vertexDescriptor = buffer.descriptors.vertex;
// mandatory buffer data
if ( buffer.indices.length > 0 ) {
geometry.setIndex( buffer.indices );
}
const PositionClass = getBufferAttributeClass( vertexDescriptor ? vertexDescriptor.position.type : 'float32' );
geometry.setAttribute( 'position', new PositionClass( buffer.vertices, 3 ) );
// optional buffer data
if ( buffer.normals.length > 0 ) {
const NormalClass = getBufferAttributeClass( vertexDescriptor.normal.type );
geometry.setAttribute( 'normal', new NormalClass( buffer.normals, 3 ) );
}
if ( buffer.uvs.length > 0 ) {
const UvClass = getBufferAttributeClass( vertexDescriptor.uv.type );
geometry.setAttribute( 'uv', new UvClass( buffer.uvs, 2 ) );
}
if ( buffer.colors.length > 0 ) {
const colorType = vertexDescriptor.color.type;
const normalized = ! isFloatType( colorType );
const ColorClass = getBufferAttributeClass( colorType );
geometry.setAttribute( 'color', new ColorClass( buffer.colors, 3, normalized ) );
}
if ( buffer.faceVertexUvs.length > 0 || buffer.faceVertexColors.length > 0 ) {
geometry = geometry.toNonIndexed();
if ( buffer.faceVertexUvs.length > 0 ) {
const UvClass = getBufferAttributeClass( buffer.descriptors.face.texcoord.type );
geometry.setAttribute( 'uv', new UvClass( buffer.faceVertexUvs, 2 ) );
}
if ( buffer.faceVertexColors.length > 0 ) {
const colorType = buffer.descriptors.face.color.type;
const normalized = ! isFloatType( colorType );
const ColorClass = getBufferAttributeClass( colorType );
geometry.setAttribute( 'color', new ColorClass( buffer.faceVertexColors, 3, normalized ) );
}
}
// custom buffer data
for ( const customProperty of Object.keys( scope.customPropertyMapping ) ) {
if ( buffer[ customProperty ].length > 0 ) {
const CustomClass = getBufferAttributeClass( vertexDescriptor.custom[ customProperty ].type );
geometry.setAttribute( customProperty, new CustomClass( buffer[ customProperty ], scope.customPropertyMapping[ customProperty ].length ) );
}
}
geometry.computeBoundingSphere();
return geometry;
}
function handleElement( buffer, elementName, element, attributeDescriptor ) {
if ( elementName === 'vertex' ) {
const { position, normal, uv, color } = attributeDescriptor;
if ( position.usage ) {
buffer.vertices.push(
element[ position.names[ 0 ] ],
element[ position.names[ 1 ] ],
element[ position.names[ 2 ] ]
);
}
if ( normal.usage ) {
buffer.normals.push(
element[ normal.names[ 0 ] ],
element[ normal.names[ 1 ] ],
element[ normal.names[ 2 ] ]
);
}
if ( uv.usage ) {
buffer.uvs.push(
element[ uv.names[ 0 ] ],
element[ uv.names[ 1 ] ]
);
}
if ( color.usage ) {
const scale = getColorScale( color.type );
const isFloat = isFloatType( color.type );
// convert to float for color space conversion
_color.setRGB(
element[ color.names[ 0 ] ] * scale,
element[ color.names[ 1 ] ] * scale,
element[ color.names[ 2 ] ] * scale,
SRGBColorSpace
);
// convert back to original type
const invScale = 1 / scale;
buffer.colors.push(
isFloat ? _color.r : Math.round( _color.r * invScale ),
isFloat ? _color.g : Math.round( _color.g * invScale ),
isFloat ? _color.b : Math.round( _color.b * invScale )
);
}
for ( const customProperty of Object.keys( scope.customPropertyMapping ) ) {
for ( const elementProperty of scope.customPropertyMapping[ customProperty ] ) {
buffer[ customProperty ].push( element[ elementProperty ] );
}
}
} else if ( elementName === 'face' ) {
const vertex_indices = element.vertex_indices || element.vertex_index; // issue #9338
const texcoord = element.texcoord;
if ( vertex_indices.length === 3 ) {
buffer.indices.push( vertex_indices[ 0 ], vertex_indices[ 1 ], vertex_indices[ 2 ] );
if ( texcoord && texcoord.length === 6 ) {
buffer.faceVertexUvs.push( texcoord[ 0 ], texcoord[ 1 ] );
buffer.faceVertexUvs.push( texcoord[ 2 ], texcoord[ 3 ] );
buffer.faceVertexUvs.push( texcoord[ 4 ], texcoord[ 5 ] );
}
} else if ( vertex_indices.length === 4 ) {
buffer.indices.push( vertex_indices[ 0 ], vertex_indices[ 1 ], vertex_indices[ 3 ] );
buffer.indices.push( vertex_indices[ 1 ], vertex_indices[ 2 ], vertex_indices[ 3 ] );
}
// face colors
const { color } = attributeDescriptor;
if ( color.usage ) {
// convert to float for color space conversion
const scale = getColorScale( color.type );
_color.setRGB(
element[ color.names[ 0 ] ] * scale,
element[ color.names[ 1 ] ] * scale,
element[ color.names[ 2 ] ] * scale,
SRGBColorSpace
);
// convert back to original type
const invScale = 1 / scale;
const r = _color.r * invScale;
const g = _color.g * invScale;
const b = _color.b * invScale;
buffer.faceVertexColors.push( r, g, b );
buffer.faceVertexColors.push( r, g, b );
buffer.faceVertexColors.push( r, g, b );
}
}
}
function binaryReadElement( at, properties ) {
const element = {};
let read = 0;
for ( let i = 0; i < properties.length; i ++ ) {
const property = properties[ i ];
const valueReader = property.valueReader;
if ( property.type === 'list' ) {
const list = [];
const n = property.countReader.read( at + read );
read += property.countReader.size;
for ( let j = 0; j < n; j ++ ) {
list.push( valueReader.read( at + read ) );
read += valueReader.size;
}
element[ property.name ] = list;
} else {
element[ property.name ] = valueReader.read( at + read );
read += valueReader.size;
}
}
return [ element, read ];
}
function setPropertyBinaryReaders( properties, body, little_endian ) {
function getBinaryReader( dataview, type, little_endian ) {
switch ( type ) {
// correspondences for non-specific length types here match rply:
case 'int8': case 'char': return { read: ( at ) => {
return dataview.getInt8( at );
}, size: 1 };
case 'uint8': case 'uchar': return { read: ( at ) => {
return dataview.getUint8( at );
}, size: 1 };
case 'int16': case 'short': return { read: ( at ) => {
return dataview.getInt16( at, little_endian );
}, size: 2 };
case 'uint16': case 'ushort': return { read: ( at ) => {
return dataview.getUint16( at, little_endian );
}, size: 2 };
case 'int32': case 'int': return { read: ( at ) => {
return dataview.getInt32( at, little_endian );
}, size: 4 };
case 'uint32': case 'uint': return { read: ( at ) => {
return dataview.getUint32( at, little_endian );
}, size: 4 };
case 'float32': case 'float': return { read: ( at ) => {
return dataview.getFloat32( at, little_endian );
}, size: 4 };
case 'float64': case 'double': return { read: ( at ) => {
return dataview.getFloat64( at, little_endian );
}, size: 8 };
}
}
for ( let i = 0, l = properties.length; i < l; i ++ ) {
const property = properties[ i ];
if ( property.type === 'list' ) {
property.countReader = getBinaryReader( body, property.countType, little_endian );
property.valueReader = getBinaryReader( body, property.itemType, little_endian );
} else {
property.valueReader = getBinaryReader( body, property.type, little_endian );
}
}
}
function parseBinary( data, header ) {
const buffer = createBuffer();
const little_endian = ( header.format === 'binary_little_endian' );
const body = new DataView( data, header.headerLength );
let result, loc = 0;
for ( let currentElement = 0; currentElement < header.elements.length; currentElement ++ ) {
const elementDesc = header.elements[ currentElement ];
const properties = elementDesc.properties;
const attributeDescriptor = getAttributeDescriptor( properties );
buffer.descriptors[ elementDesc.name ] = attributeDescriptor;
setPropertyBinaryReaders( properties, body, little_endian );
for ( let currentElementCount = 0; currentElementCount < elementDesc.count; currentElementCount ++ ) {
result = binaryReadElement( loc, properties );
loc += result[ 1 ];
const element = result[ 0 ];
handleElement( buffer, elementDesc.name, element, attributeDescriptor );
}
}
return postProcess( buffer );
}
function extractHeaderText( bytes ) {
let i = 0;
let cont = true;
let line = '';
const lines = [];
const startLine = new TextDecoder().decode( bytes.subarray( 0, 5 ) );
const hasCRNL = /^ply\r\n/.test( startLine );
do {
const c = String.fromCharCode( bytes[ i ++ ] );
if ( c !== '\n' && c !== '\r' ) {
line += c;
} else {
if ( line === 'end_header' ) cont = false;
if ( line !== '' ) {
lines.push( line );
line = '';
}
}
} while ( cont && i < bytes.length );
// ascii section using \r\n as line endings
if ( hasCRNL === true ) i ++;
return { headerText: lines.join( '\r' ) + '\r', headerLength: i };
}
//
let geometry;
const scope = this;
if ( data instanceof ArrayBuffer ) {
const bytes = new Uint8Array( data );
const { headerText, headerLength } = extractHeaderText( bytes );
const header = parseHeader( headerText, headerLength );
if ( header.format === 'ascii' ) {
const text = new TextDecoder().decode( bytes );
geometry = parseASCII( text, header );
} else {
geometry = parseBinary( data, header );
}
} else {
geometry = parseASCII( data, parseHeader( data ) );
}
return geometry;
}
}
class Float64BufferAttribute extends BufferAttribute {
constructor( array, itemSize, normalized ) {
super( new Float64Array( array ), itemSize, normalized );
}
}
class ArrayStream {
constructor( arr ) {
this.arr = arr;
this.i = 0;
}
empty() {
return this.i >= this.arr.length;
}
next() {
return this.arr[ this.i ++ ];
}
}
export { PLYLoader };

View File

@ -0,0 +1,191 @@
import {
BufferGeometry,
Float32BufferAttribute,
OrthographicCamera,
Mesh
} from 'three';
/**
* Abstract base class for all post processing passes.
*
* This module is only relevant for post processing with {@link WebGLRenderer}.
*
* @abstract
* @three_import import { Pass } from 'three/addons/postprocessing/Pass.js';
*/
class Pass {
/**
* Constructs a new pass.
*/
constructor() {
/**
* This flag can be used for type testing.
*
* @type {boolean}
* @readonly
* @default true
*/
this.isPass = true;
/**
* If set to `true`, the pass is processed by the composer.
*
* @type {boolean}
* @default true
*/
this.enabled = true;
/**
* If set to `true`, the pass indicates to swap read and write buffer after rendering.
*
* @type {boolean}
* @default true
*/
this.needsSwap = true;
/**
* If set to `true`, the pass clears its buffer before rendering
*
* @type {boolean}
* @default false
*/
this.clear = false;
/**
* If set to `true`, the result of the pass is rendered to screen. The last pass in the composers
* pass chain gets automatically rendered to screen, no matter how this property is configured.
*
* @type {boolean}
* @default false
*/
this.renderToScreen = false;
}
/**
* Sets the size of the pass.
*
* @abstract
* @param {number} width - The width to set.
* @param {number} height - The height to set.
*/
setSize( /* width, height */ ) {}
/**
* This method holds the render logic of a pass. It must be implemented in all derived classes.
*
* @abstract
* @param {WebGLRenderer} renderer - The renderer.
* @param {WebGLRenderTarget} writeBuffer - The write buffer. This buffer is intended as the rendering
* destination for the pass.
* @param {WebGLRenderTarget} readBuffer - The read buffer. The pass can access the result from the
* previous pass from this buffer.
* @param {number} deltaTime - The delta time in seconds.
* @param {boolean} maskActive - Whether masking is active or not.
*/
render( /* renderer, writeBuffer, readBuffer, deltaTime, maskActive */ ) {
console.error( 'THREE.Pass: .render() must be implemented in derived pass.' );
}
/**
* Frees the GPU-related resources allocated by this instance. Call this
* method whenever the pass is no longer used in your app.
*
* @abstract
*/
dispose() {}
}
// Helper for passes that need to fill the viewport with a single quad.
const _camera = new OrthographicCamera( - 1, 1, 1, - 1, 0, 1 );
// https://github.com/mrdoob/three.js/pull/21358
class FullscreenTriangleGeometry extends BufferGeometry {
constructor() {
super();
this.setAttribute( 'position', new Float32BufferAttribute( [ - 1, 3, 0, - 1, - 1, 0, 3, - 1, 0 ], 3 ) );
this.setAttribute( 'uv', new Float32BufferAttribute( [ 0, 2, 0, 0, 2, 0 ], 2 ) );
}
}
const _geometry = new FullscreenTriangleGeometry();
/**
* This module is a helper for passes which need to render a full
* screen effect which is quite common in context of post processing.
*
* The intended usage is to reuse a single full screen quad for rendering
* subsequent passes by just reassigning the `material` reference.
*
* This module can only be used with {@link WebGLRenderer}.
*
* @augments Mesh
* @three_import import { FullScreenQuad } from 'three/addons/postprocessing/Pass.js';
*/
class FullScreenQuad {
/**
* Constructs a new full screen quad.
*
* @param {?Material} material - The material to render te full screen quad with.
*/
constructor( material ) {
this._mesh = new Mesh( _geometry, material );
}
/**
* Frees the GPU-related resources allocated by this instance. Call this
* method whenever the instance is no longer used in your app.
*/
dispose() {
this._mesh.geometry.dispose();
}
/**
* Renders the full screen quad.
*
* @param {WebGLRenderer} renderer - The renderer.
*/
render( renderer ) {
renderer.render( this._mesh, _camera );
}
/**
* The quad's material.
*
* @type {?Material}
*/
get material() {
return this._mesh.material;
}
set material( value ) {
this._mesh.material = value;
}
}
export { Pass, FullScreenQuad };

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,496 @@
import {
AnimationClip,
AnimationMixer,
Matrix4,
Quaternion,
QuaternionKeyframeTrack,
SkeletonHelper,
Vector3,
VectorKeyframeTrack
} from 'three';
/**
* @module SkeletonUtils
* @three_import import * as SkeletonUtils from 'three/addons/utils/SkeletonUtils.js';
*/
function getBoneName( bone, options ) {
if ( options.getBoneName !== undefined ) {
return options.getBoneName( bone );
}
return options.names[ bone.name ];
}
/**
* Retargets the skeleton from the given source to the target.
*
* Both `target` and `source` can be a 3D object with a skeleton property (e.g. a skinned mesh)
* or a {@link Skeleton} directly.
*
* @param {Object3D|Skeleton} target - The target object.
* @param {Object3D|Skeleton} source - The source object.
* @param {module:SkeletonUtils~RetargetOptions} options - The options.
*/
function retarget( target, source, options = {} ) {
const quat = new Quaternion(),
scale = new Vector3(),
relativeMatrix = new Matrix4(),
globalMatrix = new Matrix4();
options.preserveBoneMatrix = options.preserveBoneMatrix !== undefined ? options.preserveBoneMatrix : true;
options.preserveBonePositions = options.preserveBonePositions !== undefined ? options.preserveBonePositions : true;
options.useTargetMatrix = options.useTargetMatrix !== undefined ? options.useTargetMatrix : false;
options.hip = options.hip !== undefined ? options.hip : 'hip';
options.hipInfluence = options.hipInfluence !== undefined ? options.hipInfluence : new Vector3( 1, 1, 1 );
options.scale = options.scale !== undefined ? options.scale : 1;
options.names = options.names || {};
const sourceBones = source.isObject3D ? source.skeleton.bones : getBones( source ),
bones = target.isObject3D ? target.skeleton.bones : getBones( target );
let bone, name, boneTo,
bonesPosition;
// reset bones
if ( target.isObject3D ) {
target.skeleton.pose();
} else {
options.useTargetMatrix = true;
options.preserveBoneMatrix = false;
}
if ( options.preserveBonePositions ) {
bonesPosition = [];
for ( let i = 0; i < bones.length; i ++ ) {
bonesPosition.push( bones[ i ].position.clone() );
}
}
if ( options.preserveBoneMatrix ) {
// reset matrix
target.updateMatrixWorld();
target.matrixWorld.identity();
// reset children matrix
for ( let i = 0; i < target.children.length; ++ i ) {
target.children[ i ].updateMatrixWorld( true );
}
}
for ( let i = 0; i < bones.length; ++ i ) {
bone = bones[ i ];
name = getBoneName( bone, options );
boneTo = getBoneByName( name, sourceBones );
globalMatrix.copy( bone.matrixWorld );
if ( boneTo ) {
boneTo.updateMatrixWorld();
if ( options.useTargetMatrix ) {
relativeMatrix.copy( boneTo.matrixWorld );
} else {
relativeMatrix.copy( target.matrixWorld ).invert();
relativeMatrix.multiply( boneTo.matrixWorld );
}
// ignore scale to extract rotation
scale.setFromMatrixScale( relativeMatrix );
relativeMatrix.scale( scale.set( 1 / scale.x, 1 / scale.y, 1 / scale.z ) );
// apply to global matrix
globalMatrix.makeRotationFromQuaternion( quat.setFromRotationMatrix( relativeMatrix ) );
if ( target.isObject3D ) {
if ( options.localOffsets ) {
if ( options.localOffsets[ bone.name ] ) {
globalMatrix.multiply( options.localOffsets[ bone.name ] );
}
}
}
globalMatrix.copyPosition( relativeMatrix );
}
if ( name === options.hip ) {
globalMatrix.elements[ 12 ] *= options.scale * options.hipInfluence.x;
globalMatrix.elements[ 13 ] *= options.scale * options.hipInfluence.y;
globalMatrix.elements[ 14 ] *= options.scale * options.hipInfluence.z;
if ( options.hipPosition !== undefined ) {
globalMatrix.elements[ 12 ] += options.hipPosition.x * options.scale;
globalMatrix.elements[ 13 ] += options.hipPosition.y * options.scale;
globalMatrix.elements[ 14 ] += options.hipPosition.z * options.scale;
}
}
if ( bone.parent ) {
bone.matrix.copy( bone.parent.matrixWorld ).invert();
bone.matrix.multiply( globalMatrix );
} else {
bone.matrix.copy( globalMatrix );
}
bone.matrix.decompose( bone.position, bone.quaternion, bone.scale );
bone.updateMatrixWorld();
}
if ( options.preserveBonePositions ) {
for ( let i = 0; i < bones.length; ++ i ) {
bone = bones[ i ];
name = getBoneName( bone, options ) || bone.name;
if ( name !== options.hip ) {
bone.position.copy( bonesPosition[ i ] );
}
}
}
if ( options.preserveBoneMatrix ) {
// restore matrix
target.updateMatrixWorld( true );
}
}
/**
* Retargets the animation clip of the source to the target 3D object.
*
* The `source` can be a 3D object with a skeleton property (e.g. a skinned mesh)
* or a {@link Skeleton} directly.
*
* @param {Object3D} target - The target 3D object. Must have a `skeleton` property.
* @param {Object3D|Skeleton} source - The source object.
* @param {AnimationClip} clip - The animation clip.
* @param {module:SkeletonUtils~RetargetOptions} options - The options.
* @return {AnimationClip} The retargeted animation clip.
*/
function retargetClip( target, source, clip, options = {} ) {
options.useFirstFramePosition = options.useFirstFramePosition !== undefined ? options.useFirstFramePosition : false;
// Calculate the fps from the source clip based on the track with the most frames, unless fps is already provided.
options.fps = options.fps !== undefined ? options.fps : ( Math.max( ...clip.tracks.map( track => track.times.length ) ) / clip.duration );
options.names = options.names || [];
if ( ! source.isObject3D ) {
source = getHelperFromSkeleton( source );
}
const numFrames = Math.round( clip.duration * ( options.fps / 1000 ) * 1000 ),
delta = clip.duration / ( numFrames - 1 ),
convertedTracks = [],
mixer = new AnimationMixer( source ),
bones = getBones( target.skeleton ),
boneDatas = [];
let positionOffset,
bone, boneTo, boneData,
name;
mixer.clipAction( clip ).play();
// trim
let start = 0, end = numFrames;
if ( options.trim !== undefined ) {
start = Math.round( options.trim[ 0 ] * options.fps );
end = Math.min( Math.round( options.trim[ 1 ] * options.fps ), numFrames ) - start;
mixer.update( options.trim[ 0 ] );
} else {
mixer.update( 0 );
}
source.updateMatrixWorld();
//
for ( let frame = 0; frame < end; ++ frame ) {
const time = frame * delta;
retarget( target, source, options );
for ( let j = 0; j < bones.length; ++ j ) {
bone = bones[ j ];
name = getBoneName( bone, options ) || bone.name;
boneTo = getBoneByName( name, source.skeleton );
if ( boneTo ) {
boneData = boneDatas[ j ] = boneDatas[ j ] || { bone: bone };
if ( options.hip === name ) {
if ( ! boneData.pos ) {
boneData.pos = {
times: new Float32Array( end ),
values: new Float32Array( end * 3 )
};
}
if ( options.useFirstFramePosition ) {
if ( frame === 0 ) {
positionOffset = bone.position.clone();
}
bone.position.sub( positionOffset );
}
boneData.pos.times[ frame ] = time;
bone.position.toArray( boneData.pos.values, frame * 3 );
}
if ( ! boneData.quat ) {
boneData.quat = {
times: new Float32Array( end ),
values: new Float32Array( end * 4 )
};
}
boneData.quat.times[ frame ] = time;
bone.quaternion.toArray( boneData.quat.values, frame * 4 );
}
}
if ( frame === end - 2 ) {
// last mixer update before final loop iteration
// make sure we do not go over or equal to clip duration
mixer.update( delta - 0.0000001 );
} else {
mixer.update( delta );
}
source.updateMatrixWorld();
}
for ( let i = 0; i < boneDatas.length; ++ i ) {
boneData = boneDatas[ i ];
if ( boneData ) {
if ( boneData.pos ) {
convertedTracks.push( new VectorKeyframeTrack(
'.bones[' + boneData.bone.name + '].position',
boneData.pos.times,
boneData.pos.values
) );
}
convertedTracks.push( new QuaternionKeyframeTrack(
'.bones[' + boneData.bone.name + '].quaternion',
boneData.quat.times,
boneData.quat.values
) );
}
}
mixer.uncacheAction( clip );
return new AnimationClip( clip.name, - 1, convertedTracks );
}
/**
* Clones the given 3D object and its descendants, ensuring that any `SkinnedMesh` instances are
* correctly associated with their bones. Bones are also cloned, and must be descendants of the
* object passed to this method. Other data, like geometries and materials, are reused by reference.
*
* @param {Object3D} source - The 3D object to clone.
* @return {Object3D} The cloned 3D object.
*/
function clone( source ) {
const sourceLookup = new Map();
const cloneLookup = new Map();
const clone = source.clone();
parallelTraverse( source, clone, function ( sourceNode, clonedNode ) {
sourceLookup.set( clonedNode, sourceNode );
cloneLookup.set( sourceNode, clonedNode );
} );
clone.traverse( function ( node ) {
if ( ! node.isSkinnedMesh ) return;
const clonedMesh = node;
const sourceMesh = sourceLookup.get( node );
const sourceBones = sourceMesh.skeleton.bones;
clonedMesh.skeleton = sourceMesh.skeleton.clone();
clonedMesh.bindMatrix.copy( sourceMesh.bindMatrix );
clonedMesh.skeleton.bones = sourceBones.map( function ( bone ) {
return cloneLookup.get( bone );
} );
clonedMesh.bind( clonedMesh.skeleton, clonedMesh.bindMatrix );
} );
return clone;
}
// internal helper
function getBoneByName( name, skeleton ) {
for ( let i = 0, bones = getBones( skeleton ); i < bones.length; i ++ ) {
if ( name === bones[ i ].name )
return bones[ i ];
}
}
function getBones( skeleton ) {
return Array.isArray( skeleton ) ? skeleton : skeleton.bones;
}
function getHelperFromSkeleton( skeleton ) {
const source = new SkeletonHelper( skeleton.bones[ 0 ] );
source.skeleton = skeleton;
return source;
}
function parallelTraverse( a, b, callback ) {
callback( a, b );
for ( let i = 0; i < a.children.length; i ++ ) {
parallelTraverse( a.children[ i ], b.children[ i ], callback );
}
}
/**
* Retarget options of `SkeletonUtils`.
*
* @typedef {Object} module:SkeletonUtils~RetargetOptions
* @property {boolean} [useFirstFramePosition=false] - Whether to use the position of the first frame or not.
* @property {number} [fps] - The FPS of the clip.
* @property {Object<string,string>} [names] - A dictionary for mapping target to source bone names.
* @property {function(string):string} [getBoneName] - A function for mapping bone names. Alternative to `names`.
* @property {Array<number>} [trim] - Whether to trim the clip or not. If set the array should hold two values for the start and end.
* @property {boolean} [preserveBoneMatrix=true] - Whether to preserve bone matrices or not.
* @property {boolean} [preserveBonePositions=true] - Whether to preserve bone positions or not.
* @property {boolean} [useTargetMatrix=false] - Whether to use the target matrix or not.
* @property {string} [hip='hip'] - The name of the source's hip bone.
* @property {Vector3} [hipInfluence=(1,1,1)] - The hip influence.
* @property {number} [scale=1] - The scale.
* @property {Object<string,Matrix4>} [localOffsets] - Per-bone local offset matrices, keyed by bone name.
* @property {Vector3} [hipPosition] - An additional position offset applied to the hip bone.
**/
export {
retarget,
retargetClip,
clone,
};

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -0,0 +1,393 @@
from __future__ import annotations
import json
import sqlite3
import time
import uuid
from pathlib import Path
from typing import Any
class PlatformStore:
def __init__(self, db_path: Path) -> None:
self.db_path = db_path
self.db_path.parent.mkdir(parents=True, exist_ok=True)
self._init()
def create_dataset(
self,
*,
name: str,
path: Path,
scan: dict[str, Any],
item_id: str | None = None,
source_type: str = "local",
source_uri: str = "",
version: str = "",
parent_task_id: str | None = None,
manifest_path: Path | None = None,
metrics: dict[str, Any] | None = None,
tags: list[str] | None = None,
) -> dict[str, Any]:
item_id = item_id or uuid.uuid4().hex
now = _now()
with self._connect() as db:
db.execute(
"insert into datasets(id,name,path,scan_json,source_type,source_uri,version,parent_task_id,manifest_path,metrics_json,tags_json,created_at,updated_at) values(?,?,?,?,?,?,?,?,?,?,?,?,?)",
(item_id, name, str(path), _json(scan), source_type, source_uri, version, parent_task_id, str(manifest_path or ""), _json(metrics or {}), _json(tags or []), now, now),
)
return self.get_dataset(item_id)
def list_datasets(self) -> list[dict[str, Any]]:
return self._list("datasets", self._dataset_row)
def get_dataset(self, item_id: str) -> dict[str, Any]:
return self._get("datasets", item_id, self._dataset_row)
def update_dataset(self, item_id: str, *, name: str | None = None, tags: list[str] | None = None, version: str | None = None) -> dict[str, Any]:
self._update_entity("datasets", item_id, name=name, tags=tags, version=version)
return self.get_dataset(item_id)
def delete_dataset(self, item_id: str) -> None:
self._delete("datasets", item_id)
def create_scene(
self,
*,
name: str,
path: Path,
metadata: dict[str, Any],
item_id: str | None = None,
source_type: str = "generated",
source_uri: str = "",
version: str = "",
parent_task_id: str | None = None,
manifest_path: Path | None = None,
metrics: dict[str, Any] | None = None,
tags: list[str] | None = None,
) -> dict[str, Any]:
item_id = item_id or uuid.uuid4().hex
now = _now()
with self._connect() as db:
db.execute(
"insert into scenes(id,name,path,metadata_json,source_type,source_uri,version,parent_task_id,manifest_path,metrics_json,tags_json,created_at,updated_at) values(?,?,?,?,?,?,?,?,?,?,?,?,?)",
(item_id, name, str(path), _json(metadata), source_type, source_uri, version, parent_task_id, str(manifest_path or ""), _json(metrics or {}), _json(tags or []), now, now),
)
return self.get_scene(item_id)
def list_scenes(self) -> list[dict[str, Any]]:
return self._list("scenes", self._scene_row)
def get_scene(self, item_id: str) -> dict[str, Any]:
return self._get("scenes", item_id, self._scene_row)
def update_scene(
self,
item_id: str,
*,
name: str | None = None,
tags: list[str] | None = None,
version: str | None = None,
metadata: dict[str, Any] | None = None,
metrics: dict[str, Any] | None = None,
) -> dict[str, Any]:
self._update_entity("scenes", item_id, name=name, tags=tags, version=version)
if metadata is not None or metrics is not None:
assignments: list[str] = []
values: list[Any] = []
if metadata is not None:
assignments.append("metadata_json=?")
values.append(_json(metadata))
if metrics is not None:
assignments.append("metrics_json=?")
values.append(_json(metrics))
assignments.append("updated_at=?")
values.append(_now())
values.append(item_id)
with self._connect() as db:
db.execute(f"update scenes set {','.join(assignments)} where id=?", values)
return self.get_scene(item_id)
def delete_scene(self, item_id: str) -> None:
self._delete("scenes", item_id)
def create_model(
self,
*,
name: str,
path: Path,
metadata: dict[str, Any],
item_id: str | None = None,
source_type: str = "training",
source_uri: str = "",
version: str = "",
parent_task_id: str | None = None,
manifest_path: Path | None = None,
metrics: dict[str, Any] | None = None,
tags: list[str] | None = None,
) -> dict[str, Any]:
item_id = item_id or uuid.uuid4().hex
now = _now()
with self._connect() as db:
db.execute(
"insert into models(id,name,path,metadata_json,source_type,source_uri,version,parent_task_id,manifest_path,metrics_json,tags_json,created_at,updated_at) values(?,?,?,?,?,?,?,?,?,?,?,?,?)",
(item_id, name, str(path), _json(metadata), source_type, source_uri, version, parent_task_id, str(manifest_path or ""), _json(metrics or {}), _json(tags or []), now, now),
)
return self.get_model(item_id)
def list_models(self) -> list[dict[str, Any]]:
return self._list("models", self._model_row)
def get_model(self, item_id: str) -> dict[str, Any]:
return self._get("models", item_id, self._model_row)
def update_model(self, item_id: str, *, name: str | None = None, tags: list[str] | None = None, version: str | None = None) -> dict[str, Any]:
self._update_entity("models", item_id, name=name, tags=tags, version=version)
return self.get_model(item_id)
def delete_model(self, item_id: str) -> None:
self._delete("models", item_id)
def create_task(self, *, task_type: str, params: dict[str, Any], run_dir: Path) -> dict[str, Any]:
item_id = uuid.uuid4().hex
now = _now()
with self._connect() as db:
db.execute(
"insert into tasks(id,type,status,params_json,result_json,run_dir,created_at,updated_at) values(?,?,?,?,?,?,?,?)",
(item_id, task_type, "queued", _json(params), "{}", str(run_dir), now, now),
)
return self.get_task(item_id)
def list_tasks(self) -> list[dict[str, Any]]:
return self._list("tasks", self._task_row)
def get_task(self, item_id: str) -> dict[str, Any]:
return self._get("tasks", item_id, self._task_row)
def next_queued_task(self) -> dict[str, Any] | None:
with self._connect() as db:
row = db.execute("select * from tasks where status='queued' order by created_at limit 1").fetchone()
return self._task_row(row) if row else None
def update_task(
self,
task_id: str,
*,
status: str | None = None,
result: dict[str, Any] | None = None,
error: str | None = None,
progress: int | None = None,
stage: str | None = None,
message: str | None = None,
) -> None:
task = self.get_task(task_id)
next_progress = task["progress"] if progress is None else max(0, min(100, int(progress)))
with self._connect() as db:
db.execute(
"update tasks set status=?, result_json=?, error=?, progress=?, stage=?, message=?, updated_at=? where id=?",
(
status or task["status"],
_json(result if result is not None else task["result"]),
error,
next_progress,
task["stage"] if stage is None else stage,
task["message"] if message is None else message,
_now(),
task_id,
),
)
def set_task_pid(self, task_id: str, pid: int | None) -> None:
with self._connect() as db:
db.execute("update tasks set pid=?, updated_at=? where id=?", (pid, _now(), task_id))
def cancel_task(self, task_id: str) -> dict[str, Any]:
task = self.get_task(task_id)
with self._connect() as db:
if task["status"] == "queued":
db.execute(
"update tasks set status='canceled', cancel_requested=1, progress=0, stage='已取消', message='任务启动前已取消', updated_at=? where id=?",
(_now(), task_id),
)
elif task["status"] == "running":
db.execute(
"update tasks set status='canceling', cancel_requested=1, stage='取消中', message='正在终止运行进程', updated_at=? where id=?",
(_now(), task_id),
)
return self.get_task(task_id)
def recover_interrupted_tasks(self) -> int:
with self._connect() as db:
cursor = db.execute(
"update tasks set status='failed_recovered', error='service restarted while task was running', pid=null, stage='恢复失败', message='服务重启时任务仍在运行,已标记为可重试', updated_at=? where status in ('running','canceling')",
(_now(),),
)
return cursor.rowcount
def retry_task(self, task_id: str) -> dict[str, Any]:
task = self.get_task(task_id)
base = Path(task["run_dir"])
index = 1
retry_dir = base.with_name(f"{base.name}_retry{index}")
while retry_dir.exists():
index += 1
retry_dir = base.with_name(f"{base.name}_retry{index}")
return self.create_task(task_type=task["type"], params=task["params"], run_dir=retry_dir)
def update_task_params(self, task_id: str, *, params: dict[str, Any]) -> dict[str, Any]:
task = self.get_task(task_id)
if task["status"] not in {"queued", "failed", "failed_recovered", "canceled"}:
raise ValueError("only queued or terminal tasks can be edited")
with self._connect() as db:
db.execute("update tasks set params_json=?, updated_at=? where id=?", (_json(params), _now(), task_id))
return self.get_task(task_id)
def delete_task(self, task_id: str) -> None:
task = self.get_task(task_id)
if task["status"] in {"running", "canceling"}:
raise ValueError("running tasks cannot be deleted")
self._delete("tasks", task_id)
def add_artifact(self, *, task_id: str | None, name: str, path: Path, kind: str) -> dict[str, Any]:
item_id = uuid.uuid4().hex
now = _now()
with self._connect() as db:
db.execute(
"insert into artifacts(id,task_id,name,path,kind,created_at) values(?,?,?,?,?,?)",
(item_id, task_id, name, str(path), kind, now),
)
return self.get_artifact(item_id)
def list_task_artifacts(self, task_id: str) -> list[dict[str, Any]]:
with self._connect() as db:
rows = db.execute("select * from artifacts where task_id=? order by created_at", (task_id,)).fetchall()
return [self._artifact_row(row) for row in rows]
def get_artifact(self, item_id: str) -> dict[str, Any]:
return self._get("artifacts", item_id, self._artifact_row)
def _init(self) -> None:
with self._connect() as db:
db.execute("create table if not exists datasets(id text primary key,name text,path text,scan_json text,created_at real,updated_at real)")
db.execute("create table if not exists scenes(id text primary key,name text,path text,metadata_json text,created_at real,updated_at real)")
db.execute("create table if not exists models(id text primary key,name text,path text,metadata_json text,created_at real,updated_at real)")
db.execute(
"create table if not exists tasks("
"id text primary key,type text,status text,params_json text,result_json text not null default '{}',"
"run_dir text,error text,cancel_requested integer not null default 0,created_at real,updated_at real)"
)
db.execute("create table if not exists artifacts(id text primary key,task_id text,name text,path text,kind text,created_at real)")
for table in ["datasets", "scenes", "models"]:
self._ensure_column(db, table, "source_type", "text not null default 'local'")
self._ensure_column(db, table, "source_uri", "text not null default ''")
self._ensure_column(db, table, "version", "text not null default ''")
self._ensure_column(db, table, "parent_task_id", "text")
self._ensure_column(db, table, "manifest_path", "text not null default ''")
self._ensure_column(db, table, "metrics_json", "text not null default '{}'")
self._ensure_column(db, table, "tags_json", "text not null default '[]'")
self._ensure_column(db, "tasks", "pid", "integer")
self._ensure_column(db, "tasks", "progress", "integer not null default 0")
self._ensure_column(db, "tasks", "stage", "text not null default ''")
self._ensure_column(db, "tasks", "message", "text not null default ''")
def _connect(self) -> sqlite3.Connection:
db = sqlite3.connect(self.db_path)
db.row_factory = sqlite3.Row
return db
def _ensure_column(self, db: sqlite3.Connection, table: str, column: str, spec: str) -> None:
columns = {row["name"] for row in db.execute(f"pragma table_info({table})").fetchall()}
if column not in columns:
db.execute(f"alter table {table} add column {column} {spec}")
def _list(self, table: str, mapper) -> list[dict[str, Any]]:
with self._connect() as db:
rows = db.execute(f"select * from {table} order by created_at desc").fetchall()
return [mapper(row) for row in rows]
def _get(self, table: str, item_id: str, mapper) -> dict[str, Any]:
with self._connect() as db:
row = db.execute(f"select * from {table} where id=?", (item_id,)).fetchone()
if row is None:
raise KeyError(item_id)
return mapper(row)
def _update_entity(self, table: str, item_id: str, *, name: str | None, tags: list[str] | None, version: str | None) -> None:
self._get(table, item_id, lambda row: row)
fields: list[str] = []
values: list[Any] = []
if name is not None:
fields.append("name=?")
values.append(name)
if tags is not None:
fields.append("tags_json=?")
values.append(_json(tags))
if version is not None:
fields.append("version=?")
values.append(version)
if not fields:
return
fields.append("updated_at=?")
values.append(_now())
values.append(item_id)
with self._connect() as db:
db.execute(f"update {table} set {', '.join(fields)} where id=?", values)
def _delete(self, table: str, item_id: str) -> None:
self._get(table, item_id, lambda row: row)
with self._connect() as db:
db.execute(f"delete from {table} where id=?", (item_id,))
def _dataset_row(self, row: sqlite3.Row) -> dict[str, Any]:
return {**self._entity_common(row), "scan": json.loads(row["scan_json"])}
def _scene_row(self, row: sqlite3.Row) -> dict[str, Any]:
return {**self._entity_common(row), "metadata": json.loads(row["metadata_json"])}
def _model_row(self, row: sqlite3.Row) -> dict[str, Any]:
return {**self._entity_common(row), "metadata": json.loads(row["metadata_json"])}
def _entity_common(self, row: sqlite3.Row) -> dict[str, Any]:
return {
"id": row["id"],
"name": row["name"],
"path": row["path"],
"source_type": row["source_type"],
"source_uri": row["source_uri"],
"version": row["version"],
"parent_task_id": row["parent_task_id"],
"manifest_path": row["manifest_path"],
"metrics": json.loads(row["metrics_json"] or "{}"),
"tags": json.loads(row["tags_json"] or "[]"),
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
def _task_row(self, row: sqlite3.Row) -> dict[str, Any]:
return {
"id": row["id"],
"type": row["type"],
"status": row["status"],
"params": json.loads(row["params_json"]),
"result": json.loads(row["result_json"] or "{}"),
"run_dir": row["run_dir"],
"error": row["error"],
"progress": int(row["progress"] or 0),
"stage": row["stage"] or "",
"message": row["message"] or "",
"cancel_requested": bool(row["cancel_requested"]),
"pid": row["pid"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
def _artifact_row(self, row: sqlite3.Row) -> dict[str, Any]:
return {"id": row["id"], "task_id": row["task_id"], "name": row["name"], "path": row["path"], "kind": row["kind"], "created_at": row["created_at"]}
def _json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False)
def _now() -> float:
return time.time()

View File

@ -0,0 +1,67 @@
from __future__ import annotations
import threading
import time
import traceback
from typing import Any
from pathlib import Path
from .adapters import TaskCanceled, execute_task, index_artifacts_for_task
from .config import PlatformConfig
from .store import PlatformStore
class TaskWorker:
def __init__(self, store: PlatformStore, config: PlatformConfig) -> None:
self.store = store
self.config = config
self._stop = threading.Event()
self._thread: threading.Thread | None = None
def start(self) -> None:
if self._thread and self._thread.is_alive():
return
self.store.recover_interrupted_tasks()
self._thread = threading.Thread(target=self._loop, name="real2sim-platform-worker", daemon=True)
self._thread.start()
def stop(self) -> None:
self._stop.set()
def _loop(self) -> None:
while not self._stop.is_set():
task = self.store.next_queued_task()
if task is None:
time.sleep(0.5)
continue
self._run(task)
def _run(self, task: dict[str, Any]) -> None:
if self.store.get_task(task["id"])["cancel_requested"]:
self.store.update_task(task["id"], status="canceled", error="task canceled before start", progress=0, stage="已取消", message="任务启动前已取消")
return
self.store.update_task(task["id"], status="running", progress=5, stage="启动中", message="任务已进入执行队列")
try:
result = execute_task(self.config, self.store, task)
except TaskCanceled as exc:
self._write_trace(task, exc)
index_artifacts_for_task(self.store, task["id"], Path(task["run_dir"]))
self.store.update_task(task["id"], status="canceled", error=str(exc), progress=0, stage="已取消", message=str(exc))
return
except Exception as exc:
self._write_trace(task, exc)
index_artifacts_for_task(self.store, task["id"], Path(task["run_dir"]))
self.store.update_task(task["id"], status="failed", error=str(exc), stage="执行失败", message=str(exc))
return
status = "canceled" if self.store.get_task(task["id"])["cancel_requested"] else "succeeded"
self.store.update_task(task["id"], status=status, result=result, progress=100 if status == "succeeded" else 0, stage="已完成" if status == "succeeded" else "已取消", message="任务完成" if status == "succeeded" else "任务已取消")
def _write_trace(self, task: dict[str, Any], exc: Exception) -> None:
log_path = self.config.runs_dir / task["id"] / "logs" / "task.log"
if "run_dir" in task:
log_path = __import__("pathlib").Path(task["run_dir"]) / "logs" / "task.log"
log_path.parent.mkdir(parents=True, exist_ok=True)
with log_path.open("a", encoding="utf-8") as handle:
handle.write("\nERROR: " + str(exc) + "\n")
handle.write(traceback.format_exc() + "\n")

499
tests/test_platform.py Normal file
View File

@ -0,0 +1,499 @@
from __future__ import annotations
import shutil
import struct
import subprocess
import time
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from PIL import Image
from real2sim_platform.adapters import _export_3dgs_rgb_fallback, collect_task_reports
from real2sim_platform.config import PlatformConfig
from real2sim_platform.server import create_app
from real2sim_platform.store import PlatformStore
ROOT = Path(__file__).resolve().parents[2]
def test_health_and_doctor(tmp_path: Path) -> None:
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
client = TestClient(create_app(cfg))
assert client.get("/api/health").json()["ok"] is True
version = client.get("/api/version").json()
assert version["features"]["media_prepare"] is True
assert version["features"]["reconstruction_preflight"] is True
assert "runtime_paths" in version
assert "config_digest" in version
doctor = client.get("/api/doctor").json()
assert "checks" in doctor
assert "media_prepare_deps" in doctor["checks"]
assert "anysplat_root" in doctor["checks"]
assert "three_bundle" in doctor["checks"]
def test_dry_run_scene_task(tmp_path: Path) -> None:
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
with TestClient(create_app(cfg)) as client:
response = client.post(
"/api/tasks",
json={
"type": "scene_reconstruction",
"params": {
"input_path": str(ROOT / "real2sim-expert-policy/examples"),
"dry_run": True,
"renderer": "spark",
"simulators": ["mujoco", "motrixsim"],
},
},
)
task = response.json()
for _ in range(40):
current = client.get(f"/api/tasks/{task['id']}").json()
if current["status"] in {"succeeded", "failed"}:
break
time.sleep(0.1)
assert current["status"] == "succeeded"
artifacts = client.get(f"/api/tasks/{task['id']}/artifacts").json()
assert any(item["name"] == "scene.json" for item in artifacts)
scene_id = current["result"]["scene_id"]
manifest = client.get(f"/api/scenes/{scene_id}/manifest").json()
assert manifest["schema"] == "real2sim.scene_manifest.v1"
assert manifest["renderer"] == "spark"
assert all("size_bytes" in item and "autoload" in item for item in manifest["assets"])
scene_artifact = next(item for item in artifacts if item["name"] == "scene.json")
assert client.get(scene_artifact["url"]).status_code == 200
def test_task_payload_validation(tmp_path: Path) -> None:
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
client = TestClient(create_app(cfg))
response = client.post("/api/tasks", json={"type": "hf_import", "params": {"repo_id": "owner/name", "repo_type": "model"}})
assert response.status_code == 400
response = client.post("/api/tasks", json={"type": "media_prepare", "params": {}})
assert response.status_code == 400
response = client.post("/api/tasks", json={"type": "reconstruction_preflight", "params": {}})
assert response.status_code == 400
response = client.post("/api/tasks", json={"type": "scene_reconstruction", "params": {"input_path": str(ROOT), "renderer": "bad"}})
assert response.status_code == 400
response = client.post("/api/tasks", json={"type": "simulator_validation", "params": {"scene_path": str(ROOT), "level": "bad"}})
assert response.status_code == 400
response = client.post(
"/api/tasks",
json={
"type": "expert_collection",
"params": {"scene_path": str(ROOT), "policy_preset": "navigation_v1"},
},
)
assert response.status_code == 400
response = client.post(
"/api/tasks",
json={
"type": "expert_collection",
"params": {"scene_path": str(ROOT), "policy_preset": "pick_and_place_v1", "workers": 2},
},
)
assert response.status_code == 400
response = client.post(
"/api/tasks",
json={"type": "training", "params": {"dataset_id": "missing", "recipe": "diffusion_policy"}},
)
assert response.status_code == 400
response = client.post(
"/api/tasks",
json={"type": "inference", "params": {"model_id": "missing", "scene_id": "missing", "workers": 2}},
)
assert response.status_code == 400
def test_crud_metadata_and_task_params(tmp_path: Path) -> None:
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(ROOT,))
client = TestClient(create_app(cfg))
dataset = client.post("/api/datasets/import-path", json={"path": str(ROOT / "real2sim-expert-policy/examples"), "name": "examples"}).json()
updated = client.patch(f"/api/datasets/{dataset['id']}", json={"name": "examples edited", "tags": ["demo", "crud"], "version": "v2"}).json()
assert updated["name"] == "examples edited"
assert updated["tags"] == ["demo", "crud"]
assert updated["version"] == "v2"
task = client.post("/api/tasks", json={"type": "renderer_benchmark", "params": {}}).json()
edited = client.patch(f"/api/tasks/{task['id']}", json={"params": {"note": "edited before run"}}).json()
assert edited["params"]["note"] == "edited before run"
assert client.delete(f"/api/tasks/{task['id']}").json()["ok"] is True
assert client.delete(f"/api/datasets/{dataset['id']}").json()["ok"] is True
def test_task_progress_fields_and_bulk_actions(tmp_path: Path) -> None:
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
with TestClient(create_app(cfg)) as client:
task = client.post("/api/tasks", json={"type": "renderer_benchmark", "params": {}}).json()
current = _wait(client, task["id"])
assert current["status"] == "succeeded"
assert current["progress"] == 100
assert current["stage"] == "已完成"
retry = client.post("/api/tasks/bulk", json={"action": "retry", "ids": [task["id"]]}).json()
assert retry["ok"] is True
retry_id = retry["results"][0]["result"]["id"]
cancel = client.post("/api/tasks/bulk", json={"action": "cancel", "ids": [retry_id]}).json()
assert cancel["ok"] is True
delete = client.post("/api/tasks/bulk", json={"action": "delete", "ids": [task["id"]]}).json()
assert delete["ok"] is True
def test_dataset_scene_model_bulk_interfaces(tmp_path: Path) -> None:
package = tmp_path / "scene_package"
package.mkdir()
scene_file = package / "scene.json"
scene_file.write_text('{"schema":"motrixsim.scene.v1"}\n', encoding="utf-8")
card_dir = tmp_path / "model"
card_dir.mkdir()
(card_dir / "model_card.json").write_text('{"metrics":{"success_rate":0.91}}\n', encoding="utf-8")
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(ROOT, tmp_path))
client = TestClient(create_app(cfg))
dataset = client.post("/api/datasets/import-path", json={"path": str(ROOT / "real2sim-expert-policy/examples"), "name": "examples"}).json()
dataset_bulk = client.post("/api/datasets/bulk", json={"action": "preflight", "ids": [dataset["id"]], "params": {"min_frames": 1}}).json()
assert dataset_bulk["ok"] is True
assert dataset_bulk["results"][0]["result"]["type"] == "reconstruction_preflight"
scene = client.post("/api/scenes/import", json={"path": str(scene_file), "name": "fixture scene"}).json()
scene_bulk = client.post("/api/scenes/bulk", json={"action": "validate", "ids": [scene["id"]], "params": {"level": "smoke"}}).json()
assert scene_bulk["ok"] is True
assert scene_bulk["results"][0]["result"]["type"] == "simulator_validation"
export_bulk = client.post("/api/scenes/bulk", json={"action": "export", "ids": [scene["id"]]}).json()
assert export_bulk["results"][0]["result"]["scene_path"].endswith("scene.json")
rejected_training = client.post(
"/api/tasks",
json={"type": "training", "params": {"dataset_id": dataset["id"], "epochs": 1}},
)
assert rejected_training.status_code == 400
assert "专家采集" in rejected_training.json()["detail"]
# Directly index a model fixture for bulk model API coverage without running a trainer.
store = PlatformStore(cfg.db_path)
expert_dir = tmp_path / "expert_collection"
expert_dir.mkdir()
(expert_dir / "rollouts.jsonl").write_text(
'{"episode":0,"step":0,"observation":[0,0,0],"action":[0,0,0]}\n',
encoding="utf-8",
)
expert_dataset = store.create_dataset(
name="expert rollouts",
path=expert_dir,
scan={"input_kind": "expert_rollouts", "files": 1},
source_type="expert_collection",
)
model = client.post(
"/api/tasks",
json={"type": "training", "params": {"dataset_id": expert_dataset["id"], "epochs": 1, "recipe": "bc_fast"}},
)
assert model.status_code == 200, model.text
model_entity = store.create_model(name="fixture model", path=card_dir, metadata={"metrics": {"success_rate": 0.91}}, manifest_path=card_dir / "model_card.json", metrics={"success_rate": 0.91})
model_bulk = client.post("/api/models/bulk", json={"action": "compare", "ids": [model_entity["id"]]}).json()
assert model_bulk["ok"] is True
assert model_bulk["results"][0]["result"]["metrics"]["success_rate"] == 0.91
assert model.json()["type"] == "training"
def test_scene_asset_append_updates_manifest_and_artifact(tmp_path: Path) -> None:
package = tmp_path / "scene_package"
package.mkdir()
scene_file = package / "scene.json"
scene_file.write_text('{"schema":"motrixsim.scene.v1"}\n', encoding="utf-8")
asset = package / "added_asset.json"
asset.write_text('{"name":"fixture asset"}\n', encoding="utf-8")
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(tmp_path,))
client = TestClient(create_app(cfg))
scene = client.post("/api/scenes/import", json={"path": str(scene_file), "name": "editable scene"}).json()
response = client.post(
f"/api/scenes/{scene['id']}/assets",
json={
"path": str(asset),
"name": "fixture metadata",
"role": "simulator_import",
"transform": {"translation": [1, 2, 3], "scale": 1.5},
},
)
assert response.status_code == 200, response.text
payload = response.json()
assert payload["asset"]["name"] == "fixture metadata"
manifest = client.get(f"/api/scenes/{scene['id']}/manifest").json()
added = [item for item in manifest["assets"] if item["name"] == "fixture metadata"]
assert added
assert added[0]["role"] == "simulator_import"
assert added[0]["transform"]["translation"] == [1, 2, 3]
assert client.get(added[0]["url"]).status_code == 200
def test_scene_runtime_metadata_update(tmp_path: Path) -> None:
store = PlatformStore(tmp_path / "state.db")
scene_file = tmp_path / "scene.xml"
scene_file.write_text("<mujoco/>", encoding="utf-8")
scene = store.create_scene(
name="runtime scene",
path=scene_file,
metadata={"status": "adapter_pending", "adapter_pending": True},
tags=["motrixsim", "adapter-pending"],
)
updated = store.update_scene(
scene["id"],
metadata={
"status": "runtime_validated",
"adapter_pending": False,
"runtime_validation": {"mujoco": True, "motrixsim": True},
},
metrics={"runtime_count": 2},
tags=["motrixsim", "runtime-validated", "motrixsim-ready"],
)
assert updated["metadata"]["runtime_validation"] == {"mujoco": True, "motrixsim": True}
assert updated["metadata"]["adapter_pending"] is False
assert updated["metrics"]["runtime_count"] == 2
assert "adapter-pending" not in updated["tags"]
def test_media_prepare_video_task(tmp_path: Path) -> None:
if shutil.which("ffmpeg") is None:
pytest.skip("ffmpeg is required for media_prepare video fixture")
video = tmp_path / "input.mp4"
_write_fixture_video(video, tmp_path / "frames")
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
with TestClient(create_app(cfg)) as client:
task = client.post(
"/api/tasks",
json={
"type": "media_prepare",
"params": {"input_path": str(video), "sample_fps": 2, "max_frames_per_video": 5, "max_videos": 1},
},
).json()
current = _wait(client, task["id"])
assert current["status"] == "succeeded", current
manifest = client.get(f"/api/datasets/{current['result']['dataset_id']}/manifest").json()
assert manifest["schema"] == "real2sim.prepared_manifest.v1"
assert manifest["frame_count"] > 0
assert manifest["video_frame_counts"][0]["frame_count"] > 0
assert Path(manifest["frames_dir"]).is_dir()
def test_scene_reconstruction_uses_prepared_dataset(tmp_path: Path) -> None:
gs_root = tmp_path / "gs"
script = gs_root / "scripts" / "real2sim_video_motrix_scene.py"
script.parent.mkdir(parents=True)
script.write_text(
"import argparse, json\n"
"from pathlib import Path\n"
"p=argparse.ArgumentParser(); p.add_argument('--output'); p.add_argument('--work-dir'); p.add_argument('--frames-dir'); p.add_argument('--quality-preset'); p.add_argument('--sfm-report', action='store_true'); p.add_argument('--capture-acceptance', action='store_true'); p.add_argument('--pose-init-train', action='store_true'); p.add_argument('--anysplat-root'); p.add_argument('--anysplat-python'); args,_=p.parse_known_args()\n"
"out=Path(args.output); out.mkdir(parents=True, exist_ok=True)\n"
"(out/'scene.json').write_text(json.dumps({'schema':'fake.scene','coordinate_frame':'gs_metric_zup'}), encoding='utf-8')\n"
"(out/'scene_mujoco.xml').write_text('<mujoco model=\"fake\"><worldbody /></mujoco>\\n', encoding='utf-8')\n"
"(out/'motrix_import.json').write_text(json.dumps({'schema':'motrixsim.scene.v1'}), encoding='utf-8')\n",
encoding="utf-8",
)
frames = tmp_path / "prepared" / "frames"
frames.mkdir(parents=True)
for index in range(30):
Image.new("RGB", (640, 480), (index * 3, 48, 120)).save(frames / f"frame_{index:06d}.jpg")
manifest = {
"schema": "real2sim.prepared_manifest.v1",
"frames_dir": str(frames),
"frame_count": 30,
"source_uri": str(tmp_path),
}
prepared_root = frames.parent
(prepared_root / "prepared_manifest.json").write_text(__import__("json").dumps(manifest), encoding="utf-8")
anysplat_root = tmp_path / "AnySplat"
anysplat_root.mkdir()
cfg = PlatformConfig(
app_root=ROOT,
workspace_dir=tmp_path / "workspace",
gs_playground_root=gs_root,
gsplat_python=Path(__import__("sys").executable),
anysplat_root=anysplat_root,
anysplat_python=Path(__import__("sys").executable),
allowed_path_roots=(tmp_path,),
)
with TestClient(create_app(cfg)) as client:
dataset = client.post("/api/datasets/import-path", json={"path": str(prepared_root), "name": "prepared"}).json()
task = client.post(
"/api/tasks",
json={"type": "scene_reconstruction", "params": {"dataset_id": dataset["id"], "dry_run": False, "renderer": "spark", "simulators": ["mujoco", "motrixsim"], "preflight_required": False}},
).json()
current = _wait(client, task["id"])
assert current["status"] == "succeeded", current
artifacts = client.get(f"/api/tasks/{task['id']}/artifacts").json()
assert any(item["name"] == "reconstruction_report.json" for item in artifacts)
reports = client.get(f"/api/tasks/{task['id']}/reports").json()
assert any(item["name"] == "capture_preflight_report.json" for item in reports["reports"])
def test_reconstruction_preflight_blocks_short_dataset(tmp_path: Path) -> None:
frames = tmp_path / "prepared" / "frames"
frames.mkdir(parents=True)
Image.new("RGB", (320, 240), "red").save(frames / "frame_000001.jpg")
prepared_root = frames.parent
(prepared_root / "prepared_manifest.json").write_text(
__import__("json").dumps({"schema": "real2sim.prepared_manifest.v1", "frames_dir": str(frames), "frame_count": 1}),
encoding="utf-8",
)
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", anysplat_root=tmp_path / "missing-anysplat", allowed_path_roots=(tmp_path,))
with TestClient(create_app(cfg)) as client:
dataset = client.post("/api/datasets/import-path", json={"path": str(prepared_root), "name": "short"}).json()
task = client.post("/api/tasks", json={"type": "reconstruction_preflight", "params": {"dataset_id": dataset["id"]}}).json()
current = _wait(client, task["id"])
assert current["status"] == "succeeded"
assert current["result"]["ok"] is False
assert "enough_frames" in current["result"]["report"]["failed_gates"]
assert current["result"]["report"]["readiness_score"] < 100
assert current["result"]["report"]["next_action"] == "run_media_prepare_or_recapture_more_frames"
def test_scene_reconstruction_unprepared_media_error(tmp_path: Path) -> None:
source = tmp_path / "hf_video_manifest"
source.mkdir()
(source / "0000.parquet").write_bytes(b"not a real parquet")
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
with TestClient(create_app(cfg)) as client:
task = client.post("/api/tasks", json={"type": "scene_reconstruction", "params": {"input_path": str(source), "dry_run": False}}).json()
current = _wait(client, task["id"])
assert current["status"] == "failed"
assert "media_prepare" in current["error"]
def test_simulator_validation_task(tmp_path: Path) -> None:
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace")
with TestClient(create_app(cfg)) as client:
scene_task = client.post(
"/api/tasks",
json={
"type": "scene_reconstruction",
"params": {
"input_path": str(ROOT / "real2sim-expert-policy/examples"),
"dry_run": True,
"renderer": "auto",
"simulators": ["mujoco", "motrixsim"],
},
},
).json()
scene_task = _wait(client, scene_task["id"])
assert scene_task["status"] == "succeeded"
validation = client.post(
"/api/tasks",
json={"type": "simulator_validation", "params": {"scene_id": scene_task["result"]["scene_id"], "simulators": ["mujoco", "motrixsim"], "level": "smoke"}},
).json()
validation = _wait(client, validation["id"])
assert validation["status"] == "succeeded"
assert validation["result"]["ok"] is True
assert validation["result"]["report"]["level"] == "smoke"
def test_simulator_validation_accepts_scene_package_layout(tmp_path: Path) -> None:
package = tmp_path / "scene_package"
(package / "mjcf").mkdir(parents=True)
(package / "mjcf" / "scene.xml").write_text('<mujoco model="fixture"><worldbody /></mujoco>\n', encoding="utf-8")
(package / "motrix_import.json").write_text('{"schema":"motrixsim.scene.v1"}\n', encoding="utf-8")
cfg = PlatformConfig(app_root=ROOT, workspace_dir=tmp_path / "workspace", allowed_path_roots=(tmp_path,))
with TestClient(create_app(cfg)) as client:
task = client.post(
"/api/tasks",
json={
"type": "simulator_validation",
"params": {"scene_path": str(package), "simulators": ["mujoco", "motrixsim"], "level": "smoke"},
},
).json()
current = _wait(client, task["id"])
assert current["status"] == "succeeded"
report = current["result"]["report"]
assert report["ok"] is True
assert report["simulators"]["mujoco"]["path"].endswith("mjcf/scene.xml")
assert report["simulators"]["motrixsim"]["path"].endswith("motrix_import.json")
def test_task_reports_prefer_capture_acceptance_failure(tmp_path: Path) -> None:
store = PlatformStore(tmp_path / "state.db")
run_dir = tmp_path / "run"
work_dir = run_dir / "work"
work_dir.mkdir(parents=True)
task = store.create_task(task_type="scene_reconstruction", params={}, run_dir=run_dir)
store.update_task(task["id"], status="failed", error="command failed with exit code 1; see logs")
(work_dir / "capture_acceptance_report.json").write_text(
__import__("json").dumps(
{
"schema": "real2sim.capture_acceptance.v1",
"accepted": False,
"decision": "recapture_required_before_high_fidelity_reconstruction",
"blockers": ["enough_registered_images", "registered_ratio"],
}
),
encoding="utf-8",
)
reports = collect_task_reports(store, task["id"])
assert reports["readable_error"].startswith("capture acceptance failed")
assert "registered_ratio" in reports["readable_error"]
def test_recover_interrupted_tasks(tmp_path: Path) -> None:
store = PlatformStore(tmp_path / "state.db")
task = store.create_task(task_type="renderer_benchmark", params={}, run_dir=tmp_path / "run")
store.update_task(task["id"], status="running")
assert store.recover_interrupted_tasks() == 1
recovered = store.get_task(task["id"])
assert recovered["status"] == "failed_recovered"
def test_discoverse_3dgs_rgb_fallback_export(tmp_path: Path) -> None:
source = tmp_path / "scene.ply"
target = tmp_path / "fallback" / "scene_threejs_rgb.ply"
report_path = tmp_path / "fallback" / "scene_threejs_rgb_report.json"
fields = ["x", "y", "z", "f_dc_0", "f_dc_1", "f_dc_2", "opacity", "scale_0", "scale_1", "scale_2", "rot_0", "rot_1", "rot_2", "rot_3"]
source.write_bytes(
(
"ply\n"
"format binary_little_endian 1.0\n"
"element vertex 2\n"
+ "".join(f"property float {name}\n" for name in fields)
+ "end_header\n"
).encode("ascii")
+ struct.pack("<" + "f" * len(fields), 0, 0, 0, 0.5, 0.0, -0.5, 4.0, 0.01, 0.01, 0.01, 1, 0, 0, 0)
+ struct.pack("<" + "f" * len(fields), 1, 2, 3, -0.5, 0.2, 0.5, 4.0, 0.01, 0.01, 0.01, 1, 0, 0, 0)
)
report = _export_3dgs_rgb_fallback(source, target, report_path)
assert report["schema"] == "real2sim.discoverse_exact_3dgs_rgb_fallback.v1"
assert report["vertices"] == 2
assert target.is_file()
assert report_path.is_file()
header = target.read_bytes().split(b"end_header\n", 1)[0].decode("ascii")
assert "property uchar red" in header
def _wait(client: TestClient, task_id: str) -> dict:
current = {}
for _ in range(60):
current = client.get(f"/api/tasks/{task_id}").json()
if current["status"] in {"succeeded", "failed", "canceled", "failed_recovered"}:
break
time.sleep(0.1)
return current
def _write_fixture_video(video_path: Path, frame_dir: Path) -> None:
frame_dir.mkdir(parents=True, exist_ok=True)
for index in range(1, 9):
Image.new("RGB", (96, 64), (index * 20, 80, 120)).save(frame_dir / f"frame_{index:03d}.jpg")
subprocess.run(
[
"ffmpeg",
"-hide_banner",
"-y",
"-framerate",
"8",
"-i",
str(frame_dir / "frame_%03d.jpg"),
"-pix_fmt",
"yuv420p",
str(video_path),
],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)