# MotrixSim Universal Three.js Preview This repository now contains a working universal-preview path: ```text MotrixSim / MJCF scene -> scene manifest -> Three.js WebGL renderer -> replay / polling / websocket state source ``` The first verified version supports primitives, OBJ/STL/GLTF/GLB mesh loading, Z-up MotrixSim coordinate conversion, body-name state binding, desktop/mobile responsive layout, MJCF `` expansion, HTTP asset serving, headless MotrixSim bridge controls, and basic Franka joint/gripper operation. The current exporter/renderer path also covers higher-variance MJCF assets: nested `` inheritance for visible-vs-collision geoms, `assetdir` / `meshdir` / `texturedir` resolution, file textures, builtin checker textures, mesh scale, `fromto`, `xyaxes`, `zaxis`, inline convex meshes, and browser-side mesh caching for repeated assets. ## Run Locally ```bash npm install npm run dev -- --port 5173 ``` Open: ```text http://127.0.0.1:5173/ ``` For the real MotrixSim backend used by the G1 store scene, use the repeatable dev helper instead of hand-running SSH commands: ```bash # Print local tunnel and bridge health. npm run real-backend:status # Check local Node, SSH access, remote MotrixSim paths/imports, running bridge, # and manifest grouping before a handoff or demo. npm run real-backend:doctor # Sync server/headless_motrix_bridge.py to Wuying, restart the active physical # scene paused, ensure the 18090 tunnel exists, then run real backend checks. npm run real-backend:up # Run the product-readiness gate: real backend doctor, backend edit sync, # Scene Builder compose / share-link restore / config export-import restore / # desktop / mobile screenshots, Chinese-default UI, UI-downloaded acceptance # package/report, external simulation labeling, backend-physics labeling, and # object physics-tier labeling, plus pause/resume latency checks. npm run product:acceptance # Full handoff/release gate: typecheck, production build, gzip bundle budget, # and production-preview product acceptance. npm run product:release-check ``` Useful overrides: ```bash WUYING_HOST=39.106.101.95 WUYING_USER=root WUYING_SSH_KEY=/path/to/wuying_ed25519 MOTRIXSIM_REMOTE_SCENE=/tmp/motrixsim-web-composed/active_g1_store_scene.xml MOTRIXSIM_LOCAL_PORT=18090 ``` The first screen is now a Scene Builder for the backend-required product path. The default `G1 + 货架商品后端物理场景` preset uses `http://127.0.0.1:18090/manifest`; it loads one real MotrixSim MJCF runtime and binds every visible shelf/product/robot object to `/state`. When the matching bridge exposes `POST /compose`, Scene Builder object Add/Remove/Clear operations regenerate the combined MJCF on the backend and reload the live manifest. Those objects are real MotrixSim bodies with free joints and collision geoms, not front-end-only Three.js meshes. For physical store scenes, the bridge also publishes product-level `editableGroup` metadata derived from real runtime body poses. Store shelf frames, plates, and pre-stocked products are fixed environment bodies so they keep their collision shape and do not avalanche when the simulator resumes. Backend-compatible added props remain real dynamic bodies when the backend adapter creates them. The current default product scene starts with no added props; uploaded mesh assets are disabled until a backend adapter can turn them into real MJCF bodies. Repositioning a fixed shelf layout should be handled by rebuilding the composed scene, not by keeping the shelf as a free runtime body. The UI, diagnostics panel, JSON acceptance package, and HTML acceptance report now expose object physics tiers: backend dynamic bodies, backend fixed bodies, backend state-only bodies, and visual-only objects. Product acceptance requires `visual-only=0`. To load a remote manifest: ```text http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest ``` ## Verify ```bash npm run build npm run verify:exporter npm run verify:preview ``` For a MotrixSim bridge with only primitive objects: ```bash MIN_OBJECTS=2 MIN_MESHES=0 npm run verify:preview -- \ 'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest' \ test-results/remote-motrix ``` For a larger robot scene with many mesh files, raise the verification timeout: ```bash VERIFY_TIMEOUT_MS=45000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:preview -- \ 'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \ test-results/robot-panda ``` For the backend control path: ```bash VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:control -- \ 'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \ test-results/control-panda ``` For the productized real backend compose and editing path on the G1 store scene: ```bash npm run verify:real-backend -- \ 'http://127.0.0.1:18090/manifest' \ test-results/real-backend-editing ``` This probe calls the live MotrixSim bridge, not a mock. It checks editable entity groups in `/manifest`, calls real backend `/compose` for a controlled legacy dynamic-body edit test, verifies the composed object bodies in `/state`, writes a pose for one composed dynamic body, verifies fixed pre-stocked product bodies reject `set_body_pose`, resumes the simulator, confirms fixed shelf anchors do not drift, checks a lifted dynamic prop drops under backend physics, and measures compact pause/resume control latency. After verification, `npm run real-backend:up` restores the default stocked shelf scene with no added test objects. For handoff or acceptance, prefer the single product gate: ```bash npm run product:acceptance ``` It writes `test-results/product-readiness/product-readiness-report.json` plus Scene Builder compose, share-link restore, config import restore, desktop, and mobile screenshots, plus an auditable `test-results/product-readiness/product-readiness-evidence.json` manifest with artifact hashes and git provenance. The gate also clicks the in-page export buttons, saves the downloaded builder config JSON, JSON acceptance package, and HTML report, parses them, imports the builder config through a real browser file chooser, opens the downloaded HTML report in Chromium, screenshots it, and records their hashes. If `127.0.0.1:5173` is not already running, it starts the Vite dev server for the duration of the check. It does not use mock scene data: backend checks call the live MotrixSim bridge, and browser checks load `http://127.0.0.1:18090/manifest` by default. The browser gate also clicks the Scene Builder `Physical Scene` control in an isolated browser context to prove the product UI triggers backend `/compose`, then checks the runtime diagnostics panel for manifest/state/control endpoints, external simulation mode, object bindings, asset health, and control latency. It also clicks the product copy-link action, reads the browser clipboard, opens the share URL in a clean browser context, and verifies that the restored scene uses the same builder config, backend physics manifest, `/control` endpoint, and object/body binding counts. It also exports the builder config, imports that file in a clean browser context, and verifies the same backend physical scene is restored. The gate also validates the object physics-tier evidence. The product physical scene must include backend fixed environment bodies and backend state-only robot links while keeping `visual-only=0`. The product gate also validates builder physics coverage: the default stocked shelf path has no added props, and any uploaded OBJ asset without a backend adapter is blocked instead of being added as a visual-only object. For release handoff, `npm run product:release-check` first builds `dist/`, checks the JavaScript gzip bundle budget, then forces the browser gate through `vite preview` on an isolated localhost port instead of reusing the development server. Release checks fail on warnings. For demo and handoff details, see [`docs/product-readiness.md`](docs/product-readiness.md). That document records the real backend topology, acceptance criteria, current product gate baseline, evidence paths, screenshot hashes, and known product boundaries. For the Franka joint/gripper control path: ```bash VERIFY_TIMEOUT_MS=70000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:joint -- \ 'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \ test-results/joint-panda ``` For a large textured store scene on the remote MotrixSim bridge: ```bash VERIFY_TIMEOUT_MS=120000 MIN_OBJECTS=500 MIN_MESHES=20 npm run verify:preview -- \ 'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18085%2Fmanifest' \ test-results/store-textured ``` For a Go2 quadruped robot scene: ```bash VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=30 MIN_MESHES=20 npm run verify:preview -- \ 'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest' \ test-results/go2-robot ``` ## Headless MotrixSim Bridge Run this on a Linux server with MotrixSim available: ```bash python server/headless_motrix_bridge.py \ --scene /path/to/scene.xml \ --host 127.0.0.1 \ --port 18083 \ --hz 120 ``` It exposes: - `GET /manifest`: Three.js preview manifest generated from the XML scene. - `GET /state`: live MotrixSim body pose state. - `GET /assets/...`: mesh sidecar assets referenced by the manifest. - `POST /control`: `pause`, `resume`, `reset`, `step`, `home`, `set_joint`, `set_body_pose`, `set_body_poses`, and `gripper`. - `POST /compose`: for supported physical presets, generate a new combined MJCF scene from selected robot/scene/object choices, hot-swap the runtime, and return the refreshed manifest. - `GET /health`: service health. No display server is required for this state-driven preview path. ## Current Boundary This is a real Three.js/WebGL preview path, not the earlier Canvas 2.5D page. It is not yet a perfect renderer for every MotrixSim feature. Unsupported or private features still need explicit adapters or a server-side render/video fallback. Scene Builder product composition must stay connected to a MotrixSim backend state source. Added objects are accepted only when a backend adapter creates real MJCF bodies. The current default physical path is the stocked shelf scene on port `18090`; assets without adapters are visible in the asset library but cannot be inserted into the product physics scene. ## 页面内上传与组装(已支持) - 左侧「场景上传 / 上传场景包」支持一次选择 `.json` manifest 和关联的 mesh / texture / bin 文件。manifest 至少需要包含 `version` 与 `objects`。 - 上传后会自动插入到场景下拉列表首位,并把 manifest 内相对路径(例如 `meshes/robot.obj`、`textures/base.png`)匹配到同批上传的本地文件,改写为浏览器可加载的 `blob:` URL。 - 左侧「资产库 / 上传资产」支持 `.gltf/.glb/.obj/.stl`,也可以同时选择 `.bin/.png/.jpg/.webp/.ktx2/.mtl` 等 sidecar 文件;`.gltf` 内部的 buffer / image 相对引用会同步改写。 - 已有后端适配的资产才能加入产品物理场景,并可继续调整位置(X/Y/Z 输入框)和通过左键拖拽编辑(暂停时可拖拽)。 - 未接入后端适配的上传 mesh 会显示“需后端适配”,添加按钮保持禁用,不会作为“仅视觉”物体混入产品验收场景。 - 上传面板支持拖放文件;「清空自定义」会清理上传资产;「清空场景」会移除当前上传场景并回到默认场景列表。 边界:本地上传 mesh 不会自动变成 MotrixSim 后端碰撞体。需要真实碰撞时,必须先通过后端适配把对象转成 MJCF body / geom;适配完成前产品模式不允许仅视觉加入。 The Franka Panda MJCF robot scene has been verified as a real robot asset preview: 59 manifest objects, 58 loaded OBJ/STL meshes, and zero mesh fallback. Joint-level operation is now wired for this scene through MotrixSim `joint_names`, `link_names`, and link pose streaming. Broader task-level robot actions still need scene-specific policy/action adapters. Additional remote scenes verified: - Store shelf scene: 569 objects, 568 loaded meshes, 17 textured materials, zero fallback, desktop/mobile nonblank. - Go2 quadruped scene: 34 objects, 33 loaded meshes, 12 joints exposed, zero fallback, desktop/mobile nonblank. - G1 + stocked shelf backend physical scene: served from bridge port 18090 with shared backend collision bodies for the robot, shelves, and fixed stock products. Store shelf/product groups are exposed for inspection, and product acceptance requires `631/631` body bindings and `visual-only=0`.