MotrixSim Universal Three.js Preview
This repository now contains a working universal-preview path:
MotrixSim / MJCF scene
-> scene manifest
-> Three.js WebGL renderer
-> replay / polling / websocket state source
The first verified version supports primitives, OBJ/STL/GLTF/GLB mesh loading,
Z-up MotrixSim coordinate conversion, body-name state binding, desktop/mobile
responsive layout, MJCF <include> expansion, HTTP asset serving, headless
MotrixSim bridge controls, and basic Franka joint/gripper operation.
The current exporter/renderer path also covers higher-variance MJCF assets:
nested <default> inheritance for visible-vs-collision geoms, assetdir /
meshdir / texturedir resolution, file textures, builtin checker textures,
mesh scale, fromto, xyaxes, zaxis, inline convex meshes, and browser-side
mesh caching for repeated assets.
Run Locally
npm install
npm run dev -- --port 5173
Open:
http://127.0.0.1:5173/
For the real MotrixSim backend used by the G1 store scene, use the repeatable dev helper instead of hand-running SSH commands:
# Print local tunnel and bridge health.
npm run real-backend:status
# Check local Node, SSH access, remote MotrixSim paths/imports, running bridge,
# and manifest grouping before a handoff or demo.
npm run real-backend:doctor
# Sync server/headless_motrix_bridge.py to Wuying, restart the active physical
# scene paused, ensure the 18090 tunnel exists, then run real backend checks.
npm run real-backend:up
# Run the product-readiness gate: real backend doctor, backend edit sync,
# desktop/mobile browser screenshots, Chinese-default UI, external simulation
# labeling, backend-physics labeling, and pause/resume latency checks.
npm run product:acceptance
# Full handoff/release gate: typecheck, production build, and product acceptance.
npm run product:release-check
Useful overrides:
WUYING_HOST=39.106.101.95
WUYING_USER=root
WUYING_SSH_KEY=/path/to/wuying_ed25519
MOTRIXSIM_REMOTE_SCENE=/tmp/motrixsim-web-composed/active_g1_store_scene.xml
MOTRIXSIM_LOCAL_PORT=18090
The first screen is now a Scene Builder. It can compose a base scene, one live
robot manifest, and front-end primitive objects into a single Three.js preview.
Use Build Preview for visual composition, or Physical Scene to load the
native Franka + cube MotrixSim scene where collision is computed by the backend.
The G1 + Store + blocks physics preset uses http://127.0.0.1:18090/manifest
and is a backend-composed MJCF scene, so G1, store shelves/products, and demo
objects share one MotrixSim physics world.
When the matching bridge exposes POST /compose, Scene Builder object
Add/Remove/Clear operations regenerate the combined MJCF on the backend and
reload the live manifest. Those objects are real MotrixSim bodies with
free joints and collision geoms, not front-end-only Three.js meshes.
For physical store scenes, the bridge also publishes product-level
editableGroup metadata derived from real runtime body poses. While paused,
dragging a shelf anchor writes a batched set_body_poses command back to
MotrixSim so the shelf and its products move together; dragging a product member
directly still edits that product's own real body pose.
To load a remote manifest:
http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest
Verify
npm run build
npm run verify:exporter
npm run verify:preview
For a MotrixSim bridge with only primitive objects:
MIN_OBJECTS=2 MIN_MESHES=0 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest' \
test-results/remote-motrix
For a larger robot scene with many mesh files, raise the verification timeout:
VERIFY_TIMEOUT_MS=45000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
test-results/robot-panda
For the backend control path:
VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:control -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
test-results/control-panda
For the productized real backend editing path on the G1 store scene:
npm run verify:real-backend -- \
'http://127.0.0.1:18090/manifest' \
test-results/real-backend-editing
This probe calls the live MotrixSim bridge, not a mock. It checks editable
entity groups in /manifest, verifies paused /state, writes a no-op pose for
one member, writes the full shelf group through set_body_poses, and measures
compact pause/resume control latency.
For handoff or acceptance, prefer the single product gate:
npm run product:acceptance
It writes test-results/product-readiness/product-readiness-report.json plus
desktop/mobile screenshots. If 127.0.0.1:5173 is not already running, it
starts the Vite dev server for the duration of the check. It does not use mock
scene data: backend checks call the live MotrixSim bridge, and browser checks
load http://127.0.0.1:18090/manifest by default.
For demo and handoff details, see
docs/product-readiness.md. That document records
the real backend topology, acceptance criteria, current 20/20 baseline, evidence
paths, and known product boundaries.
For the Franka joint/gripper control path:
VERIFY_TIMEOUT_MS=70000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:joint -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
test-results/joint-panda
For a large textured store scene on the remote MotrixSim bridge:
VERIFY_TIMEOUT_MS=120000 MIN_OBJECTS=500 MIN_MESHES=20 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18085%2Fmanifest' \
test-results/store-textured
For a Go2 quadruped robot scene:
VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=30 MIN_MESHES=20 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest' \
test-results/go2-robot
Headless MotrixSim Bridge
Run this on a Linux server with MotrixSim available:
python server/headless_motrix_bridge.py \
--scene /path/to/scene.xml \
--host 127.0.0.1 \
--port 18083 \
--hz 120
It exposes:
GET /manifest: Three.js preview manifest generated from the XML scene.GET /state: live MotrixSim body pose state.GET /assets/...: mesh sidecar assets referenced by the manifest.POST /control:pause,resume,reset,step,home,set_joint,set_body_pose,set_body_poses, andgripper.POST /compose: for supported physical presets, generate a new combined MJCF scene from selected robot/scene/object choices, hot-swap the runtime, and return the refreshed manifest.GET /health: service health.
No display server is required for this state-driven preview path.
Current Boundary
This is a real Three.js/WebGL preview path, not the earlier Canvas 2.5D page. It is not yet a perfect renderer for every MotrixSim feature. Unsupported or private features still need explicit adapters or a server-side render/video fallback.
Scene Builder visual composition does not create a new MotrixSim physics world.
Added front-end objects are preview-only unless they are sent through a physical
compose endpoint. The current physical compose path supports the
G1 + Store + blocks physics preset and the built-in primitive object set.
页面内上传与组装(已支持)
- 左侧「场景上传 / 上传场景包」支持一次选择
.jsonmanifest 和关联的 mesh / texture / bin 文件。manifest 至少需要包含version与objects。 - 上传后会自动插入到场景下拉列表首位,并把 manifest 内相对路径(例如
meshes/robot.obj、textures/base.png)匹配到同批上传的本地文件,改写为浏览器可加载的blob:URL。 - 左侧「资产库 / 上传资产」支持
.gltf/.glb/.obj/.stl,也可以同时选择.bin/.png/.jpg/.webp/.ktx2/.mtl等 sidecar 文件;.gltf内部的 buffer / image 相对引用会同步改写。 - 点击资产卡片会把该对象加入当前场景列表,可继续调整位置(X/Y/Z 输入框)和通过左键拖拽编辑(暂停时可拖拽)。
- 上传面板支持拖放文件;「清空自定义」会清理上传资产;「清空场景」会移除当前上传场景并回到默认场景列表。
边界:本地上传 mesh 默认是 Three.js 可视资产。它可以预览和摆放,但不会自动变成 MotrixSim 后端碰撞体。需要真实碰撞时,仍要通过后端 /compose 把对象转成 MJCF body / geom;当前后端物理组合只覆盖内置 primitive 对象。
The Franka Panda MJCF robot scene has been verified as a real robot asset
preview: 59 manifest objects, 58 loaded OBJ/STL meshes, and zero mesh fallback.
Joint-level operation is now wired for this scene through MotrixSim
joint_names, link_names, and link pose streaming. Broader task-level robot
actions still need scene-specific policy/action adapters.
Additional remote scenes verified:
- Store shelf scene: 569 objects, 568 loaded meshes, 17 textured materials, zero fallback, desktop/mobile nonblank.
- Go2 quadruped scene: 34 objects, 33 loaded meshes, 12 joints exposed, zero fallback, desktop/mobile nonblank.
- G1 + Store + demo blocks physical scene: served from bridge port 18090 with shared backend collision bodies for the robot, shelves, products, and demo primitives. Store shelf/product groups are exposed as editable entity groups so the web UI can synchronize paused layout edits back into the MotrixSim runtime instead of only moving Three.js visuals.