Initial MotrixSim Web preview productization

This commit is contained in:
eust-w 2026-06-24 06:23:35 +08:00
commit e2ef0b2aa1
157 changed files with 39163 additions and 0 deletions

10
.gitignore vendored Normal file
View File

@ -0,0 +1,10 @@
node_modules/
dist/
.vite/
*.log
.DS_Store
__pycache__/
*.py[cod]
/tmp.json
test-results/
!test-results/.gitkeep

172
README.md Normal file
View File

@ -0,0 +1,172 @@
# MotrixSim Universal Three.js Preview
This repository now contains a working universal-preview path:
```text
MotrixSim / MJCF scene
-> scene manifest
-> Three.js WebGL renderer
-> replay / polling / websocket state source
```
The first verified version supports primitives, OBJ/STL/GLTF/GLB mesh loading,
Z-up MotrixSim coordinate conversion, body-name state binding, desktop/mobile
responsive layout, MJCF `<include>` expansion, HTTP asset serving, headless
MotrixSim bridge controls, and basic Franka joint/gripper operation.
The current exporter/renderer path also covers higher-variance MJCF assets:
nested `<default>` inheritance for visible-vs-collision geoms, `assetdir` /
`meshdir` / `texturedir` resolution, file textures, builtin checker textures,
mesh scale, `fromto`, `xyaxes`, `zaxis`, inline convex meshes, and browser-side
mesh caching for repeated assets.
## Run Locally
```bash
npm install
npm run dev -- --port 5173
```
Open:
```text
http://127.0.0.1:5173/
```
The first screen is now a Scene Builder. It can compose a base scene, one live
robot manifest, and front-end primitive objects into a single Three.js preview.
Use `Build Preview` for visual composition, or `Physical Scene` to load the
native Franka + cube MotrixSim scene where collision is computed by the backend.
The `G1 + Store + blocks physics` preset uses `http://127.0.0.1:18090/manifest`
and is a backend-composed MJCF scene, so G1, store shelves/products, and demo
objects share one MotrixSim physics world.
When the matching bridge exposes `POST /compose`, Scene Builder object
Add/Remove/Clear operations regenerate the combined MJCF on the backend and
reload the live manifest. Those objects are real MotrixSim bodies with
free joints and collision geoms, not front-end-only Three.js meshes.
To load a remote manifest:
```text
http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest
```
## Verify
```bash
npm run build
npm run verify:exporter
npm run verify:preview
```
For a MotrixSim bridge with only primitive objects:
```bash
MIN_OBJECTS=2 MIN_MESHES=0 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest' \
test-results/remote-motrix
```
For a larger robot scene with many mesh files, raise the verification timeout:
```bash
VERIFY_TIMEOUT_MS=45000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
test-results/robot-panda
```
For the backend control path:
```bash
VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:control -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
test-results/control-panda
```
For the Franka joint/gripper control path:
```bash
VERIFY_TIMEOUT_MS=70000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:joint -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
test-results/joint-panda
```
For a large textured store scene on the remote MotrixSim bridge:
```bash
VERIFY_TIMEOUT_MS=120000 MIN_OBJECTS=500 MIN_MESHES=20 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18085%2Fmanifest' \
test-results/store-textured
```
For a Go2 quadruped robot scene:
```bash
VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=30 MIN_MESHES=20 npm run verify:preview -- \
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest' \
test-results/go2-robot
```
## Headless MotrixSim Bridge
Run this on a Linux server with MotrixSim available:
```bash
python server/headless_motrix_bridge.py \
--scene /path/to/scene.xml \
--host 127.0.0.1 \
--port 18083 \
--hz 120
```
It exposes:
- `GET /manifest`: Three.js preview manifest generated from the XML scene.
- `GET /state`: live MotrixSim body pose state.
- `GET /assets/...`: mesh sidecar assets referenced by the manifest.
- `POST /control`: `pause`, `resume`, `reset`, `step`, `home`,
`set_joint`, and `gripper`.
- `POST /compose`: for supported physical presets, generate a new combined
MJCF scene from selected robot/scene/object choices, hot-swap the runtime,
and return the refreshed manifest.
- `GET /health`: service health.
No display server is required for this state-driven preview path.
## Current Boundary
This is a real Three.js/WebGL preview path, not the earlier Canvas 2.5D page.
It is not yet a perfect renderer for every MotrixSim feature. Unsupported or
private features still need explicit adapters or a server-side render/video
fallback.
Scene Builder visual composition does not create a new MotrixSim physics world.
Added front-end objects are preview-only unless they are sent through a physical
compose endpoint. The current physical compose path supports the
`G1 + Store + blocks physics` preset and the built-in primitive object set.
## 页面内上传与组装(已支持)
- 左侧「场景上传 / 上传场景包」支持一次选择 `.json` manifest 和关联的 mesh / texture / bin 文件。manifest 至少需要包含 `version` 与 `objects`。
- 上传后会自动插入到场景下拉列表首位,并把 manifest 内相对路径(例如 `meshes/robot.obj`、`textures/base.png`)匹配到同批上传的本地文件,改写为浏览器可加载的 `blob:` URL。
- 左侧「资产库 / 上传资产」支持 `.gltf/.glb/.obj/.stl`,也可以同时选择 `.bin/.png/.jpg/.webp/.ktx2/.mtl` 等 sidecar 文件;`.gltf` 内部的 buffer / image 相对引用会同步改写。
- 点击资产卡片会把该对象加入当前场景列表,可继续调整位置(X/Y/Z 输入框)和通过左键拖拽编辑(暂停时可拖拽)。
- 上传面板支持拖放文件;「清空自定义」会清理上传资产;「清空场景」会移除当前上传场景并回到默认场景列表。
边界:本地上传 mesh 默认是 Three.js 可视资产。它可以预览和摆放,但不会自动变成 MotrixSim 后端碰撞体。需要真实碰撞时,仍要通过后端 `/compose` 把对象转成 MJCF body / geom;当前后端物理组合只覆盖内置 primitive 对象。
The Franka Panda MJCF robot scene has been verified as a real robot asset
preview: 59 manifest objects, 58 loaded OBJ/STL meshes, and zero mesh fallback.
Joint-level operation is now wired for this scene through MotrixSim
`joint_names`, `link_names`, and link pose streaming. Broader task-level robot
actions still need scene-specific policy/action adapters.
Additional remote scenes verified:
- Store shelf scene: 569 objects, 568 loaded meshes, 17 textured materials, zero
fallback, desktop/mobile nonblank.
- Go2 quadruped scene: 34 objects, 33 loaded meshes, 12 joints exposed, zero
fallback, desktop/mobile nonblank.
- G1 + Store + demo blocks physical scene: served from bridge port 18090 with
shared backend collision bodies for the robot, shelves, products, and demo
primitives.

117
docs/manifest-spec.md Normal file
View File

@ -0,0 +1,117 @@
# MotrixSim Universal Three.js Manifest
This project uses a small scene manifest as the boundary between MotrixSim/MJCF
assets and the browser renderer.
The manifest does not try to duplicate every engine feature. It carries enough
data for a browser to load visible assets, create stable Three.js nodes, and bind
runtime state by body name.
## Minimal Shape
```json
{
"version": "0.1",
"name": "demo",
"coordinateSystem": { "up": "Z", "handedness": "right" },
"materials": [
{ "id": "ball", "color": "#4f8cff" },
{ "id": "checker_floor", "textureKind": "checker", "textureColors": ["#334155", "#e2e8f0"], "textureRepeat": [4, 4] },
{ "id": "label", "color": "#ffffff", "textureUrl": "/assets/label.png", "textureRepeat": [1, 1] }
],
"objects": [
{
"id": "ball_visual",
"body": "_unnamed_link_0",
"geometry": { "kind": "primitive", "primitive": "sphere", "radius": 0.5 },
"material": "ball"
},
{
"id": "inline_convex",
"geometry": { "kind": "convexMesh", "vertices": [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1] },
"material": "checker_floor"
}
],
"state": { "mode": "poll", "url": "/state", "controlUrl": "/control", "pollIntervalMs": 50 },
"controls": {
"joints": [
{ "name": "joint1", "index": 0, "range": [-2.8973, 2.8973] }
],
"actuators": []
}
}
```
## Supported Visuals
- Primitives: `box`, `sphere`, `capsule`, `cylinder`, `plane`, `axes`.
- Meshes: `gltf`, `glb`, `obj`, `stl`, plus small inline convex meshes.
- Materials: solid color, transparent color, file texture, and generated
checker texture with repeat.
- State binding: `object.body` maps a visual node to a runtime body name.
- Runtime body names may be MotrixSim body names or link names. The headless
bridge publishes link poses first, then fills non-duplicate body poses.
- MJCF export: the current exporter expands `<include>` files, resolves mesh
names from either `name` or file stem, resolves `assetdir` / `meshdir` /
`texturedir`, applies nested `<default>` geom inheritance, skips
collision-only geoms, handles `fromto`, `xyaxes`, `zaxis`, mesh `scale`, file
textures, builtin checker textures, and emits an internal asset map for the
headless bridge to serve `/assets/...`.
Repeated mesh URLs are cached by the browser renderer and cloned into the scene,
so large scenes with hundreds of repeated OBJ assets do not refetch or reparse
the same asset for every instance.
## Runtime State
For `state.mode = "poll"`, the browser repeatedly fetches `state.url` and
applies the returned pose data to body-bound objects. A typical response is:
```json
{
"step": 12,
"time": 0.012,
"paused": true,
"running": false,
"link_names": ["link0", "link1", "hand"],
"joint_names": ["joint1", "joint2"],
"bodies": {
"hand": {
"pos": [0.4, 0.0, 0.77],
"quat": [-0.88, -0.36, 0.27, 0.11]
}
},
"joints": {
"joint2": { "pos": 0.6, "vel": 0.0, "range": [-1.7628, 1.7628], "index": 1 }
},
"actuators": {}
}
```
## Control Endpoint
When `state.controlUrl` is present, the UI can POST JSON commands to the
bridge. The current bridge supports:
- `{"action": "pause"}` / `{"action": "resume"}` / `{"action": "toggle"}`
- `{"action": "step"}` for one paused physics step
- `{"action": "reset"}` for engine reset
- `{"action": "home"}` for zeroing exposed joints
- `{"action": "set_joint", "joint": "joint2", "value": 0.6}`
- `{"action": "set_joints", "values": {"joint1": 0, "joint2": 0.6}}`
- `{"action": "gripper", "value": 0.04}` for known finger joints
The bridge returns the same frame shape as `/state`, so the frontend can update
the renderer and the control panel from one response.
## Coordinate Policy
MotrixSim/MJCF scenes are assumed to be Z-up. Three.js is Y-up. The renderer
converts positions, rotations, primitive dimensions, and loaded mesh local
coordinate bases when `coordinateSystem.up` is `Z`.
## Security Boundary
If the browser renders the real mesh, that mesh is downloaded to the browser.
Private assets that must not leave the server should use a server-side video or
image stream instead of Three.js mesh delivery.

13
index.html Normal file
View File

@ -0,0 +1,13 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>MotrixSim Universal Three.js Preview</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.ts"></script>
</body>
</html>

1171
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

28
package.json Normal file
View File

@ -0,0 +1,28 @@
{
"name": "motrixsim-web",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"dev": "vite --host 127.0.0.1",
"build": "tsc && vite build",
"preview": "vite preview --host 127.0.0.1",
"typecheck": "tsc --noEmit",
"verify:preview": "node scripts/verify-universal-preview.mjs",
"verify:control": "node scripts/verify-control-preview.mjs",
"verify:joint": "node scripts/verify-joint-control.mjs",
"verify:motrix-input": "node scripts/verify-motrix-input.mjs",
"verify:go2-policy": "node scripts/verify-go2-policy-motion.mjs",
"verify:exporter": "python3 scripts/verify_exporter_features.py",
"acceptance:probe": "node scripts/acceptance-probe.mjs"
},
"dependencies": {
"three": "^0.181.2",
"vite": "^7.3.0"
},
"devDependencies": {
"@types/three": "^0.184.1",
"playwright": "^1.61.0",
"typescript": "^5.9.3"
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,12 @@
o unknown_mesh_marker
v 0.000 0.000 0.700
v -0.600 -0.600 0.000
v 0.600 -0.600 0.000
v 0.600 0.600 0.000
v -0.600 0.600 0.000
f 1 2 3
f 1 3 4
f 1 4 5
f 1 5 2
f 2 5 4 3

View File

@ -0,0 +1,18 @@
{
"frames": [
{ "step": 0, "time": 0.00, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 4.98], "quat": [0, 0, 0, 1] } } },
{ "step": 20, "time": 0.16, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 4.22], "quat": [0, 0, 0.02, 0.9998] } } },
{ "step": 40, "time": 0.32, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 3.02], "quat": [0, 0, 0.04, 0.9992] } } },
{ "step": 60, "time": 0.48, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 1.42], "quat": [0, 0, 0.06, 0.9982] } } },
{ "step": 80, "time": 0.64, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.52], "quat": [0, 0, 0.08, 0.9968] } } },
{ "step": 100, "time": 0.80, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.70], "quat": [0, 0, 0.10, 0.9950] } } },
{ "step": 120, "time": 0.96, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.50], "quat": [0, 0, 0.12, 0.9928] } } },
{ "step": 140, "time": 1.12, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.50], "quat": [0, 0, 0.14, 0.9902] } } },
{ "step": 160, "time": 1.28, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 4.95], "quat": [0, 0, 0.16, 0.9871] } } },
{ "step": 180, "time": 1.44, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 3.78], "quat": [0, 0, 0.18, 0.9837] } } },
{ "step": 200, "time": 1.60, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 2.18], "quat": [0, 0, 0.20, 0.9798] } } },
{ "step": 220, "time": 1.76, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.66], "quat": [0, 0, 0.22, 0.9755] } } },
{ "step": 240, "time": 1.92, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.50], "quat": [0, 0, 0.24, 0.9708] } } }
]
}

View File

@ -0,0 +1,73 @@
{
"version": "0.1",
"name": "Universal Three.js Preview Demo",
"description": "Motrix-style Z-up scene manifest with primitives, mesh loading, and replayed body state.",
"units": "m",
"coordinateSystem": { "up": "Z", "handedness": "right" },
"materials": [
{ "id": "floor", "color": "#263449", "roughness": 0.92 },
{ "id": "ball", "color": "#4f8cff", "roughness": 0.36, "metalness": 0.05 },
{ "id": "capsule", "color": "#f59e0b", "roughness": 0.48 },
{ "id": "mesh", "color": "#a7f3d0", "roughness": 0.62 },
{ "id": "fallback", "color": "#fb7185", "wireframe": true }
],
"objects": [
{
"id": "floor",
"name": "factory_floor",
"geometry": { "kind": "primitive", "primitive": "box", "size": [9, 9, 0.08] },
"pose": { "position": [0, 0, -0.04] },
"material": "floor",
"receiveShadow": true
},
{
"id": "falling_ball",
"name": "MotrixSim falling ball body",
"body": "_unnamed_link_0",
"geometry": { "kind": "primitive", "primitive": "sphere", "radius": 0.5, "segments": 48 },
"material": "ball",
"castShadow": true
},
{
"id": "capsule_demo",
"name": "capsule primitive support",
"geometry": { "kind": "primitive", "primitive": "capsule", "radius": 0.22, "length": 1.4, "segments": 24 },
"pose": { "position": [-2.8, 1.6, 0.75], "quaternion": [0, 0, 0, 1] },
"material": "capsule",
"castShadow": true
},
{
"id": "obj_mesh_demo",
"name": "OBJ mesh asset support",
"geometry": { "kind": "mesh", "url": "/samples/assets/pyramid.obj", "format": "obj" },
"pose": { "position": [2.6, -1.8, 0.02], "scale": [1.1, 1.1, 1.1] },
"material": "mesh",
"castShadow": true
},
{
"id": "axes",
"name": "scene axes",
"geometry": { "kind": "primitive", "primitive": "axes", "size": [1.4, 1.4, 1.4] },
"pose": { "position": [-4.1, -4.1, 0.03] }
}
],
"lights": [
{ "type": "hemisphere", "color": "#dbeafe", "intensity": 0.65 },
{ "type": "directional", "color": "#ffffff", "intensity": 2.4, "position": [4, -6, 7] },
{ "type": "ambient", "color": "#9fb4d1", "intensity": 0.22 }
],
"camera": {
"position": [5.5, -7.2, 4.6],
"target": [0, 0, 1.2],
"fov": 46,
"near": 0.05,
"far": 200
},
"state": {
"mode": "replay",
"url": "/samples/universal-demo-state.json",
"fps": 12,
"loop": true
}
}

View File

@ -0,0 +1,49 @@
# MotrixSim Web Preview Research Test
This folder contains the 2026-06-22 investigation for MotrixSim Web preview architecture.
## Reports
- `research-report.md`: test evidence and implementation recommendation.
- `comparison-report.md`: WASM vs backend streaming comparison.
- `motrixsim-operation-consistency-report.md`: MotrixSim-style keyboard, mouse camera, actuator, pause/reset operation verification.
- `evidence/`: latest Go2 and Franka screenshots plus JSON evidence for MotrixSim operation consistency.
- `figures/real-web-preview/motrix-live-web-preview.jpg`: real browser screenshot of the MotrixSim Web preview.
- `figures/real-web-preview/motrix-live-web-preview-recording.mp4`: simple screen recording generated from browser frames.
- `figures/real-web-preview/motrix-live-web-preview-recording.gif`: lightweight animated preview for inline reports.
- `figures/real-web-preview/universal-three-preview-sample-desktop.jpg`: real Three.js WebGL sample preview, desktop viewport.
- `figures/real-web-preview/universal-three-preview-sample-mobile.jpg`: real Three.js WebGL sample preview, mobile viewport.
- `figures/real-web-preview/universal-three-preview-remote-motrix-desktop.jpg`: real MotrixSim bridge scene rendered by Three.js, desktop viewport.
- `figures/real-web-preview/universal-three-preview-remote-motrix-mobile.jpg`: real MotrixSim bridge scene rendered by Three.js, mobile viewport.
- `figures/real-web-preview/universal-three-preview-robot-panda-desktop.jpg`: real Franka Panda robot MJCF scene rendered by Three.js, desktop viewport.
- `figures/real-web-preview/universal-three-preview-robot-panda-mobile.jpg`: real Franka Panda robot MJCF scene rendered by Three.js, mobile viewport.
- `figures/real-web-preview/universal-three-preview-control-desktop.jpg`: real backend pause/step/reset/resume control verification.
- `figures/real-web-preview/universal-three-preview-joint-control-desktop.jpg`: real Franka joint and gripper control verification.
- `figures/real-web-preview/universal-three-preview-store-textured-desktop.jpg`: real Store scene with OBJ/PNG textured assets.
- `figures/real-web-preview/universal-three-preview-store-textured-mobile.jpg`: Store scene mobile viewport verification.
- `figures/real-web-preview/universal-three-preview-go2-robot-desktop.jpg`: real Go2 quadruped robot scene.
- `figures/real-web-preview/universal-three-preview-go2-robot-mobile.jpg`: Go2 mobile viewport verification.
- `artifacts/motrix_live_web_preview_browser_evidence_2026-06-22.json`: DOM/state evidence captured from the browser page.
- `artifacts/motrix_live_web_preview_recording_evidence_2026-06-22.json`: frame-level state evidence for the recording.
- `artifacts/universal_three_preview_sample_evidence_2026-06-22.json`: Playwright evidence for the bundled universal Three.js sample.
- `artifacts/universal_three_preview_remote_motrix_evidence_2026-06-22.json`: Playwright evidence for the remote MotrixSim bridge scene.
- `artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json`: Playwright evidence for the real Franka Panda robot scene.
- `artifacts/universal_three_preview_control_evidence_2026-06-22.json`: Playwright evidence for pause/step/reset/resume control.
- `artifacts/universal_three_preview_joint_control_evidence_2026-06-22.json`: Playwright evidence for joint2 and gripper control.
- `artifacts/universal_three_preview_store_textured_evidence_2026-06-22.json`: Playwright evidence for Store textured scene.
- `artifacts/universal_three_preview_go2_robot_evidence_2026-06-22.json`: Playwright evidence for Go2 quadruped scene.
- `figures/test-screenshots/`: PNG screenshots generated from the real test artifacts.
- Feishu Wiki: https://horizonrobotics.feishu.cn/wiki/UKxEwcYjwiuSapkDeaXcqRlUnVd
## Reproduce
```bash
bash scripts/wuying_probe.sh
scp scripts/motrix_state_stream_probe.py root@39.106.101.95:/tmp/motrix_state_stream_probe.py
scp scripts/motrix_live_web_preview.py root@39.106.101.95:/tmp/motrix_live_web_preview.py
ssh root@39.106.101.95 '/opt/gs_playground_real2sim_48q/.venv/bin/python /tmp/motrix_live_web_preview.py --host 127.0.0.1 --port 18081'
ssh -L 18081:127.0.0.1:18081 root@39.106.101.95
cd web-wasm-smoke && npm install && node smoke.mjs
```
The Wuying SSH key is intentionally referenced from the private document repo and is not copied into this project.

View File

@ -0,0 +1,52 @@
{
"ok": true,
"repo": "/Users/d-robotics/workSpace/motrix-lerobot",
"files": {
"readme": "/Users/d-robotics/workSpace/motrix-lerobot/Readme.md",
"env": "/Users/d-robotics/workSpace/motrix-lerobot/motrix_lerobot/env.py",
"runner": "/Users/d-robotics/workSpace/motrix-lerobot/scripts/runner.py",
"so101_scene": "/Users/d-robotics/workSpace/motrix-lerobot/assets/SO101/scene_with_ball.xml",
"g1_scene": "/Users/d-robotics/workSpace/motrix-lerobot/assets/g1/scene_real.xml"
},
"findings": [
{
"check": "imports_motrixsim",
"status": "pass"
},
{
"check": "loads_so101_scene",
"status": "pass"
},
{
"check": "has_render_app",
"status": "pass"
},
{
"check": "captures_two_cameras",
"status": "pass"
},
{
"check": "extracts_joint_state",
"status": "pass"
},
{
"check": "steps_model",
"status": "pass"
}
],
"checks": {
"imports_motrixsim": true,
"loads_so101_scene": true,
"has_render_app": true,
"captures_two_cameras": true,
"extracts_joint_state": true,
"steps_model": true
},
"exists": {
"readme": true,
"env": true,
"runner": true,
"so101_scene": true,
"g1_scene": true
}
}

View File

@ -0,0 +1,109 @@
{
"captured_at": "2026-06-22T04:43:10.882Z",
"url": "http://127.0.0.1:18081/",
"remote_host": "39.106.101.95",
"screenshot": "/Users/d-robotics/workSpace/MotrixSimWeb/research-tests/2026-06-22-motrixsim-web-preview/figures/real-web-preview/motrix-live-web-preview.jpg",
"selected_state": {
"bodyName": "_unnamed_link_0",
"canvas": {
"clientHeight": 720,
"clientWidth": 920,
"height": 1440,
"width": 1840
},
"connection": "connected",
"heading": "Real Runtime",
"posText": "0.0000, 0.0000, 4.6429",
"state": {
"bodies": {
"_unnamed_link_0": {
"pos": [
0,
0,
4.642916202545166
],
"quat": [
0,
0,
0,
1
]
}
},
"body_names": [
"_unnamed_link_0"
],
"host": "1nh2jhpso5kckmx",
"ok": true,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
"server_time": 1782103390.8206172,
"step": 3584,
"step_ms": {
"max": 0.17769599799066782,
"mean": 0.06107135443814252,
"p95": 0.09495299309492111
}
},
"stepText": "3584",
"title": "MotrixSim Live Web Preview",
"z": 4.642916202545166
},
"page_state_samples": [
{
"step": 3584,
"z": 4.642916202545166,
"posText": "0.0000, 0.0000, 4.6429",
"connection": "connected"
}
],
"verification": {
"source": "browser DOM raw live state rendered from /state endpoint",
"requirement": "real MotrixSim Web preview screenshot, not terminal output or smoke placeholder",
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml"
},
"post_screenshot_dom_state": {
"captured_at": "2026-06-22T04:43:26.711Z",
"parsed_state": {
"bodies": {
"_unnamed_link_0": {
"pos": [
0,
0,
4.9882283210754395
],
"quat": [
0,
0,
0,
1
]
}
},
"body_names": [
"_unnamed_link_0"
],
"host": "1nh2jhpso5kckmx",
"ok": true,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
"server_time": 1782103406.7485473,
"step": 5477,
"step_ms": {
"max": 0.2323179505765438,
"mean": 0.06990823318024558,
"p95": 0.12352498015388846
}
},
"visible": {
"body": "_unnamed_link_0",
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"mean_step_ms": "0.06991",
"position_xyz": "0.0000, 0.0000, 4.9882",
"runtime_heading": "Real Runtime",
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
"server_step": "5477",
"title": "MotrixSim Web Preview"
},
"z": 4.9882283210754395
}
}

View File

@ -0,0 +1,589 @@
{
"captured_at": "2026-06-22T07:30:38.135Z",
"url": "http://127.0.0.1:18081/",
"remote_host": "39.106.101.95",
"frame_count": 48,
"frame_interval_ms": 100,
"frame_dir": "/Users/d-robotics/workSpace/MotrixSimWeb/research-tests/2026-06-22-motrixsim-web-preview/figures/real-web-preview/recording-frames",
"samples": [
{
"frame": 0,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113428.8410916,
"step": 1196860,
"title": "MotrixSim Web Preview",
"visibleStep": "1196860",
"z": 0.4996328353881836
},
{
"frame": 1,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113429.0350401,
"step": 1196883,
"title": "MotrixSim Web Preview",
"visibleStep": "1196883",
"z": 0.4996328353881836
},
{
"frame": 2,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113429.3060741,
"step": 1196915,
"title": "MotrixSim Web Preview",
"visibleStep": "1196915",
"z": 0.4996328353881836
},
{
"frame": 3,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.8234",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113429.5065846,
"step": 1196939,
"title": "MotrixSim Web Preview",
"visibleStep": "1196939",
"z": 4.82342004776001
},
{
"frame": 4,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 3.2931",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113429.6738975,
"step": 1196959,
"title": "MotrixSim Web Preview",
"visibleStep": "1196959",
"z": 3.293060541152954
},
{
"frame": 5,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.3607",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113429.846431,
"step": 1196979,
"title": "MotrixSim Web Preview",
"visibleStep": "1196979",
"z": 0.36073562502861023
},
{
"frame": 6,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4773",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113430.0658035,
"step": 1197005,
"title": "MotrixSim Web Preview",
"visibleStep": "1197005",
"z": 0.47731032967567444
},
{
"frame": 7,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113430.2270908,
"step": 1197025,
"title": "MotrixSim Web Preview",
"visibleStep": "1197025",
"z": 0.49963366985321045
},
{
"frame": 8,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113430.4802213,
"step": 1197055,
"title": "MotrixSim Web Preview",
"visibleStep": "1197055",
"z": 0.4996328353881836
},
{
"frame": 9,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113430.614984,
"step": 1197071,
"title": "MotrixSim Web Preview",
"visibleStep": "1197071",
"z": 0.4996328353881836
},
{
"frame": 10,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113430.7318132,
"step": 1197085,
"title": "MotrixSim Web Preview",
"visibleStep": "1197085",
"z": 0.4996328353881836
},
{
"frame": 11,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113430.9516282,
"step": 1197111,
"title": "MotrixSim Web Preview",
"visibleStep": "1197111",
"z": 0.4996328353881836
},
{
"frame": 12,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113431.236581,
"step": 1197145,
"title": "MotrixSim Web Preview",
"visibleStep": "1197145",
"z": 0.4996328353881836
},
{
"frame": 13,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113431.428369,
"step": 1197167,
"title": "MotrixSim Web Preview",
"visibleStep": "1197167",
"z": 0.4996328353881836
},
{
"frame": 14,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.3290",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113431.5858905,
"step": 1197186,
"title": "MotrixSim Web Preview",
"visibleStep": "1197186",
"z": 4.328996181488037
},
{
"frame": 15,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.5737",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113431.827208,
"step": 1197215,
"title": "MotrixSim Web Preview",
"visibleStep": "1197215",
"z": 0.5737295150756836
},
{
"frame": 16,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.6533",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113431.9960675,
"step": 1197235,
"title": "MotrixSim Web Preview",
"visibleStep": "1197235",
"z": 0.6532669067382812
},
{
"frame": 17,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113432.225468,
"step": 1197262,
"title": "MotrixSim Web Preview",
"visibleStep": "1197262",
"z": 0.49963316321372986
},
{
"frame": 18,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113432.3864737,
"step": 1197281,
"title": "MotrixSim Web Preview",
"visibleStep": "1197281",
"z": 0.4996328353881836
},
{
"frame": 19,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113432.6327212,
"step": 1197311,
"title": "MotrixSim Web Preview",
"visibleStep": "1197311",
"z": 0.4996328353881836
},
{
"frame": 20,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113432.7867239,
"step": 1197329,
"title": "MotrixSim Web Preview",
"visibleStep": "1197329",
"z": 0.4996328353881836
},
{
"frame": 21,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113433.0273411,
"step": 1197358,
"title": "MotrixSim Web Preview",
"visibleStep": "1197358",
"z": 0.4996328353881836
},
{
"frame": 22,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113433.1892936,
"step": 1197377,
"title": "MotrixSim Web Preview",
"visibleStep": "1197377",
"z": 0.4996328353881836
},
{
"frame": 23,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113433.3487034,
"step": 1197396,
"title": "MotrixSim Web Preview",
"visibleStep": "1197396",
"z": 0.4996328353881836
},
{
"frame": 24,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.3290",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113433.587853,
"step": 1197424,
"title": "MotrixSim Web Preview",
"visibleStep": "1197424",
"z": 4.328996181488037
},
{
"frame": 25,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 2.0923",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113433.755578,
"step": 1197444,
"title": "MotrixSim Web Preview",
"visibleStep": "1197444",
"z": 2.0923168659210205
},
{
"frame": 26,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.6697",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113433.9922316,
"step": 1197472,
"title": "MotrixSim Web Preview",
"visibleStep": "1197472",
"z": 0.6696948409080505
},
{
"frame": 27,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4997",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113434.1470935,
"step": 1197491,
"title": "MotrixSim Web Preview",
"visibleStep": "1197491",
"z": 0.49972307682037354
},
{
"frame": 28,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113434.3138952,
"step": 1197511,
"title": "MotrixSim Web Preview",
"visibleStep": "1197511",
"z": 0.4996328353881836
},
{
"frame": 29,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113434.4939082,
"step": 1197532,
"title": "MotrixSim Web Preview",
"visibleStep": "1197532",
"z": 0.4996328353881836
},
{
"frame": 30,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113434.6086948,
"step": 1197546,
"title": "MotrixSim Web Preview",
"visibleStep": "1197546",
"z": 0.4996328353881836
},
{
"frame": 31,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113434.9620519,
"step": 1197588,
"title": "MotrixSim Web Preview",
"visibleStep": "1197588",
"z": 0.4996328353881836
},
{
"frame": 32,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113435.1847568,
"step": 1197614,
"title": "MotrixSim Web Preview",
"visibleStep": "1197614",
"z": 0.4996328353881836
},
{
"frame": 33,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113435.347539,
"step": 1197634,
"title": "MotrixSim Web Preview",
"visibleStep": "1197634",
"z": 0.4996328353881836
},
{
"frame": 34,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.8587",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113435.504579,
"step": 1197652,
"title": "MotrixSim Web Preview",
"visibleStep": "1197652",
"z": 4.858736038208008
},
{
"frame": 35,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 3.4069",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113435.6657073,
"step": 1197672,
"title": "MotrixSim Web Preview",
"visibleStep": "1197672",
"z": 3.4068565368652344
},
{
"frame": 36,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.6182",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113435.9040775,
"step": 1197700,
"title": "MotrixSim Web Preview",
"visibleStep": "1197700",
"z": 0.6181541681289673
},
{
"frame": 37,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4879",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113436.0745072,
"step": 1197720,
"title": "MotrixSim Web Preview",
"visibleStep": "1197720",
"z": 0.4878646731376648
},
{
"frame": 38,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113436.3046973,
"step": 1197748,
"title": "MotrixSim Web Preview",
"visibleStep": "1197748",
"z": 0.4996328353881836
},
{
"frame": 39,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113436.468162,
"step": 1197767,
"title": "MotrixSim Web Preview",
"visibleStep": "1197767",
"z": 0.4996328353881836
},
{
"frame": 40,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113436.7040653,
"step": 1197795,
"title": "MotrixSim Web Preview",
"visibleStep": "1197795",
"z": 0.4996328353881836
},
{
"frame": 41,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113436.8651109,
"step": 1197814,
"title": "MotrixSim Web Preview",
"visibleStep": "1197814",
"z": 0.4996328353881836
},
{
"frame": 42,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113437.1052198,
"step": 1197843,
"title": "MotrixSim Web Preview",
"visibleStep": "1197843",
"z": 0.4996328353881836
},
{
"frame": 43,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 0.4996",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113437.2693567,
"step": 1197862,
"title": "MotrixSim Web Preview",
"visibleStep": "1197862",
"z": 0.4996328353881836
},
{
"frame": 44,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.9765",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113437.471499,
"step": 1197886,
"title": "MotrixSim Web Preview",
"visibleStep": "1197886",
"z": 4.976456165313721
},
{
"frame": 45,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.5291",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113437.5684204,
"step": 1197898,
"title": "MotrixSim Web Preview",
"visibleStep": "1197898",
"z": 4.529120445251465
},
{
"frame": 46,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.5291",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113437.5684204,
"step": 1197898,
"title": "MotrixSim Web Preview",
"visibleStep": "1197898",
"z": 4.529120445251465
},
{
"frame": 47,
"connection": "connected",
"host": "1nh2jhpso5kckmx",
"posText": "0.0000, 0.0000, 4.5291",
"runtimeHeading": "Real Runtime",
"serverTime": 1782113437.5684204,
"step": 1197898,
"title": "MotrixSim Web Preview",
"visibleStep": "1197898",
"z": 4.529120445251465
}
],
"z_min": 0.36073562502861023,
"z_max": 4.976456165313721,
"explanation": "Frames captured from the actual browser page connected to remote MotrixSim /state, then converted to simple recording assets."
}

View File

@ -0,0 +1,50 @@
{
"ok": true,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
"steps": 240,
"body_names": [
"_unnamed_link_0"
],
"sample_count": 30,
"step_ms": {
"mean": 0.014092,
"p95": 0.014621,
"max": 0.132174
},
"first_sample": {
"step": 0,
"bodies": {
"_unnamed_link_0": {
"pos": [
0.0,
0.0,
5.0
],
"quat": [
-0.0,
-0.0,
-0.0,
1.0
]
}
}
},
"last_sample": {
"step": 232,
"bodies": {
"_unnamed_link_0": {
"pos": [
0.0,
0.0,
0.499633
],
"quat": [
-0.0,
-0.0,
-0.0,
1.0
]
}
}
}
}

View File

@ -0,0 +1,27 @@
{
"ok": true,
"package": "@mujoco/mujoco",
"packageVersion": "3.9.0",
"node": "v23.9.0",
"model": {
"nq": 7,
"nv": 6,
"nbody": 2,
"ngeom": 2
},
"timingMs": {
"wasmLoad": 55.728,
"modelCompile": 21.673,
"totalStep": 25.15,
"perStep": 0.012575
},
"qposHead": [
0,
0,
0.049633,
1,
0,
0,
0
]
}

View File

@ -0,0 +1,53 @@
{
"captured_at": "2026-06-22T09:29:02.013Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
"pause": {
"step": 0,
"paused": "true",
"control": "control: pause ok",
"meshes": "58",
"objects": "59",
"stateStatus": "state: poll connected"
},
"pauseLater": {
"step": 0,
"paused": "true",
"control": "control: pause ok",
"meshes": "58",
"objects": "59",
"stateStatus": "state: poll connected"
},
"step": {
"step": 1,
"paused": "true",
"control": "control: step ok",
"meshes": "58",
"objects": "59",
"stateStatus": "state: poll connected"
},
"reset": {
"step": 0,
"paused": "true",
"control": "control: reset ok",
"meshes": "58",
"objects": "59",
"stateStatus": "state: poll connected"
},
"resume": {
"step": 8,
"paused": "false",
"control": "control: resume ok",
"meshes": "58",
"objects": "59",
"stateStatus": "state: poll connected"
},
"verification": {
"pause_stopped": true,
"step_once_incremented": true,
"reset_zero": true,
"resume_advanced": true,
"min_objects": 50,
"min_meshes": 40,
"timeout_ms": 60000
}
}

View File

@ -0,0 +1,83 @@
{
"captured_at": "2026-06-22T10:16:38.826Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest",
"desktop": {
"before": {
"step": "2551",
"pixels": "30"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 34 objects",
"stateStatus": "state: poll connected",
"manifest": "scene_flat.xml",
"objects": "34",
"bindings": "33",
"meshes": "33",
"fallbacks": "0",
"step": "2551",
"pixels": "30",
"viewport": {
"width": 1280,
"height": 720
},
"canvas": {
"width": 920,
"height": 720,
"clientWidth": 920,
"clientHeight": 720,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "360px 920px",
"shellRows": "720px"
}
},
"mobile": {
"before": {
"step": "2551",
"pixels": "30"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 34 objects",
"stateStatus": "state: poll connected",
"manifest": "scene_flat.xml",
"objects": "34",
"bindings": "33",
"meshes": "33",
"fallbacks": "0",
"step": "2551",
"pixels": "30",
"viewport": {
"width": 390,
"height": 844
},
"canvas": {
"width": 780,
"height": 1040,
"clientWidth": 390,
"clientHeight": 520,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "390px",
"shellRows": "1234.59px 520px"
}
},
"verification": {
"desktop_nonblank": true,
"desktop_mesh_requirement_met": true,
"desktop_replay_moved": false,
"mobile_actual_viewport": true,
"mobile_nonblank": true,
"mobile_mesh_requirement_met": true,
"mobile_replay_moved": false,
"mobile_stacked_layout": true,
"min_objects": 30,
"min_meshes": 20,
"timeout_ms": 60000
}
}

View File

@ -0,0 +1,93 @@
{
"captured_at": "2026-06-22T09:37:26.294Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
"joint_name": "joint2",
"joint_target": 0.6,
"dom": {
"objects": "59",
"meshes": "58",
"paused": "true",
"control": "control: gripper ok",
"jointStatus": "joints: gripper open",
"selectedJoint": "joint2",
"jointValue": "0.60"
},
"before": {
"paused": true,
"joint": {
"pos": 0,
"vel": 0,
"range": [
-1.7627999782562256,
1.7627999782562256
],
"index": 1
},
"hand": {
"pos": [
0.08799997717142105,
0,
0.9260001182556152
],
"quat": [
-0.9238792657852173,
-0.3826832175254822,
0,
0
]
}
},
"after_joint": {
"paused": true,
"joint": {
"pos": 0.6000000238418579,
"vel": 0,
"range": [
-1.7627999782562256,
1.7627999782562256
],
"index": 1
},
"hand": {
"pos": [
0.4074625074863434,
0,
0.7727354764938354
],
"quat": [
-0.8826155662536621,
-0.36559122800827026,
0.27302485704421997,
0.11309058964252472
]
}
},
"after_gripper": {
"finger_joint1": {
"pos": 0.03999999910593033,
"vel": 0,
"range": [
0,
0.03999999910593033
],
"index": 7
},
"finger_joint2": {
"pos": 0.03999999910593033,
"vel": 0,
"range": [
0,
0.03999999910593033
],
"index": 8
}
},
"verification": {
"joint_value_applied": true,
"hand_pose_changed": true,
"gripper_opened": true,
"min_objects": 50,
"min_meshes": 40,
"timeout_ms": 70000
}
}

View File

@ -0,0 +1,80 @@
{
"captured_at": "2026-06-22T08:48:03.112Z",
"url": "http://127.0.0.1:5173/",
"desktop": {
"before": {
"step": "120",
"pixels": "4"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 5 objects",
"stateStatus": "state: replay 13 frames @ 12 fps",
"manifest": "Universal Three.js Preview Demo",
"objects": "5",
"bindings": "1",
"meshes": "1",
"fallbacks": "0",
"step": "160",
"pixels": "4",
"viewport": {
"width": 1280,
"height": 720
},
"canvas": {
"width": 920,
"height": 720,
"clientWidth": 920,
"clientHeight": 720,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "360px 920px",
"shellRows": "720px"
}
},
"mobile": {
"before": {
"step": "100",
"pixels": "4"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 5 objects",
"stateStatus": "state: replay 13 frames @ 12 fps",
"manifest": "Universal Three.js Preview Demo",
"objects": "5",
"bindings": "1",
"meshes": "1",
"fallbacks": "0",
"step": "60",
"pixels": "4",
"viewport": {
"width": 390,
"height": 844
},
"canvas": {
"width": 780,
"height": 1040,
"clientWidth": 390,
"clientHeight": 520,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "390px",
"shellRows": "710.219px 520px"
}
},
"verification": {
"desktop_nonblank": true,
"desktop_mesh_loaded": true,
"desktop_replay_moved": true,
"mobile_actual_viewport": true,
"mobile_nonblank": true,
"mobile_mesh_loaded": true,
"mobile_replay_moved": true,
"mobile_stacked_layout": true
}
}

View File

@ -0,0 +1,82 @@
{
"captured_at": "2026-06-22T08:55:18.655Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest",
"desktop": {
"before": {
"step": "19078",
"pixels": "4"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 2 objects",
"stateStatus": "state: poll connected",
"manifest": "falling_ball.xml",
"objects": "2",
"bindings": "1",
"meshes": "0",
"fallbacks": "0",
"step": "19253",
"pixels": "4",
"viewport": {
"width": 1280,
"height": 720
},
"canvas": {
"width": 920,
"height": 720,
"clientWidth": 920,
"clientHeight": 720,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "360px 920px",
"shellRows": "720px"
}
},
"mobile": {
"before": {
"step": "19421",
"pixels": "4"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 2 objects",
"stateStatus": "state: poll connected",
"manifest": "falling_ball.xml",
"objects": "2",
"bindings": "1",
"meshes": "0",
"fallbacks": "0",
"step": "19668",
"pixels": "4",
"viewport": {
"width": 390,
"height": 844
},
"canvas": {
"width": 780,
"height": 1040,
"clientWidth": 390,
"clientHeight": 520,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "390px",
"shellRows": "696.219px 520px"
}
},
"verification": {
"desktop_nonblank": true,
"desktop_mesh_requirement_met": true,
"desktop_replay_moved": true,
"mobile_actual_viewport": true,
"mobile_nonblank": true,
"mobile_mesh_requirement_met": true,
"mobile_replay_moved": true,
"mobile_stacked_layout": true,
"min_objects": 2,
"min_meshes": 0
}
}

View File

@ -0,0 +1,83 @@
{
"captured_at": "2026-06-22T09:12:16.061Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
"desktop": {
"before": {
"step": "19907",
"pixels": "2"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 59 objects",
"stateStatus": "state: poll connected",
"manifest": "scene.xml",
"objects": "59",
"bindings": "58",
"meshes": "58",
"fallbacks": "0",
"step": "20063",
"pixels": "2",
"viewport": {
"width": 1280,
"height": 720
},
"canvas": {
"width": 920,
"height": 720,
"clientWidth": 920,
"clientHeight": 720,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "360px 920px",
"shellRows": "720px"
}
},
"mobile": {
"before": {
"step": "22473",
"pixels": "2"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 59 objects",
"stateStatus": "state: poll connected",
"manifest": "scene.xml",
"objects": "59",
"bindings": "58",
"meshes": "58",
"fallbacks": "0",
"step": "22635",
"pixels": "2",
"viewport": {
"width": 390,
"height": 844
},
"canvas": {
"width": 780,
"height": 1040,
"clientWidth": 390,
"clientHeight": 520,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "390px",
"shellRows": "696.219px 520px"
}
},
"verification": {
"desktop_nonblank": true,
"desktop_mesh_requirement_met": true,
"desktop_replay_moved": true,
"mobile_actual_viewport": true,
"mobile_nonblank": true,
"mobile_mesh_requirement_met": true,
"mobile_replay_moved": true,
"mobile_stacked_layout": true,
"min_objects": 50,
"min_meshes": 40,
"timeout_ms": 45000
}
}

View File

@ -0,0 +1,82 @@
{
"captured_at": "2026-06-22T08:55:20.520Z",
"url": "http://127.0.0.1:5173/",
"desktop": {
"before": {
"step": "0",
"pixels": "4"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 5 objects",
"stateStatus": "state: replay 13 frames @ 12 fps",
"manifest": "Universal Three.js Preview Demo",
"objects": "5",
"bindings": "1",
"meshes": "1",
"fallbacks": "0",
"step": "220",
"pixels": "4",
"viewport": {
"width": 1280,
"height": 720
},
"canvas": {
"width": 920,
"height": 720,
"clientWidth": 920,
"clientHeight": 720,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "360px 920px",
"shellRows": "720px"
}
},
"mobile": {
"before": {
"step": "0",
"pixels": "4"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 5 objects",
"stateStatus": "state: replay 13 frames @ 12 fps",
"manifest": "Universal Three.js Preview Demo",
"objects": "5",
"bindings": "1",
"meshes": "1",
"fallbacks": "0",
"step": "120",
"pixels": "4",
"viewport": {
"width": 390,
"height": 844
},
"canvas": {
"width": 780,
"height": 1040,
"clientWidth": 390,
"clientHeight": 520,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "390px",
"shellRows": "710.219px 520px"
}
},
"verification": {
"desktop_nonblank": true,
"desktop_mesh_requirement_met": true,
"desktop_replay_moved": true,
"mobile_actual_viewport": true,
"mobile_nonblank": true,
"mobile_mesh_requirement_met": true,
"mobile_replay_moved": true,
"mobile_stacked_layout": true,
"min_objects": 5,
"min_meshes": 1
}
}

View File

@ -0,0 +1,83 @@
{
"captured_at": "2026-06-22T10:13:13.543Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18085%2Fmanifest",
"desktop": {
"before": {
"step": "2234",
"pixels": "22"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 569 objects",
"stateStatus": "state: poll connected",
"manifest": "scene.xml",
"objects": "569",
"bindings": "548",
"meshes": "568",
"fallbacks": "0",
"step": "2234",
"pixels": "22",
"viewport": {
"width": 1280,
"height": 720
},
"canvas": {
"width": 920,
"height": 720,
"clientWidth": 920,
"clientHeight": 720,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "360px 920px",
"shellRows": "720px"
}
},
"mobile": {
"before": {
"step": "2234",
"pixels": "18"
},
"after": {
"title": "MotrixSim Universal Three.js Preview",
"h1": "Universal Three.js Preview",
"status": "manifest loaded: 569 objects",
"stateStatus": "state: poll connected",
"manifest": "scene.xml",
"objects": "569",
"bindings": "548",
"meshes": "568",
"fallbacks": "0",
"step": "2234",
"pixels": "18",
"viewport": {
"width": 390,
"height": 844
},
"canvas": {
"width": 780,
"height": 1040,
"clientWidth": 390,
"clientHeight": 520,
"tag": "CANVAS",
"className": "preview-canvas"
},
"shellColumns": "390px",
"shellRows": "1234.59px 520px"
}
},
"verification": {
"desktop_nonblank": true,
"desktop_mesh_requirement_met": true,
"desktop_replay_moved": false,
"mobile_actual_viewport": true,
"mobile_nonblank": true,
"mobile_mesh_requirement_met": true,
"mobile_replay_moved": false,
"mobile_stacked_layout": true,
"min_objects": 500,
"min_meshes": 20,
"timeout_ms": 120000
}
}

View File

@ -0,0 +1,246 @@
# MotrixSim Web 预览方案对比报告
日期:2026-06-22
## 最终建议
**主线选择方案二:后端状态流。**
当前目标不是做一个独立的 MuJoCo demo,而是让真实 MotrixSim 环境能在 Web 前端预览。无影云桌面上的 `/opt/gs_playground_real2sim_48q/.venv/bin/python` 已实测可 import `motrixsim`、`mujoco` 和 `torch`,并能从 MotrixSim 场景中抽取 body 位姿。这个链路最贴近真实生产环境。
**方案一 MuJoCo WASM 保留为后续轻量模式。**
WASM smoke 已通过,适合公开模型、离线单机 demo、低服务器成本展示,但不适合作为 MotrixSim/Python/GPU 栈的第一版替代品。
![决策矩阵](figures/decision-matrix.svg)
## 核心对比
| 维度 | 方案一:MuJoCo WASM | 方案二:后端状态流 |
|---|---|---|
| 计算位置 | 浏览器 CPU/WASM | 无影云桌面 Python/MotrixSim/GPU |
| 与 MotrixSim 一致性 | 中低,需要重新适配 | 高,直接复用真实环境 |
| Python/torch/ROS/RL 集成 | 弱,需要移植 | 强,原生保留 |
| 首版落地复杂度 | 中,需处理 WASM、MEMFS、资产转换 | 中低,先抽状态再传输 |
| 服务器成本 | 低 | 中高,取决于并发 |
| 网络依赖 | 下载后弱依赖 | 强依赖 |
| 模型资产保护 | 弱,资产下发浏览器 | 强,核心资产留后端 |
| 复杂场景上限 | 浏览器受限 | 后端硬件决定 |
| 适合定位 | 轻量 demo / 离线展示 | 当前 Web 预览主线 |
## 实测证据
| 测试项 | 方案 | 结果 |
|---|---|---|
| 无影 SSH | 后端状态流 | `39.106.101.95:22` 成功 |
| GPU | 后端状态流 | `NVIDIA RTX5880-Ada-48Q, 49152 MiB` |
| MotrixSim import | 后端状态流 | `/opt/gs_playground_real2sim_48q/.venv/bin/python` 下成功 |
| MotrixSim step | 后端状态流 | 240 step 平均 `0.014092 ms/step` |
| body pose 抽取 | 后端状态流 | position/quaternion 成功 |
| 真实 Web 预览 | 后端状态流 | 浏览器打开 `127.0.0.1:18081`,页面显示 live MotrixSim ball、step、position、raw state |
| 通用 Three.js 预览 | 后端状态流 | `falling_ball.xml` 通过 `/manifest` + `/state` 在 Three.js/WebGL 中渲染成功 |
| 机器人配置 Three.js 预览 | 后端状态流 | `franka_emika_panda/scene.xml` 展开 `panda.xml` 后加载 59 对象、58 个 OBJ/STL mesh,0 fallback |
| Web 控制闭环 | 后端状态流 | Pause/Step/Reset/Resume 已通过页面按钮控制 MotrixSim backend |
| Franka 关节控制 | 后端状态流 | `joint2=0.6 rad` 后 hand pose 变化,`Open Grip` 后 finger joints 到 `0.04 m` |
| WASM 加载 | MuJoCo WASM | `@mujoco/mujoco@3.9.0` 成功 |
| WASM step | MuJoCo WASM | 2000 step 平均 `0.012575 ms/step` |
## 真实 Web 预览重测
本次按用户要求重新测试了真实 Web 页面,不再把终端输出或 smoke 结果当作前端效果图。
![MotrixSim 真实 Web 预览](figures/real-web-preview/motrix-live-web-preview.jpg)
| 项目 | 结果 |
|---|---|
| 远端运行 | 无影 `39.106.101.95`,主机 `1nh2jhpso5kckmx` |
| 运行时 | `/opt/gs_playground_real2sim_48q/.venv/bin/python` |
| 场景 | `/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml` |
| 前端 | 浏览器页面 `MotrixSim Web Preview`,Canvas 渲染 |
| 后端 | Python 服务真实加载 MotrixSim 并持续 `model.step(data)` |
| 页面证据 | 显示 `connected`、server step、body、position xyz、Raw live state |
| 本地截图 | `figures/real-web-preview/motrix-live-web-preview.jpg` |
| 证据 JSON | `artifacts/motrix_live_web_preview_browser_evidence_2026-06-22.json` |
### 简单录像
![MotrixSim Web 预览动图](figures/real-web-preview/motrix-live-web-preview-recording.gif)
| 项目 | 结果 |
|---|---|
| 录像来源 | 浏览器真实页面连续截帧 |
| 帧数 | 48 |
| 页面 z 范围 | `0.3607 m` 到 `4.9765 m` |
| MP4 | `figures/real-web-preview/motrix-live-web-preview-recording.mp4` |
| GIF | `figures/real-web-preview/motrix-live-web-preview-recording.gif` |
这个截图和录像把方案二的 MVP 链路从“后端可抽状态”推进到了“浏览器可看实时预览”。因此方案二作为主线的权重进一步提高:它能直接复用 MotrixSim/Python 环境,且已经能通过 Web 页面展示真实仿真状态。
需要明确的是:**当前接到 Web 的是真实 3D 状态,不是完整 3D 网格预览。** 现在的页面用 Canvas 做 2.5D 可视化,数据来自 MotrixSim 的 3D position/quaternion;要达到完整“真实三维预览”,下一步应接入 Three.js/GLTF 或 MJCF mesh 转换、场景 manifest 和 orbit camera。
## 通用 Three.js 预览器实测
已实现第一版通用 Three.js/WebGL 预览器,当前能力是:
- 读取 scene manifest,不再绑定固定 demo 代码。
- 支持 primitive:box、sphere、capsule、cylinder、plane、axes。
- 支持 mesh loader:OBJ、STL、GLTF、GLB。
- 支持状态源:replay、HTTP poll、websocket。
- 支持 Motrix/MJCF Z-up 到 Three.js Y-up 的 position/quaternion 转换。
- 支持 `object.body` 到 runtime body state 的绑定。
### Sample manifest 验证
![通用 Three.js sample desktop](figures/real-web-preview/universal-three-preview-sample-desktop.jpg)
| 项目 | 结果 |
|---|---|
| objects | 5 |
| body bindings | 1 |
| meshes | 1 |
| fallbacks | 0 |
| desktop replay step | `0` → `220` |
| mobile replay step | `0` → `120` |
| mobile viewport | `390 x 844` |
| mobile layout | 单列堆叠 |
### 远端 MotrixSim bridge 验证
![远端 MotrixSim bridge desktop](figures/real-web-preview/universal-three-preview-remote-motrix-desktop.jpg)
| 项目 | 结果 |
|---|---|
| 远端 | `39.106.101.95` |
| bridge | `server/headless_motrix_bridge.py` |
| scene | `/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml` |
| manifest | `falling_ball.xml` |
| objects | 2 |
| body bindings | 1 |
| state | `poll connected` |
| desktop step | `19078` → `19253` |
| mobile step | `19421` → `19668` |
| evidence | `artifacts/universal_three_preview_remote_motrix_evidence_2026-06-22.json` |
![远端 MotrixSim bridge mobile](figures/real-web-preview/universal-three-preview-remote-motrix-mobile.jpg)
### 机器人配置验证:Franka Panda
![Franka Panda robot desktop](figures/real-web-preview/universal-three-preview-robot-panda-desktop.jpg)
| 项目 | 结果 |
|---|---|
| 远端 | `39.106.101.95` |
| scene | `/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml` |
| include | `panda.xml` 已展开 |
| manifest objects | 59 |
| mesh objects | 58 |
| fallback | 0 |
| state | `poll connected` |
| desktop step | `19907` → `20063` |
| mobile step | `22473` → `22635` |
| evidence | `artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json` |
![Franka Panda robot mobile](figures/real-web-preview/universal-three-preview-robot-panda-mobile.jpg)
### Web 控制闭环验证
![Web 控制闭环](figures/real-web-preview/universal-three-preview-control-desktop.jpg)
| 项目 | 结果 |
|---|---|
| Pause | step 停止 |
| Step once | step 单步增长 |
| Reset | step 回到 `0` |
| Resume | step 继续增长 |
| evidence | `artifacts/universal_three_preview_control_evidence_2026-06-22.json` |
![Franka joint control](figures/real-web-preview/universal-three-preview-joint-control-desktop.jpg)
| 项目 | 结果 |
|---|---|
| controls.joints | 9 |
| state link poses | 11 |
| joint2 | `0` → `0.6000000238` |
| hand pose | 已变化 |
| gripper | finger joints → `0.04 m` |
| evidence | `artifacts/universal_three_preview_joint_control_evidence_2026-06-22.json` |
结论更新:方案二已经不只是“状态流 + Canvas 预览”,现在已经走通“MotrixSim/MJCF scene → manifest → Three.js/WebGL → live state → Web 控制命令 → MotrixSim joint/link 更新”的基础闭环,并且真实机器人 OBJ/STL 资产已能在浏览器里预览和低层操作。复杂材质纹理与大规模重复资产也完成了 store 货架场景验证。剩余工作重点是高层 task/action adapter、更多 MJCF 特性的长尾覆盖和性能优化,而不是证明 Three.js 路线是否可行。
## 通用资产覆盖补充验证(2026-06-22 18:17)
| 场景 | 类型 | Objects | Meshes | Fallbacks | 额外覆盖 |
|---|---|---:|---:|---:|---|
| Store `scene.xml` | 大规模贴图货架 | 569 | 568 | 0 | `assetdir`、PNG texture、OBJ 复用、17 个 textured materials |
| Go2 `scene_flat.xml` | 四足机器人 | 34 | 33 | 0 | 12 joints、link pose、OBJ robot mesh、checker ground |
这轮实现补齐了 `nested <default>`、`fromto`、`xyaxes`、`zaxis`、mesh scale、file texture、builtin checker、inline convex mesh 与 mesh cache。Store 首次验证时曾因 569 个对象重复加载 OBJ 超时,加入 mesh cache 后桌面/移动均通过非空渲染检查;这说明后续性能优化应优先围绕实例化、LOD 和渐进加载,而不是回退到视频流。
## 方案二 MVP 设计
![后端状态流架构](figures/backend-stream-architecture.svg)
### 后端
建议进程分成三层:
1. `SimulationWorker`
- 加载 MotrixSim scene。
- 定频执行 `model.step(data)`。
- 抽取 body pose、qpos、sensor、task status。
2. `StateBroker`
- 将状态压缩为 JSON 或二进制 buffer。
- 按前端渲染频率抽样,例如 20-60 Hz。
- 支持 pause/reset/action/camera 命令。
3. `WebServer`
- 提供 scene manifest。
- 提供 WebSocket 状态流。
- 可选提供 MotrixSim RenderApp 截图或视频流。
### 前端
建议前端第一版只做“监视器 + 控制器”:
1. 加载 `scene_manifest.json`。
2. 根据 manifest 创建 Three.js 节点或加载 GLTF。
3. 建立 WebSocket。
4. 按 body index 更新 `position` 和 `quaternion`。
5. 提供 reset、pause、step、action、camera 控件。
## 方案一定位
![MuJoCo WASM 架构](figures/wasm-architecture.svg)
MuJoCo WASM 路线已证明可以跑通最小 XML、step 和 qpos 读取。它的工程价值是:
- 无服务器公开 demo。
- 离线模型教学。
- 单机器人轻量展示。
- 前端交互原型。
不建议把它作为第一阶段主线,因为:
- MotrixSim 的 Python API、渲染、控制器和训练链路不能直接复用。
- 私有 MJCF、mesh、texture 必须下发到浏览器。
- 多机器人、大场景、复杂接触在浏览器中风险更高。
- 多线程 WASM 还需要跨源隔离响应头,部署复杂度不为零。
## 里程碑建议
| 阶段 | 时间 | 目标 | 验收标准 |
|---|---|---|---|
| M0 | 0.5 天 | 固化无影接入和环境脚本 | 一键输出 GPU、MotrixSim import、scene step |
| M1 | 1-2 天 | 状态流 API MVP | 浏览器可看到 falling ball / robot body pose 更新 |
| M2 | 2-4 天 | Three.js 场景 manifest | 前端节点与 MotrixSim body 名稳定映射 |
| M3 | 3-5 天 | 控制闭环 | 前端 reset/pause/action 生效 |
| M4 | 3-5 天 | 真实画面对照 | RenderApp 截图或视频流与 Three.js 状态画面可对比 |
| M5 | 后续 | WASM 轻量模式 | 单模型可离线加载、step、渲染 |
## 决策
| 决策项 | 结论 |
|---|---|
| 当前主线 | 后端状态流 |
| 兜底能力 | 后端真实画面/视频流 |
| 后续增强 | MuJoCo WASM 轻量模式 |
| 不建议 | 第一版直接把 MotrixSim 完整迁到浏览器 |

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

View File

@ -0,0 +1,76 @@
{
"captured_at": "2026-06-22T10:33:23.139Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
"initial": {
"step": 11275,
"paused": false,
"command": [
0,
0,
0
]
},
"after_toggle": {
"step": 11334,
"paused": true
},
"after_reset": {
"step": 0,
"paused": true,
"command": [
0,
0,
0
]
},
"after_forward_key": {
"command": [
1,
0,
0
]
},
"after_stop_key": {
"command": [
0,
0,
0
]
},
"actuator": {
"name": "actuator1",
"target": 1.0140550017356873,
"state": {
"ctrl": 1.0140550136566162,
"range": [
-2.8973000049591064,
2.8973000049591064
],
"index": 0,
"targetName": "joint1"
}
},
"dom": {
"objects": "59",
"meshes": "58",
"paused": "true",
"step": "0",
"control": "control: set_actuator ok",
"inputStatus": "input: command sent 0.00 / 0.00 / 0.00",
"inputKeys": "-",
"inputCommand": "0.00 / 0.00 / 0.00",
"actuatorStatus": "actuators: actuator1 set",
"selectedActuator": "actuator1",
"actuatorValue": "1.014"
},
"verification": {
"space_toggled_pause": true,
"r_reset_step_zero": true,
"w_sent_forward_command": true,
"keyup_sent_stop_command": true,
"actuator_ctrl_applied": true,
"min_objects": 50,
"min_meshes": 40,
"timeout_ms": 60000
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

View File

@ -0,0 +1,76 @@
{
"captured_at": "2026-06-22T10:32:12.848Z",
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest",
"initial": {
"step": 5303,
"paused": false,
"command": [
0,
0,
0
]
},
"after_toggle": {
"step": 5419,
"paused": true
},
"after_reset": {
"step": 0,
"paused": true,
"command": [
0,
0,
0
]
},
"after_forward_key": {
"command": [
1,
0,
0
]
},
"after_stop_key": {
"command": [
0,
0,
0
]
},
"actuator": {
"name": "FL_hip",
"target": 0.33152000010013577,
"state": {
"ctrl": 0.3315199911594391,
"range": [
-0.9472000002861023,
0.9472000002861023
],
"index": 0,
"targetName": "FL_hip_joint"
}
},
"dom": {
"objects": "34",
"meshes": "33",
"paused": "true",
"step": "0",
"control": "control: set_actuator ok",
"inputStatus": "input: command sent 0.00 / 0.00 / 0.00",
"inputKeys": "-",
"inputCommand": "0.00 / 0.00 / 0.00",
"actuatorStatus": "actuators: FL_hip set",
"selectedActuator": "FL_hip",
"actuatorValue": "0.332"
},
"verification": {
"space_toggled_pause": true,
"r_reset_step_zero": true,
"w_sent_forward_command": true,
"keyup_sent_stop_command": true,
"actuator_ctrl_applied": true,
"min_objects": 30,
"min_meshes": 20,
"timeout_ms": 60000
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

View File

@ -0,0 +1,115 @@
<callout emoji="✅" background-color="light-green" border-color="green">
<p><b>推荐结论:</b>当前主线选择“MotrixSim 后端计算 + WebSocket 状态流 + 前端 Three.js 渲染”。MuJoCo WASM 已通过最小 smoke,但更适合作为后续离线/公开轻量 demo,不适合作为第一阶段真实 MotrixSim Web 预览主线。</p>
</callout>
<h1>一、测试范围</h1>
<table>
<thead><tr><th background-color="light-gray">项目</th><th background-color="light-gray">结果</th></tr></thead>
<tbody>
<tr><td>本地调研目录</td><td><code>research-tests/2026-06-22-motrixsim-web-preview</code></td></tr>
<tr><td>无影 SSH 主机</td><td><code>39.106.101.95</code></td></tr>
<tr><td>无影 GPU</td><td><code>NVIDIA RTX5880-Ada-48Q, 49152 MiB</code></td></tr>
<tr><td>MotrixSim Python</td><td><code>/opt/gs_playground_real2sim_48q/.venv/bin/python</code></td></tr>
<tr><td>本地 WASM 包</td><td><code>@mujoco/mujoco@3.9.0</code></td></tr>
</tbody>
</table>
<h1>二、无影 MotrixSim 实测</h1>
<p>修正 SSH 主机后,TCP 22 和 SSH 均成功。远端默认系统 Python 未安装 MotrixSim,但 <code>/opt/gs_playground_real2sim_48q/.venv/bin/python</code> 可以成功 import <code>motrixsim</code>、<code>mujoco</code> 和 <code>torch</code>。</p>
<table>
<thead><tr><th background-color="light-gray">指标</th><th background-color="light-gray">实测结果</th></tr></thead>
<tbody>
<tr><td>MotrixSim scene</td><td><code>/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml</code></td></tr>
<tr><td>step 数</td><td>240</td></tr>
<tr><td>采样数</td><td>30</td></tr>
<tr><td>平均 step 耗时</td><td><code>0.014092 ms</code></td></tr>
<tr><td>p95 step 耗时</td><td><code>0.014621 ms</code></td></tr>
<tr><td>body 位姿</td><td>position / quaternion 抽取成功</td></tr>
<tr><td>body 高度变化</td><td><code>z=5.0</code> 到 <code>z=0.499633</code></td></tr>
</tbody>
</table>
<whiteboard type="mermaid">
flowchart LR
A["无影云桌面 GPU"] --> B["MotrixSim Python worker"]
B --> C["scene.step"]
C --> D["body pose / qpos / sensor"]
D --> E["WebSocket state stream"]
E --> F["Three.js Web viewer"]
F -. control .-> B
</whiteboard>
<h1>三、MuJoCo WASM 实测</h1>
<p>本地使用官方 <code>@mujoco/mujoco@3.9.0</code> 完成最小 XML 加载、模型编译、<code>mj_step</code> 和 <code>qpos</code> 读取。这个测试证明纯前端物理 demo 可行,但没有覆盖 MotrixSim Python 生态、远端 GPU、私有资产保护和真实渲染对照。</p>
<table>
<thead><tr><th background-color="light-gray">指标</th><th background-color="light-gray">实测结果</th></tr></thead>
<tbody>
<tr><td>单线程 WASM 文件</td><td><code>8.7 MB</code></td></tr>
<tr><td>多线程 WASM 文件</td><td><code>8.8 MB</code></td></tr>
<tr><td>WASM 加载耗时</td><td><code>55.728 ms</code></td></tr>
<tr><td>模型编译耗时</td><td><code>21.673 ms</code></td></tr>
<tr><td>2000 step 总耗时</td><td><code>25.15 ms</code></td></tr>
<tr><td>单 step 耗时</td><td><code>0.012575 ms</code></td></tr>
</tbody>
</table>
<h1>四、方案对比</h1>
<table>
<thead><tr><th background-color="light-gray">维度</th><th background-color="light-gray">MuJoCo WASM</th><th background-color="light-gray">后端状态流</th></tr></thead>
<tbody>
<tr><td>与真实 MotrixSim 一致性</td><td>中低,需要重新适配</td><td><b>高,直接复用远端环境</b></td></tr>
<tr><td>Python / torch / RL 集成</td><td>弱,需要移植</td><td><b>强,原生保留</b></td></tr>
<tr><td>服务器成本</td><td><b>低</b></td><td>中高,随并发增长</td></tr>
<tr><td>资产保护</td><td>弱,资产下发浏览器</td><td><b>强,资产留后端</b></td></tr>
<tr><td>MVP 落地</td><td>适合轻量 demo</td><td><b>适合当前目标</b></td></tr>
<tr><td>复杂场景上限</td><td>浏览器受限</td><td><b>由后端硬件决定</b></td></tr>
</tbody>
</table>
<whiteboard type="mermaid">
flowchart TD
Start["当前目标:真实 MotrixSim Web 预览"] --> Q1{"是否必须复用 Python / MotrixSim / GPU?"}
Q1 -->|是| Backend["选择后端状态流"]
Q1 -->|否| Q2{"是否需要离线和低服务器成本?"}
Q2 -->|是| Wasm["选择 MuJoCo WASM"]
Q2 -->|否| Hybrid["后端状态流优先,WASM 作为补充"]
Backend --> Mvp["MVP:state stream + Three.js"]
Mvp --> Fallback["兜底:真实渲染截图或视频流"]
</whiteboard>
<h1>五、推荐路线</h1>
<ol>
<li seq="auto">M0:固化无影接入脚本,持续输出 GPU、MotrixSim import、scene step 证据。</li>
<li seq="auto">M1:在 MotrixSim venv 中补充 WebSocket 服务依赖,启动后端状态流 API。</li>
<li seq="auto">M2:前端加载 scene manifest,并按 body index 更新 Three.js 节点。</li>
<li seq="auto">M3:加入 reset、pause、action、camera 控制闭环。</li>
<li seq="auto">M4:增加真实 MotrixSim 渲染截图或视频流作为画面对照。</li>
<li seq="auto">M5:后续补 MuJoCo WASM 轻量离线模式。</li>
</ol>
<h1>六、风险和待验证</h1>
<table>
<thead><tr><th background-color="light-gray">风险</th><th background-color="light-gray">影响</th><th background-color="light-gray">建议</th></tr></thead>
<tbody>
<tr><td>MotrixSim venv 未装 FastAPI / websockets</td><td>不能直接启动 API 服务</td><td>最小安装并锁版本,或复用已有服务框架</td></tr>
<tr><td>当前只测了轻量 falling ball</td><td>复杂机器人和碰撞场景带宽未知</td><td>增加 Go1/Go2/G1 或 Real2Sim package benchmark</td></tr>
<tr><td>前端资产映射未完成</td><td>Web 节点可能和 MotrixSim body 不一致</td><td>生成 scene manifest,固定 body/node 索引</td></tr>
<tr><td>真实画面对照未接入</td><td>无法判断 Three.js 重建是否偏离</td><td>接入 RenderApp 截图、MJPEG 或 WebRTC 兜底</td></tr>
</tbody>
</table>
<h1>七、本地证据文件</h1>
<ul>
<li><code>artifacts/wuying_probe_2026-06-22.log</code></li>
<li><code>artifacts/motrix_state_probe_remote_2026-06-22.json</code></li>
<li><code>artifacts/mujoco_wasm_smoke_2026-06-22.json</code></li>
<li><code>research-report.md</code></li>
<li><code>comparison-report.md</code></li>
</ul>
<h1>八、参考资料</h1>
<ul>
<li><a href="https://github.com/google-deepmind/mujoco/tree/main/wasm">MuJoCo JavaScript bindings</a></li>
<li><a href="https://mujoco.readthedocs.io/en/stable/python.html">MuJoCo Python docs</a></li>
<li><a href="https://mujoco.readthedocs.io/en/stable/programming/visualization.html">MuJoCo visualization docs</a></li>
</ul>

View File

@ -0,0 +1,24 @@
<h2>机器人配置验证:Franka Panda(2026-06-22)</h2>
<callout emoji="✅" background-color="light-green" border-color="green">
<p>本节是按“找一个带有机器人的配置再测试一下”重新执行的真实浏览器验证。测试对象不是 falling ball,也不是占位模型,而是无影 MotrixSim 示例里的 Franka Panda 机器人 MJCF 配置。</p>
</callout>
<table>
<thead>
<tr><th background-color="light-gray">验证项</th><th background-color="light-gray">结果</th></tr>
</thead>
<tbody>
<tr><td>远端环境</td><td>无影 39.106.101.95</td></tr>
<tr><td>场景</td><td>/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml</td></tr>
<tr><td>include</td><td>scene.xml 中的 panda.xml 已展开</td></tr>
<tr><td>资产</td><td>真实 OBJ/STL 机器人 mesh,通过 /assets HTTP 服务给浏览器</td></tr>
<tr><td>manifest objects</td><td>59</td></tr>
<tr><td>loaded meshes</td><td>58</td></tr>
<tr><td>fallbacks</td><td>0</td></tr>
<tr><td>state</td><td>poll connected,step 持续增长</td></tr>
<tr><td>桌面 step</td><td>19907 → 20063</td></tr>
<tr><td>移动 step</td><td>22473 → 22635</td></tr>
<tr><td>证据文件</td><td>artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json</td></tr>
</tbody>
</table>
<p>本轮同时修复了 MJCF include 展开、mesh 资产服务、body/geom 静态位姿累乘、Z-up 到 Three.js Y-up 的 mesh 本地坐标基转换,以及自动相机取景。截图中可以看到浏览器 WebGL 画面里真实渲染出了 Franka Panda 机器人本体。</p>
<p><b>边界说明:</b>该场景在当前 MotrixSim runtime 暴露的 body state 只有 link0,所以本轮证明的是“真实机器人三维资产预览 + 后端状态连接”,还不是完整关节动画驱动。下一步要补 qpos / joint 层级映射,才能让机器人各关节按 MotrixSim 状态实时运动。</p>

View File

@ -0,0 +1,157 @@
{
"ok": true,
"synced_at": "2026-06-22T12:16:34+08:00",
"space_id": "7530104418265350163",
"parent_title": "具身",
"parent_node_token": "PHQpwGXHOiqTm3kN96tcwEmOnGg",
"title": "MotrixSim Web 预览方案调研与测试报告(2026-06-22)",
"node_token": "UKxEwcYjwiuSapkDeaXcqRlUnVd",
"obj_token": "WZCfdtn8Jokb50xX3F3cJziingf",
"obj_type": "docx",
"wiki_url": "https://horizonrobotics.feishu.cn/wiki/UKxEwcYjwiuSapkDeaXcqRlUnVd",
"docx_url": "https://horizonrobotics.feishu.cn/docx/WZCfdtn8Jokb50xX3F3cJziingf",
"screenshot_update": {
"synced_at": "2026-06-22T12:28:28+08:00",
"section_title": "九、测试输出截图(非 Web 预览)",
"image_count": 4,
"images": [
"figures/test-screenshots/wuying-ssh-gpu.png",
"figures/test-screenshots/motrix-state-probe.png",
"figures/test-screenshots/mujoco-wasm-smoke.png",
"figures/test-screenshots/metrics-summary.png"
]
},
"real_web_preview_update": {
"synced_at": "2026-06-22T12:47:15+08:00",
"section_title": "十、真实 MotrixSim Web 预览重测(2026-06-22)",
"remote_host": "39.106.101.95",
"remote_hostname": "1nh2jhpso5kckmx",
"remote_pid": 496408,
"doc_image_block_id": "doxcnyD3OtCs1ZtVrVO04mh7CBe",
"doc_file_token": "HAZKbAG4oonXMgxCGf2cP5zrnde",
"image": "figures/real-web-preview/motrix-live-web-preview.jpg",
"evidence": "artifacts/motrix_live_web_preview_browser_evidence_2026-06-22.json"
},
"recording_and_3d_scope_update": {
"synced_at": "2026-06-22T15:33:00+08:00",
"section_title": "十一、简单录像说明与 3D 预览界定",
"gif_block_id": "doxcnnJklKLzQvOK14UELhYzMwc",
"gif_file_token": "YA6ybzu1ToXA7yxcd9bcKNe8n1f",
"mp4_block_id": "doxcnCMS14uQSvIkdSTVtRyoxZc",
"mp4_file_token": "WXtib2BedoUjVdxshL2cKVAqnyf",
"gif": "figures/real-web-preview/motrix-live-web-preview-recording.gif",
"mp4": "figures/real-web-preview/motrix-live-web-preview-recording.mp4",
"evidence": "artifacts/motrix_live_web_preview_recording_evidence_2026-06-22.json",
"scope": "real MotrixSim 3D state stream rendered by Canvas 2.5D MVP; not yet a full Three.js/WebGL mesh preview"
},
"universal_three_preview_update": {
"synced_at": "2026-06-22T16:58:46+08:00",
"section_title": "十二、通用 Three.js 预览器实测",
"sample_desktop_block_id": "doxcnV5fGlC32xHSMPK4gMEmqeh",
"sample_desktop_file_token": "NKuYb1hLDochIox1as0cRKoAnec",
"remote_desktop_block_id": "doxcnZhqFCyLgXzMAffsZxNeSAd",
"remote_desktop_file_token": "DJoWbHonaoAfbuxy95OcB9ZGnBg",
"remote_mobile_block_id": "doxcnaOwM2dcbafqalvcUqAW80f",
"remote_mobile_file_token": "Bx4HbZxqJoHDfuxmg00cIJbwn0c",
"sample_evidence": "artifacts/universal_three_preview_sample_evidence_2026-06-22.json",
"remote_motrix_evidence": "artifacts/universal_three_preview_remote_motrix_evidence_2026-06-22.json",
"remote_bridge": {
"host": "39.106.101.95",
"port": 18083,
"pid": 573064,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml"
}
},
"robot_panda_preview_update": {
"synced_at": "2026-06-22T17:17:09+08:00",
"section_title": "机器人配置验证:Franka Panda(2026-06-22)",
"desktop_block_id": "doxcn43wErQYuTQoOYDp5D00sif",
"desktop_file_token": "WOsFbGrhnoaDnPxMzEWcIPVvn6d",
"mobile_block_id": "doxcnrbK9mOWN9hmn6Pz39lsTng",
"mobile_file_token": "XbUQbNtBEoJN64xaB4lcMzdanah",
"desktop_image": "figures/real-web-preview/universal-three-preview-robot-panda-desktop.jpg",
"mobile_image": "figures/real-web-preview/universal-three-preview-robot-panda-mobile.jpg",
"evidence": "artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json",
"remote_bridge": {
"host": "39.106.101.95",
"port": 18084,
"pid": 575258,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml",
"runtime_body_names": ["link0"]
}
},
"web_control_update": {
"synced_at": "2026-06-22T17:44:23+08:00",
"section_title": "Web 控制闭环与 Franka 关节控制",
"document_revision_id": 42,
"section_block_id": "doxcnDbwfdvEnm9nTotbKxHih0e",
"control_image_block_id": "doxcn7FVVDPqwoYkVzJDujtumcg",
"control_image_file_token": "HeTJbySytos09gxZn2dceptbnhe",
"joint_control_image_block_id": "doxcnqihfLxHpjFUocxi4ANLxce",
"joint_control_image_file_token": "LqfUbWnWjo6hYwxDtbVcqqZen5e",
"control_image": "figures/real-web-preview/universal-three-preview-control-desktop.jpg",
"joint_control_image": "figures/real-web-preview/universal-three-preview-joint-control-desktop.jpg",
"control_evidence": "artifacts/universal_three_preview_control_evidence_2026-06-22.json",
"joint_control_evidence": "artifacts/universal_three_preview_joint_control_evidence_2026-06-22.json",
"remote_bridge": {
"host": "39.106.101.95",
"port": 18084,
"pid": 622164,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml",
"link_pose_count": 11,
"joint_count": 9
},
"verified_controls": ["pause", "step", "reset", "resume", "set_joint", "gripper"]
},
"universal_asset_matrix_update": {
"synced_at": "2026-06-22T18:17:43+08:00",
"section_title": "通用资产矩阵复测:Store 与 Go2",
"document_revision_id": 47,
"section_block_id": "doxcnuSHCNhVwmDPgtkGRAiNpbb",
"store_desktop_block_id": "doxcn2m2HyaZOYAHOxF6w3FGHmd",
"store_desktop_file_token": "ViGCb8SCfownU1xTAmHcBCScn8b",
"go2_desktop_block_id": "doxcnbO5xuxkLiH2QfmC9PcLSKh",
"go2_desktop_file_token": "OolqbSpaVoFi0bxmf27cMKGCnLa",
"store_desktop_image": "figures/real-web-preview/universal-three-preview-store-textured-desktop.jpg",
"store_mobile_image": "figures/real-web-preview/universal-three-preview-store-textured-mobile.jpg",
"go2_desktop_image": "figures/real-web-preview/universal-three-preview-go2-robot-desktop.jpg",
"go2_mobile_image": "figures/real-web-preview/universal-three-preview-go2-robot-mobile.jpg",
"store_evidence": "artifacts/universal_three_preview_store_textured_evidence_2026-06-22.json",
"go2_evidence": "artifacts/universal_three_preview_go2_robot_evidence_2026-06-22.json",
"remote_bridges": {
"store": {
"host": "39.106.101.95",
"port": 18085,
"pid": 661691,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/store/scene.xml",
"objects": 569,
"meshes": 568,
"fallbacks": 0,
"textured_materials": 17
},
"go2": {
"host": "39.106.101.95",
"port": 18086,
"pid": 664826,
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/go2/scene_flat.xml",
"objects": 34,
"meshes": 33,
"fallbacks": 0,
"joint_count": 12,
"link_pose_count": 13
}
},
"new_coverage": [
"nested default inheritance",
"assetdir mesh/texture resolution",
"file texture",
"builtin checker texture",
"mesh scale",
"fromto",
"xyaxes",
"zaxis",
"inline convex mesh",
"mesh cache clone"
]
}
}

View File

@ -0,0 +1,19 @@
<hr/>
<h2>通用资产矩阵复测:Store 与 Go2(2026-06-22)</h2>
<callout emoji="✅" background-color="light-green" border-color="green">
<p>本轮继续推进“万能 Three.js 预览”:从 Franka 单机器人扩展到大规模贴图货架 Store 和 Go2 四足机器人,均在无影 MotrixSim bridge + 本地浏览器中完成真实 WebGL 预览验证。</p>
</callout>
<table>
<colgroup><col span="1" width="150"/><col span="1" width="150"/><col span="1" width="90"/><col span="1" width="90"/><col span="1" width="90"/><col span="1" width="300"/></colgroup>
<thead>
<tr><th background-color="light-gray">场景</th><th background-color="light-gray">类型</th><th background-color="light-gray">Objects</th><th background-color="light-gray">Meshes</th><th background-color="light-gray">Fallbacks</th><th background-color="light-gray">额外覆盖</th></tr>
</thead>
<tbody>
<tr><td>Store scene.xml</td><td>大规模贴图货架</td><td>569</td><td>568</td><td>0</td><td>assetdir、PNG texture、OBJ 复用、17 个 textured materials、桌面/移动非空</td></tr>
<tr><td>Go2 scene_flat.xml</td><td>四足机器人</td><td>34</td><td>33</td><td>0</td><td>12 joints、13 link poses、OBJ robot mesh、checker ground、桌面/移动非空</td></tr>
</tbody>
</table>
<p>本轮代码新增覆盖:nested default inheritance、assetdir/meshdir/texturedir 解析、file texture、builtin checker texture、mesh scale、fromto、xyaxes、zaxis、inline convex mesh,以及 mesh cache + clone。</p>
<callout emoji="⚠️" background-color="light-yellow" border-color="yellow">
<p>Store 首次全量验证曾因 569 个对象重复加载 OBJ 超时;加入 mesh cache 后通过。后续优化重点应转向实例化、LOD、渐进加载和更完整材质系统,而不是回退到视频流。</p>
</callout>

View File

@ -0,0 +1,23 @@
<hr/>
<h2>Web 控制闭环与 Franka 关节控制(2026-06-22)</h2>
<callout emoji="✅" background-color="light-green" border-color="green">
<p>本轮已从只读 Three.js 预览推进到 Web 可操作预览:浏览器按钮可以真实控制无影云桌面上的 MotrixSim 后端,覆盖暂停、单步、重置、恢复、Franka 关节设置和夹爪开合。</p>
</callout>
<table>
<colgroup><col span="1" width="180"/><col span="1" width="540"/></colgroup>
<thead>
<tr><th background-color="light-gray">验证项</th><th background-color="light-gray">真实结果</th></tr>
</thead>
<tbody>
<tr><td>Web 入口</td><td>http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest</td></tr>
<tr><td>后端环境</td><td>无影云桌面 39.106.101.95,MotrixSim bridge 端口 18084,场景为 Franka Emika Panda MJCF。</td></tr>
<tr><td>渲染规模</td><td>Three.js 页面加载 59 个 objects、58 个 meshes,不再是占位 smoke 图。</td></tr>
<tr><td>仿真控制</td><td>Pause 后 step 保持 0;Step 后 step=1;Reset 后 step=0;Resume 后 step 推进到 8。</td></tr>
<tr><td>关节控制</td><td>页面选择 joint2 并设置 0.6,后端 state 返回 joint2=0.6000000238418579。</td></tr>
<tr><td>3D 姿态变化</td><td>joint2 设置前 hand pos=[0.088, 0, 0.926],设置后 hand pos=[0.407, 0, 0.773],说明 Web 控制驱动了真实三维机器人姿态。</td></tr>
<tr><td>夹爪控制</td><td>Open Grip 后 finger_joint1 与 finger_joint2 均打开到 0.03999999910593033 m。</td></tr>
</tbody>
</table>
<callout emoji="⚠️" background-color="light-yellow" border-color="yellow">
<p>边界说明:当前完成的是底层 joint/gripper 操作闭环。抓取、移动到目标点、轨迹跟随等高层任务还需要按具体场景补 policy/action adapter。</p>
</callout>

View File

@ -0,0 +1,46 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="560" viewBox="0 0 1200 560">
<rect width="1200" height="560" fill="#f7f9fb"/>
<text x="60" y="70" font-family="Arial, sans-serif" font-size="34" font-weight="700" fill="#17202a">方案二:MotrixSim 后端状态流架构</text>
<text x="60" y="108" font-family="Arial, sans-serif" font-size="18" fill="#52616f">真实 MotrixSim / Python / GPU 留在无影云桌面,Web 前端只接收状态并渲染。</text>
<rect x="60" y="170" width="270" height="210" rx="10" fill="#ffffff" stroke="#2e86de" stroke-width="3"/>
<text x="95" y="215" font-family="Arial, sans-serif" font-size="24" font-weight="700" fill="#1b4f72">无影云桌面</text>
<text x="95" y="252" font-family="Arial, sans-serif" font-size="17" fill="#17202a">39.106.101.95</text>
<text x="95" y="282" font-family="Arial, sans-serif" font-size="17" fill="#17202a">RTX5880-Ada-48Q</text>
<text x="95" y="312" font-family="Arial, sans-serif" font-size="17" fill="#17202a">MotrixSim venv</text>
<text x="95" y="342" font-family="Arial, sans-serif" font-size="17" fill="#17202a">Python worker</text>
<rect x="455" y="170" width="290" height="210" rx="10" fill="#ffffff" stroke="#27ae60" stroke-width="3"/>
<text x="492" y="215" font-family="Arial, sans-serif" font-size="24" font-weight="700" fill="#196f3d">状态流服务</text>
<text x="492" y="252" font-family="Arial, sans-serif" font-size="17" fill="#17202a">scene.step()</text>
<text x="492" y="282" font-family="Arial, sans-serif" font-size="17" fill="#17202a">body position</text>
<text x="492" y="312" font-family="Arial, sans-serif" font-size="17" fill="#17202a">body quaternion</text>
<text x="492" y="342" font-family="Arial, sans-serif" font-size="17" fill="#17202a">qpos / sensors</text>
<rect x="870" y="170" width="270" height="210" rx="10" fill="#ffffff" stroke="#8e44ad" stroke-width="3"/>
<text x="910" y="215" font-family="Arial, sans-serif" font-size="24" font-weight="700" fill="#633974">Web 前端</text>
<text x="910" y="252" font-family="Arial, sans-serif" font-size="17" fill="#17202a">Three.js / R3F</text>
<text x="910" y="282" font-family="Arial, sans-serif" font-size="17" fill="#17202a">GLTF / mesh viewer</text>
<text x="910" y="312" font-family="Arial, sans-serif" font-size="17" fill="#17202a">WebSocket state</text>
<text x="910" y="342" font-family="Arial, sans-serif" font-size="17" fill="#17202a">UI controls</text>
<path d="M330 275 H455" stroke="#34495e" stroke-width="4" fill="none" marker-end="url(#arrow)"/>
<text x="354" y="252" font-family="Arial, sans-serif" font-size="15" fill="#34495e">Python API</text>
<path d="M745 275 H870" stroke="#34495e" stroke-width="4" fill="none" marker-end="url(#arrow)"/>
<text x="760" y="252" font-family="Arial, sans-serif" font-size="15" fill="#34495e">WebSocket</text>
<path d="M870 330 H745" stroke="#7f8c8d" stroke-width="3" fill="none" stroke-dasharray="8 6" marker-end="url(#arrowGray)"/>
<text x="766" y="358" font-family="Arial, sans-serif" font-size="15" fill="#7f8c8d">reset / action / camera</text>
<rect x="175" y="445" width="850" height="58" rx="8" fill="#eafaf1" stroke="#a9dfbf"/>
<text x="210" y="481" font-family="Arial, sans-serif" font-size="20" font-weight="700" fill="#196f3d">推荐 MVP:</text>
<text x="318" y="481" font-family="Arial, sans-serif" font-size="20" fill="#17202a">MotrixSim 后端计算 + 20-60 Hz 状态流 + 前端 Three.js 重建渲染</text>
<defs>
<marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#34495e"/>
</marker>
<marker id="arrowGray" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#7f8c8d"/>
</marker>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 3.8 KiB

View File

@ -0,0 +1,58 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="640" viewBox="0 0 1200 640">
<rect width="1200" height="640" fill="#f8fafc"/>
<text x="60" y="70" font-family="Arial, sans-serif" font-size="34" font-weight="700" fill="#17202a">MotrixSim Web 预览方案决策矩阵</text>
<text x="60" y="108" font-family="Arial, sans-serif" font-size="18" fill="#52616f">评分 1-5,越高越适合当前目标;当前目标是复用真实 MotrixSim/Python/GPU 环境做 Web 预览。</text>
<g transform="translate(80,150)">
<rect x="0" y="0" width="1040" height="420" rx="10" fill="#ffffff" stroke="#d5dce5"/>
<rect x="0" y="0" width="1040" height="58" rx="10" fill="#eaf2f8"/>
<text x="30" y="37" font-family="Arial, sans-serif" font-size="19" font-weight="700" fill="#17202a">维度</text>
<text x="455" y="37" font-family="Arial, sans-serif" font-size="19" font-weight="700" fill="#a04000">MuJoCo WASM</text>
<text x="750" y="37" font-family="Arial, sans-serif" font-size="19" font-weight="700" fill="#196f3d">后端状态流</text>
<g font-family="Arial, sans-serif" font-size="17" fill="#17202a">
<text x="30" y="95">真实 MotrixSim 一致性</text>
<text x="515" y="95">2</text>
<text x="815" y="95">5</text>
<text x="30" y="145">现有 Python/GPU 生态复用</text>
<text x="515" y="145">1</text>
<text x="815" y="145">5</text>
<text x="30" y="195">前端离线/低服务器成本</text>
<text x="515" y="195">5</text>
<text x="815" y="195">2</text>
<text x="30" y="245">私有资产和算法保护</text>
<text x="515" y="245">1</text>
<text x="815" y="245">5</text>
<text x="30" y="295">MVP 落地速度</text>
<text x="515" y="295">3</text>
<text x="815" y="295">4</text>
<text x="30" y="345">复杂场景扩展上限</text>
<text x="515" y="345">2</text>
<text x="815" y="345">5</text>
<text x="30" y="395" font-weight="700">综合建议</text>
<text x="455" y="395" fill="#a04000">后续轻量 demo</text>
<text x="750" y="395" fill="#196f3d" font-weight="700">当前主线 MVP</text>
</g>
<g stroke="#d5dce5">
<line x1="0" y1="58" x2="1040" y2="58"/>
<line x1="0" y1="110" x2="1040" y2="110"/>
<line x1="0" y1="160" x2="1040" y2="160"/>
<line x1="0" y1="210" x2="1040" y2="210"/>
<line x1="0" y1="260" x2="1040" y2="260"/>
<line x1="0" y1="310" x2="1040" y2="310"/>
<line x1="0" y1="360" x2="1040" y2="360"/>
<line x1="390" y1="0" x2="390" y2="420"/>
<line x1="670" y1="0" x2="670" y2="420"/>
</g>
</g>
<rect x="80" y="590" width="1040" height="34" rx="6" fill="#eafaf1" stroke="#a9dfbf"/>
<text x="105" y="613" font-family="Arial, sans-serif" font-size="17" fill="#196f3d">结论:先做后端状态流,视频流作为真实画面兜底;WASM 只作为离线/公开轻量模式。</text>
</svg>

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 325 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 113 KiB

Some files were not shown because too many files have changed in this diff Show More