Initial MotrixSim Web preview productization
10
.gitignore
vendored
Normal file
@ -0,0 +1,10 @@
|
||||
node_modules/
|
||||
dist/
|
||||
.vite/
|
||||
*.log
|
||||
.DS_Store
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
/tmp.json
|
||||
test-results/
|
||||
!test-results/.gitkeep
|
||||
172
README.md
Normal file
@ -0,0 +1,172 @@
|
||||
# MotrixSim Universal Three.js Preview
|
||||
|
||||
This repository now contains a working universal-preview path:
|
||||
|
||||
```text
|
||||
MotrixSim / MJCF scene
|
||||
-> scene manifest
|
||||
-> Three.js WebGL renderer
|
||||
-> replay / polling / websocket state source
|
||||
```
|
||||
|
||||
The first verified version supports primitives, OBJ/STL/GLTF/GLB mesh loading,
|
||||
Z-up MotrixSim coordinate conversion, body-name state binding, desktop/mobile
|
||||
responsive layout, MJCF `<include>` expansion, HTTP asset serving, headless
|
||||
MotrixSim bridge controls, and basic Franka joint/gripper operation.
|
||||
|
||||
The current exporter/renderer path also covers higher-variance MJCF assets:
|
||||
nested `<default>` inheritance for visible-vs-collision geoms, `assetdir` /
|
||||
`meshdir` / `texturedir` resolution, file textures, builtin checker textures,
|
||||
mesh scale, `fromto`, `xyaxes`, `zaxis`, inline convex meshes, and browser-side
|
||||
mesh caching for repeated assets.
|
||||
|
||||
## Run Locally
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run dev -- --port 5173
|
||||
```
|
||||
|
||||
Open:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:5173/
|
||||
```
|
||||
|
||||
The first screen is now a Scene Builder. It can compose a base scene, one live
|
||||
robot manifest, and front-end primitive objects into a single Three.js preview.
|
||||
Use `Build Preview` for visual composition, or `Physical Scene` to load the
|
||||
native Franka + cube MotrixSim scene where collision is computed by the backend.
|
||||
The `G1 + Store + blocks physics` preset uses `http://127.0.0.1:18090/manifest`
|
||||
and is a backend-composed MJCF scene, so G1, store shelves/products, and demo
|
||||
objects share one MotrixSim physics world.
|
||||
When the matching bridge exposes `POST /compose`, Scene Builder object
|
||||
Add/Remove/Clear operations regenerate the combined MJCF on the backend and
|
||||
reload the live manifest. Those objects are real MotrixSim bodies with
|
||||
free joints and collision geoms, not front-end-only Three.js meshes.
|
||||
|
||||
To load a remote manifest:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest
|
||||
```
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
npm run verify:exporter
|
||||
npm run verify:preview
|
||||
```
|
||||
|
||||
For a MotrixSim bridge with only primitive objects:
|
||||
|
||||
```bash
|
||||
MIN_OBJECTS=2 MIN_MESHES=0 npm run verify:preview -- \
|
||||
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest' \
|
||||
test-results/remote-motrix
|
||||
```
|
||||
|
||||
For a larger robot scene with many mesh files, raise the verification timeout:
|
||||
|
||||
```bash
|
||||
VERIFY_TIMEOUT_MS=45000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:preview -- \
|
||||
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
|
||||
test-results/robot-panda
|
||||
```
|
||||
|
||||
For the backend control path:
|
||||
|
||||
```bash
|
||||
VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:control -- \
|
||||
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
|
||||
test-results/control-panda
|
||||
```
|
||||
|
||||
For the Franka joint/gripper control path:
|
||||
|
||||
```bash
|
||||
VERIFY_TIMEOUT_MS=70000 MIN_OBJECTS=50 MIN_MESHES=40 npm run verify:joint -- \
|
||||
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest' \
|
||||
test-results/joint-panda
|
||||
```
|
||||
|
||||
For a large textured store scene on the remote MotrixSim bridge:
|
||||
|
||||
```bash
|
||||
VERIFY_TIMEOUT_MS=120000 MIN_OBJECTS=500 MIN_MESHES=20 npm run verify:preview -- \
|
||||
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18085%2Fmanifest' \
|
||||
test-results/store-textured
|
||||
```
|
||||
|
||||
For a Go2 quadruped robot scene:
|
||||
|
||||
```bash
|
||||
VERIFY_TIMEOUT_MS=60000 MIN_OBJECTS=30 MIN_MESHES=20 npm run verify:preview -- \
|
||||
'http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest' \
|
||||
test-results/go2-robot
|
||||
```
|
||||
|
||||
## Headless MotrixSim Bridge
|
||||
|
||||
Run this on a Linux server with MotrixSim available:
|
||||
|
||||
```bash
|
||||
python server/headless_motrix_bridge.py \
|
||||
--scene /path/to/scene.xml \
|
||||
--host 127.0.0.1 \
|
||||
--port 18083 \
|
||||
--hz 120
|
||||
```
|
||||
|
||||
It exposes:
|
||||
|
||||
- `GET /manifest`: Three.js preview manifest generated from the XML scene.
|
||||
- `GET /state`: live MotrixSim body pose state.
|
||||
- `GET /assets/...`: mesh sidecar assets referenced by the manifest.
|
||||
- `POST /control`: `pause`, `resume`, `reset`, `step`, `home`,
|
||||
`set_joint`, and `gripper`.
|
||||
- `POST /compose`: for supported physical presets, generate a new combined
|
||||
MJCF scene from selected robot/scene/object choices, hot-swap the runtime,
|
||||
and return the refreshed manifest.
|
||||
- `GET /health`: service health.
|
||||
|
||||
No display server is required for this state-driven preview path.
|
||||
|
||||
## Current Boundary
|
||||
|
||||
This is a real Three.js/WebGL preview path, not the earlier Canvas 2.5D page.
|
||||
It is not yet a perfect renderer for every MotrixSim feature. Unsupported or
|
||||
private features still need explicit adapters or a server-side render/video
|
||||
fallback.
|
||||
|
||||
Scene Builder visual composition does not create a new MotrixSim physics world.
|
||||
Added front-end objects are preview-only unless they are sent through a physical
|
||||
compose endpoint. The current physical compose path supports the
|
||||
`G1 + Store + blocks physics` preset and the built-in primitive object set.
|
||||
|
||||
## 页面内上传与组装(已支持)
|
||||
|
||||
- 左侧「场景上传 / 上传场景包」支持一次选择 `.json` manifest 和关联的 mesh / texture / bin 文件。manifest 至少需要包含 `version` 与 `objects`。
|
||||
- 上传后会自动插入到场景下拉列表首位,并把 manifest 内相对路径(例如 `meshes/robot.obj`、`textures/base.png`)匹配到同批上传的本地文件,改写为浏览器可加载的 `blob:` URL。
|
||||
- 左侧「资产库 / 上传资产」支持 `.gltf/.glb/.obj/.stl`,也可以同时选择 `.bin/.png/.jpg/.webp/.ktx2/.mtl` 等 sidecar 文件;`.gltf` 内部的 buffer / image 相对引用会同步改写。
|
||||
- 点击资产卡片会把该对象加入当前场景列表,可继续调整位置(X/Y/Z 输入框)和通过左键拖拽编辑(暂停时可拖拽)。
|
||||
- 上传面板支持拖放文件;「清空自定义」会清理上传资产;「清空场景」会移除当前上传场景并回到默认场景列表。
|
||||
|
||||
边界:本地上传 mesh 默认是 Three.js 可视资产。它可以预览和摆放,但不会自动变成 MotrixSim 后端碰撞体。需要真实碰撞时,仍要通过后端 `/compose` 把对象转成 MJCF body / geom;当前后端物理组合只覆盖内置 primitive 对象。
|
||||
|
||||
The Franka Panda MJCF robot scene has been verified as a real robot asset
|
||||
preview: 59 manifest objects, 58 loaded OBJ/STL meshes, and zero mesh fallback.
|
||||
Joint-level operation is now wired for this scene through MotrixSim
|
||||
`joint_names`, `link_names`, and link pose streaming. Broader task-level robot
|
||||
actions still need scene-specific policy/action adapters.
|
||||
|
||||
Additional remote scenes verified:
|
||||
|
||||
- Store shelf scene: 569 objects, 568 loaded meshes, 17 textured materials, zero
|
||||
fallback, desktop/mobile nonblank.
|
||||
- Go2 quadruped scene: 34 objects, 33 loaded meshes, 12 joints exposed, zero
|
||||
fallback, desktop/mobile nonblank.
|
||||
- G1 + Store + demo blocks physical scene: served from bridge port 18090 with
|
||||
shared backend collision bodies for the robot, shelves, products, and demo
|
||||
primitives.
|
||||
117
docs/manifest-spec.md
Normal file
@ -0,0 +1,117 @@
|
||||
# MotrixSim Universal Three.js Manifest
|
||||
|
||||
This project uses a small scene manifest as the boundary between MotrixSim/MJCF
|
||||
assets and the browser renderer.
|
||||
|
||||
The manifest does not try to duplicate every engine feature. It carries enough
|
||||
data for a browser to load visible assets, create stable Three.js nodes, and bind
|
||||
runtime state by body name.
|
||||
|
||||
## Minimal Shape
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "0.1",
|
||||
"name": "demo",
|
||||
"coordinateSystem": { "up": "Z", "handedness": "right" },
|
||||
"materials": [
|
||||
{ "id": "ball", "color": "#4f8cff" },
|
||||
{ "id": "checker_floor", "textureKind": "checker", "textureColors": ["#334155", "#e2e8f0"], "textureRepeat": [4, 4] },
|
||||
{ "id": "label", "color": "#ffffff", "textureUrl": "/assets/label.png", "textureRepeat": [1, 1] }
|
||||
],
|
||||
"objects": [
|
||||
{
|
||||
"id": "ball_visual",
|
||||
"body": "_unnamed_link_0",
|
||||
"geometry": { "kind": "primitive", "primitive": "sphere", "radius": 0.5 },
|
||||
"material": "ball"
|
||||
},
|
||||
{
|
||||
"id": "inline_convex",
|
||||
"geometry": { "kind": "convexMesh", "vertices": [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1] },
|
||||
"material": "checker_floor"
|
||||
}
|
||||
],
|
||||
"state": { "mode": "poll", "url": "/state", "controlUrl": "/control", "pollIntervalMs": 50 },
|
||||
"controls": {
|
||||
"joints": [
|
||||
{ "name": "joint1", "index": 0, "range": [-2.8973, 2.8973] }
|
||||
],
|
||||
"actuators": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Supported Visuals
|
||||
|
||||
- Primitives: `box`, `sphere`, `capsule`, `cylinder`, `plane`, `axes`.
|
||||
- Meshes: `gltf`, `glb`, `obj`, `stl`, plus small inline convex meshes.
|
||||
- Materials: solid color, transparent color, file texture, and generated
|
||||
checker texture with repeat.
|
||||
- State binding: `object.body` maps a visual node to a runtime body name.
|
||||
- Runtime body names may be MotrixSim body names or link names. The headless
|
||||
bridge publishes link poses first, then fills non-duplicate body poses.
|
||||
- MJCF export: the current exporter expands `<include>` files, resolves mesh
|
||||
names from either `name` or file stem, resolves `assetdir` / `meshdir` /
|
||||
`texturedir`, applies nested `<default>` geom inheritance, skips
|
||||
collision-only geoms, handles `fromto`, `xyaxes`, `zaxis`, mesh `scale`, file
|
||||
textures, builtin checker textures, and emits an internal asset map for the
|
||||
headless bridge to serve `/assets/...`.
|
||||
|
||||
Repeated mesh URLs are cached by the browser renderer and cloned into the scene,
|
||||
so large scenes with hundreds of repeated OBJ assets do not refetch or reparse
|
||||
the same asset for every instance.
|
||||
|
||||
## Runtime State
|
||||
|
||||
For `state.mode = "poll"`, the browser repeatedly fetches `state.url` and
|
||||
applies the returned pose data to body-bound objects. A typical response is:
|
||||
|
||||
```json
|
||||
{
|
||||
"step": 12,
|
||||
"time": 0.012,
|
||||
"paused": true,
|
||||
"running": false,
|
||||
"link_names": ["link0", "link1", "hand"],
|
||||
"joint_names": ["joint1", "joint2"],
|
||||
"bodies": {
|
||||
"hand": {
|
||||
"pos": [0.4, 0.0, 0.77],
|
||||
"quat": [-0.88, -0.36, 0.27, 0.11]
|
||||
}
|
||||
},
|
||||
"joints": {
|
||||
"joint2": { "pos": 0.6, "vel": 0.0, "range": [-1.7628, 1.7628], "index": 1 }
|
||||
},
|
||||
"actuators": {}
|
||||
}
|
||||
```
|
||||
|
||||
## Control Endpoint
|
||||
|
||||
When `state.controlUrl` is present, the UI can POST JSON commands to the
|
||||
bridge. The current bridge supports:
|
||||
|
||||
- `{"action": "pause"}` / `{"action": "resume"}` / `{"action": "toggle"}`
|
||||
- `{"action": "step"}` for one paused physics step
|
||||
- `{"action": "reset"}` for engine reset
|
||||
- `{"action": "home"}` for zeroing exposed joints
|
||||
- `{"action": "set_joint", "joint": "joint2", "value": 0.6}`
|
||||
- `{"action": "set_joints", "values": {"joint1": 0, "joint2": 0.6}}`
|
||||
- `{"action": "gripper", "value": 0.04}` for known finger joints
|
||||
|
||||
The bridge returns the same frame shape as `/state`, so the frontend can update
|
||||
the renderer and the control panel from one response.
|
||||
|
||||
## Coordinate Policy
|
||||
|
||||
MotrixSim/MJCF scenes are assumed to be Z-up. Three.js is Y-up. The renderer
|
||||
converts positions, rotations, primitive dimensions, and loaded mesh local
|
||||
coordinate bases when `coordinateSystem.up` is `Z`.
|
||||
|
||||
## Security Boundary
|
||||
|
||||
If the browser renders the real mesh, that mesh is downloaded to the browser.
|
||||
Private assets that must not leave the server should use a server-side video or
|
||||
image stream instead of Three.js mesh delivery.
|
||||
13
index.html
Normal file
@ -0,0 +1,13 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>MotrixSim Universal Three.js Preview</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="/src/main.ts"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
1171
package-lock.json
generated
Normal file
28
package.json
Normal file
@ -0,0 +1,28 @@
|
||||
{
|
||||
"name": "motrixsim-web",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite --host 127.0.0.1",
|
||||
"build": "tsc && vite build",
|
||||
"preview": "vite preview --host 127.0.0.1",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"verify:preview": "node scripts/verify-universal-preview.mjs",
|
||||
"verify:control": "node scripts/verify-control-preview.mjs",
|
||||
"verify:joint": "node scripts/verify-joint-control.mjs",
|
||||
"verify:motrix-input": "node scripts/verify-motrix-input.mjs",
|
||||
"verify:go2-policy": "node scripts/verify-go2-policy-motion.mjs",
|
||||
"verify:exporter": "python3 scripts/verify_exporter_features.py",
|
||||
"acceptance:probe": "node scripts/acceptance-probe.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"three": "^0.181.2",
|
||||
"vite": "^7.3.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/three": "^0.184.1",
|
||||
"playwright": "^1.61.0",
|
||||
"typescript": "^5.9.3"
|
||||
}
|
||||
}
|
||||
5546
public/generated/g1-store-blocks-lite.manifest.json
Normal file
17467
public/generated/g1-store-blocks.manifest.json
Normal file
12
public/samples/assets/pyramid.obj
Normal file
@ -0,0 +1,12 @@
|
||||
o unknown_mesh_marker
|
||||
v 0.000 0.000 0.700
|
||||
v -0.600 -0.600 0.000
|
||||
v 0.600 -0.600 0.000
|
||||
v 0.600 0.600 0.000
|
||||
v -0.600 0.600 0.000
|
||||
f 1 2 3
|
||||
f 1 3 4
|
||||
f 1 4 5
|
||||
f 1 5 2
|
||||
f 2 5 4 3
|
||||
|
||||
18
public/samples/universal-demo-state.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"frames": [
|
||||
{ "step": 0, "time": 0.00, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 4.98], "quat": [0, 0, 0, 1] } } },
|
||||
{ "step": 20, "time": 0.16, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 4.22], "quat": [0, 0, 0.02, 0.9998] } } },
|
||||
{ "step": 40, "time": 0.32, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 3.02], "quat": [0, 0, 0.04, 0.9992] } } },
|
||||
{ "step": 60, "time": 0.48, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 1.42], "quat": [0, 0, 0.06, 0.9982] } } },
|
||||
{ "step": 80, "time": 0.64, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.52], "quat": [0, 0, 0.08, 0.9968] } } },
|
||||
{ "step": 100, "time": 0.80, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.70], "quat": [0, 0, 0.10, 0.9950] } } },
|
||||
{ "step": 120, "time": 0.96, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.50], "quat": [0, 0, 0.12, 0.9928] } } },
|
||||
{ "step": 140, "time": 1.12, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.50], "quat": [0, 0, 0.14, 0.9902] } } },
|
||||
{ "step": 160, "time": 1.28, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 4.95], "quat": [0, 0, 0.16, 0.9871] } } },
|
||||
{ "step": 180, "time": 1.44, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 3.78], "quat": [0, 0, 0.18, 0.9837] } } },
|
||||
{ "step": 200, "time": 1.60, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 2.18], "quat": [0, 0, 0.20, 0.9798] } } },
|
||||
{ "step": 220, "time": 1.76, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.66], "quat": [0, 0, 0.22, 0.9755] } } },
|
||||
{ "step": 240, "time": 1.92, "bodies": { "_unnamed_link_0": { "pos": [0, 0, 0.50], "quat": [0, 0, 0.24, 0.9708] } } }
|
||||
]
|
||||
}
|
||||
|
||||
73
public/samples/universal-demo.manifest.json
Normal file
@ -0,0 +1,73 @@
|
||||
{
|
||||
"version": "0.1",
|
||||
"name": "Universal Three.js Preview Demo",
|
||||
"description": "Motrix-style Z-up scene manifest with primitives, mesh loading, and replayed body state.",
|
||||
"units": "m",
|
||||
"coordinateSystem": { "up": "Z", "handedness": "right" },
|
||||
"materials": [
|
||||
{ "id": "floor", "color": "#263449", "roughness": 0.92 },
|
||||
{ "id": "ball", "color": "#4f8cff", "roughness": 0.36, "metalness": 0.05 },
|
||||
{ "id": "capsule", "color": "#f59e0b", "roughness": 0.48 },
|
||||
{ "id": "mesh", "color": "#a7f3d0", "roughness": 0.62 },
|
||||
{ "id": "fallback", "color": "#fb7185", "wireframe": true }
|
||||
],
|
||||
"objects": [
|
||||
{
|
||||
"id": "floor",
|
||||
"name": "factory_floor",
|
||||
"geometry": { "kind": "primitive", "primitive": "box", "size": [9, 9, 0.08] },
|
||||
"pose": { "position": [0, 0, -0.04] },
|
||||
"material": "floor",
|
||||
"receiveShadow": true
|
||||
},
|
||||
{
|
||||
"id": "falling_ball",
|
||||
"name": "MotrixSim falling ball body",
|
||||
"body": "_unnamed_link_0",
|
||||
"geometry": { "kind": "primitive", "primitive": "sphere", "radius": 0.5, "segments": 48 },
|
||||
"material": "ball",
|
||||
"castShadow": true
|
||||
},
|
||||
{
|
||||
"id": "capsule_demo",
|
||||
"name": "capsule primitive support",
|
||||
"geometry": { "kind": "primitive", "primitive": "capsule", "radius": 0.22, "length": 1.4, "segments": 24 },
|
||||
"pose": { "position": [-2.8, 1.6, 0.75], "quaternion": [0, 0, 0, 1] },
|
||||
"material": "capsule",
|
||||
"castShadow": true
|
||||
},
|
||||
{
|
||||
"id": "obj_mesh_demo",
|
||||
"name": "OBJ mesh asset support",
|
||||
"geometry": { "kind": "mesh", "url": "/samples/assets/pyramid.obj", "format": "obj" },
|
||||
"pose": { "position": [2.6, -1.8, 0.02], "scale": [1.1, 1.1, 1.1] },
|
||||
"material": "mesh",
|
||||
"castShadow": true
|
||||
},
|
||||
{
|
||||
"id": "axes",
|
||||
"name": "scene axes",
|
||||
"geometry": { "kind": "primitive", "primitive": "axes", "size": [1.4, 1.4, 1.4] },
|
||||
"pose": { "position": [-4.1, -4.1, 0.03] }
|
||||
}
|
||||
],
|
||||
"lights": [
|
||||
{ "type": "hemisphere", "color": "#dbeafe", "intensity": 0.65 },
|
||||
{ "type": "directional", "color": "#ffffff", "intensity": 2.4, "position": [4, -6, 7] },
|
||||
{ "type": "ambient", "color": "#9fb4d1", "intensity": 0.22 }
|
||||
],
|
||||
"camera": {
|
||||
"position": [5.5, -7.2, 4.6],
|
||||
"target": [0, 0, 1.2],
|
||||
"fov": 46,
|
||||
"near": 0.05,
|
||||
"far": 200
|
||||
},
|
||||
"state": {
|
||||
"mode": "replay",
|
||||
"url": "/samples/universal-demo-state.json",
|
||||
"fps": 12,
|
||||
"loop": true
|
||||
}
|
||||
}
|
||||
|
||||
49
research-tests/2026-06-22-motrixsim-web-preview/README.md
Normal file
@ -0,0 +1,49 @@
|
||||
# MotrixSim Web Preview Research Test
|
||||
|
||||
This folder contains the 2026-06-22 investigation for MotrixSim Web preview architecture.
|
||||
|
||||
## Reports
|
||||
|
||||
- `research-report.md`: test evidence and implementation recommendation.
|
||||
- `comparison-report.md`: WASM vs backend streaming comparison.
|
||||
- `motrixsim-operation-consistency-report.md`: MotrixSim-style keyboard, mouse camera, actuator, pause/reset operation verification.
|
||||
- `evidence/`: latest Go2 and Franka screenshots plus JSON evidence for MotrixSim operation consistency.
|
||||
- `figures/real-web-preview/motrix-live-web-preview.jpg`: real browser screenshot of the MotrixSim Web preview.
|
||||
- `figures/real-web-preview/motrix-live-web-preview-recording.mp4`: simple screen recording generated from browser frames.
|
||||
- `figures/real-web-preview/motrix-live-web-preview-recording.gif`: lightweight animated preview for inline reports.
|
||||
- `figures/real-web-preview/universal-three-preview-sample-desktop.jpg`: real Three.js WebGL sample preview, desktop viewport.
|
||||
- `figures/real-web-preview/universal-three-preview-sample-mobile.jpg`: real Three.js WebGL sample preview, mobile viewport.
|
||||
- `figures/real-web-preview/universal-three-preview-remote-motrix-desktop.jpg`: real MotrixSim bridge scene rendered by Three.js, desktop viewport.
|
||||
- `figures/real-web-preview/universal-three-preview-remote-motrix-mobile.jpg`: real MotrixSim bridge scene rendered by Three.js, mobile viewport.
|
||||
- `figures/real-web-preview/universal-three-preview-robot-panda-desktop.jpg`: real Franka Panda robot MJCF scene rendered by Three.js, desktop viewport.
|
||||
- `figures/real-web-preview/universal-three-preview-robot-panda-mobile.jpg`: real Franka Panda robot MJCF scene rendered by Three.js, mobile viewport.
|
||||
- `figures/real-web-preview/universal-three-preview-control-desktop.jpg`: real backend pause/step/reset/resume control verification.
|
||||
- `figures/real-web-preview/universal-three-preview-joint-control-desktop.jpg`: real Franka joint and gripper control verification.
|
||||
- `figures/real-web-preview/universal-three-preview-store-textured-desktop.jpg`: real Store scene with OBJ/PNG textured assets.
|
||||
- `figures/real-web-preview/universal-three-preview-store-textured-mobile.jpg`: Store scene mobile viewport verification.
|
||||
- `figures/real-web-preview/universal-three-preview-go2-robot-desktop.jpg`: real Go2 quadruped robot scene.
|
||||
- `figures/real-web-preview/universal-three-preview-go2-robot-mobile.jpg`: Go2 mobile viewport verification.
|
||||
- `artifacts/motrix_live_web_preview_browser_evidence_2026-06-22.json`: DOM/state evidence captured from the browser page.
|
||||
- `artifacts/motrix_live_web_preview_recording_evidence_2026-06-22.json`: frame-level state evidence for the recording.
|
||||
- `artifacts/universal_three_preview_sample_evidence_2026-06-22.json`: Playwright evidence for the bundled universal Three.js sample.
|
||||
- `artifacts/universal_three_preview_remote_motrix_evidence_2026-06-22.json`: Playwright evidence for the remote MotrixSim bridge scene.
|
||||
- `artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json`: Playwright evidence for the real Franka Panda robot scene.
|
||||
- `artifacts/universal_three_preview_control_evidence_2026-06-22.json`: Playwright evidence for pause/step/reset/resume control.
|
||||
- `artifacts/universal_three_preview_joint_control_evidence_2026-06-22.json`: Playwright evidence for joint2 and gripper control.
|
||||
- `artifacts/universal_three_preview_store_textured_evidence_2026-06-22.json`: Playwright evidence for Store textured scene.
|
||||
- `artifacts/universal_three_preview_go2_robot_evidence_2026-06-22.json`: Playwright evidence for Go2 quadruped scene.
|
||||
- `figures/test-screenshots/`: PNG screenshots generated from the real test artifacts.
|
||||
- Feishu Wiki: https://horizonrobotics.feishu.cn/wiki/UKxEwcYjwiuSapkDeaXcqRlUnVd
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
bash scripts/wuying_probe.sh
|
||||
scp scripts/motrix_state_stream_probe.py root@39.106.101.95:/tmp/motrix_state_stream_probe.py
|
||||
scp scripts/motrix_live_web_preview.py root@39.106.101.95:/tmp/motrix_live_web_preview.py
|
||||
ssh root@39.106.101.95 '/opt/gs_playground_real2sim_48q/.venv/bin/python /tmp/motrix_live_web_preview.py --host 127.0.0.1 --port 18081'
|
||||
ssh -L 18081:127.0.0.1:18081 root@39.106.101.95
|
||||
cd web-wasm-smoke && npm install && node smoke.mjs
|
||||
```
|
||||
|
||||
The Wuying SSH key is intentionally referenced from the private document repo and is not copied into this project.
|
||||
@ -0,0 +1,52 @@
|
||||
{
|
||||
"ok": true,
|
||||
"repo": "/Users/d-robotics/workSpace/motrix-lerobot",
|
||||
"files": {
|
||||
"readme": "/Users/d-robotics/workSpace/motrix-lerobot/Readme.md",
|
||||
"env": "/Users/d-robotics/workSpace/motrix-lerobot/motrix_lerobot/env.py",
|
||||
"runner": "/Users/d-robotics/workSpace/motrix-lerobot/scripts/runner.py",
|
||||
"so101_scene": "/Users/d-robotics/workSpace/motrix-lerobot/assets/SO101/scene_with_ball.xml",
|
||||
"g1_scene": "/Users/d-robotics/workSpace/motrix-lerobot/assets/g1/scene_real.xml"
|
||||
},
|
||||
"findings": [
|
||||
{
|
||||
"check": "imports_motrixsim",
|
||||
"status": "pass"
|
||||
},
|
||||
{
|
||||
"check": "loads_so101_scene",
|
||||
"status": "pass"
|
||||
},
|
||||
{
|
||||
"check": "has_render_app",
|
||||
"status": "pass"
|
||||
},
|
||||
{
|
||||
"check": "captures_two_cameras",
|
||||
"status": "pass"
|
||||
},
|
||||
{
|
||||
"check": "extracts_joint_state",
|
||||
"status": "pass"
|
||||
},
|
||||
{
|
||||
"check": "steps_model",
|
||||
"status": "pass"
|
||||
}
|
||||
],
|
||||
"checks": {
|
||||
"imports_motrixsim": true,
|
||||
"loads_so101_scene": true,
|
||||
"has_render_app": true,
|
||||
"captures_two_cameras": true,
|
||||
"extracts_joint_state": true,
|
||||
"steps_model": true
|
||||
},
|
||||
"exists": {
|
||||
"readme": true,
|
||||
"env": true,
|
||||
"runner": true,
|
||||
"so101_scene": true,
|
||||
"g1_scene": true
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,109 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T04:43:10.882Z",
|
||||
"url": "http://127.0.0.1:18081/",
|
||||
"remote_host": "39.106.101.95",
|
||||
"screenshot": "/Users/d-robotics/workSpace/MotrixSimWeb/research-tests/2026-06-22-motrixsim-web-preview/figures/real-web-preview/motrix-live-web-preview.jpg",
|
||||
"selected_state": {
|
||||
"bodyName": "_unnamed_link_0",
|
||||
"canvas": {
|
||||
"clientHeight": 720,
|
||||
"clientWidth": 920,
|
||||
"height": 1440,
|
||||
"width": 1840
|
||||
},
|
||||
"connection": "connected",
|
||||
"heading": "Real Runtime",
|
||||
"posText": "0.0000, 0.0000, 4.6429",
|
||||
"state": {
|
||||
"bodies": {
|
||||
"_unnamed_link_0": {
|
||||
"pos": [
|
||||
0,
|
||||
0,
|
||||
4.642916202545166
|
||||
],
|
||||
"quat": [
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
1
|
||||
]
|
||||
}
|
||||
},
|
||||
"body_names": [
|
||||
"_unnamed_link_0"
|
||||
],
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"ok": true,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
|
||||
"server_time": 1782103390.8206172,
|
||||
"step": 3584,
|
||||
"step_ms": {
|
||||
"max": 0.17769599799066782,
|
||||
"mean": 0.06107135443814252,
|
||||
"p95": 0.09495299309492111
|
||||
}
|
||||
},
|
||||
"stepText": "3584",
|
||||
"title": "MotrixSim Live Web Preview",
|
||||
"z": 4.642916202545166
|
||||
},
|
||||
"page_state_samples": [
|
||||
{
|
||||
"step": 3584,
|
||||
"z": 4.642916202545166,
|
||||
"posText": "0.0000, 0.0000, 4.6429",
|
||||
"connection": "connected"
|
||||
}
|
||||
],
|
||||
"verification": {
|
||||
"source": "browser DOM raw live state rendered from /state endpoint",
|
||||
"requirement": "real MotrixSim Web preview screenshot, not terminal output or smoke placeholder",
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml"
|
||||
},
|
||||
"post_screenshot_dom_state": {
|
||||
"captured_at": "2026-06-22T04:43:26.711Z",
|
||||
"parsed_state": {
|
||||
"bodies": {
|
||||
"_unnamed_link_0": {
|
||||
"pos": [
|
||||
0,
|
||||
0,
|
||||
4.9882283210754395
|
||||
],
|
||||
"quat": [
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
1
|
||||
]
|
||||
}
|
||||
},
|
||||
"body_names": [
|
||||
"_unnamed_link_0"
|
||||
],
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"ok": true,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
|
||||
"server_time": 1782103406.7485473,
|
||||
"step": 5477,
|
||||
"step_ms": {
|
||||
"max": 0.2323179505765438,
|
||||
"mean": 0.06990823318024558,
|
||||
"p95": 0.12352498015388846
|
||||
}
|
||||
},
|
||||
"visible": {
|
||||
"body": "_unnamed_link_0",
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"mean_step_ms": "0.06991",
|
||||
"position_xyz": "0.0000, 0.0000, 4.9882",
|
||||
"runtime_heading": "Real Runtime",
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
|
||||
"server_step": "5477",
|
||||
"title": "MotrixSim Web Preview"
|
||||
},
|
||||
"z": 4.9882283210754395
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,589 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T07:30:38.135Z",
|
||||
"url": "http://127.0.0.1:18081/",
|
||||
"remote_host": "39.106.101.95",
|
||||
"frame_count": 48,
|
||||
"frame_interval_ms": 100,
|
||||
"frame_dir": "/Users/d-robotics/workSpace/MotrixSimWeb/research-tests/2026-06-22-motrixsim-web-preview/figures/real-web-preview/recording-frames",
|
||||
"samples": [
|
||||
{
|
||||
"frame": 0,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113428.8410916,
|
||||
"step": 1196860,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1196860",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 1,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113429.0350401,
|
||||
"step": 1196883,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1196883",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 2,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113429.3060741,
|
||||
"step": 1196915,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1196915",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 3,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.8234",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113429.5065846,
|
||||
"step": 1196939,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1196939",
|
||||
"z": 4.82342004776001
|
||||
},
|
||||
{
|
||||
"frame": 4,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 3.2931",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113429.6738975,
|
||||
"step": 1196959,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1196959",
|
||||
"z": 3.293060541152954
|
||||
},
|
||||
{
|
||||
"frame": 5,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.3607",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113429.846431,
|
||||
"step": 1196979,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1196979",
|
||||
"z": 0.36073562502861023
|
||||
},
|
||||
{
|
||||
"frame": 6,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4773",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113430.0658035,
|
||||
"step": 1197005,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197005",
|
||||
"z": 0.47731032967567444
|
||||
},
|
||||
{
|
||||
"frame": 7,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113430.2270908,
|
||||
"step": 1197025,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197025",
|
||||
"z": 0.49963366985321045
|
||||
},
|
||||
{
|
||||
"frame": 8,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113430.4802213,
|
||||
"step": 1197055,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197055",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 9,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113430.614984,
|
||||
"step": 1197071,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197071",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 10,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113430.7318132,
|
||||
"step": 1197085,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197085",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 11,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113430.9516282,
|
||||
"step": 1197111,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197111",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 12,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113431.236581,
|
||||
"step": 1197145,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197145",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 13,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113431.428369,
|
||||
"step": 1197167,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197167",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 14,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.3290",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113431.5858905,
|
||||
"step": 1197186,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197186",
|
||||
"z": 4.328996181488037
|
||||
},
|
||||
{
|
||||
"frame": 15,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.5737",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113431.827208,
|
||||
"step": 1197215,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197215",
|
||||
"z": 0.5737295150756836
|
||||
},
|
||||
{
|
||||
"frame": 16,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.6533",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113431.9960675,
|
||||
"step": 1197235,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197235",
|
||||
"z": 0.6532669067382812
|
||||
},
|
||||
{
|
||||
"frame": 17,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113432.225468,
|
||||
"step": 1197262,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197262",
|
||||
"z": 0.49963316321372986
|
||||
},
|
||||
{
|
||||
"frame": 18,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113432.3864737,
|
||||
"step": 1197281,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197281",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 19,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113432.6327212,
|
||||
"step": 1197311,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197311",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 20,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113432.7867239,
|
||||
"step": 1197329,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197329",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 21,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113433.0273411,
|
||||
"step": 1197358,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197358",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 22,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113433.1892936,
|
||||
"step": 1197377,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197377",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 23,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113433.3487034,
|
||||
"step": 1197396,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197396",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 24,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.3290",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113433.587853,
|
||||
"step": 1197424,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197424",
|
||||
"z": 4.328996181488037
|
||||
},
|
||||
{
|
||||
"frame": 25,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 2.0923",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113433.755578,
|
||||
"step": 1197444,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197444",
|
||||
"z": 2.0923168659210205
|
||||
},
|
||||
{
|
||||
"frame": 26,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.6697",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113433.9922316,
|
||||
"step": 1197472,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197472",
|
||||
"z": 0.6696948409080505
|
||||
},
|
||||
{
|
||||
"frame": 27,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4997",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113434.1470935,
|
||||
"step": 1197491,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197491",
|
||||
"z": 0.49972307682037354
|
||||
},
|
||||
{
|
||||
"frame": 28,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113434.3138952,
|
||||
"step": 1197511,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197511",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 29,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113434.4939082,
|
||||
"step": 1197532,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197532",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 30,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113434.6086948,
|
||||
"step": 1197546,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197546",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 31,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113434.9620519,
|
||||
"step": 1197588,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197588",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 32,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113435.1847568,
|
||||
"step": 1197614,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197614",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 33,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113435.347539,
|
||||
"step": 1197634,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197634",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 34,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.8587",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113435.504579,
|
||||
"step": 1197652,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197652",
|
||||
"z": 4.858736038208008
|
||||
},
|
||||
{
|
||||
"frame": 35,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 3.4069",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113435.6657073,
|
||||
"step": 1197672,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197672",
|
||||
"z": 3.4068565368652344
|
||||
},
|
||||
{
|
||||
"frame": 36,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.6182",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113435.9040775,
|
||||
"step": 1197700,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197700",
|
||||
"z": 0.6181541681289673
|
||||
},
|
||||
{
|
||||
"frame": 37,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4879",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113436.0745072,
|
||||
"step": 1197720,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197720",
|
||||
"z": 0.4878646731376648
|
||||
},
|
||||
{
|
||||
"frame": 38,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113436.3046973,
|
||||
"step": 1197748,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197748",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 39,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113436.468162,
|
||||
"step": 1197767,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197767",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 40,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113436.7040653,
|
||||
"step": 1197795,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197795",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 41,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113436.8651109,
|
||||
"step": 1197814,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197814",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 42,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113437.1052198,
|
||||
"step": 1197843,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197843",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 43,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 0.4996",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113437.2693567,
|
||||
"step": 1197862,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197862",
|
||||
"z": 0.4996328353881836
|
||||
},
|
||||
{
|
||||
"frame": 44,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.9765",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113437.471499,
|
||||
"step": 1197886,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197886",
|
||||
"z": 4.976456165313721
|
||||
},
|
||||
{
|
||||
"frame": 45,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.5291",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113437.5684204,
|
||||
"step": 1197898,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197898",
|
||||
"z": 4.529120445251465
|
||||
},
|
||||
{
|
||||
"frame": 46,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.5291",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113437.5684204,
|
||||
"step": 1197898,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197898",
|
||||
"z": 4.529120445251465
|
||||
},
|
||||
{
|
||||
"frame": 47,
|
||||
"connection": "connected",
|
||||
"host": "1nh2jhpso5kckmx",
|
||||
"posText": "0.0000, 0.0000, 4.5291",
|
||||
"runtimeHeading": "Real Runtime",
|
||||
"serverTime": 1782113437.5684204,
|
||||
"step": 1197898,
|
||||
"title": "MotrixSim Web Preview",
|
||||
"visibleStep": "1197898",
|
||||
"z": 4.529120445251465
|
||||
}
|
||||
],
|
||||
"z_min": 0.36073562502861023,
|
||||
"z_max": 4.976456165313721,
|
||||
"explanation": "Frames captured from the actual browser page connected to remote MotrixSim /state, then converted to simple recording assets."
|
||||
}
|
||||
@ -0,0 +1,50 @@
|
||||
{
|
||||
"ok": true,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml",
|
||||
"steps": 240,
|
||||
"body_names": [
|
||||
"_unnamed_link_0"
|
||||
],
|
||||
"sample_count": 30,
|
||||
"step_ms": {
|
||||
"mean": 0.014092,
|
||||
"p95": 0.014621,
|
||||
"max": 0.132174
|
||||
},
|
||||
"first_sample": {
|
||||
"step": 0,
|
||||
"bodies": {
|
||||
"_unnamed_link_0": {
|
||||
"pos": [
|
||||
0.0,
|
||||
0.0,
|
||||
5.0
|
||||
],
|
||||
"quat": [
|
||||
-0.0,
|
||||
-0.0,
|
||||
-0.0,
|
||||
1.0
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"last_sample": {
|
||||
"step": 232,
|
||||
"bodies": {
|
||||
"_unnamed_link_0": {
|
||||
"pos": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.499633
|
||||
],
|
||||
"quat": [
|
||||
-0.0,
|
||||
-0.0,
|
||||
-0.0,
|
||||
1.0
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,27 @@
|
||||
{
|
||||
"ok": true,
|
||||
"package": "@mujoco/mujoco",
|
||||
"packageVersion": "3.9.0",
|
||||
"node": "v23.9.0",
|
||||
"model": {
|
||||
"nq": 7,
|
||||
"nv": 6,
|
||||
"nbody": 2,
|
||||
"ngeom": 2
|
||||
},
|
||||
"timingMs": {
|
||||
"wasmLoad": 55.728,
|
||||
"modelCompile": 21.673,
|
||||
"totalStep": 25.15,
|
||||
"perStep": 0.012575
|
||||
},
|
||||
"qposHead": [
|
||||
0,
|
||||
0,
|
||||
0.049633,
|
||||
1,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,53 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T09:29:02.013Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
|
||||
"pause": {
|
||||
"step": 0,
|
||||
"paused": "true",
|
||||
"control": "control: pause ok",
|
||||
"meshes": "58",
|
||||
"objects": "59",
|
||||
"stateStatus": "state: poll connected"
|
||||
},
|
||||
"pauseLater": {
|
||||
"step": 0,
|
||||
"paused": "true",
|
||||
"control": "control: pause ok",
|
||||
"meshes": "58",
|
||||
"objects": "59",
|
||||
"stateStatus": "state: poll connected"
|
||||
},
|
||||
"step": {
|
||||
"step": 1,
|
||||
"paused": "true",
|
||||
"control": "control: step ok",
|
||||
"meshes": "58",
|
||||
"objects": "59",
|
||||
"stateStatus": "state: poll connected"
|
||||
},
|
||||
"reset": {
|
||||
"step": 0,
|
||||
"paused": "true",
|
||||
"control": "control: reset ok",
|
||||
"meshes": "58",
|
||||
"objects": "59",
|
||||
"stateStatus": "state: poll connected"
|
||||
},
|
||||
"resume": {
|
||||
"step": 8,
|
||||
"paused": "false",
|
||||
"control": "control: resume ok",
|
||||
"meshes": "58",
|
||||
"objects": "59",
|
||||
"stateStatus": "state: poll connected"
|
||||
},
|
||||
"verification": {
|
||||
"pause_stopped": true,
|
||||
"step_once_incremented": true,
|
||||
"reset_zero": true,
|
||||
"resume_advanced": true,
|
||||
"min_objects": 50,
|
||||
"min_meshes": 40,
|
||||
"timeout_ms": 60000
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,83 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T10:16:38.826Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest",
|
||||
"desktop": {
|
||||
"before": {
|
||||
"step": "2551",
|
||||
"pixels": "30"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 34 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "scene_flat.xml",
|
||||
"objects": "34",
|
||||
"bindings": "33",
|
||||
"meshes": "33",
|
||||
"fallbacks": "0",
|
||||
"step": "2551",
|
||||
"pixels": "30",
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"canvas": {
|
||||
"width": 920,
|
||||
"height": 720,
|
||||
"clientWidth": 920,
|
||||
"clientHeight": 720,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "360px 920px",
|
||||
"shellRows": "720px"
|
||||
}
|
||||
},
|
||||
"mobile": {
|
||||
"before": {
|
||||
"step": "2551",
|
||||
"pixels": "30"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 34 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "scene_flat.xml",
|
||||
"objects": "34",
|
||||
"bindings": "33",
|
||||
"meshes": "33",
|
||||
"fallbacks": "0",
|
||||
"step": "2551",
|
||||
"pixels": "30",
|
||||
"viewport": {
|
||||
"width": 390,
|
||||
"height": 844
|
||||
},
|
||||
"canvas": {
|
||||
"width": 780,
|
||||
"height": 1040,
|
||||
"clientWidth": 390,
|
||||
"clientHeight": 520,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "390px",
|
||||
"shellRows": "1234.59px 520px"
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"desktop_nonblank": true,
|
||||
"desktop_mesh_requirement_met": true,
|
||||
"desktop_replay_moved": false,
|
||||
"mobile_actual_viewport": true,
|
||||
"mobile_nonblank": true,
|
||||
"mobile_mesh_requirement_met": true,
|
||||
"mobile_replay_moved": false,
|
||||
"mobile_stacked_layout": true,
|
||||
"min_objects": 30,
|
||||
"min_meshes": 20,
|
||||
"timeout_ms": 60000
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,93 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T09:37:26.294Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
|
||||
"joint_name": "joint2",
|
||||
"joint_target": 0.6,
|
||||
"dom": {
|
||||
"objects": "59",
|
||||
"meshes": "58",
|
||||
"paused": "true",
|
||||
"control": "control: gripper ok",
|
||||
"jointStatus": "joints: gripper open",
|
||||
"selectedJoint": "joint2",
|
||||
"jointValue": "0.60"
|
||||
},
|
||||
"before": {
|
||||
"paused": true,
|
||||
"joint": {
|
||||
"pos": 0,
|
||||
"vel": 0,
|
||||
"range": [
|
||||
-1.7627999782562256,
|
||||
1.7627999782562256
|
||||
],
|
||||
"index": 1
|
||||
},
|
||||
"hand": {
|
||||
"pos": [
|
||||
0.08799997717142105,
|
||||
0,
|
||||
0.9260001182556152
|
||||
],
|
||||
"quat": [
|
||||
-0.9238792657852173,
|
||||
-0.3826832175254822,
|
||||
0,
|
||||
0
|
||||
]
|
||||
}
|
||||
},
|
||||
"after_joint": {
|
||||
"paused": true,
|
||||
"joint": {
|
||||
"pos": 0.6000000238418579,
|
||||
"vel": 0,
|
||||
"range": [
|
||||
-1.7627999782562256,
|
||||
1.7627999782562256
|
||||
],
|
||||
"index": 1
|
||||
},
|
||||
"hand": {
|
||||
"pos": [
|
||||
0.4074625074863434,
|
||||
0,
|
||||
0.7727354764938354
|
||||
],
|
||||
"quat": [
|
||||
-0.8826155662536621,
|
||||
-0.36559122800827026,
|
||||
0.27302485704421997,
|
||||
0.11309058964252472
|
||||
]
|
||||
}
|
||||
},
|
||||
"after_gripper": {
|
||||
"finger_joint1": {
|
||||
"pos": 0.03999999910593033,
|
||||
"vel": 0,
|
||||
"range": [
|
||||
0,
|
||||
0.03999999910593033
|
||||
],
|
||||
"index": 7
|
||||
},
|
||||
"finger_joint2": {
|
||||
"pos": 0.03999999910593033,
|
||||
"vel": 0,
|
||||
"range": [
|
||||
0,
|
||||
0.03999999910593033
|
||||
],
|
||||
"index": 8
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"joint_value_applied": true,
|
||||
"hand_pose_changed": true,
|
||||
"gripper_opened": true,
|
||||
"min_objects": 50,
|
||||
"min_meshes": 40,
|
||||
"timeout_ms": 70000
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,80 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T08:48:03.112Z",
|
||||
"url": "http://127.0.0.1:5173/",
|
||||
"desktop": {
|
||||
"before": {
|
||||
"step": "120",
|
||||
"pixels": "4"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 5 objects",
|
||||
"stateStatus": "state: replay 13 frames @ 12 fps",
|
||||
"manifest": "Universal Three.js Preview Demo",
|
||||
"objects": "5",
|
||||
"bindings": "1",
|
||||
"meshes": "1",
|
||||
"fallbacks": "0",
|
||||
"step": "160",
|
||||
"pixels": "4",
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"canvas": {
|
||||
"width": 920,
|
||||
"height": 720,
|
||||
"clientWidth": 920,
|
||||
"clientHeight": 720,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "360px 920px",
|
||||
"shellRows": "720px"
|
||||
}
|
||||
},
|
||||
"mobile": {
|
||||
"before": {
|
||||
"step": "100",
|
||||
"pixels": "4"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 5 objects",
|
||||
"stateStatus": "state: replay 13 frames @ 12 fps",
|
||||
"manifest": "Universal Three.js Preview Demo",
|
||||
"objects": "5",
|
||||
"bindings": "1",
|
||||
"meshes": "1",
|
||||
"fallbacks": "0",
|
||||
"step": "60",
|
||||
"pixels": "4",
|
||||
"viewport": {
|
||||
"width": 390,
|
||||
"height": 844
|
||||
},
|
||||
"canvas": {
|
||||
"width": 780,
|
||||
"height": 1040,
|
||||
"clientWidth": 390,
|
||||
"clientHeight": 520,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "390px",
|
||||
"shellRows": "710.219px 520px"
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"desktop_nonblank": true,
|
||||
"desktop_mesh_loaded": true,
|
||||
"desktop_replay_moved": true,
|
||||
"mobile_actual_viewport": true,
|
||||
"mobile_nonblank": true,
|
||||
"mobile_mesh_loaded": true,
|
||||
"mobile_replay_moved": true,
|
||||
"mobile_stacked_layout": true
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,82 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T08:55:18.655Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18083%2Fmanifest",
|
||||
"desktop": {
|
||||
"before": {
|
||||
"step": "19078",
|
||||
"pixels": "4"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 2 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "falling_ball.xml",
|
||||
"objects": "2",
|
||||
"bindings": "1",
|
||||
"meshes": "0",
|
||||
"fallbacks": "0",
|
||||
"step": "19253",
|
||||
"pixels": "4",
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"canvas": {
|
||||
"width": 920,
|
||||
"height": 720,
|
||||
"clientWidth": 920,
|
||||
"clientHeight": 720,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "360px 920px",
|
||||
"shellRows": "720px"
|
||||
}
|
||||
},
|
||||
"mobile": {
|
||||
"before": {
|
||||
"step": "19421",
|
||||
"pixels": "4"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 2 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "falling_ball.xml",
|
||||
"objects": "2",
|
||||
"bindings": "1",
|
||||
"meshes": "0",
|
||||
"fallbacks": "0",
|
||||
"step": "19668",
|
||||
"pixels": "4",
|
||||
"viewport": {
|
||||
"width": 390,
|
||||
"height": 844
|
||||
},
|
||||
"canvas": {
|
||||
"width": 780,
|
||||
"height": 1040,
|
||||
"clientWidth": 390,
|
||||
"clientHeight": 520,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "390px",
|
||||
"shellRows": "696.219px 520px"
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"desktop_nonblank": true,
|
||||
"desktop_mesh_requirement_met": true,
|
||||
"desktop_replay_moved": true,
|
||||
"mobile_actual_viewport": true,
|
||||
"mobile_nonblank": true,
|
||||
"mobile_mesh_requirement_met": true,
|
||||
"mobile_replay_moved": true,
|
||||
"mobile_stacked_layout": true,
|
||||
"min_objects": 2,
|
||||
"min_meshes": 0
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,83 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T09:12:16.061Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
|
||||
"desktop": {
|
||||
"before": {
|
||||
"step": "19907",
|
||||
"pixels": "2"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 59 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "scene.xml",
|
||||
"objects": "59",
|
||||
"bindings": "58",
|
||||
"meshes": "58",
|
||||
"fallbacks": "0",
|
||||
"step": "20063",
|
||||
"pixels": "2",
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"canvas": {
|
||||
"width": 920,
|
||||
"height": 720,
|
||||
"clientWidth": 920,
|
||||
"clientHeight": 720,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "360px 920px",
|
||||
"shellRows": "720px"
|
||||
}
|
||||
},
|
||||
"mobile": {
|
||||
"before": {
|
||||
"step": "22473",
|
||||
"pixels": "2"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 59 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "scene.xml",
|
||||
"objects": "59",
|
||||
"bindings": "58",
|
||||
"meshes": "58",
|
||||
"fallbacks": "0",
|
||||
"step": "22635",
|
||||
"pixels": "2",
|
||||
"viewport": {
|
||||
"width": 390,
|
||||
"height": 844
|
||||
},
|
||||
"canvas": {
|
||||
"width": 780,
|
||||
"height": 1040,
|
||||
"clientWidth": 390,
|
||||
"clientHeight": 520,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "390px",
|
||||
"shellRows": "696.219px 520px"
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"desktop_nonblank": true,
|
||||
"desktop_mesh_requirement_met": true,
|
||||
"desktop_replay_moved": true,
|
||||
"mobile_actual_viewport": true,
|
||||
"mobile_nonblank": true,
|
||||
"mobile_mesh_requirement_met": true,
|
||||
"mobile_replay_moved": true,
|
||||
"mobile_stacked_layout": true,
|
||||
"min_objects": 50,
|
||||
"min_meshes": 40,
|
||||
"timeout_ms": 45000
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,82 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T08:55:20.520Z",
|
||||
"url": "http://127.0.0.1:5173/",
|
||||
"desktop": {
|
||||
"before": {
|
||||
"step": "0",
|
||||
"pixels": "4"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 5 objects",
|
||||
"stateStatus": "state: replay 13 frames @ 12 fps",
|
||||
"manifest": "Universal Three.js Preview Demo",
|
||||
"objects": "5",
|
||||
"bindings": "1",
|
||||
"meshes": "1",
|
||||
"fallbacks": "0",
|
||||
"step": "220",
|
||||
"pixels": "4",
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"canvas": {
|
||||
"width": 920,
|
||||
"height": 720,
|
||||
"clientWidth": 920,
|
||||
"clientHeight": 720,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "360px 920px",
|
||||
"shellRows": "720px"
|
||||
}
|
||||
},
|
||||
"mobile": {
|
||||
"before": {
|
||||
"step": "0",
|
||||
"pixels": "4"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 5 objects",
|
||||
"stateStatus": "state: replay 13 frames @ 12 fps",
|
||||
"manifest": "Universal Three.js Preview Demo",
|
||||
"objects": "5",
|
||||
"bindings": "1",
|
||||
"meshes": "1",
|
||||
"fallbacks": "0",
|
||||
"step": "120",
|
||||
"pixels": "4",
|
||||
"viewport": {
|
||||
"width": 390,
|
||||
"height": 844
|
||||
},
|
||||
"canvas": {
|
||||
"width": 780,
|
||||
"height": 1040,
|
||||
"clientWidth": 390,
|
||||
"clientHeight": 520,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "390px",
|
||||
"shellRows": "710.219px 520px"
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"desktop_nonblank": true,
|
||||
"desktop_mesh_requirement_met": true,
|
||||
"desktop_replay_moved": true,
|
||||
"mobile_actual_viewport": true,
|
||||
"mobile_nonblank": true,
|
||||
"mobile_mesh_requirement_met": true,
|
||||
"mobile_replay_moved": true,
|
||||
"mobile_stacked_layout": true,
|
||||
"min_objects": 5,
|
||||
"min_meshes": 1
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,83 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T10:13:13.543Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18085%2Fmanifest",
|
||||
"desktop": {
|
||||
"before": {
|
||||
"step": "2234",
|
||||
"pixels": "22"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 569 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "scene.xml",
|
||||
"objects": "569",
|
||||
"bindings": "548",
|
||||
"meshes": "568",
|
||||
"fallbacks": "0",
|
||||
"step": "2234",
|
||||
"pixels": "22",
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"canvas": {
|
||||
"width": 920,
|
||||
"height": 720,
|
||||
"clientWidth": 920,
|
||||
"clientHeight": 720,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "360px 920px",
|
||||
"shellRows": "720px"
|
||||
}
|
||||
},
|
||||
"mobile": {
|
||||
"before": {
|
||||
"step": "2234",
|
||||
"pixels": "18"
|
||||
},
|
||||
"after": {
|
||||
"title": "MotrixSim Universal Three.js Preview",
|
||||
"h1": "Universal Three.js Preview",
|
||||
"status": "manifest loaded: 569 objects",
|
||||
"stateStatus": "state: poll connected",
|
||||
"manifest": "scene.xml",
|
||||
"objects": "569",
|
||||
"bindings": "548",
|
||||
"meshes": "568",
|
||||
"fallbacks": "0",
|
||||
"step": "2234",
|
||||
"pixels": "18",
|
||||
"viewport": {
|
||||
"width": 390,
|
||||
"height": 844
|
||||
},
|
||||
"canvas": {
|
||||
"width": 780,
|
||||
"height": 1040,
|
||||
"clientWidth": 390,
|
||||
"clientHeight": 520,
|
||||
"tag": "CANVAS",
|
||||
"className": "preview-canvas"
|
||||
},
|
||||
"shellColumns": "390px",
|
||||
"shellRows": "1234.59px 520px"
|
||||
}
|
||||
},
|
||||
"verification": {
|
||||
"desktop_nonblank": true,
|
||||
"desktop_mesh_requirement_met": true,
|
||||
"desktop_replay_moved": false,
|
||||
"mobile_actual_viewport": true,
|
||||
"mobile_nonblank": true,
|
||||
"mobile_mesh_requirement_met": true,
|
||||
"mobile_replay_moved": false,
|
||||
"mobile_stacked_layout": true,
|
||||
"min_objects": 500,
|
||||
"min_meshes": 20,
|
||||
"timeout_ms": 120000
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,246 @@
|
||||
# MotrixSim Web 预览方案对比报告
|
||||
|
||||
日期:2026-06-22
|
||||
|
||||
## 最终建议
|
||||
|
||||
**主线选择方案二:后端状态流。**
|
||||
当前目标不是做一个独立的 MuJoCo demo,而是让真实 MotrixSim 环境能在 Web 前端预览。无影云桌面上的 `/opt/gs_playground_real2sim_48q/.venv/bin/python` 已实测可 import `motrixsim`、`mujoco` 和 `torch`,并能从 MotrixSim 场景中抽取 body 位姿。这个链路最贴近真实生产环境。
|
||||
|
||||
**方案一 MuJoCo WASM 保留为后续轻量模式。**
|
||||
WASM smoke 已通过,适合公开模型、离线单机 demo、低服务器成本展示,但不适合作为 MotrixSim/Python/GPU 栈的第一版替代品。
|
||||
|
||||

|
||||
|
||||
## 核心对比
|
||||
|
||||
| 维度 | 方案一:MuJoCo WASM | 方案二:后端状态流 |
|
||||
|---|---|---|
|
||||
| 计算位置 | 浏览器 CPU/WASM | 无影云桌面 Python/MotrixSim/GPU |
|
||||
| 与 MotrixSim 一致性 | 中低,需要重新适配 | 高,直接复用真实环境 |
|
||||
| Python/torch/ROS/RL 集成 | 弱,需要移植 | 强,原生保留 |
|
||||
| 首版落地复杂度 | 中,需处理 WASM、MEMFS、资产转换 | 中低,先抽状态再传输 |
|
||||
| 服务器成本 | 低 | 中高,取决于并发 |
|
||||
| 网络依赖 | 下载后弱依赖 | 强依赖 |
|
||||
| 模型资产保护 | 弱,资产下发浏览器 | 强,核心资产留后端 |
|
||||
| 复杂场景上限 | 浏览器受限 | 后端硬件决定 |
|
||||
| 适合定位 | 轻量 demo / 离线展示 | 当前 Web 预览主线 |
|
||||
|
||||
## 实测证据
|
||||
|
||||
| 测试项 | 方案 | 结果 |
|
||||
|---|---|---|
|
||||
| 无影 SSH | 后端状态流 | `39.106.101.95:22` 成功 |
|
||||
| GPU | 后端状态流 | `NVIDIA RTX5880-Ada-48Q, 49152 MiB` |
|
||||
| MotrixSim import | 后端状态流 | `/opt/gs_playground_real2sim_48q/.venv/bin/python` 下成功 |
|
||||
| MotrixSim step | 后端状态流 | 240 step 平均 `0.014092 ms/step` |
|
||||
| body pose 抽取 | 后端状态流 | position/quaternion 成功 |
|
||||
| 真实 Web 预览 | 后端状态流 | 浏览器打开 `127.0.0.1:18081`,页面显示 live MotrixSim ball、step、position、raw state |
|
||||
| 通用 Three.js 预览 | 后端状态流 | `falling_ball.xml` 通过 `/manifest` + `/state` 在 Three.js/WebGL 中渲染成功 |
|
||||
| 机器人配置 Three.js 预览 | 后端状态流 | `franka_emika_panda/scene.xml` 展开 `panda.xml` 后加载 59 对象、58 个 OBJ/STL mesh,0 fallback |
|
||||
| Web 控制闭环 | 后端状态流 | Pause/Step/Reset/Resume 已通过页面按钮控制 MotrixSim backend |
|
||||
| Franka 关节控制 | 后端状态流 | `joint2=0.6 rad` 后 hand pose 变化,`Open Grip` 后 finger joints 到 `0.04 m` |
|
||||
| WASM 加载 | MuJoCo WASM | `@mujoco/mujoco@3.9.0` 成功 |
|
||||
| WASM step | MuJoCo WASM | 2000 step 平均 `0.012575 ms/step` |
|
||||
|
||||
## 真实 Web 预览重测
|
||||
|
||||
本次按用户要求重新测试了真实 Web 页面,不再把终端输出或 smoke 结果当作前端效果图。
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| 远端运行 | 无影 `39.106.101.95`,主机 `1nh2jhpso5kckmx` |
|
||||
| 运行时 | `/opt/gs_playground_real2sim_48q/.venv/bin/python` |
|
||||
| 场景 | `/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml` |
|
||||
| 前端 | 浏览器页面 `MotrixSim Web Preview`,Canvas 渲染 |
|
||||
| 后端 | Python 服务真实加载 MotrixSim 并持续 `model.step(data)` |
|
||||
| 页面证据 | 显示 `connected`、server step、body、position xyz、Raw live state |
|
||||
| 本地截图 | `figures/real-web-preview/motrix-live-web-preview.jpg` |
|
||||
| 证据 JSON | `artifacts/motrix_live_web_preview_browser_evidence_2026-06-22.json` |
|
||||
|
||||
### 简单录像
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| 录像来源 | 浏览器真实页面连续截帧 |
|
||||
| 帧数 | 48 |
|
||||
| 页面 z 范围 | `0.3607 m` 到 `4.9765 m` |
|
||||
| MP4 | `figures/real-web-preview/motrix-live-web-preview-recording.mp4` |
|
||||
| GIF | `figures/real-web-preview/motrix-live-web-preview-recording.gif` |
|
||||
|
||||
这个截图和录像把方案二的 MVP 链路从“后端可抽状态”推进到了“浏览器可看实时预览”。因此方案二作为主线的权重进一步提高:它能直接复用 MotrixSim/Python 环境,且已经能通过 Web 页面展示真实仿真状态。
|
||||
|
||||
需要明确的是:**当前接到 Web 的是真实 3D 状态,不是完整 3D 网格预览。** 现在的页面用 Canvas 做 2.5D 可视化,数据来自 MotrixSim 的 3D position/quaternion;要达到完整“真实三维预览”,下一步应接入 Three.js/GLTF 或 MJCF mesh 转换、场景 manifest 和 orbit camera。
|
||||
|
||||
## 通用 Three.js 预览器实测
|
||||
|
||||
已实现第一版通用 Three.js/WebGL 预览器,当前能力是:
|
||||
|
||||
- 读取 scene manifest,不再绑定固定 demo 代码。
|
||||
- 支持 primitive:box、sphere、capsule、cylinder、plane、axes。
|
||||
- 支持 mesh loader:OBJ、STL、GLTF、GLB。
|
||||
- 支持状态源:replay、HTTP poll、websocket。
|
||||
- 支持 Motrix/MJCF Z-up 到 Three.js Y-up 的 position/quaternion 转换。
|
||||
- 支持 `object.body` 到 runtime body state 的绑定。
|
||||
|
||||
### Sample manifest 验证
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| objects | 5 |
|
||||
| body bindings | 1 |
|
||||
| meshes | 1 |
|
||||
| fallbacks | 0 |
|
||||
| desktop replay step | `0` → `220` |
|
||||
| mobile replay step | `0` → `120` |
|
||||
| mobile viewport | `390 x 844` |
|
||||
| mobile layout | 单列堆叠 |
|
||||
|
||||
### 远端 MotrixSim bridge 验证
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| 远端 | `39.106.101.95` |
|
||||
| bridge | `server/headless_motrix_bridge.py` |
|
||||
| scene | `/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml` |
|
||||
| manifest | `falling_ball.xml` |
|
||||
| objects | 2 |
|
||||
| body bindings | 1 |
|
||||
| state | `poll connected` |
|
||||
| desktop step | `19078` → `19253` |
|
||||
| mobile step | `19421` → `19668` |
|
||||
| evidence | `artifacts/universal_three_preview_remote_motrix_evidence_2026-06-22.json` |
|
||||
|
||||

|
||||
|
||||
### 机器人配置验证:Franka Panda
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| 远端 | `39.106.101.95` |
|
||||
| scene | `/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml` |
|
||||
| include | `panda.xml` 已展开 |
|
||||
| manifest objects | 59 |
|
||||
| mesh objects | 58 |
|
||||
| fallback | 0 |
|
||||
| state | `poll connected` |
|
||||
| desktop step | `19907` → `20063` |
|
||||
| mobile step | `22473` → `22635` |
|
||||
| evidence | `artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json` |
|
||||
|
||||

|
||||
|
||||
### Web 控制闭环验证
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| Pause | step 停止 |
|
||||
| Step once | step 单步增长 |
|
||||
| Reset | step 回到 `0` |
|
||||
| Resume | step 继续增长 |
|
||||
| evidence | `artifacts/universal_three_preview_control_evidence_2026-06-22.json` |
|
||||
|
||||

|
||||
|
||||
| 项目 | 结果 |
|
||||
|---|---|
|
||||
| controls.joints | 9 |
|
||||
| state link poses | 11 |
|
||||
| joint2 | `0` → `0.6000000238` |
|
||||
| hand pose | 已变化 |
|
||||
| gripper | finger joints → `0.04 m` |
|
||||
| evidence | `artifacts/universal_three_preview_joint_control_evidence_2026-06-22.json` |
|
||||
|
||||
结论更新:方案二已经不只是“状态流 + Canvas 预览”,现在已经走通“MotrixSim/MJCF scene → manifest → Three.js/WebGL → live state → Web 控制命令 → MotrixSim joint/link 更新”的基础闭环,并且真实机器人 OBJ/STL 资产已能在浏览器里预览和低层操作。复杂材质纹理与大规模重复资产也完成了 store 货架场景验证。剩余工作重点是高层 task/action adapter、更多 MJCF 特性的长尾覆盖和性能优化,而不是证明 Three.js 路线是否可行。
|
||||
|
||||
## 通用资产覆盖补充验证(2026-06-22 18:17)
|
||||
|
||||
| 场景 | 类型 | Objects | Meshes | Fallbacks | 额外覆盖 |
|
||||
|---|---|---:|---:|---:|---|
|
||||
| Store `scene.xml` | 大规模贴图货架 | 569 | 568 | 0 | `assetdir`、PNG texture、OBJ 复用、17 个 textured materials |
|
||||
| Go2 `scene_flat.xml` | 四足机器人 | 34 | 33 | 0 | 12 joints、link pose、OBJ robot mesh、checker ground |
|
||||
|
||||
这轮实现补齐了 `nested <default>`、`fromto`、`xyaxes`、`zaxis`、mesh scale、file texture、builtin checker、inline convex mesh 与 mesh cache。Store 首次验证时曾因 569 个对象重复加载 OBJ 超时,加入 mesh cache 后桌面/移动均通过非空渲染检查;这说明后续性能优化应优先围绕实例化、LOD 和渐进加载,而不是回退到视频流。
|
||||
|
||||
## 方案二 MVP 设计
|
||||
|
||||

|
||||
|
||||
### 后端
|
||||
|
||||
建议进程分成三层:
|
||||
|
||||
1. `SimulationWorker`
|
||||
- 加载 MotrixSim scene。
|
||||
- 定频执行 `model.step(data)`。
|
||||
- 抽取 body pose、qpos、sensor、task status。
|
||||
|
||||
2. `StateBroker`
|
||||
- 将状态压缩为 JSON 或二进制 buffer。
|
||||
- 按前端渲染频率抽样,例如 20-60 Hz。
|
||||
- 支持 pause/reset/action/camera 命令。
|
||||
|
||||
3. `WebServer`
|
||||
- 提供 scene manifest。
|
||||
- 提供 WebSocket 状态流。
|
||||
- 可选提供 MotrixSim RenderApp 截图或视频流。
|
||||
|
||||
### 前端
|
||||
|
||||
建议前端第一版只做“监视器 + 控制器”:
|
||||
|
||||
1. 加载 `scene_manifest.json`。
|
||||
2. 根据 manifest 创建 Three.js 节点或加载 GLTF。
|
||||
3. 建立 WebSocket。
|
||||
4. 按 body index 更新 `position` 和 `quaternion`。
|
||||
5. 提供 reset、pause、step、action、camera 控件。
|
||||
|
||||
## 方案一定位
|
||||
|
||||

|
||||
|
||||
MuJoCo WASM 路线已证明可以跑通最小 XML、step 和 qpos 读取。它的工程价值是:
|
||||
|
||||
- 无服务器公开 demo。
|
||||
- 离线模型教学。
|
||||
- 单机器人轻量展示。
|
||||
- 前端交互原型。
|
||||
|
||||
不建议把它作为第一阶段主线,因为:
|
||||
|
||||
- MotrixSim 的 Python API、渲染、控制器和训练链路不能直接复用。
|
||||
- 私有 MJCF、mesh、texture 必须下发到浏览器。
|
||||
- 多机器人、大场景、复杂接触在浏览器中风险更高。
|
||||
- 多线程 WASM 还需要跨源隔离响应头,部署复杂度不为零。
|
||||
|
||||
## 里程碑建议
|
||||
|
||||
| 阶段 | 时间 | 目标 | 验收标准 |
|
||||
|---|---|---|---|
|
||||
| M0 | 0.5 天 | 固化无影接入和环境脚本 | 一键输出 GPU、MotrixSim import、scene step |
|
||||
| M1 | 1-2 天 | 状态流 API MVP | 浏览器可看到 falling ball / robot body pose 更新 |
|
||||
| M2 | 2-4 天 | Three.js 场景 manifest | 前端节点与 MotrixSim body 名稳定映射 |
|
||||
| M3 | 3-5 天 | 控制闭环 | 前端 reset/pause/action 生效 |
|
||||
| M4 | 3-5 天 | 真实画面对照 | RenderApp 截图或视频流与 Three.js 状态画面可对比 |
|
||||
| M5 | 后续 | WASM 轻量模式 | 单模型可离线加载、step、渲染 |
|
||||
|
||||
## 决策
|
||||
|
||||
| 决策项 | 结论 |
|
||||
|---|---|
|
||||
| 当前主线 | 后端状态流 |
|
||||
| 兜底能力 | 后端真实画面/视频流 |
|
||||
| 后续增强 | MuJoCo WASM 轻量模式 |
|
||||
| 不建议 | 第一版直接把 MotrixSim 完整迁到浏览器 |
|
||||
|
After Width: | Height: | Size: 52 KiB |
|
After Width: | Height: | Size: 54 KiB |
@ -0,0 +1,76 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T10:33:23.139Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest",
|
||||
"initial": {
|
||||
"step": 11275,
|
||||
"paused": false,
|
||||
"command": [
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"after_toggle": {
|
||||
"step": 11334,
|
||||
"paused": true
|
||||
},
|
||||
"after_reset": {
|
||||
"step": 0,
|
||||
"paused": true,
|
||||
"command": [
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"after_forward_key": {
|
||||
"command": [
|
||||
1,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"after_stop_key": {
|
||||
"command": [
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"actuator": {
|
||||
"name": "actuator1",
|
||||
"target": 1.0140550017356873,
|
||||
"state": {
|
||||
"ctrl": 1.0140550136566162,
|
||||
"range": [
|
||||
-2.8973000049591064,
|
||||
2.8973000049591064
|
||||
],
|
||||
"index": 0,
|
||||
"targetName": "joint1"
|
||||
}
|
||||
},
|
||||
"dom": {
|
||||
"objects": "59",
|
||||
"meshes": "58",
|
||||
"paused": "true",
|
||||
"step": "0",
|
||||
"control": "control: set_actuator ok",
|
||||
"inputStatus": "input: command sent 0.00 / 0.00 / 0.00",
|
||||
"inputKeys": "-",
|
||||
"inputCommand": "0.00 / 0.00 / 0.00",
|
||||
"actuatorStatus": "actuators: actuator1 set",
|
||||
"selectedActuator": "actuator1",
|
||||
"actuatorValue": "1.014"
|
||||
},
|
||||
"verification": {
|
||||
"space_toggled_pause": true,
|
||||
"r_reset_step_zero": true,
|
||||
"w_sent_forward_command": true,
|
||||
"keyup_sent_stop_command": true,
|
||||
"actuator_ctrl_applied": true,
|
||||
"min_objects": 50,
|
||||
"min_meshes": 40,
|
||||
"timeout_ms": 60000
|
||||
}
|
||||
}
|
||||
|
After Width: | Height: | Size: 54 KiB |
|
After Width: | Height: | Size: 60 KiB |
|
After Width: | Height: | Size: 59 KiB |
@ -0,0 +1,76 @@
|
||||
{
|
||||
"captured_at": "2026-06-22T10:32:12.848Z",
|
||||
"url": "http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18086%2Fmanifest",
|
||||
"initial": {
|
||||
"step": 5303,
|
||||
"paused": false,
|
||||
"command": [
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"after_toggle": {
|
||||
"step": 5419,
|
||||
"paused": true
|
||||
},
|
||||
"after_reset": {
|
||||
"step": 0,
|
||||
"paused": true,
|
||||
"command": [
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"after_forward_key": {
|
||||
"command": [
|
||||
1,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"after_stop_key": {
|
||||
"command": [
|
||||
0,
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
"actuator": {
|
||||
"name": "FL_hip",
|
||||
"target": 0.33152000010013577,
|
||||
"state": {
|
||||
"ctrl": 0.3315199911594391,
|
||||
"range": [
|
||||
-0.9472000002861023,
|
||||
0.9472000002861023
|
||||
],
|
||||
"index": 0,
|
||||
"targetName": "FL_hip_joint"
|
||||
}
|
||||
},
|
||||
"dom": {
|
||||
"objects": "34",
|
||||
"meshes": "33",
|
||||
"paused": "true",
|
||||
"step": "0",
|
||||
"control": "control: set_actuator ok",
|
||||
"inputStatus": "input: command sent 0.00 / 0.00 / 0.00",
|
||||
"inputKeys": "-",
|
||||
"inputCommand": "0.00 / 0.00 / 0.00",
|
||||
"actuatorStatus": "actuators: FL_hip set",
|
||||
"selectedActuator": "FL_hip",
|
||||
"actuatorValue": "0.332"
|
||||
},
|
||||
"verification": {
|
||||
"space_toggled_pause": true,
|
||||
"r_reset_step_zero": true,
|
||||
"w_sent_forward_command": true,
|
||||
"keyup_sent_stop_command": true,
|
||||
"actuator_ctrl_applied": true,
|
||||
"min_objects": 30,
|
||||
"min_meshes": 20,
|
||||
"timeout_ms": 60000
|
||||
}
|
||||
}
|
||||
|
After Width: | Height: | Size: 59 KiB |
@ -0,0 +1,115 @@
|
||||
<callout emoji="✅" background-color="light-green" border-color="green">
|
||||
<p><b>推荐结论:</b>当前主线选择“MotrixSim 后端计算 + WebSocket 状态流 + 前端 Three.js 渲染”。MuJoCo WASM 已通过最小 smoke,但更适合作为后续离线/公开轻量 demo,不适合作为第一阶段真实 MotrixSim Web 预览主线。</p>
|
||||
</callout>
|
||||
|
||||
<h1>一、测试范围</h1>
|
||||
<table>
|
||||
<thead><tr><th background-color="light-gray">项目</th><th background-color="light-gray">结果</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>本地调研目录</td><td><code>research-tests/2026-06-22-motrixsim-web-preview</code></td></tr>
|
||||
<tr><td>无影 SSH 主机</td><td><code>39.106.101.95</code></td></tr>
|
||||
<tr><td>无影 GPU</td><td><code>NVIDIA RTX5880-Ada-48Q, 49152 MiB</code></td></tr>
|
||||
<tr><td>MotrixSim Python</td><td><code>/opt/gs_playground_real2sim_48q/.venv/bin/python</code></td></tr>
|
||||
<tr><td>本地 WASM 包</td><td><code>@mujoco/mujoco@3.9.0</code></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h1>二、无影 MotrixSim 实测</h1>
|
||||
<p>修正 SSH 主机后,TCP 22 和 SSH 均成功。远端默认系统 Python 未安装 MotrixSim,但 <code>/opt/gs_playground_real2sim_48q/.venv/bin/python</code> 可以成功 import <code>motrixsim</code>、<code>mujoco</code> 和 <code>torch</code>。</p>
|
||||
<table>
|
||||
<thead><tr><th background-color="light-gray">指标</th><th background-color="light-gray">实测结果</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>MotrixSim scene</td><td><code>/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml</code></td></tr>
|
||||
<tr><td>step 数</td><td>240</td></tr>
|
||||
<tr><td>采样数</td><td>30</td></tr>
|
||||
<tr><td>平均 step 耗时</td><td><code>0.014092 ms</code></td></tr>
|
||||
<tr><td>p95 step 耗时</td><td><code>0.014621 ms</code></td></tr>
|
||||
<tr><td>body 位姿</td><td>position / quaternion 抽取成功</td></tr>
|
||||
<tr><td>body 高度变化</td><td><code>z=5.0</code> 到 <code>z=0.499633</code></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<whiteboard type="mermaid">
|
||||
flowchart LR
|
||||
A["无影云桌面 GPU"] --> B["MotrixSim Python worker"]
|
||||
B --> C["scene.step"]
|
||||
C --> D["body pose / qpos / sensor"]
|
||||
D --> E["WebSocket state stream"]
|
||||
E --> F["Three.js Web viewer"]
|
||||
F -. control .-> B
|
||||
</whiteboard>
|
||||
|
||||
<h1>三、MuJoCo WASM 实测</h1>
|
||||
<p>本地使用官方 <code>@mujoco/mujoco@3.9.0</code> 完成最小 XML 加载、模型编译、<code>mj_step</code> 和 <code>qpos</code> 读取。这个测试证明纯前端物理 demo 可行,但没有覆盖 MotrixSim Python 生态、远端 GPU、私有资产保护和真实渲染对照。</p>
|
||||
<table>
|
||||
<thead><tr><th background-color="light-gray">指标</th><th background-color="light-gray">实测结果</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>单线程 WASM 文件</td><td><code>8.7 MB</code></td></tr>
|
||||
<tr><td>多线程 WASM 文件</td><td><code>8.8 MB</code></td></tr>
|
||||
<tr><td>WASM 加载耗时</td><td><code>55.728 ms</code></td></tr>
|
||||
<tr><td>模型编译耗时</td><td><code>21.673 ms</code></td></tr>
|
||||
<tr><td>2000 step 总耗时</td><td><code>25.15 ms</code></td></tr>
|
||||
<tr><td>单 step 耗时</td><td><code>0.012575 ms</code></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h1>四、方案对比</h1>
|
||||
<table>
|
||||
<thead><tr><th background-color="light-gray">维度</th><th background-color="light-gray">MuJoCo WASM</th><th background-color="light-gray">后端状态流</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>与真实 MotrixSim 一致性</td><td>中低,需要重新适配</td><td><b>高,直接复用远端环境</b></td></tr>
|
||||
<tr><td>Python / torch / RL 集成</td><td>弱,需要移植</td><td><b>强,原生保留</b></td></tr>
|
||||
<tr><td>服务器成本</td><td><b>低</b></td><td>中高,随并发增长</td></tr>
|
||||
<tr><td>资产保护</td><td>弱,资产下发浏览器</td><td><b>强,资产留后端</b></td></tr>
|
||||
<tr><td>MVP 落地</td><td>适合轻量 demo</td><td><b>适合当前目标</b></td></tr>
|
||||
<tr><td>复杂场景上限</td><td>浏览器受限</td><td><b>由后端硬件决定</b></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<whiteboard type="mermaid">
|
||||
flowchart TD
|
||||
Start["当前目标:真实 MotrixSim Web 预览"] --> Q1{"是否必须复用 Python / MotrixSim / GPU?"}
|
||||
Q1 -->|是| Backend["选择后端状态流"]
|
||||
Q1 -->|否| Q2{"是否需要离线和低服务器成本?"}
|
||||
Q2 -->|是| Wasm["选择 MuJoCo WASM"]
|
||||
Q2 -->|否| Hybrid["后端状态流优先,WASM 作为补充"]
|
||||
Backend --> Mvp["MVP:state stream + Three.js"]
|
||||
Mvp --> Fallback["兜底:真实渲染截图或视频流"]
|
||||
</whiteboard>
|
||||
|
||||
<h1>五、推荐路线</h1>
|
||||
<ol>
|
||||
<li seq="auto">M0:固化无影接入脚本,持续输出 GPU、MotrixSim import、scene step 证据。</li>
|
||||
<li seq="auto">M1:在 MotrixSim venv 中补充 WebSocket 服务依赖,启动后端状态流 API。</li>
|
||||
<li seq="auto">M2:前端加载 scene manifest,并按 body index 更新 Three.js 节点。</li>
|
||||
<li seq="auto">M3:加入 reset、pause、action、camera 控制闭环。</li>
|
||||
<li seq="auto">M4:增加真实 MotrixSim 渲染截图或视频流作为画面对照。</li>
|
||||
<li seq="auto">M5:后续补 MuJoCo WASM 轻量离线模式。</li>
|
||||
</ol>
|
||||
|
||||
<h1>六、风险和待验证</h1>
|
||||
<table>
|
||||
<thead><tr><th background-color="light-gray">风险</th><th background-color="light-gray">影响</th><th background-color="light-gray">建议</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>MotrixSim venv 未装 FastAPI / websockets</td><td>不能直接启动 API 服务</td><td>最小安装并锁版本,或复用已有服务框架</td></tr>
|
||||
<tr><td>当前只测了轻量 falling ball</td><td>复杂机器人和碰撞场景带宽未知</td><td>增加 Go1/Go2/G1 或 Real2Sim package benchmark</td></tr>
|
||||
<tr><td>前端资产映射未完成</td><td>Web 节点可能和 MotrixSim body 不一致</td><td>生成 scene manifest,固定 body/node 索引</td></tr>
|
||||
<tr><td>真实画面对照未接入</td><td>无法判断 Three.js 重建是否偏离</td><td>接入 RenderApp 截图、MJPEG 或 WebRTC 兜底</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h1>七、本地证据文件</h1>
|
||||
<ul>
|
||||
<li><code>artifacts/wuying_probe_2026-06-22.log</code></li>
|
||||
<li><code>artifacts/motrix_state_probe_remote_2026-06-22.json</code></li>
|
||||
<li><code>artifacts/mujoco_wasm_smoke_2026-06-22.json</code></li>
|
||||
<li><code>research-report.md</code></li>
|
||||
<li><code>comparison-report.md</code></li>
|
||||
</ul>
|
||||
|
||||
<h1>八、参考资料</h1>
|
||||
<ul>
|
||||
<li><a href="https://github.com/google-deepmind/mujoco/tree/main/wasm">MuJoCo JavaScript bindings</a></li>
|
||||
<li><a href="https://mujoco.readthedocs.io/en/stable/python.html">MuJoCo Python docs</a></li>
|
||||
<li><a href="https://mujoco.readthedocs.io/en/stable/programming/visualization.html">MuJoCo visualization docs</a></li>
|
||||
</ul>
|
||||
@ -0,0 +1,24 @@
|
||||
<h2>机器人配置验证:Franka Panda(2026-06-22)</h2>
|
||||
<callout emoji="✅" background-color="light-green" border-color="green">
|
||||
<p>本节是按“找一个带有机器人的配置再测试一下”重新执行的真实浏览器验证。测试对象不是 falling ball,也不是占位模型,而是无影 MotrixSim 示例里的 Franka Panda 机器人 MJCF 配置。</p>
|
||||
</callout>
|
||||
<table>
|
||||
<thead>
|
||||
<tr><th background-color="light-gray">验证项</th><th background-color="light-gray">结果</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td>远端环境</td><td>无影 39.106.101.95</td></tr>
|
||||
<tr><td>场景</td><td>/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml</td></tr>
|
||||
<tr><td>include</td><td>scene.xml 中的 panda.xml 已展开</td></tr>
|
||||
<tr><td>资产</td><td>真实 OBJ/STL 机器人 mesh,通过 /assets HTTP 服务给浏览器</td></tr>
|
||||
<tr><td>manifest objects</td><td>59</td></tr>
|
||||
<tr><td>loaded meshes</td><td>58</td></tr>
|
||||
<tr><td>fallbacks</td><td>0</td></tr>
|
||||
<tr><td>state</td><td>poll connected,step 持续增长</td></tr>
|
||||
<tr><td>桌面 step</td><td>19907 → 20063</td></tr>
|
||||
<tr><td>移动 step</td><td>22473 → 22635</td></tr>
|
||||
<tr><td>证据文件</td><td>artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>本轮同时修复了 MJCF include 展开、mesh 资产服务、body/geom 静态位姿累乘、Z-up 到 Three.js Y-up 的 mesh 本地坐标基转换,以及自动相机取景。截图中可以看到浏览器 WebGL 画面里真实渲染出了 Franka Panda 机器人本体。</p>
|
||||
<p><b>边界说明:</b>该场景在当前 MotrixSim runtime 暴露的 body state 只有 link0,所以本轮证明的是“真实机器人三维资产预览 + 后端状态连接”,还不是完整关节动画驱动。下一步要补 qpos / joint 层级映射,才能让机器人各关节按 MotrixSim 状态实时运动。</p>
|
||||
@ -0,0 +1,157 @@
|
||||
{
|
||||
"ok": true,
|
||||
"synced_at": "2026-06-22T12:16:34+08:00",
|
||||
"space_id": "7530104418265350163",
|
||||
"parent_title": "具身",
|
||||
"parent_node_token": "PHQpwGXHOiqTm3kN96tcwEmOnGg",
|
||||
"title": "MotrixSim Web 预览方案调研与测试报告(2026-06-22)",
|
||||
"node_token": "UKxEwcYjwiuSapkDeaXcqRlUnVd",
|
||||
"obj_token": "WZCfdtn8Jokb50xX3F3cJziingf",
|
||||
"obj_type": "docx",
|
||||
"wiki_url": "https://horizonrobotics.feishu.cn/wiki/UKxEwcYjwiuSapkDeaXcqRlUnVd",
|
||||
"docx_url": "https://horizonrobotics.feishu.cn/docx/WZCfdtn8Jokb50xX3F3cJziingf",
|
||||
"screenshot_update": {
|
||||
"synced_at": "2026-06-22T12:28:28+08:00",
|
||||
"section_title": "九、测试输出截图(非 Web 预览)",
|
||||
"image_count": 4,
|
||||
"images": [
|
||||
"figures/test-screenshots/wuying-ssh-gpu.png",
|
||||
"figures/test-screenshots/motrix-state-probe.png",
|
||||
"figures/test-screenshots/mujoco-wasm-smoke.png",
|
||||
"figures/test-screenshots/metrics-summary.png"
|
||||
]
|
||||
},
|
||||
"real_web_preview_update": {
|
||||
"synced_at": "2026-06-22T12:47:15+08:00",
|
||||
"section_title": "十、真实 MotrixSim Web 预览重测(2026-06-22)",
|
||||
"remote_host": "39.106.101.95",
|
||||
"remote_hostname": "1nh2jhpso5kckmx",
|
||||
"remote_pid": 496408,
|
||||
"doc_image_block_id": "doxcnyD3OtCs1ZtVrVO04mh7CBe",
|
||||
"doc_file_token": "HAZKbAG4oonXMgxCGf2cP5zrnde",
|
||||
"image": "figures/real-web-preview/motrix-live-web-preview.jpg",
|
||||
"evidence": "artifacts/motrix_live_web_preview_browser_evidence_2026-06-22.json"
|
||||
},
|
||||
"recording_and_3d_scope_update": {
|
||||
"synced_at": "2026-06-22T15:33:00+08:00",
|
||||
"section_title": "十一、简单录像说明与 3D 预览界定",
|
||||
"gif_block_id": "doxcnnJklKLzQvOK14UELhYzMwc",
|
||||
"gif_file_token": "YA6ybzu1ToXA7yxcd9bcKNe8n1f",
|
||||
"mp4_block_id": "doxcnCMS14uQSvIkdSTVtRyoxZc",
|
||||
"mp4_file_token": "WXtib2BedoUjVdxshL2cKVAqnyf",
|
||||
"gif": "figures/real-web-preview/motrix-live-web-preview-recording.gif",
|
||||
"mp4": "figures/real-web-preview/motrix-live-web-preview-recording.mp4",
|
||||
"evidence": "artifacts/motrix_live_web_preview_recording_evidence_2026-06-22.json",
|
||||
"scope": "real MotrixSim 3D state stream rendered by Canvas 2.5D MVP; not yet a full Three.js/WebGL mesh preview"
|
||||
},
|
||||
"universal_three_preview_update": {
|
||||
"synced_at": "2026-06-22T16:58:46+08:00",
|
||||
"section_title": "十二、通用 Three.js 预览器实测",
|
||||
"sample_desktop_block_id": "doxcnV5fGlC32xHSMPK4gMEmqeh",
|
||||
"sample_desktop_file_token": "NKuYb1hLDochIox1as0cRKoAnec",
|
||||
"remote_desktop_block_id": "doxcnZhqFCyLgXzMAffsZxNeSAd",
|
||||
"remote_desktop_file_token": "DJoWbHonaoAfbuxy95OcB9ZGnBg",
|
||||
"remote_mobile_block_id": "doxcnaOwM2dcbafqalvcUqAW80f",
|
||||
"remote_mobile_file_token": "Bx4HbZxqJoHDfuxmg00cIJbwn0c",
|
||||
"sample_evidence": "artifacts/universal_three_preview_sample_evidence_2026-06-22.json",
|
||||
"remote_motrix_evidence": "artifacts/universal_three_preview_remote_motrix_evidence_2026-06-22.json",
|
||||
"remote_bridge": {
|
||||
"host": "39.106.101.95",
|
||||
"port": 18083,
|
||||
"pid": 573064,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/falling_ball.xml"
|
||||
}
|
||||
},
|
||||
"robot_panda_preview_update": {
|
||||
"synced_at": "2026-06-22T17:17:09+08:00",
|
||||
"section_title": "机器人配置验证:Franka Panda(2026-06-22)",
|
||||
"desktop_block_id": "doxcn43wErQYuTQoOYDp5D00sif",
|
||||
"desktop_file_token": "WOsFbGrhnoaDnPxMzEWcIPVvn6d",
|
||||
"mobile_block_id": "doxcnrbK9mOWN9hmn6Pz39lsTng",
|
||||
"mobile_file_token": "XbUQbNtBEoJN64xaB4lcMzdanah",
|
||||
"desktop_image": "figures/real-web-preview/universal-three-preview-robot-panda-desktop.jpg",
|
||||
"mobile_image": "figures/real-web-preview/universal-three-preview-robot-panda-mobile.jpg",
|
||||
"evidence": "artifacts/universal_three_preview_robot_panda_evidence_2026-06-22.json",
|
||||
"remote_bridge": {
|
||||
"host": "39.106.101.95",
|
||||
"port": 18084,
|
||||
"pid": 575258,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml",
|
||||
"runtime_body_names": ["link0"]
|
||||
}
|
||||
},
|
||||
"web_control_update": {
|
||||
"synced_at": "2026-06-22T17:44:23+08:00",
|
||||
"section_title": "Web 控制闭环与 Franka 关节控制",
|
||||
"document_revision_id": 42,
|
||||
"section_block_id": "doxcnDbwfdvEnm9nTotbKxHih0e",
|
||||
"control_image_block_id": "doxcn7FVVDPqwoYkVzJDujtumcg",
|
||||
"control_image_file_token": "HeTJbySytos09gxZn2dceptbnhe",
|
||||
"joint_control_image_block_id": "doxcnqihfLxHpjFUocxi4ANLxce",
|
||||
"joint_control_image_file_token": "LqfUbWnWjo6hYwxDtbVcqqZen5e",
|
||||
"control_image": "figures/real-web-preview/universal-three-preview-control-desktop.jpg",
|
||||
"joint_control_image": "figures/real-web-preview/universal-three-preview-joint-control-desktop.jpg",
|
||||
"control_evidence": "artifacts/universal_three_preview_control_evidence_2026-06-22.json",
|
||||
"joint_control_evidence": "artifacts/universal_three_preview_joint_control_evidence_2026-06-22.json",
|
||||
"remote_bridge": {
|
||||
"host": "39.106.101.95",
|
||||
"port": 18084,
|
||||
"pid": 622164,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/franka_emika_panda/scene.xml",
|
||||
"link_pose_count": 11,
|
||||
"joint_count": 9
|
||||
},
|
||||
"verified_controls": ["pause", "step", "reset", "resume", "set_joint", "gripper"]
|
||||
},
|
||||
"universal_asset_matrix_update": {
|
||||
"synced_at": "2026-06-22T18:17:43+08:00",
|
||||
"section_title": "通用资产矩阵复测:Store 与 Go2",
|
||||
"document_revision_id": 47,
|
||||
"section_block_id": "doxcnuSHCNhVwmDPgtkGRAiNpbb",
|
||||
"store_desktop_block_id": "doxcn2m2HyaZOYAHOxF6w3FGHmd",
|
||||
"store_desktop_file_token": "ViGCb8SCfownU1xTAmHcBCScn8b",
|
||||
"go2_desktop_block_id": "doxcnbO5xuxkLiH2QfmC9PcLSKh",
|
||||
"go2_desktop_file_token": "OolqbSpaVoFi0bxmf27cMKGCnLa",
|
||||
"store_desktop_image": "figures/real-web-preview/universal-three-preview-store-textured-desktop.jpg",
|
||||
"store_mobile_image": "figures/real-web-preview/universal-three-preview-store-textured-mobile.jpg",
|
||||
"go2_desktop_image": "figures/real-web-preview/universal-three-preview-go2-robot-desktop.jpg",
|
||||
"go2_mobile_image": "figures/real-web-preview/universal-three-preview-go2-robot-mobile.jpg",
|
||||
"store_evidence": "artifacts/universal_three_preview_store_textured_evidence_2026-06-22.json",
|
||||
"go2_evidence": "artifacts/universal_three_preview_go2_robot_evidence_2026-06-22.json",
|
||||
"remote_bridges": {
|
||||
"store": {
|
||||
"host": "39.106.101.95",
|
||||
"port": 18085,
|
||||
"pid": 661691,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/store/scene.xml",
|
||||
"objects": 569,
|
||||
"meshes": 568,
|
||||
"fallbacks": 0,
|
||||
"textured_materials": 17
|
||||
},
|
||||
"go2": {
|
||||
"host": "39.106.101.95",
|
||||
"port": 18086,
|
||||
"pid": 664826,
|
||||
"scene": "/opt/Motphys/motrixsim-docs/examples/assets/go2/scene_flat.xml",
|
||||
"objects": 34,
|
||||
"meshes": 33,
|
||||
"fallbacks": 0,
|
||||
"joint_count": 12,
|
||||
"link_pose_count": 13
|
||||
}
|
||||
},
|
||||
"new_coverage": [
|
||||
"nested default inheritance",
|
||||
"assetdir mesh/texture resolution",
|
||||
"file texture",
|
||||
"builtin checker texture",
|
||||
"mesh scale",
|
||||
"fromto",
|
||||
"xyaxes",
|
||||
"zaxis",
|
||||
"inline convex mesh",
|
||||
"mesh cache clone"
|
||||
]
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,19 @@
|
||||
<hr/>
|
||||
<h2>通用资产矩阵复测:Store 与 Go2(2026-06-22)</h2>
|
||||
<callout emoji="✅" background-color="light-green" border-color="green">
|
||||
<p>本轮继续推进“万能 Three.js 预览”:从 Franka 单机器人扩展到大规模贴图货架 Store 和 Go2 四足机器人,均在无影 MotrixSim bridge + 本地浏览器中完成真实 WebGL 预览验证。</p>
|
||||
</callout>
|
||||
<table>
|
||||
<colgroup><col span="1" width="150"/><col span="1" width="150"/><col span="1" width="90"/><col span="1" width="90"/><col span="1" width="90"/><col span="1" width="300"/></colgroup>
|
||||
<thead>
|
||||
<tr><th background-color="light-gray">场景</th><th background-color="light-gray">类型</th><th background-color="light-gray">Objects</th><th background-color="light-gray">Meshes</th><th background-color="light-gray">Fallbacks</th><th background-color="light-gray">额外覆盖</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td>Store scene.xml</td><td>大规模贴图货架</td><td>569</td><td>568</td><td>0</td><td>assetdir、PNG texture、OBJ 复用、17 个 textured materials、桌面/移动非空</td></tr>
|
||||
<tr><td>Go2 scene_flat.xml</td><td>四足机器人</td><td>34</td><td>33</td><td>0</td><td>12 joints、13 link poses、OBJ robot mesh、checker ground、桌面/移动非空</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>本轮代码新增覆盖:nested default inheritance、assetdir/meshdir/texturedir 解析、file texture、builtin checker texture、mesh scale、fromto、xyaxes、zaxis、inline convex mesh,以及 mesh cache + clone。</p>
|
||||
<callout emoji="⚠️" background-color="light-yellow" border-color="yellow">
|
||||
<p>Store 首次全量验证曾因 569 个对象重复加载 OBJ 超时;加入 mesh cache 后通过。后续优化重点应转向实例化、LOD、渐进加载和更完整材质系统,而不是回退到视频流。</p>
|
||||
</callout>
|
||||
@ -0,0 +1,23 @@
|
||||
<hr/>
|
||||
<h2>Web 控制闭环与 Franka 关节控制(2026-06-22)</h2>
|
||||
<callout emoji="✅" background-color="light-green" border-color="green">
|
||||
<p>本轮已从只读 Three.js 预览推进到 Web 可操作预览:浏览器按钮可以真实控制无影云桌面上的 MotrixSim 后端,覆盖暂停、单步、重置、恢复、Franka 关节设置和夹爪开合。</p>
|
||||
</callout>
|
||||
<table>
|
||||
<colgroup><col span="1" width="180"/><col span="1" width="540"/></colgroup>
|
||||
<thead>
|
||||
<tr><th background-color="light-gray">验证项</th><th background-color="light-gray">真实结果</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td>Web 入口</td><td>http://127.0.0.1:5173/?manifest=http%3A%2F%2F127.0.0.1%3A18084%2Fmanifest</td></tr>
|
||||
<tr><td>后端环境</td><td>无影云桌面 39.106.101.95,MotrixSim bridge 端口 18084,场景为 Franka Emika Panda MJCF。</td></tr>
|
||||
<tr><td>渲染规模</td><td>Three.js 页面加载 59 个 objects、58 个 meshes,不再是占位 smoke 图。</td></tr>
|
||||
<tr><td>仿真控制</td><td>Pause 后 step 保持 0;Step 后 step=1;Reset 后 step=0;Resume 后 step 推进到 8。</td></tr>
|
||||
<tr><td>关节控制</td><td>页面选择 joint2 并设置 0.6,后端 state 返回 joint2=0.6000000238418579。</td></tr>
|
||||
<tr><td>3D 姿态变化</td><td>joint2 设置前 hand pos=[0.088, 0, 0.926],设置后 hand pos=[0.407, 0, 0.773],说明 Web 控制驱动了真实三维机器人姿态。</td></tr>
|
||||
<tr><td>夹爪控制</td><td>Open Grip 后 finger_joint1 与 finger_joint2 均打开到 0.03999999910593033 m。</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<callout emoji="⚠️" background-color="light-yellow" border-color="yellow">
|
||||
<p>边界说明:当前完成的是底层 joint/gripper 操作闭环。抓取、移动到目标点、轨迹跟随等高层任务还需要按具体场景补 policy/action adapter。</p>
|
||||
</callout>
|
||||
@ -0,0 +1,46 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="560" viewBox="0 0 1200 560">
|
||||
<rect width="1200" height="560" fill="#f7f9fb"/>
|
||||
<text x="60" y="70" font-family="Arial, sans-serif" font-size="34" font-weight="700" fill="#17202a">方案二:MotrixSim 后端状态流架构</text>
|
||||
<text x="60" y="108" font-family="Arial, sans-serif" font-size="18" fill="#52616f">真实 MotrixSim / Python / GPU 留在无影云桌面,Web 前端只接收状态并渲染。</text>
|
||||
|
||||
<rect x="60" y="170" width="270" height="210" rx="10" fill="#ffffff" stroke="#2e86de" stroke-width="3"/>
|
||||
<text x="95" y="215" font-family="Arial, sans-serif" font-size="24" font-weight="700" fill="#1b4f72">无影云桌面</text>
|
||||
<text x="95" y="252" font-family="Arial, sans-serif" font-size="17" fill="#17202a">39.106.101.95</text>
|
||||
<text x="95" y="282" font-family="Arial, sans-serif" font-size="17" fill="#17202a">RTX5880-Ada-48Q</text>
|
||||
<text x="95" y="312" font-family="Arial, sans-serif" font-size="17" fill="#17202a">MotrixSim venv</text>
|
||||
<text x="95" y="342" font-family="Arial, sans-serif" font-size="17" fill="#17202a">Python worker</text>
|
||||
|
||||
<rect x="455" y="170" width="290" height="210" rx="10" fill="#ffffff" stroke="#27ae60" stroke-width="3"/>
|
||||
<text x="492" y="215" font-family="Arial, sans-serif" font-size="24" font-weight="700" fill="#196f3d">状态流服务</text>
|
||||
<text x="492" y="252" font-family="Arial, sans-serif" font-size="17" fill="#17202a">scene.step()</text>
|
||||
<text x="492" y="282" font-family="Arial, sans-serif" font-size="17" fill="#17202a">body position</text>
|
||||
<text x="492" y="312" font-family="Arial, sans-serif" font-size="17" fill="#17202a">body quaternion</text>
|
||||
<text x="492" y="342" font-family="Arial, sans-serif" font-size="17" fill="#17202a">qpos / sensors</text>
|
||||
|
||||
<rect x="870" y="170" width="270" height="210" rx="10" fill="#ffffff" stroke="#8e44ad" stroke-width="3"/>
|
||||
<text x="910" y="215" font-family="Arial, sans-serif" font-size="24" font-weight="700" fill="#633974">Web 前端</text>
|
||||
<text x="910" y="252" font-family="Arial, sans-serif" font-size="17" fill="#17202a">Three.js / R3F</text>
|
||||
<text x="910" y="282" font-family="Arial, sans-serif" font-size="17" fill="#17202a">GLTF / mesh viewer</text>
|
||||
<text x="910" y="312" font-family="Arial, sans-serif" font-size="17" fill="#17202a">WebSocket state</text>
|
||||
<text x="910" y="342" font-family="Arial, sans-serif" font-size="17" fill="#17202a">UI controls</text>
|
||||
|
||||
<path d="M330 275 H455" stroke="#34495e" stroke-width="4" fill="none" marker-end="url(#arrow)"/>
|
||||
<text x="354" y="252" font-family="Arial, sans-serif" font-size="15" fill="#34495e">Python API</text>
|
||||
<path d="M745 275 H870" stroke="#34495e" stroke-width="4" fill="none" marker-end="url(#arrow)"/>
|
||||
<text x="760" y="252" font-family="Arial, sans-serif" font-size="15" fill="#34495e">WebSocket</text>
|
||||
<path d="M870 330 H745" stroke="#7f8c8d" stroke-width="3" fill="none" stroke-dasharray="8 6" marker-end="url(#arrowGray)"/>
|
||||
<text x="766" y="358" font-family="Arial, sans-serif" font-size="15" fill="#7f8c8d">reset / action / camera</text>
|
||||
|
||||
<rect x="175" y="445" width="850" height="58" rx="8" fill="#eafaf1" stroke="#a9dfbf"/>
|
||||
<text x="210" y="481" font-family="Arial, sans-serif" font-size="20" font-weight="700" fill="#196f3d">推荐 MVP:</text>
|
||||
<text x="318" y="481" font-family="Arial, sans-serif" font-size="20" fill="#17202a">MotrixSim 后端计算 + 20-60 Hz 状态流 + 前端 Three.js 重建渲染</text>
|
||||
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#34495e"/>
|
||||
</marker>
|
||||
<marker id="arrowGray" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#7f8c8d"/>
|
||||
</marker>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.8 KiB |
@ -0,0 +1,58 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="640" viewBox="0 0 1200 640">
|
||||
<rect width="1200" height="640" fill="#f8fafc"/>
|
||||
<text x="60" y="70" font-family="Arial, sans-serif" font-size="34" font-weight="700" fill="#17202a">MotrixSim Web 预览方案决策矩阵</text>
|
||||
<text x="60" y="108" font-family="Arial, sans-serif" font-size="18" fill="#52616f">评分 1-5,越高越适合当前目标;当前目标是复用真实 MotrixSim/Python/GPU 环境做 Web 预览。</text>
|
||||
|
||||
<g transform="translate(80,150)">
|
||||
<rect x="0" y="0" width="1040" height="420" rx="10" fill="#ffffff" stroke="#d5dce5"/>
|
||||
<rect x="0" y="0" width="1040" height="58" rx="10" fill="#eaf2f8"/>
|
||||
<text x="30" y="37" font-family="Arial, sans-serif" font-size="19" font-weight="700" fill="#17202a">维度</text>
|
||||
<text x="455" y="37" font-family="Arial, sans-serif" font-size="19" font-weight="700" fill="#a04000">MuJoCo WASM</text>
|
||||
<text x="750" y="37" font-family="Arial, sans-serif" font-size="19" font-weight="700" fill="#196f3d">后端状态流</text>
|
||||
|
||||
<g font-family="Arial, sans-serif" font-size="17" fill="#17202a">
|
||||
<text x="30" y="95">真实 MotrixSim 一致性</text>
|
||||
<text x="515" y="95">2</text>
|
||||
<text x="815" y="95">5</text>
|
||||
|
||||
<text x="30" y="145">现有 Python/GPU 生态复用</text>
|
||||
<text x="515" y="145">1</text>
|
||||
<text x="815" y="145">5</text>
|
||||
|
||||
<text x="30" y="195">前端离线/低服务器成本</text>
|
||||
<text x="515" y="195">5</text>
|
||||
<text x="815" y="195">2</text>
|
||||
|
||||
<text x="30" y="245">私有资产和算法保护</text>
|
||||
<text x="515" y="245">1</text>
|
||||
<text x="815" y="245">5</text>
|
||||
|
||||
<text x="30" y="295">MVP 落地速度</text>
|
||||
<text x="515" y="295">3</text>
|
||||
<text x="815" y="295">4</text>
|
||||
|
||||
<text x="30" y="345">复杂场景扩展上限</text>
|
||||
<text x="515" y="345">2</text>
|
||||
<text x="815" y="345">5</text>
|
||||
|
||||
<text x="30" y="395" font-weight="700">综合建议</text>
|
||||
<text x="455" y="395" fill="#a04000">后续轻量 demo</text>
|
||||
<text x="750" y="395" fill="#196f3d" font-weight="700">当前主线 MVP</text>
|
||||
</g>
|
||||
|
||||
<g stroke="#d5dce5">
|
||||
<line x1="0" y1="58" x2="1040" y2="58"/>
|
||||
<line x1="0" y1="110" x2="1040" y2="110"/>
|
||||
<line x1="0" y1="160" x2="1040" y2="160"/>
|
||||
<line x1="0" y1="210" x2="1040" y2="210"/>
|
||||
<line x1="0" y1="260" x2="1040" y2="260"/>
|
||||
<line x1="0" y1="310" x2="1040" y2="310"/>
|
||||
<line x1="0" y1="360" x2="1040" y2="360"/>
|
||||
<line x1="390" y1="0" x2="390" y2="420"/>
|
||||
<line x1="670" y1="0" x2="670" y2="420"/>
|
||||
</g>
|
||||
</g>
|
||||
|
||||
<rect x="80" y="590" width="1040" height="34" rx="6" fill="#eafaf1" stroke="#a9dfbf"/>
|
||||
<text x="105" y="613" font-family="Arial, sans-serif" font-size="17" fill="#196f3d">结论:先做后端状态流,视频流作为真实画面兜底;WASM 只作为离线/公开轻量模式。</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.9 KiB |
|
After Width: | Height: | Size: 325 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 53 KiB |
|
After Width: | Height: | Size: 57 KiB |
|
After Width: | Height: | Size: 52 KiB |
|
After Width: | Height: | Size: 164 KiB |
|
After Width: | Height: | Size: 49 KiB |
|
After Width: | Height: | Size: 113 KiB |
|
After Width: | Height: | Size: 48 KiB |
|
After Width: | Height: | Size: 99 KiB |