🚀 Release reproducible Real2Sim Studio alpha workbench
Some checks failed
CI / workbench (push) Has been cancelled

This commit is contained in:
eust-w 2026-10-28 01:13:52 +08:00
parent 3e4c2f444e
commit 56f124fab1
54 changed files with 1831 additions and 841 deletions

View File

@ -1,3 +1,3 @@
DASHSCOPE_API_KEY=sk-your-beijing-region-api-key
DASHSCOPE_API_BASE=https://dashscope.aliyuncs.com/api/v1
PORT=8787
PORT=8789

21
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@ -0,0 +1,21 @@
---
name: Bug report
about: A reproducible problem in the workbench or an exported scene
---
## What happened?
## Reproduction steps
## Environment
- OS / CPU / GPU:
- Node.js / browser:
- Python / Genesis version (if relevant):
- Demo or generation mode:
## Expected result
## Logs or screenshots
Remove credentials, private assets and signed download URLs before posting.

View File

@ -0,0 +1,10 @@
---
name: Feature request
about: Describe a concrete workflow that the workbench should support
---
## Workflow and user need
## Current workaround
## Proposed behavior and an example

25
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,25 @@
name: CI
on: [push, pull_request]
permissions:
contents: read
jobs:
workbench:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- run: npm run test:run
- run: npm run lint
- run: npm run build
- run: npm audit --omit=dev
- run: npx playwright install --with-deps chromium
- run: npm run test:e2e
- uses: actions/upload-artifact@v4
if: always()
with:
name: browser-evidence
path: artifacts/smoke/

4
.gitignore vendored
View File

@ -16,6 +16,10 @@ dist-ssr
!.env.example
.data
coverage
.venv
__pycache__
*.pyc
artifacts
playwright-report
test-results

21
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,21 @@
# Contributing
Use Node.js 22.12+ and `npm ci`. Start with `npm run demo`; it uses an isolated local data directory and cannot call the paid generation API.
Before submitting a pull request:
```bash
npm run test:run
npm run lint
npm run build
npm run test:e2e
npm audit
```
The browser smoke test needs Chromium (`npx playwright install chromium`). It starts and stops its own isolated servers. For changes to physical parameter mapping or the Python export, also run the example through Genesis and include `metrics.json`, the environment, and reproduction commands.
Keep changes focused. Include a regression test for changed behavior and explain visible limitations. Do not commit `.env.local`, `.data`, virtual environments, credentials, private generated assets, or account sessions.
Useful first contributions include installation reports, a reproducible cloth example, unit-aware material presets, accessible keyboard controls and additional generation-provider adapters. Open an issue before adding a new simulator backend.
Existing optional Git hooks can be enabled with `git config core.hooksPath .githooks`. Their commit-message convention requires an emoji prefix. Contributions are licensed under MIT.

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 eust-w and Real2Sim Studio contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

190
README.md
View File

@ -1,154 +1,86 @@
# real2sim 平台
# Real2Sim Studio
基于阿里云百炼 DashScope `Tripo/Tripo-H3.1` 的 3D 资产生成与柔性物体仿真工作台。前端默认中文,并内置英文切换。
**A local-first workbench for 3D assets and Genesis simulation exports.**
## 功能
[中文](docs/README.zh-CN.md) · [Offline example](examples/README.md) · [Roadmap](docs/ROADMAP.md) · [Contributing](CONTRIBUTING.md) · [MIT license](LICENSE)
- 模型生成:支持文生 3D、单图生 3D、多图生 3D。
- Tripo 参数:支持 `Tripo-H3.1` / `Tripo-P1.0`、贴图质量、几何精度、PBR、无贴图生成。
- 异步任务:后端创建 DashScope 任务,本地保存任务记录,前端支持轮询与手动刷新。
- 柔性物体:从本地资产库选择源 GLB,生成柔性版本 prompt,提交 H3.1 派生任务。
- 仿真配置:按材料、弹性、阻尼、厚度、最大形变、重力导出 `real2sim.flexProfile.v1` JSON。
- 本地资产库:DashScope 返回的预览图和 GLB 会缓存到 `.data/assets/<taskId>/`,前端预览、下载和柔性源选择优先使用本地 `/assets/...` URL。
- Genesis 导出:柔性任务可导出 `.tar.gz` 包,包含本地 GLB、`flex-profile.json`、`manifest.json`、`genesis_scene.py` 和说明文档。
![Real2Sim Studio workbench](docs/images/workbench.png)
## 技术栈
Inspect a GLB, configure a material preset and export a self-contained Genesis scene. Start with the included procedural example without an API key, or connect your own DashScope account for Tripo text/image-to-3D generation.
- 前端:React、Vite、TypeScript、Three.js、@react-three/fiber、lucide-react
- 后端:Express、Zod、DashScope HTTP API
- 测试:Vitest
## Try it
## 本地运行
Requires Node.js **22.12 or later**.
```bash
git clone https://github.com/eust-w/real2sim-platform.git
cd real2sim-platform
npm ci
npm run demo
```
Open the Vite URL printed in the terminal (normally `http://127.0.0.1:5173`). Select **Load example**, rotate the model, adjust the material controls, then **Export Genesis Package**. Demo mode cannot make paid generation calls and uses `.data/demo/`, separate from your usual tasks.
## What is included
- Text, single-image and multi-image generation through DashScope / Tripo using your own account.
- Local task history and asset caching, GLB preview and downloads.
- A self-contained, MIT-licensed procedural sample.
- Material preset controls and Genesis export using the current settings, including gravity.
- FEM elastic export and an experimental PBD cloth export path.
- English / Chinese UI, unit tests, backend type checking and browser smoke tests.
The package contains the mesh, `flex-profile.json`, `manifest.json`, `genesis_scene.py`, pinned runtime requirements and instructions. Physics runs in **Genesis**, not in the browser preview.
## Run the exported scene
See the [complete example guide](examples/README.md) for Python environment setup.
```bash
npm run example:export
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r examples/requirements.txt
python .data/genesis-packages/example-soft-puck/genesis_scene.py --steps 800 --record
```
The first run compiles solver kernels. Without `--record`, the scene runs numerical checks and saves metrics and a trajectory without requesting a camera. The recorded result uses the actual Genesis solver, not an animated preview.
**Alpha security limitation:** the optional Python runtime has known Pillow advisories constrained by an upstream MoviePy dependency. Use only the trusted example in an isolated environment; see [the audit note](SECURITY.md#known-optional-runtime-risk-2026-10-28). The Node audit and Python audit are separate.
## Optional AI generation
```bash
npm install
cp .env.example .env.local
# Set your DashScope API key in .env.local.
npm run dev
```
`.env.local` 需要配置北京地域的 DashScope API Key:
Generation uses an external paid service. Provider models, generated assets and service terms are not covered by this repository's MIT license. A flexible variant is a new prompt-based generation; it is not deformation or reconstruction of the original GLB.
```bash
DASHSCOPE_API_KEY=sk-your-beijing-region-api-key
DASHSCOPE_API_BASE=https://dashscope.aliyuncs.com/api/v1
PORT=8787
```
The local API defaults to `127.0.0.1:8789`. Set `PORT` before starting to change it; the development proxy uses the same value. This is a single-user local application, not a publicly hosted service. See [Security](SECURITY.md).
前端地址:`http://localhost:5173`
## Limits
后端地址:`http://localhost:8787`
- Material values are illustrative presets, not calibrated estimates from real objects.
- The included mesh is procedural. Arbitrary generated meshes can fail tetrahedralization and may need repair or unit correction.
- Cloth requires an appropriate surface mesh. Inflatable/soft-robot presets do not simulate pressure or actuators.
- No robot training, scene reconstruction, measured sim-to-real accuracy or universal model compatibility is claimed.
- The current exporter targets Genesis World 1.4.5. Newer releases may change APIs.
## 仓库与提交规范
当前 GitHub 远端:
- `origin`:`https://github.com/eust-w/real2sim-platform.git`
- GitHub 仓库可见性要求:private。
Gitea 远端要求:
- 远端名建议使用 `gitea`。
- Gitea 仓库可见性要求:public。
- 配置方式:`git remote add gitea <gitea-public-repo-url>`。
本仓库使用仓库级 Git hooks:
```bash
git config core.hooksPath .githooks
```
提交前 `.githooks/pre-commit` 会依次执行:
## Development
```bash
npm run test:run
npm run lint
npm run build
npx playwright install chromium
npm run test:e2e
npm audit
```
提交信息由 `.githooks/commit-msg` 校验,标题必须以 emoji 开头,例如:
The browser test uses isolated data, exports a real GLB package, verifies current slider values, checks canvas pixels and rotation, and captures desktop/mobile screenshots. Physics execution is a separate Python check, not implied by passing the TypeScript tests.
```bash
git commit -m "✨ add local asset cache"
```
## License and credits
每次代码变更后的标准流程:
```bash
git status --short
git add <changed-files>
git commit -m "✨ describe the change"
git push origin main
git push gitea main
```
## API
- `GET /api/health`
- `GET /api/tripo/tasks`
- `POST /api/tripo/tasks`
- `GET /api/tripo/tasks/:taskId`
- `POST /api/flex/tasks`
- `GET /api/genesis/packages/:taskId`
- `GET /api/genesis/packages/:taskId/download`
## 柔性物体实现原理
当前柔性流程不是本地软体物理求解器,也不会把已有 GLB 直接做网格形变再导出。
普通版本和柔性版本的区别:
- 普通版本:直接按用户 prompt / 图片生成 3D 资产,几何和贴图不带仿真语义。
- 柔性版本:以源资产描述为基础,加入软硅胶、织物、充气膜或软体机器人等材料提示,并绑定一份 `real2sim.flexProfile.v1` 物理配置。
- 两者的 GLB 都只是视觉/几何资产;真正进入仿真器后是否柔性,取决于 Genesis 脚本是否给它分配 FEM / PBD 等柔体材料。
实际流程是:
1. 前端选择源资产、材料类型和弹性/阻尼/厚度/最大形变/重力等参数。
2. 后端把这些配置转换成材料感知 prompt,例如软硅胶、织物、充气膜或软体机器人结构。
3. 后端用这个 prompt 调用 DashScope `Tripo/Tripo-H3.1` 创建一个新的 3D 生成任务。
4. 平台同时导出 `real2sim.flexProfile.v1`,把物理参数归一化为仿真配置,供后续 real-to-sim 管线或求解器使用。
5. 任务成功后,后端把远程渲染图和 GLB 缓存到本地资产库,前端 3D 视口从本地 `/assets/...` 加载 GLB 预览。
## Genesis 导出包
柔性任务结果面板会显示“导出 Genesis 包”。导出包位于 `.data/genesis-packages/<taskId>.tar.gz`,内容包括:
- `assets/model.glb`:本地缓存的柔性版本 GLB。
- `assets/preview.*`:本地渲染预览图,如果任务有预览图。
- `flex-profile.json`:UI 仿真滑块导出的物理配置。
- `manifest.json`:Genesis solver 和材料参数映射。
- `genesis_scene.py`:可直接运行的 Genesis 场景脚本。
- `README.md`:包内运行说明和网格限制说明。
默认映射规则:
- `silicone` / `inflatable` / `softRobot`:导出为 Genesis FEM Elastic,脚本使用 `gs.materials.FEM.Elastic(...)`。
- `cloth`:导出为 Genesis PBD Cloth,脚本使用 `gs.materials.PBD.Cloth(...)`。
运行方式:
```bash
tar -xzf <taskId>-genesis-flex.tar.gz
cd <taskId>
pip install genesis-world
python genesis_scene.py
```
注意:GLB 本身不携带柔体物理属性。它在 Genesis 中变成柔体,是因为导出包里的 `genesis_scene.py` 给它分配了 Genesis 柔体 solver 和 material。如果 Genesis 无法对 GLB 做 FEM 四面体化,需要先把模型转换成干净、封闭的 OBJ,再更新脚本里的 `MODEL_PATH`。
## Tripo 接口说明
DashScope 3D 生成是异步流程:
1. `POST /api/v1/services/aigc/video-generation/3d-generation` 创建任务。
2. `GET /api/v1/tasks/{task_id}` 查询任务。
官方输入方式为 `prompt`、`image`、`images` 三选一,不直接接收 GLB 作为再生成输入。因此本平台的柔性流程会绑定源 GLB 作为仿真资产,并用源 prompt / 源名称派生柔性版本生成任务。
## 验证
```bash
npm run test:run
npm run lint
npm run build
```
MIT for this workbench and its procedural sample. See [third-party notices](THIRD_PARTY_NOTICES.md). Built with React, Three.js, Express and Genesis World; optional generation is provided by Tripo through DashScope. This project is independent of those upstream projects and providers.

17
SECURITY.md Normal file
View File

@ -0,0 +1,17 @@
# Security
Real2Sim Studio v0.1 is a single-user local workbench. The API binds to `127.0.0.1`, rejects non-local Host/Origin values, and does not enable permissive CORS. It is not a multi-user hosted service. Do not expose the API or Vite dev server through a public tunnel or reverse proxy.
Keep provider credentials in `.env.local` or the process environment. Never put them in a browser build, URL, issue, recording or commit. `npm run demo` disables provider access and separates demo tasks from normal user data.
Only run exported scripts and load model files from trusted sources. Material presets are not safety-certified or calibrated for real hardware.
## Known optional runtime risk (2026-10-28)
The separate Python simulation environment is **not vulnerability-clean**. `pip-audit` reported 72 advisory entries for Pillow 11.3.0 (some identifiers overlap). Genesis 1.4.5 depends on MoviePy >=2, whose current 2.2.1 release requires Pillow <12; the audited fixes extend through Pillow 13.0.0. Installing Pillow 13 alone makes that environment dependency-inconsistent. We do not hide or suppress those audit results.
This Alpha demonstrates only the bundled, trusted procedural mesh. Do not use this Python environment to process untrusted images, fonts or models, and do not expose it as an upload service. Use an isolated environment without credentials or private files. The recording validates the sample workflow, not immunity from dependency vulnerabilities. Track upstream MoviePy compatibility before using the optional runtime on external assets.
The Node dependency audit was clean at release preparation. These are point-in-time results, not a security guarantee.
Report vulnerabilities privately through [GitHub private vulnerability reporting](https://github.com/eust-w/real2sim-platform/security/advisories/new). Do not post credentials or sensitive reproductions in public issues.

11
THIRD_PARTY_NOTICES.md Normal file
View File

@ -0,0 +1,11 @@
# Third-party software and assets
The workbench and the procedural sample in `examples/soft-puck/` are distributed under the root MIT license. `scripts/generate-example.py` constructs that mesh from an icosphere; it contains no scanned, purchased, provider-generated, or company-internal asset.
- React, Express, Vite, Three.js, React Three Fiber, Drei and Zod retain their respective upstream licenses. See their installed package license files and the exact dependency versions in `package-lock.json`.
- Lucide icons use the ISC license.
- Genesis World is a separate Apache-2.0 project, installed by the user. Its transitive dependencies retain their own licenses. This repository does not redistribute Genesis, PyTorch or their model assets.
- Tripo through Alibaba Cloud DashScope is an optional external generation service. The root MIT license does not license that service, its models or generated outputs. Users supply their own credentials and must check their service terms and input/output rights.
- Video recordings supplied with a release show this project's procedural sample and workbench. They are not evidence of real-object material identification or physical calibration.
Upstream references: [Genesis World](https://github.com/Genesis-Embodied-AI/genesis-world), [Three.js](https://github.com/mrdoob/three.js), [Lucide](https://github.com/lucide-icons/lucide).

68
docs/README.zh-CN.md Normal file
View File

@ -0,0 +1,68 @@
# Real2Sim Studio
一个本地优先的 3D 资产与 Genesis 仿真导出工作台。
[English](../README.md) | [示例复现](../examples/README.md) | [路线图](ROADMAP.md)
![工作台](images/workbench.png)
预览 GLB、配置材料预设,并导出包含网格、参数和 Python 脚本的 Genesis 场景包。内置自制样例不需要 API Key;可选接入自己的 DashScope 账号,通过 Tripo 生成 3D 资产。
## 无需密钥体验
需要 Node.js 22.12 或更新版本。
```bash
git clone https://github.com/eust-w/real2sim-platform.git
cd real2sim-platform
npm ci
npm run demo
```
打开终端打印的本地网址,点击“加载示例”,调整材料参数,再导出 Genesis 包。演示模式不读取 `.env.local` 中的服务密钥,禁止付费生成调用,并使用独立的 `.data/demo/` 数据目录。
## 运行真实仿真
```bash
npm run example:export
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r examples/requirements.txt
python .data/genesis-packages/example-soft-puck/genesis_scene.py --steps 800 --record
```
首次启动需要编译求解器。无 `--record` 时不请求相机,输出数值指标与轨迹;添加 `--record` 输出实际 Genesis 求解画面。详细环境与注意事项见[示例文档](../examples/README.md)。
**Alpha 安全限制:**可选 Python 依赖链中的 Pillow 存在已知漏洞,上游 MoviePy 的版本约束尚未兼容修复版本。仅在隔离环境中运行可信样例,不要处理外来模型或图片,详见[审计说明](../SECURITY.md)。Node 审计通过不代表 Python 环境无漏洞。
## 已有功能
- 文本、单图、多图生成接口,可选使用用户自己的 DashScope / Tripo 服务。
- 本地任务历史、资产缓存、GLB 预览和模型下载。
- 自制且可按 MIT 许可再分发的离线样例。
- 根据当前滑杆导出材料预设、重力、网格和 Genesis 运行脚本。
- FEM 弹性体导出,以及实验性的 PBD 布料导出路径。
- 中英文界面、单元测试、后端类型检查与桌面/移动端浏览器测试。
## 边界与费用
浏览器是几何预览,不执行物理求解。材料参数是示意预设,不是从真实物体测得的参数。样例是程序化网格,不是扫描重建结果。布料路径没有被这个实体样例验证;充气体和软体机器人预设也不包含压力或执行器模型。
本项目不承诺任意网格都能仿真,不宣称已实现机器人训练、场景重建或经过测量的 sim-to-real 精度。生成式柔性变体是新的提示词生成,并非对原 GLB 的物理变形。
工作台代码和自制样例采用 MIT 许可。外部模型、服务及生成资产另受提供方条款约束,开源工作台不代表底层生成模型开源或生成免费。详见[第三方声明](../THIRD_PARTY_NOTICES.md)。
连接生成服务时,复制 `.env.example` 为 `.env.local` 并设置自己的 Key,然后运行 `npm run dev`。API 只监听 `127.0.0.1:8789`,本项目不是可直接公开部署的多用户服务。不要把密钥或 `.data/` 提交到 Git。
## 开发与反馈
```bash
npm run test:run
npm run lint
npm run build
npx playwright install chromium
npm run test:e2e
npm audit
```
欢迎提交可复现的问题、安装日志和独立复现结果。请阅读[贡献指南](../CONTRIBUTING.md)与[安全说明](../SECURITY.md),不要在 Issue 中包含密钥或私人模型。

24
docs/ROADMAP.md Normal file
View File

@ -0,0 +1,24 @@
# Roadmap
## v0.1 scope
- Local generation through DashScope / Tripo, asset caching and GLB inspection.
- A procedural sample usable without an API key.
- Material presets and Genesis scene export using the current configuration.
- A reproducible FEM example with numerical checks and a recording.
- English / Chinese UI, setup guides, CI and an explicit license.
## Next
- A tested cloth mesh, PBD example and material-specific controls.
- Mesh diagnostics: watertightness, units, orientation and tetrahedralization failures.
- Import user-owned local GLB assets and retain provenance.
- Named, unit-aware material presets and tracked parameter comparisons.
- A provider interface, followed by an additional generation backend.
## Later, after validation
- Additional simulation export formats.
- Batch asset evaluation and task environments.
We do not currently claim calibrated real-to-sim reconstruction, browser physics, pressure-driven inflatable simulation, robot-policy training, or universal compatibility with arbitrary generated meshes.

BIN
docs/images/workbench.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 185 KiB

16
docs/media/README.md Normal file
View File

@ -0,0 +1,16 @@
# Launch media sources
The release videos use only this repository's original procedural sample, actual offline UI recordings, the native Genesis camera recording and original explanatory text. No private task history, provider-generated mesh, cloned voice or third-party music is included.
`storyboard.json` is the bilingual narration and shot list. `scripts/capture-media.mjs` requires a running isolated `npm run demo` server. `scripts/render-launch.py` uses FFmpeg and macOS's built-in Tingting synthetic voice. It is a local production helper, not part of the cross-platform application runtime. Voice synthesis is explicitly disclosed in the video.
```bash
node scripts/capture-media.mjs
# Put the verified simulation.mp4 and metrics.json under artifacts/media/.
uv run --with pillow==13.0.0 python scripts/render-launch.py
uv run --with pillow==13.0.0 python scripts/render-launch.py --language zh --vertical
```
The media tool runs with a separate, current Pillow environment. Do not confuse that environment with the separately audited Genesis dependency chain.
Outputs in `artifacts/release/` are intentionally excluded from Git. Publish only inspected final files as Release assets. Keep platform IDs, public URLs and publication states in a separate ledger; an upload or draft is not public playback.

View File

@ -0,0 +1,12 @@
[
{"duration":7,"title":"从模型到真实求解","enTitle":"From a mesh to a solver","source":"physics","zh":"一个三维模型,怎样真正进入仿真?先看这段真实的柔体求解。","en":"How does a 3D mesh enter a simulation? Start with a real deformable-body solver run."},
{"duration":8,"title":"Real2Sim Studio","enTitle":"Real2Sim Studio","source":"ui","zh":"这是 Real2Sim Studio。把资产预览、材料配置和场景导出,放进一个本地工作台。","en":"Real2Sim Studio brings asset inspection, material presets and scene export into a local workbench."},
{"duration":9,"title":"先跑通一个可信样例","enTitle":"Start with a trusted sample","source":"ui","zh":"加载内置的自制样例。不需要接口密钥,也不会调用付费生成服务。","en":"Load the included procedural sample. No API key, and no paid generation calls."},
{"duration":8,"title":"几何预览,不冒充物理","enTitle":"Geometry preview, not physics","source":"ui","zh":"在浏览器中旋转和检查模型。这里是几何预览,不是物理仿真。","en":"Rotate and inspect the mesh in your browser. This is geometry preview, not physics."},
{"duration":9,"title":"当前参数,进入导出包","enTitle":"Export the settings you chose","source":"settings","zh":"调整材料预设和重力。导出的配置,使用你当前选择的参数。","en":"Adjust material presets and gravity. The exported package uses your current settings."},
{"duration":10,"title":"模型、配置、脚本,一起交付","enTitle":"Mesh, configuration and script","source":"package","zh":"下载完整场景包。模型、配置、运行脚本和依赖说明,都放在一起。","en":"Download a complete scene package: mesh, manifest, material profile, Python script and requirements."},
{"duration":10,"title":"Genesis FEM · CPU","enTitle":"Genesis FEM on CPU","source":"physics","zh":"运行导出的脚本。这里的下落、接触和形变,来自 Genesis 求解器,不是预设动画。","en":"Run the exported script. Falling, contact and deformation here come from Genesis, not a canned animation."},
{"duration":9,"title":"可复现,也有明确边界","enTitle":"Reproducible, with clear limits","source":"metrics","zh":"这是有限元弹性体样例。材料数值是示意预设,不是实物标定,也没有承诺仿真到现实的精度。","en":"This is an FEM elastic sample. Presets are illustrative, not measured material properties or sim-to-real validation."},
{"duration":10,"title":"可选生成服务 · Alpha 限制","enTitle":"Optional generation · Alpha limits","source":"limits","zh":"外部生成服务可能收费。可选 Python 依赖仍有已知风险,请先在隔离环境运行可信样例。","en":"External generation may incur fees. The optional Python dependencies have known advisories. Use the trusted sample in isolation."},
{"duration":10,"title":"下载、复现、反馈","enTitle":"Download. Reproduce. Contribute.","source":"end","zh":"代码和自制样例按 MIT 许可开放。下载并运行示例,欢迎把复现结果和问题反馈给我们。","en":"Code and the procedural sample are MIT-licensed. Run the example and share reproducible results and issues."}
]

123
docs/open-source-launch.md Normal file
View File

@ -0,0 +1,123 @@
# Real2Sim 开源首发与视频方案
核查日期:2026-10-26。代码基线:`3e4c2f4`。
本文保留 2026-10-26 的发布前基线,表内“当前”均指该次核查,不是最新状态。后续实现、验证与限制见 [v0.1.0 Alpha 发布说明](releases/v0.1.0-alpha.1.md);平台实际发布证据单独记录,不以本方案作为已发布证明。
## 判断与定位
项目具备开源工具的基础,但当前应定位为早期工作台。成为有影响力的开源项目,需要真实可复现的使用价值、清楚的技术边界和持续维护,不能由一次宣传或 Star 数承诺保证。
建议展示名:**Real2Sim Studio**,暂保留仓库名 `real2sim-platform`。
当前可用介绍:
> 一个开源准备中的 3D 资产工作台,连接 Tripo 生成、本地资产预览、材料参数配置和 Genesis 场景包导出。
首发目标定位:
> 面向具身智能开发者的可视化资产到仿真工作台,让第一个可运行案例更容易复现。
首要用户是需要把单个物体带入仿真的开发者和研究者。首版围绕一个完整案例展开,不同时承诺场景重建、机器人训练和多个仿真器。
当前生成依赖 DashScope / Tripo 服务;开源工作台不等于开源底层生成模型,也不等于免费生成。项目没有验证几何重建精度、实物材料参数辨识或 sim-to-real 效果。
## 已核查的状态
| 项目 | 本轮结果 | 能证明的范围 |
| --- | --- | --- |
| GitHub | `eust-w/real2sim-platform` 当前 PRIVATE,无许可证 | 还未成为公开授权的开源仓库 |
| 工作区 | 核查前干净 | 本轮方案以当前提交为基线 |
| 功能代码 | 文本 / 单图 / 多图生成、任务管理、缓存、GLB 预览、配置和 Genesis 导出 | 代码具备这些路径,未重新调用付费生成接口 |
| 单元测试 | 3 个文件、7 个测试通过 | 资产缓存、配置和导出包逻辑;不含真实 Genesis 执行 |
| lint / build | 均通过 | 编译与静态检查通过;构建仍提示较大的 JS bundle |
| 本地任务 | 5 条历史任务,其中 2 条记录有本地模型 URL | 可作为后续检查素材的候选,不证明素材权利或仿真成功 |
| 生产依赖审计 | 5 个受影响依赖:1 critical、2 high、2 moderate | npm advisory 分级,不等同于已验证可利用 |
| 敏感文件路径历史 | 本地可达 Git 历史的指定路径检查只出现 `.env.example` | 尚未进行完整内容级历史密钥扫描 |
审计涉及 `proxy-addr`、`body-parser`、`zod`、`qs`、`fflate`;应结合实际调用路径更新和复验。
## 首发前优先处理
1. **修复本地资产到柔性任务的接口契约。** `src/App.tsx` 将本地 `/assets/...` 路径作为 `sourceModelUrl` 提交,而 `server/schemas.ts` 使用绝对 URL 校验。本轮直接调用 schema 已复现 `Invalid URL`。需要覆盖真实本地资产选择流程。
2. **让导出参数与 UI 一致。** `server/genesisPackage.ts` 的脚本生成固定使用 `-9.8`,忽略用户重力。还要核对 UI、flex profile 和 Genesis manifest 对 solver、单位和参数含义的约定。
3. **验证真正的 Genesis 执行。** 当前测试用几个字节代替 GLB,并检查生成的 Python 文本,不能证明四面体化、材质创建和求解运行成功。固定已验证的 Genesis 版本、运行环境和样例,保存运行命令、日志、视频及失败限制。
4. **明确预览与求解的区别。** `Viewport3D.tsx` 的 `SoftBody` 使用 `MeshDistortMaterial` 和周期缩放;它是视觉占位效果。已有 GLB 的 `AssetModel` 路径展示几何,并未执行浏览器物理求解。宣传中的柔体结果必须来自实际仿真录屏。
5. **补齐可复现的首次体验。** 提供可再分发的样例和无需 API Key 的样例模式;生成模式使用用户自己的 Key,并写清服务依赖和费用边界。记录全新环境的安装和运行结果。
6. **完成发布基础。** 明确代码和素材授权主体,选定 LICENSE,补第三方素材说明、英文首页与中文文档、CI、CONTRIBUTING、问题模板、路线图和版本说明。完成依赖修复及完整历史密钥检查。
开放源码与公开托管服务是不同交付。当前后端没有用户鉴权、配额或请求限流,且监听地址没有限制为 loopback;公开演示若连接维护者的付费 Key,需先解决访问和成本控制。首发优先采用本地运行与只读样例展示。
README 现在写着 GitHub 应为 private、Gitea 应为 public。确定开源发布策略后,需要同步更新这段约定;本轮未调整远端可见性。
## 差异化方向
EmbodiedGen 官方项目已经描述了多种生成后端、物理属性、跨仿真器导出及 Genesis 柔体示例。仅以“AI 生成 3D + 导出仿真”做介绍,差异不明显。
建议把工作集中在可验证的体验优势:
- 用户能从干净环境快速复现一个已验证案例,而不是自行调通许多组件。
- 同一资产的来源、尺寸、材料配置、导出版本和运行结果可以追溯。
- 失败时能明确指出问题发生在生成、网格、导出还是求解阶段。
- 后续将生成后端做成可替换适配器;首发先交付一个可靠路径。
这些是产品方向,尚不是当前已经实现的能力。
## 分阶段交付
| 阶段 | 具体交付 | 进入下一阶段的条件 |
| --- | --- | --- |
| 可运行案例 | 修复主流程、锁定仿真版本、一个有授权的样例、运行日志和实录 | 他人在干净环境按说明复现 |
| 开源发布包 | LICENSE、英文 README、中文指南、CI、样例、v0.1.0 说明 | 功能声明与证据一致,授权和密钥检查完成 |
| 视频成片 | 90 秒横版母片、竖版短片、中英文字幕、封面、发布文案 | 所有镜头来自本项目可验证的行为 |
| 对外首发 | GitHub Release 与选定账号发布;记录实际链接 | 仓库和下载入口可访问,各平台发布结果可核对 |
| 用户迭代 | 收集安装失败、仿真失败和复现反馈,完善贡献入口 | 用真实使用与贡献衡量价值 |
首轮观察目标可设为 10 位独立用户成功复现、3 条具体使用反馈,再争取外部贡献。它们是内部验证目标,不是效果保证。
## 90 秒主视频分镜
主题:**从 3D 资产到 Genesis:把生成、配置和导出串起来。**
以下分镜用于主流程修复和真实仿真验证之后。尚未录制的结果不能用生成式视频或浏览器占位动画替代。
| 时长 | 画面 | 旁白方向 |
| --- | --- | --- |
| 0–6 秒 | 同一个样例的输入、GLB 和 Genesis 运行结果并列 | “生成一个 3D 模型之后,怎样把它带进仿真?” |
| 6–16 秒 | 工作台全景与样例选择 | “Real2Sim Studio 把资产预览、材料配置和场景导出放在一起。” |
| 16–32 秒 | 实际生成记录、缓存结果和模型旋转 | “使用 Tripo 生成资产,再把模型保存在本地。” |
| 32–48 秒 | 参数配置、单位、导出的配置文件 | “选择材料并调整参数,导出模型、配置和 Genesis 脚本。” |
| 48–68 秒 | 执行包内命令,展示真实求解画面 | “这是这个样例在已验证环境中的运行结果。” |
| 68–80 秒 | 完整下载包、示例和复现命令 | “同一份样例和配置可以下载复现。” |
| 80–90 秒 | GitHub 地址、版本号、示例入口 | “代码与示例已开放,欢迎复现并反馈。” |
制作要求:
- 录屏时保留同一资产的身份和过程连续性;等待过程可剪辑,但标注加速或省略,耗时只报实测值。
- 参数对比必须重新运行真实仿真;未验证前不使用“物理准确”“一键任意物体”“世界首个”等描述。
- 把 `API generation`、`3D preview`、`Genesis simulation` 的画面阶段标清。
- 画面不包含 API Key、服务令牌、私人任务内容或内部页面。
- 片尾“已开源”只在仓库可见性和许可证发布完成后使用。
- 留存无字幕母片、中文 SRT、英文 SRT、配音音轨和素材来源记录,便于后续适配。
## 平台版本建议
以下是编辑方案,不是平台上传限制或流量保证;实际上传时再核对各账号当时支持的规格。
| 渠道 | 建议版本 | 内容侧重 |
| --- | --- | --- |
| GitHub README / Release | 10–20 秒循环演示,加 90 秒完整视频入口 | 能做什么、如何复现、当前限制 |
| B 站 / YouTube | 90 秒介绍,加独立 3–5 分钟复现教程 | 实际操作、环境和失败排查 |
| 抖音 / 视频号 / 小红书 | 30–45 秒竖版,重新排版而非直接裁切横屏 | 先展示结果,再讲一个使用场景 |
| X / LinkedIn | 30–60 秒英文字幕版本,加项目链接 | 开发者价值、贡献方向 |
| 公众号 / 知乎 | 视频配技术文章与运行步骤 | 技术边界、设计取舍和可复现证据 |
发布次序建议:先让 GitHub 代码和示例可用,再发布完整演示与教程,随后分发短版。每次发布保留平台、账号、文案版本、成片版本、实际 URL 和状态。
具体账号和首发平台尚未指定。本轮未连接社交账号、未上传视频。
## 参考依据
- [GitHub:仓库许可说明](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository):公开仓库需要明确开源许可,不能将可见性视为完整授权。
- [EmbodiedGen 官方仓库](https://github.com/HorizonRobotics/EmbodiedGen):用于核对相邻项目的能力与本项目差异化方向,不代表已完成运行对比。
- 本地证据:`README.md`、`src/App.tsx`、`src/components/Viewport3D.tsx`、`server/schemas.ts`、`server/flex.ts`、`server/genesisPackage.ts`、`server/genesisPackage.test.ts`。

View File

@ -0,0 +1,45 @@
# Real2Sim Studio v0.1.0 Alpha
A local-first asset-to-simulation workbench with a reproducible, original example.
## Included
- Offline mode with a self-authored MIT procedural GLB, isolated task storage and disabled paid generation calls.
- Geometry preview, rotation, English/Chinese UI and material presets.
- Genesis archive export using the current sliders, including gravity. Downloads work from the default hidden `.data` directory.
- A Genesis 1.4.5 FEM runtime, implicit CPU solver, camera recording, trajectory and numerical metrics.
- Optional DashScope / Tripo text, image and multi-image integration. Paid generation was not called during this release verification.
- MIT license, third-party notices, contribution guide, security notes, CI and bilingual documentation.
## Verification on 2026-10-28
Reference environment: macOS 15.7.7 ARM64, Python 3.11.15, Genesis World 1.4.5, PyTorch 2.14.1, Node 24.14.0.
- 16 TypeScript tests and 2 numerical-metric unit tests passed.
- Lint and production build passed. The Three.js-containing JS bundle remains large (about 1.2 MB before gzip).
- Browser smoke tests verified a real GLB archive, gravity 3.7, hidden-directory download, nonblank canvas pixels, rotation, and mobile layout without horizontal overflow.
- The original sample ran for 800 steps of 0.002 s on CPU. All sampled positions were finite; the body fell onto the ground and changed shape. See [metrics](../verification/genesis-cpu.json).
- Shape deformation is measured after removing translation and rigid rotation with Kabsch alignment. It is not an accuracy benchmark against a real object.
- The native recording contains 1.6 seconds of simulated time. The introduction clearly labels slowed replay. Kernel compilation time is not presented as real-time performance.
- Node `npm audit`: 0 reported vulnerabilities at verification time. The optional Python audit is not clean; see below.
## Important limitations
This is an **Alpha**, not a calibrated digital-twin or robot-training system. Material values are illustrative presets; arbitrary meshes may fail tetrahedralization. Cloth export is experimental and is not validated by the solid example. Inflatable/soft-robot presets do not model pressure or actuators. A prompt-generated flexible variant is a new generation, not physical modification of its source mesh.
The optional Python stack has known Pillow advisories constrained by MoviePy's Pillow <12 requirement. Do not process untrusted assets. Use the bundled example in an isolated environment without credentials or private data. See [SECURITY.md](../../SECURITY.md) for the unresolved dependency issue; no advisory was silently ignored.
The browser is only a geometry preview. It does not run Genesis physics. The workbench is a local single-user application; do not expose its development servers publicly.
## Reproduce
```bash
npm ci
npm run demo
# In a separate terminal, after configuring the Python environment:
npm run example:export
python .data/genesis-packages/example-soft-puck/genesis_scene.py --steps 800 --record
python scripts/verify-physics.py
```
See the [example guide](../../examples/README.md). The media uses the original procedural sample, actual UI captures and native Genesis recording, with synthetic narration and no cloned voice or third-party music.

View File

@ -0,0 +1,19 @@
{
"checks": [
"offline import",
"cross-origin rejection",
"real GLB archive",
"current gravity",
"solver mapping",
"desktop canvas pixels",
"interactive rotation",
"browser export current sliders",
"mobile no overflow"
],
"browserErrors": [],
"canvas": {
"colors": 282,
"width": 840,
"height": 578
}
}

View File

@ -0,0 +1,15 @@
{
"genesis_version": "1.4.5",
"backend": "cpu",
"solver": "fem_elastic",
"steps": 800,
"dt": 0.002,
"gravity": 9.8,
"vertex_count": 226,
"initial_center_z": 0.3496624529361725,
"final_center_z": 0.04337442293763161,
"min_vertex_z": 0.0009183835936710238,
"all_finite": true,
"recorded": true,
"max_shape_deformation_m": 0.041962653398513794
}

View File

@ -6,7 +6,7 @@ import tseslint from 'typescript-eslint'
import { defineConfig, globalIgnores } from 'eslint/config'
export default defineConfig([
globalIgnores(['dist']),
globalIgnores(['dist', '.venv', '.data', 'artifacts']),
{
files: ['**/*.{ts,tsx}'],
extends: [

50
examples/README.md Normal file
View File

@ -0,0 +1,50 @@
# Reproduce the procedural example
This sample is an original, watertight mesh generated from an icosphere. The model and generator are MIT-licensed with the repository. It is not a scanned object or a paid generation result. GLB uses metres and Y-up coordinates.
## Inspect and export
```bash
npm ci
npm run demo
```
Open the local Vite URL, select **Load example**, then **Export Genesis Package**. The archive includes the current sliders, mesh, Python entry point, requirements and manifest. Unpack it before running. To export the default example without the UI:
```bash
npm run example:export
```
The default export goes to `.data/genesis-packages/example-soft-puck/`. Re-exporting replaces this generated folder, including previous run results. Keep any results you need elsewhere first.
## Run Genesis
The reference environment is Python 3.11, Genesis World 1.4.5 and PyTorch 2.14.1. Use a virtual environment; the Python dependencies are separate from the Node application.
```bash
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -r examples/requirements.txt
python .data/genesis-packages/example-soft-puck/genesis_scene.py --steps 800
```
The default CPU run writes `metrics.json` and `trajectory.npz` next to the script. The first run may take several minutes to compile kernels. `--backend gpu` requests the Genesis GPU backend but is not covered by the reference CPU verification.
For a native Genesis camera recording:
```bash
python .data/genesis-packages/example-soft-puck/genesis_scene.py --steps 800 --record
```
This writes `simulation.mp4`. Rendering needs a supported graphics environment; a headless numerical success does not imply camera support. `--viewer` additionally opens the interactive Genesis viewer.
## Interpret the result
- FEM elastic material presets use an implicit solver with a 0.002 s step and four substeps. The sample is dropped onto a plane.
- The browser displays geometry only. Deformation footage comes from the exported Genesis scene.
- Gravity comes from the exported manifest, not a fixed value in the Python script.
- Presets are illustrative, not measured silicone properties. No physical calibration or sim-to-real accuracy is claimed.
- Cloth uses a separate experimental PBD path and requires a suitable surface mesh. This solid sample validates the FEM path only.
- Arbitrary generated models may have wrong units, disconnected surfaces or invalid topology. Inspect and repair them before simulation.
To regenerate the source geometry after installing the Python requirements, run `python scripts/generate-example.py`. This overwrites only `examples/soft-puck/model.glb`.

View File

@ -0,0 +1,3 @@
genesis-world==1.4.5
torch==2.14.1
imageio-ffmpeg==0.6.0

Binary file not shown.

874
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@ -1,23 +1,29 @@
{
"name": "real2sim-platform",
"private": true,
"version": "0.0.0",
"version": "0.1.0",
"description": "A local-first 3D asset and Genesis simulation workbench",
"license": "MIT",
"repository": "github:eust-w/real2sim-platform",
"engines": { "node": ">=22.12" },
"type": "module",
"scripts": {
"dev": "concurrently -k \"npm:server\" \"npm:client\"",
"dev": "node scripts/dev.mjs",
"demo": "node scripts/dev.mjs --demo",
"client": "vite",
"server": "tsx watch server/index.ts",
"start": "tsx server/index.ts",
"build": "tsc -b && vite build",
"build": "tsc -b && tsc -p tsconfig.server.json && vite build",
"lint": "eslint .",
"preview": "vite preview",
"test": "vitest",
"test:run": "vitest run"
"test:run": "vitest run",
"test:e2e": "node scripts/smoke.mjs",
"example:export": "tsx scripts/export-example.ts"
},
"dependencies": {
"@react-three/drei": "^10.7.7",
"@react-three/fiber": "^9.6.1",
"cors": "^2.8.6",
"dotenv": "^17.4.2",
"express": "^5.2.1",
"lucide-react": "^1.17.0",
@ -28,7 +34,6 @@
},
"devDependencies": {
"@eslint/js": "^10.0.1",
"@types/cors": "^2.8.19",
"@types/express": "^5.0.6",
"@types/node": "^24.12.3",
"@types/react": "^19.2.14",

56
scripts/capture-media.mjs Normal file
View File

@ -0,0 +1,56 @@
import { chromium } from 'playwright'
import { mkdir, copyFile, writeFile } from 'node:fs/promises'
import path from 'node:path'
const base = process.env.DEMO_URL || 'http://127.0.0.1:5173'
const health = await (await fetch(`${base}/api/health`)).json()
if (!health.demo || health.apiConfigured) throw new Error('Media capture requires isolated offline demo mode.')
const root = path.resolve('artifacts/media')
await mkdir(root, { recursive: true })
await mkdir('docs/images', { recursive: true })
const browser = await chromium.launch({ args: ['--enable-webgl', '--use-angle=swiftshader', '--enable-unsafe-swiftshader'] })
try {
const context = await browser.newContext({ viewport: { width: 1600, height: 900 }, recordVideo: { dir: root, size: { width: 1600, height: 900 } } })
const page = await context.newPage()
await page.addInitScript(() => localStorage.setItem('real2sim.locale', 'en'))
await page.goto(base)
await page.getByRole('button', { name: 'Load example', exact: true }).click()
await page.getByRole('heading', { name: 'Material & Simulation Export' }).waitFor()
await page.waitForTimeout(3000)
await page.screenshot({ path: 'docs/images/workbench.png' })
await page.getByRole('button', { name: 'Rotate model' }).click()
await page.waitForTimeout(8000)
await page.getByRole('button', { name: 'Rotate model' }).click()
const gravity = page.getByLabel('Gravity', { exact: true })
await gravity.focus()
await gravity.press('Home')
for (let i = 0; i < 37; i++) await gravity.press('ArrowRight')
await page.waitForTimeout(2000)
const downloadPromise = page.waitForEvent('download')
await page.getByRole('button', { name: 'Export Genesis Package', exact: true }).click()
await (await downloadPromise).saveAs(path.join(root, 'gravity-3.7-export.tar.gz'))
await page.screenshot({ path: path.join(root, 'settings.png') })
await page.waitForTimeout(3000)
const video = page.video()
await context.close()
await video.saveAs(path.join(root, 'workbench.webm'))
const mobile = await browser.newContext({ viewport: { width: 540, height: 960 }, recordVideo: { dir: root, size: { width: 540, height: 960 } } })
const phone = await mobile.newPage()
await phone.addInitScript(() => localStorage.setItem('real2sim.locale', 'zh'))
await phone.goto(base)
await phone.getByRole('button', { name: '加载样例', exact: true }).click()
await phone.waitForTimeout(3000)
await phone.getByRole('button', { name: 'Rotate model' }).click()
await phone.waitForTimeout(6000)
await phone.getByRole('button', { name: 'Rotate model' }).click()
await phone.getByLabel('重力', { exact: true }).scrollIntoViewIfNeeded()
await phone.waitForTimeout(5000)
const mobileVideo = phone.video()
await mobile.close()
await mobileVideo.saveAs(path.join(root, 'mobile.webm'))
await copyFile('docs/images/workbench.png', path.join(root, 'workbench.png'))
await writeFile(path.join(root, 'capture.json'), JSON.stringify({ capturedAt: new Date().toISOString(), source: 'isolated local offline example', model: 'examples/soft-puck/model.glb', noPaidGeneration: true }, null, 2))
} finally {
await browser.close()
}

30
scripts/dev.mjs Normal file
View File

@ -0,0 +1,30 @@
import { spawn } from 'node:child_process'
import { fileURLToPath } from 'node:url'
const root = fileURLToPath(new URL('../', import.meta.url))
const demo = process.argv.includes('--demo')
const env = { ...process.env }
if (demo) {
env.REAL2SIM_DEMO = '1'
env.REAL2SIM_DATA_ROOT = `${root}.data/demo`
delete env.DASHSCOPE_API_KEY
delete env.REAL2SIM_ASSET_ROOT
delete env.REAL2SIM_GENESIS_PACKAGE_ROOT
}
const children = [
spawn(process.execPath, ['--import', 'tsx', 'server/index.ts'], { cwd: root, env, stdio: 'inherit' }),
spawn(process.execPath, ['node_modules/vite/bin/vite.js', '--host', '127.0.0.1'], { cwd: root, env, stdio: 'inherit' }),
]
let closing = false
function stop(code = 0) {
if (closing) return
closing = true
for (const child of children) child.kill('SIGTERM')
process.exitCode = code
}
for (const child of children) {
child.on('error', (error) => { console.error(error.message); stop(1) })
child.on('exit', (code) => stop(code || 0))
}
process.on('SIGINT', () => stop())
process.on('SIGTERM', () => stop())

View File

@ -0,0 +1,6 @@
import { createGenesisPackage } from '../server/genesisPackage.js'
import { importExample } from '../server/examples.js'
const task = await importExample()
const exported = await createGenesisPackage(task)
console.log(exported.packageDir)

View File

@ -0,0 +1,17 @@
"""Regenerate the MIT-licensed sample from primitives, without external assets."""
from pathlib import Path
import trimesh
root = Path(__file__).resolve().parents[1] / "examples" / "soft-puck"
root.mkdir(parents=True, exist_ok=True)
mesh = trimesh.creation.icosphere(subdivisions=2, radius=0.08)
mesh.apply_scale([1, 0.65, 1])
mesh.visual = trimesh.visual.TextureVisuals(
material=trimesh.visual.material.PBRMaterial(
baseColorFactor=[31, 184, 158, 255], metallicFactor=0.0, roughnessFactor=0.38
)
)
assert mesh.is_watertight and mesh.volume > 0
mesh.export(root / "model.glb")
print(f"Generated watertight sample: {len(mesh.vertices)} vertices, {len(mesh.faces)} faces")

157
scripts/render-launch.py Normal file
View File

@ -0,0 +1,157 @@
"""Build launch videos from captured UI and actual Genesis footage. macOS + FFmpeg."""
import argparse
import json
from pathlib import Path
import shutil
import subprocess
from PIL import Image, ImageDraw, ImageFont
ROOT = Path(__file__).resolve().parents[1]
MEDIA = ROOT / "artifacts/media"
OUT = ROOT / "artifacts/release"
OUT.mkdir(parents=True, exist_ok=True)
FONT = "/System/Library/Fonts/Hiragino Sans GB.ttc"
MONO = "/System/Library/Fonts/Menlo.ttc"
SCENES = json.loads((ROOT / "docs/media/storyboard.json").read_text())
INK, MUTED, GREEN = "#172624", "#566b66", "#087f70"
SIZE = (1920, 1080)
def run(*args):
subprocess.run([str(a) for a in args], check=True)
def probe(file):
return json.loads(subprocess.check_output(["ffprobe", "-v", "error", "-show_format", "-show_streams", "-of", "json", str(file)]))
def text(draw, value, x, y, size=42, fill=INK, width=1740, center=False, mono=False):
font = ImageFont.truetype(MONO if mono else FONT, size)
lines, line = [], ""
for char in value:
if char == "\n" or (line and draw.textlength(line + char, font=font) > width):
lines.append(line)
line = "" if char == "\n" else char
else:
line += char
if line:
lines.append(line)
for line in lines:
px = x + (width - draw.textlength(line, font=font)) / 2 if center else x
draw.text((px, y), line, font=font, fill=fill)
y += int(size * 1.4)
return y
def slide(kind):
image = Image.new("RGB", SIZE, "#edf3f2")
draw = ImageDraw.Draw(image)
if kind == "package":
text(draw, "A REPRODUCIBLE SCENE PACKAGE", 110, 180, 38, GREEN)
rows = [("assets/model.glb", "MIT procedural mesh"), ("manifest.json", "Gravity + solver parameters"), ("flex-profile.json", "Current material preset"), ("genesis_scene.py", "Runnable Genesis entry point"), ("requirements.txt", "Pinned direct dependencies")]
for i, (name, detail) in enumerate(rows):
y = 290 + i * 100
draw.line((110, y + 78, 1800, y + 78), fill="#cad8d4", width=2)
text(draw, name, 110, y, 40, mono=True, width=800)
text(draw, detail, 980, y, 34, MUTED, width=810)
elif kind == "metrics":
metrics = json.loads((MEDIA / "metrics.json").read_text())
rows = [f"Genesis {metrics['genesis_version']} / CPU", f"{metrics['steps']} steps x {metrics['dt']:.3f} s", f"{metrics['vertex_count']} simulated vertices", f"{metrics['max_shape_deformation_m'] * 100:.1f} cm max shape deformation", "Finite trajectory / ground contact"]
text(draw, "MEASURED IN THIS SAMPLE RUN", 110, 180, 38, GREEN)
for i, value in enumerate(rows):
text(draw, value, 110, 290 + i * 100, 47)
text(draw, "Not a material calibration or real-world accuracy benchmark.", 110, 830, 30, MUTED)
elif kind == "limits":
text(draw, "v0.1.0 ALPHA", 110, 180, 40, GREEN)
rows = ["Local single-user workbench", "Optional DashScope / Tripo service fees", "Known Python dependency advisories", "Trusted sample only, in an isolated environment", "Cloth export is experimental"]
for i, value in enumerate(rows):
text(draw, value, 110, 290 + i * 100, 43)
else:
text(draw, "Real2Sim Studio", 110, 190, 100)
text(draw, "MIT-LICENSED WORKBENCH + PROCEDURAL SAMPLE", 115, 355, 31, GREEN)
text(draw, "npm ci\nnpm run demo", 115, 455, 53, mono=True)
text(draw, "github.com/eust-w/real2sim-platform", 115, 690, 48, GREEN)
text(draw, "v0.1.0 Alpha | See README and SECURITY.md before running.", 115, 810, 29, MUTED)
file = MEDIA / f"slide-{kind}.png"
image.save(file)
return file
def overlay(index, scene, lang, vertical=False):
width, height = (1080, 1920) if vertical else SIZE
image = Image.new("RGBA", (width, height), (0, 0, 0, 0))
draw = ImageDraw.Draw(image)
header, footer = (260, 390) if vertical else (132, 184)
draw.rectangle((0, 0, width, header), fill="#f6f9f8")
draw.rectangle((0, height - footer, width, height), fill="#172624")
title = scene["title"] if lang == "zh" else scene["enTitle"]
text(draw, "Real2Sim Studio / v0.1.0 Alpha", 48, 24, 25, GREEN, width=width-96)
text(draw, title, 48, 77, 48 if vertical else 46, width=width - 96)
caption = scene[lang]
text(draw, caption, 60, height - footer + 30, 42 if vertical else 36, "#ffffff", width=width-120, center=True)
label = "Genesis FEM / CPU / 0.2x replay" if scene["source"] == "physics" else "Actual workbench capture" if scene["source"] in ("ui", "settings") else "Reproducible Alpha / see repository"
text(draw, label, 48, height - footer - 43, 23, INK, width=width-96)
text(draw, "Synthetic narration / no cloned voice" if lang == "en" else "合成配音 · 非真人仿声", 48, height-55, 23, "#b5ccc5", width=width-96)
draw.rectangle((0, height - 7, int(width * (index + 1) / (5 if vertical else 10)), height), fill="#24bfa8")
file = MEDIA / f"overlay-{lang}-{'v' if vertical else 'h'}-{index}.png"
image.save(file)
return file
def audio(index, scene):
file = MEDIA / f"voice-{index}.aiff"
if not file.exists():
run("say", "-v", "Tingting", "-r", "190", "-o", file, scene["zh"])
return file
def render(lang="zh", vertical=False):
selection = [0, 2, 4, 6, 9] if vertical else list(range(10))
segments, subtitles, cursor = [], [], 0
for out_index, index in enumerate(selection):
scene = SCENES[index]
duration = scene["duration"]
voice = audio(index, scene)
voice_duration = float(probe(voice)["format"]["duration"])
tempo = max(1, voice_duration / (duration - 0.7))
source = scene["source"]
inputs = []
if source == "physics":
inputs = ["-stream_loop", "-1", "-i", MEDIA / "simulation.mp4"]
transform = "setpts=5*PTS,crop=520:640:380:40,scale=1080:1330,pad=1080:1920:0:270:color=0xedf3f2" if vertical else "setpts=5*PTS,crop=1000:500:140:140,scale=1720:860,pad=1920:1080:100:132:color=0xedf3f2"
elif source in ("ui", "settings"):
file = MEDIA / ("mobile.webm" if vertical else "workbench.webm")
inputs = ["-stream_loop", "-1", "-ss", "10" if source == "settings" else "4", "-i", file]
transform = "scale=900:1600,pad=1080:1920:90:200:color=0xedf3f2" if vertical else "scale=1568:882,pad=1920:1080:176:132:color=0xedf3f2"
else:
file = slide(source)
inputs = ["-loop", "1", "-framerate", "30", "-i", file]
transform = "scale=1080:608,pad=1080:1920:0:520:color=0xedf3f2" if vertical else "null"
cover = overlay(out_index, scene, lang, vertical)
output = MEDIA / f"segment-{lang}-{'v' if vertical else 'h'}-{out_index}.mp4"
run("ffmpeg", "-y", "-loglevel", "error", *inputs, "-loop", "1", "-i", cover, "-i", voice,
"-filter_complex", f"[0:v]{transform},setsar=1,fps=30[base];[base][1:v]overlay=0:0,format=yuv420p[v];[2:a]atempo={tempo:.5f},apad,atrim=duration={duration},loudnorm=I=-16:TP=-1.5:LRA=11[a]",
"-map", "[v]", "-map", "[a]", "-t", duration, "-c:v", "libx264", "-preset", "fast", "-crf", "20", "-threads", "4", "-c:a", "aac", "-b:a", "160k", "-ar", "48000", "-movflags", "+faststart", output)
segments.append(output)
subtitles.append((cursor, cursor + duration, scene[lang]))
cursor += duration
print(f"Rendered {lang} {'vertical' if vertical else 'landscape'} {out_index+1}/{len(selection)}", flush=True)
name = f"real2sim-{'short' if vertical else 'intro'}-{lang}"
listing = MEDIA / f"{name}-concat.txt"
listing.write_text("".join(f"file '{p}'\n" for p in segments))
run("ffmpeg", "-y", "-loglevel", "error", "-f", "concat", "-safe", "0", "-i", listing, "-c", "copy", "-movflags", "+faststart", OUT / f"{name}.mp4")
def timestamp(seconds):
return f"00:{seconds // 60:02}:{seconds % 60:02},000"
(OUT / f"{name}.srt").write_text("\n\n".join(f"{i+1}\n{timestamp(a)} --> {timestamp(b)}\n{caption}" for i, (a, b, caption) in enumerate(subtitles)) + "\n")
run("ffmpeg", "-y", "-loglevel", "error", "-ss", "2", "-i", OUT / f"{name}.mp4", "-frames:v", "1", OUT / f"{name}-cover.jpg")
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--language", choices=["zh", "en", "all"], default="all")
parser.add_argument("--vertical", action="store_true")
args = parser.parse_args()
for language in (["zh", "en"] if args.language == "all" else [args.language]):
render(language, args.vertical)
shutil.copyfile(ROOT / "docs/media/storyboard.json", OUT / "storyboard.json")

107
scripts/smoke.mjs Normal file
View File

@ -0,0 +1,107 @@
import assert from 'node:assert/strict'
import { spawn, execFileSync } from 'node:child_process'
import { createServer as createNetServer } from 'node:net'
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import os from 'node:os'
import path from 'node:path'
import { createServer } from 'vite'
import { chromium } from 'playwright'
const scratch = await mkdtemp(path.join(os.tmpdir(), 'real2sim-smoke-'))
const data = path.join(scratch, '.data')
await mkdir(data)
const evidence = path.resolve('artifacts/smoke')
await mkdir(evidence, { recursive: true })
const portProbe = createNetServer()
await new Promise(resolve => portProbe.listen(0, '127.0.0.1', resolve))
const port = portProbe.address().port
await new Promise(resolve => portProbe.close(resolve))
const base = `http://127.0.0.1:${port}`
const backend = spawn(process.execPath, ['--import', 'tsx', 'server/index.ts'], {
env: { ...process.env, PORT: String(port), REAL2SIM_DEMO: '1', REAL2SIM_DATA_ROOT: data, REAL2SIM_ASSET_ROOT: `${data}/assets`, REAL2SIM_GENESIS_PACKAGE_ROOT: `${data}/genesis-packages`, DASHSCOPE_API_KEY: '' },
stdio: ['ignore', 'pipe', 'pipe'],
})
let logs = ''
backend.stdout.on('data', chunk => { logs += chunk })
backend.stderr.on('data', chunk => { logs += chunk })
let vite, browser
const report = { checks: [], browserErrors: [] }
try {
for (let i = 0; i < 100; i++) {
if (backend.exitCode !== null) throw new Error(`Backend exited: ${logs}`)
try { if ((await fetch(`${base}/api/health`)).ok) break } catch { /* Startup race. */ }
await new Promise(resolve => setTimeout(resolve, 100))
}
assert.equal((await (await fetch(`${base}/api/health`)).json()).apiConfigured, false)
assert.equal((await fetch(`${base}/api/examples/soft-puck`, { method: 'POST', headers: { origin: 'https://untrusted.example' } })).status, 403)
const imported = await (await fetch(`${base}/api/examples/soft-puck`, { method: 'POST' })).json()
assert.ok(imported.task, JSON.stringify(imported))
assert.equal(imported.task.kind, 'sample')
const sim = { ...imported.task.request.simulation, gravity: 3.7 }
const exported = await (await fetch(`${base}/api/genesis/packages`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ sourceTaskId: imported.task.taskId, simulation: sim }) })).json()
assert.equal(exported.manifest.genesis.gravity, 3.7)
const archive = path.join(data, 'sample.tar.gz')
await writeFile(archive, Buffer.from(await (await fetch(base + exported.archiveUrl)).arrayBuffer()))
execFileSync('tar', ['-xzf', archive, '-C', data])
const manifest = JSON.parse(await readFile(path.join(data, 'manifest.json'), 'utf8'))
assert.equal(manifest.genesis.gravity, 3.7)
const model = await readFile(path.join(data, manifest.assets.model))
assert.equal(model.toString('ascii', 0, 4), 'glTF')
assert.equal(JSON.parse(await readFile(path.join(data, 'flex-profile.json'), 'utf8')).solver.type, 'fem_elastic')
report.checks.push('offline import', 'cross-origin rejection', 'real GLB archive', 'current gravity', 'solver mapping')
vite = await createServer({ server: { host: '127.0.0.1', port: 0, proxy: { '/api': base, '/assets': base } } })
await vite.listen()
browser = await chromium.launch({ args: ['--enable-webgl', '--use-angle=swiftshader', '--enable-unsafe-swiftshader'] })
const page = await browser.newPage({ viewport: { width: 1600, height: 1000 } })
await page.addInitScript(() => localStorage.setItem('real2sim.locale', 'en'))
page.on('pageerror', error => report.browserErrors.push(error.message))
await page.goto(vite.resolvedUrls.local[0])
await page.getByRole('button', { name: 'Load example', exact: true }).click()
await page.getByRole('heading', { name: 'Material & Simulation Export' }).waitFor()
await page.locator('canvas').waitFor()
await page.waitForTimeout(1500)
const pixels = await page.locator('canvas').evaluate(canvas => {
const gl = canvas.getContext('webgl2') || canvas.getContext('webgl')
if (!gl) return { colors: 0 }
const pixels = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight * 4)
gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, gl.RGBA, gl.UNSIGNED_BYTE, pixels)
const colors = new Set()
for (let i = 0; i < pixels.length; i += 400) colors.add(`${pixels[i]},${pixels[i + 1]},${pixels[i + 2]}`)
return { colors: colors.size, width: gl.drawingBufferWidth, height: gl.drawingBufferHeight }
})
assert.ok(pixels.colors > 15, `Blank canvas: ${JSON.stringify(pixels)}`)
await page.screenshot({ path: path.join(evidence, 'desktop.png'), fullPage: true })
const before = await page.locator('canvas').screenshot()
await page.getByRole('button', { name: 'Rotate model' }).click()
await page.waitForTimeout(3000)
assert.notDeepEqual(await page.locator('canvas').screenshot(), before)
await page.getByRole('button', { name: 'Rotate model' }).click()
const gravity = page.getByLabel('Gravity', { exact: true })
await gravity.focus()
await gravity.press('Home')
for (let i = 0; i < 37; i++) await gravity.press('ArrowRight')
const downloadPromise = page.waitForEvent('download')
await page.getByRole('button', { name: 'Export Genesis Package', exact: true }).click()
const download = await downloadPromise
await download.saveAs(path.join(evidence, 'browser-export.tar.gz'))
const extractDir = path.join(data, 'browser-export')
await mkdir(extractDir)
execFileSync('tar', ['-xzf', path.join(evidence, 'browser-export.tar.gz'), '-C', extractDir])
assert.equal(JSON.parse(await readFile(path.join(extractDir, 'manifest.json'), 'utf8')).genesis.gravity, 3.7)
await page.setViewportSize({ width: 390, height: 844 })
await page.waitForTimeout(500)
assert.ok(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth))
await page.screenshot({ path: path.join(evidence, 'mobile.png'), fullPage: true })
report.checks.push('desktop canvas pixels', 'interactive rotation', 'browser export current sliders', 'mobile no overflow')
report.canvas = pixels
assert.deepEqual(report.browserErrors, [])
await writeFile(path.join(evidence, 'report.json'), JSON.stringify(report, null, 2))
console.log(JSON.stringify(report, null, 2))
} finally {
await browser?.close()
await vite?.close()
backend.kill('SIGTERM')
if (backend.exitCode === null) await new Promise(resolve => backend.once('exit', resolve))
await rm(scratch, { recursive: true, force: true })
}

View File

@ -0,0 +1,23 @@
import sys
from pathlib import Path
import unittest
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "server"))
from physics_metrics import shape_deformation
class MetricsTests(unittest.TestCase):
def setUp(self):
self.vertices = np.array([[0., 0., 0.], [1., 0., 0.], [0., 1., 0.], [0., 0., 1.]])
def test_rigid_motion_is_not_deformation(self):
rotation = np.array([[0., -1., 0.], [1., 0., 0.], [0., 0., 1.]])
self.assertLess(shape_deformation(self.vertices, self.vertices @ rotation + 5), 1e-12)
def test_compression_is_deformation(self):
self.assertGreater(shape_deformation(self.vertices, self.vertices * [1., 1., 0.5]), 0.1)
if __name__ == "__main__":
unittest.main()

20
scripts/verify-physics.py Normal file
View File

@ -0,0 +1,20 @@
"""Check the bundled drop-test evidence, not arbitrary materials or meshes."""
import json
from pathlib import Path
import sys
import numpy as np
root = Path(sys.argv[1] if len(sys.argv) > 1 else ".data/genesis-packages/example-soft-puck")
metrics = json.loads((root / "metrics.json").read_text())
trajectory = np.load(root / "trajectory.npz")["positions"]
assert metrics["solver"] == "fem_elastic"
assert metrics["gravity"] == 9.8
assert metrics["steps"] >= 500
assert np.isfinite(trajectory).all()
assert np.abs(trajectory).max() < 1
assert 0.30 < metrics["initial_center_z"] < 0.40
assert 0.0 < metrics["final_center_z"] < 0.15
assert metrics["min_vertex_z"] > -0.01
assert 0.001 < metrics["max_shape_deformation_m"] < 0.2
print("PASS: finite trajectory, falling body, ground contact, nonzero deformation")

View File

@ -38,7 +38,7 @@ const assetSpecs = [
export const assetRoot = process.env.REAL2SIM_ASSET_ROOT
? path.resolve(process.env.REAL2SIM_ASSET_ROOT)
: path.join(process.cwd(), '.data', 'assets')
: path.resolve(process.env.REAL2SIM_DATA_ROOT || '.data', 'assets')
function safePathSegment(value: string) {
return value.replace(/[^a-zA-Z0-9._-]/g, '_')

7
server/config.ts Normal file
View File

@ -0,0 +1,7 @@
import dotenv from 'dotenv'
// A demo must never inherit the user's paid provider key or private task history.
if (process.env.REAL2SIM_DEMO !== '1') {
dotenv.config({ path: '.env.local', quiet: true })
dotenv.config({ quiet: true })
}

33
server/examples.ts Normal file
View File

@ -0,0 +1,33 @@
import { copyFile, mkdir } from 'node:fs/promises'
import path from 'node:path'
import { assetRoot } from './assetStore.js'
import { upsertTask, type StoredTask } from './taskStore.js'
export async function importExample() {
const taskId = 'example-soft-puck'
const dir = path.join(assetRoot, taskId)
await mkdir(dir, { recursive: true })
await copyFile(new URL('../examples/soft-puck/model.glb', import.meta.url), path.join(dir, 'model.glb'))
const now = new Date().toISOString()
const task: StoredTask = {
taskId,
kind: 'sample',
title: 'Sample / Silicone puck',
status: 'SUCCEEDED',
request: {
sourceName: 'Procedural silicone puck (MIT example)',
sourceModelUrl: '/assets/example-soft-puck/model.glb',
model: 'Tripo/Tripo-H3.1',
parameters: { texture_quality: 'standard', geometry_quality: 'standard', pbr: true, texture: true },
simulation: { material: 'silicone', elasticity: 0, damping: 10, thickness: 20, deformation: 70, gravity: 9.8 },
},
output: {
task_id: taskId,
task_status: 'SUCCEEDED',
results: [{ local_model_url: '/assets/example-soft-puck/model.glb' }],
},
createdAt: now,
updatedAt: now,
}
return upsertTask(task)
}

View File

@ -53,6 +53,7 @@ describe('flex profile helpers', () => {
expect(profile.schema).toBe('real2sim.flexProfile.v1')
expect(profile.physics.elasticity).toBe(0.5)
expect(profile.solver.preserveVolume).toBe(false)
expect(profile.solver.type).toBe('pbd_cloth')
expect(profile.solver.parametersArePresets).toBe(true)
})
})

View File

@ -44,10 +44,9 @@ export function createSimulationExport(input: CreateFlexTaskInput) {
gravity: input.simulation.gravity,
},
solver: {
type: 'position-based-dynamics',
substeps: 8,
selfCollision: true,
preserveVolume: input.simulation.material !== 'cloth',
type: input.simulation.material === 'cloth' ? 'pbd_cloth' : 'fem_elastic',
substeps: 4,
parametersArePresets: true,
},
generatedAt: new Date().toISOString(),
}

View File

@ -33,7 +33,7 @@ function createFlexTask(material: 'silicone' | 'cloth' = 'silicone'): StoredTask
damping: 42,
thickness: 3.2,
deformation: 64,
gravity: 9.8,
gravity: 3.7,
},
},
output: {
@ -93,7 +93,9 @@ describe('Genesis package export', () => {
const script = await readFile(path.join(exported.packageDir, 'genesis_scene.py'), 'utf8')
expect(script).toContain('gs.materials.FEM.Elastic')
expect(script).toContain('force_retet=True')
expect(script).toContain('E=70400')
expect(exported.manifest.genesis.parameters.E).toBe(70400)
expect(exported.manifest.genesis.gravity).toBe(3.7)
expect(script).toContain('gravity = -float(config["gravity"])')
const profile = JSON.parse(await readFile(path.join(exported.packageDir, 'flex-profile.json'), 'utf8'))
expect(profile.schema).toBe('real2sim.flexProfile.v1')
@ -111,6 +113,6 @@ describe('Genesis package export', () => {
const script = await readFile(path.join(exported.packageDir, 'genesis_scene.py'), 'utf8')
expect(script).toContain('gs.materials.PBD.Cloth')
expect(script).toContain('stretch_stiffness=0.754')
expect(exported.manifest.genesis.parameters.stretchCompliance).toBeCloseTo(1e-6 / 73, 12)
})
})

View File

@ -13,8 +13,8 @@ type GenesisParameters = {
nu?: number
rho?: number
damping: number
stretchStiffness?: number
bendStiffness?: number
stretchCompliance?: number
bendCompliance?: number
maxVolume?: number
}
@ -30,6 +30,7 @@ export type GenesisPackageManifest = {
script: string
solver: GenesisSolver
material: SimulationProfile['material']
gravity: number
parameters: GenesisParameters
}
notes: string[]
@ -44,14 +45,15 @@ export type GenesisPackageResult = {
export const genesisPackageRoot = process.env.REAL2SIM_GENESIS_PACKAGE_ROOT
? path.resolve(process.env.REAL2SIM_GENESIS_PACKAGE_ROOT)
: path.join(process.cwd(), '.data', 'genesis-packages')
: path.resolve(process.env.REAL2SIM_DATA_ROOT || '.data', 'genesis-packages')
function safePathSegment(value: string) {
return value.replace(/[^a-zA-Z0-9._-]/g, '_')
if (!/^[a-zA-Z0-9_-][a-zA-Z0-9._-]{0,199}$/.test(value)) throw new Error('Invalid task ID.')
return value
}
function isFlexTask(task: StoredTask): task is StoredTask & { request: CreateFlexTaskInput } {
return task.kind === 'flex' && typeof (task.request as Partial<CreateFlexTaskInput>).simulation === 'object'
return (task.kind === 'flex' || task.kind === 'sample') && typeof (task.request as Partial<CreateFlexTaskInput>).simulation === 'object'
}
function round(value: number, digits = 3) {
@ -82,7 +84,7 @@ function localAssetPath(localUrl?: string) {
function chooseModelAsset(task: StoredTask) {
const result = task.output?.results?.[0]
return localAssetPath(result?.local_pbr_model_url) || localAssetPath(result?.local_base_model_url)
return localAssetPath(result?.local_pbr_model_url) || localAssetPath(result?.local_base_model_url) || localAssetPath(result?.local_model_url)
}
function choosePreviewAsset(task: StoredTask) {
@ -103,9 +105,9 @@ function materialParameters(simulation: SimulationProfile): { solver: GenesisSol
return {
solver: 'pbd_cloth',
parameters: {
damping: round(0.85 + (simulation.damping / 100) * 0.14),
stretchStiffness: round(clamp(0.25 + simulation.elasticity * 0.007, 0.25, 0.98)),
bendStiffness: round(clamp(0.04 + simulation.thickness * 0.012 + simulation.damping * 0.001, 0.04, 0.45)),
damping: round(0.001 + simulation.damping * 0.0001, 5),
stretchCompliance: Number((1e-6 / (1 + simulation.elasticity)).toPrecision(6)),
bendCompliance: Number((1e-4 / (1 + simulation.thickness)).toPrecision(6)),
},
}
}
@ -128,104 +130,6 @@ function materialParameters(simulation: SimulationProfile): { solver: GenesisSol
}
}
function pythonString(value: string) {
return JSON.stringify(value)
}
function genesisScript(manifest: GenesisPackageManifest) {
const gravity = manifest.genesis.material === 'cloth' ? -9.8 : -9.8
const params = manifest.genesis.parameters
if (manifest.genesis.solver === 'pbd_cloth') {
return `from pathlib import Path
import genesis as gs
ROOT = Path(__file__).resolve().parent
MODEL_PATH = ROOT / "assets" / ${pythonString(path.basename(manifest.assets.model))}
gs.init()
scene = gs.Scene(
sim_options=gs.options.SimOptions(
dt=0.01,
substeps=4,
gravity=(0, 0, ${gravity}),
),
pbd_options=gs.options.PBDOptions(
iterations=10,
damping=${params.damping},
),
)
scene.add_entity(gs.morphs.Plane())
soft_body = scene.add_entity(
gs.morphs.Mesh(
file=str(MODEL_PATH),
pos=(0, 0, 0.5),
scale=1.0,
parse_glb_with_zup=True,
),
material=gs.materials.PBD.Cloth(
stretch_stiffness=${params.stretchStiffness},
bend_stiffness=${params.bendStiffness},
),
)
scene.build()
for _ in range(1000):
scene.step()
`
}
return `from pathlib import Path
import genesis as gs
ROOT = Path(__file__).resolve().parent
MODEL_PATH = ROOT / "assets" / ${pythonString(path.basename(manifest.assets.model))}
gs.init()
scene = gs.Scene(
sim_options=gs.options.SimOptions(
dt=0.001,
substeps=4,
gravity=(0, 0, ${gravity}),
),
fem_options=gs.options.FEMOptions(
dt=1e-3,
damping=${params.damping},
),
)
scene.add_entity(gs.morphs.Plane())
soft_body = scene.add_entity(
gs.morphs.Mesh(
file=str(MODEL_PATH),
pos=(0, 0, 0.5),
scale=1.0,
parse_glb_with_zup=True,
force_retet=True,
maxvolume=${params.maxVolume},
),
material=gs.materials.FEM.Elastic(
E=${params.E},
nu=${params.nu},
rho=${params.rho},
),
)
scene.build()
for _ in range(1000):
scene.step()
`
}
function packageReadme(manifest: GenesisPackageManifest) {
return `# Genesis flexible object package
@ -234,8 +138,9 @@ Task: ${manifest.title}
## Run
\`\`\`bash
pip install genesis-world
python genesis_scene.py
pip install -r requirements.txt
python genesis_scene.py --steps 600
# Optional: --record saves simulation.mp4; --viewer opens an interactive viewer.
\`\`\`
## Contents
@ -247,7 +152,7 @@ python genesis_scene.py
## Important
The GLB stores geometry and visual material only. It becomes deformable in Genesis because \`genesis_scene.py\` loads it with Genesis deformable material settings. If Genesis cannot tetrahedralize the GLB for FEM, convert the model to a clean watertight OBJ and update \`MODEL_PATH\` in the script.
The GLB stores geometry and visual material only. It becomes deformable in Genesis because \`genesis_scene.py\` loads it with Genesis deformable material settings. If Genesis cannot tetrahedralize the GLB for FEM, convert the model to a clean watertight OBJ and update \`assets.model\` in \`manifest.json\`. Material values are illustrative presets, not calibrated measurements. External asset rights remain with their respective owners.
`
}
@ -355,6 +260,7 @@ export async function createGenesisPackage(task: StoredTask): Promise<GenesisPac
script: 'genesis_scene.py',
solver: genesis.solver,
material: simulation.material,
gravity: simulation.gravity,
parameters: genesis.parameters,
},
notes: [
@ -366,7 +272,9 @@ export async function createGenesisPackage(task: StoredTask): Promise<GenesisPac
await writeFile(path.join(packageDir, 'manifest.json'), JSON.stringify(manifest, null, 2))
await writeFile(path.join(packageDir, 'flex-profile.json'), JSON.stringify(createSimulationExport(task.request), null, 2))
await writeFile(path.join(packageDir, 'genesis_scene.py'), genesisScript(manifest))
await copyFile(new URL('./genesis_scene.py', import.meta.url), path.join(packageDir, 'genesis_scene.py'))
await copyFile(new URL('./physics_metrics.py', import.meta.url), path.join(packageDir, 'physics_metrics.py'))
await copyFile(new URL('../examples/requirements.txt', import.meta.url), path.join(packageDir, 'requirements.txt'))
await writeFile(path.join(packageDir, 'README.md'), packageReadme(manifest))
await mkdir(genesisPackageRoot, { recursive: true })

97
server/genesis_scene.py Normal file
View File

@ -0,0 +1,97 @@
"""Run an exported Real2Sim scene. All geometry and parameters come from the package."""
import argparse
import json
from pathlib import Path
import genesis as gs
import numpy as np
from physics_metrics import shape_deformation
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--steps", type=int, default=600)
parser.add_argument("--backend", choices=["cpu", "gpu"], default="cpu")
parser.add_argument("--record", action="store_true")
parser.add_argument("--viewer", action="store_true")
parser.add_argument("--capture-every", type=int, default=5)
args = parser.parse_args()
if args.steps < 1 or args.capture_every < 1:
parser.error("steps and capture-every must be positive")
root = Path(__file__).resolve().parent
manifest = json.loads((root / "manifest.json").read_text())
config = manifest["genesis"]
params = config["parameters"]
gravity = -float(config["gravity"])
is_cloth = config["solver"] == "pbd_cloth"
dt = 0.005 if is_cloth else 0.002
gs.init(backend=gs.cpu if args.backend == "cpu" else gs.gpu, seed=0, logging_level="warning")
options = dict(
sim_options=gs.options.SimOptions(dt=dt, substeps=4, gravity=(0, 0, gravity)),
show_viewer=args.viewer,
)
if is_cloth:
options["pbd_options"] = gs.options.PBDOptions(max_stretch_solver_iterations=10)
else:
options["fem_options"] = gs.options.FEMOptions(
use_implicit_solver=True, n_newton_iterations=4, n_pcg_iterations=100,
damping=params["damping"], damping_alpha=params["damping"],
)
scene = gs.Scene(**options)
scene.add_entity(gs.morphs.Plane(), surface=gs.surfaces.Default(color=(0.84, 0.87, 0.88)))
mesh_options = dict(file=str(root / manifest["assets"]["model"]), pos=(0, 0, 0.35), scale=1.0)
if not is_cloth:
mesh_options.update(force_retet=True, maxvolume=params["maxVolume"])
material = (
gs.materials.PBD.Cloth(
stretch_compliance=params["stretchCompliance"], bending_compliance=params["bendCompliance"],
air_resistance=params["damping"],
)
if is_cloth else gs.materials.FEM.Elastic(E=params["E"], nu=params["nu"], rho=params["rho"])
)
body = scene.add_entity(
gs.morphs.Mesh(**mesh_options), material=material,
surface=gs.surfaces.Default(color=(0.12, 0.72, 0.62)),
)
camera = None
if args.record:
camera = scene.add_camera(res=(1280, 720), pos=(0.65, -0.85, 0.55), lookat=(0, 0, 0.16), fov=40, GUI=False)
print("Building Genesis scene...", flush=True)
scene.build()
print("Running solver...", flush=True)
def positions():
return body.get_state().pos.detach().cpu().numpy().reshape(-1, 3).copy()
initial = positions()
frames = [initial]
max_deformation = 0.0
min_z = float(initial[:, 2].min())
if camera:
camera.start_recording(save_to_filename=str(root / "simulation.mp4"), fps=50)
for step in range(args.steps):
scene.step()
if (step + 1) % args.capture_every == 0 or step == args.steps - 1:
pos = positions()
if not np.isfinite(pos).all() or np.abs(pos).max() > 100:
raise RuntimeError(f"Unstable solver state at step {step + 1}")
max_deformation = max(max_deformation, shape_deformation(initial, pos))
min_z = min(min_z, float(pos[:, 2].min()))
frames.append(pos)
if camera:
camera.stop_recording()
final = positions()
np.savez_compressed(root / "trajectory.npz", positions=np.asarray(frames), dt=dt * args.capture_every)
metrics = {
"genesis_version": gs.__version__, "backend": args.backend, "solver": config["solver"],
"steps": args.steps, "dt": dt, "gravity": config["gravity"], "vertex_count": len(initial),
"initial_center_z": float(initial[:, 2].mean()), "final_center_z": float(final[:, 2].mean()),
"min_vertex_z": min_z, "max_shape_deformation_m": max_deformation,
"all_finite": bool(np.isfinite(final).all()), "recorded": bool(camera),
}
(root / "metrics.json").write_text(json.dumps(metrics, indent=2) + "\n")
print(json.dumps(metrics, indent=2))
if __name__ == "__main__":
main()

View File

@ -1,24 +1,37 @@
import cors from 'cors'
import dotenv from 'dotenv'
import './config.js'
import express from 'express'
import { createReadStream } from 'node:fs'
import { stat } from 'node:fs/promises'
import path from 'node:path'
import { randomUUID } from 'node:crypto'
import { ZodError } from 'zod'
import { assetRoot, cacheTaskAssets, cacheTasksAssets } from './assetStore.js'
import { buildFlexiblePrompt, createSimulationExport } from './flex.js'
import { createGenesisPackage } from './genesisPackage.js'
import { createFlexTaskSchema, createTripoTaskSchema } from './schemas.js'
import { createGenesisPackage, genesisPackageRoot } from './genesisPackage.js'
import { createFlexTaskSchema, createTripoTaskSchema, exportGenesisSchema } from './schemas.js'
import { importExample } from './examples.js'
import { getTask, mergeTaskOutput, readTasks, upsertTask, writeTasks, type StoredTask } from './taskStore.js'
import { createTripoTask, queryTripoTask } from './tripoClient.js'
dotenv.config({ path: '.env.local' })
dotenv.config()
const app = express()
const port = Number(process.env.PORT || 8787)
const port = Number(process.env.PORT || 8789)
const demo = process.env.REAL2SIM_DEMO === '1'
app.use(cors())
app.disable('x-powered-by')
app.use((req, res, next) => {
const localNames = new Set(['localhost', '127.0.0.1', '[::1]'])
const origin = req.get('origin')
try {
if (!localNames.has(req.hostname) || (origin && !localNames.has(new URL(origin).hostname))) {
res.status(403).json({ error: 'This workbench accepts local requests only.' })
return
}
} catch {
res.status(403).json({ error: 'Invalid request origin.' })
return
}
next()
})
app.use(express.json({ limit: '1mb' }))
app.use('/assets', express.static(assetRoot))
@ -41,11 +54,58 @@ app.get('/api/health', (_req, res) => {
res.json({
ok: true,
service: 'real2sim-platform',
apiConfigured: Boolean(process.env.DASHSCOPE_API_KEY),
apiConfigured: !demo && Boolean(process.env.DASHSCOPE_API_KEY),
demo,
defaultModel: 'Tripo/Tripo-H3.1',
})
})
app.post('/api/examples/soft-puck', async (_req, res, next) => {
try {
res.status(201).json({ task: await importExample() })
} catch (error) { next(error) }
})
app.post('/api/genesis/packages', async (req, res, next) => {
try {
const input = exportGenesisSchema.parse(req.body)
const source = await getTask(input.sourceTaskId)
if (!source) {
res.status(404).json({ error: 'Source asset not found.' })
return
}
const task: StoredTask = {
...source,
taskId: `export-${randomUUID()}`,
kind: 'flex',
request: {
sourceTaskId: source.taskId,
sourceName: source.title,
model: 'Tripo/Tripo-H3.1',
parameters: { texture_quality: 'standard', geometry_quality: 'standard', pbr: true, texture: true },
simulation: input.simulation,
},
}
const exported = await createGenesisPackage(task)
res.status(201).json({ manifest: exported.manifest, archiveUrl: `/api/genesis/exports/${task.taskId}/download` })
} catch (error) { next(error) }
})
app.get('/api/genesis/exports/:exportId/download', async (req, res, next) => {
if (!/^export-[0-9a-f-]{36}$/.test(req.params.exportId)) {
res.status(400).json({ error: 'Invalid export ID.' })
return
}
try {
const archive = path.join(genesisPackageRoot, `${req.params.exportId}.tar.gz`)
await stat(archive)
res.download(archive, `${req.params.exportId}.tar.gz`, { dotfiles: 'allow' })
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') res.status(404).json({ error: 'Export not found.' })
else next(error)
}
})
app.get('/api/tripo/tasks', async (_req, res, next) => {
try {
res.json({ tasks: await readTasksWithLocalAssets() })
@ -88,6 +148,11 @@ app.post('/api/tripo/tasks', async (req, res, next) => {
app.get('/api/tripo/tasks/:taskId', async (req, res, next) => {
try {
const stored = await getTask(req.params.taskId)
if (stored?.kind === 'sample') {
res.json({ task: stored })
return
}
const response = await queryTripoTask(req.params.taskId)
if (!response.output) {
throw new Error('DashScope did not return task output.')
@ -218,7 +283,7 @@ app.use((error: unknown, _req: express.Request, res: express.Response, _next: ex
res.status(500).json({ error: message })
})
app.listen(port, () => {
app.listen(port, '127.0.0.1', () => {
console.log(`real2sim assets served from ${path.relative(process.cwd(), assetRoot) || assetRoot}`)
console.log(`real2sim API listening on http://localhost:${port}`)
})

13
server/physics_metrics.py Normal file
View File

@ -0,0 +1,13 @@
"""Separate shape change from translation and rigid rotation in a vertex trajectory."""
import numpy as np
def shape_deformation(initial, current):
reference = initial - initial.mean(axis=0)
observed = current - current.mean(axis=0)
# Kabsch alignment removes rigid motion without fitting scale or shear.
u, _, vh = np.linalg.svd(reference.T @ observed)
if np.linalg.det(u @ vh) < 0:
u[:, -1] *= -1
aligned = reference @ (u @ vh)
return float(np.linalg.norm(observed - aligned, axis=1).max())

14
server/schemas.test.ts Normal file
View File

@ -0,0 +1,14 @@
import { describe, expect, it } from 'vitest'
import { createFlexTaskSchema } from './schemas.js'
const simulation = { material: 'silicone', elasticity: 72, damping: 42, thickness: 3.2, deformation: 64, gravity: 3.7 }
describe('flex source asset validation', () => {
it.each(['/assets/sample/model.glb', 'https://example.com/model.glb'])('accepts an asset URL: %s', (sourceModelUrl) => {
expect(createFlexTaskSchema.safeParse({ sourceModelUrl, simulation }).success).toBe(true)
})
it.each(['/assets/../secret', '/assets/%2e%2e/secret', '/assets/a%2fb/model.glb', '//example.com/a.glb', 'file:///etc/passwd', 'javascript:alert(1)', '/other/model.glb'])('rejects unsafe source: %s', (sourceModelUrl) => {
expect(createFlexTaskSchema.safeParse({ sourceModelUrl, simulation }).success).toBe(false)
})
})

View File

@ -1,5 +1,17 @@
import { z } from 'zod'
function isAssetUrl(value: string) {
if (value.startsWith('/assets/')) {
return /^\/assets\/[a-zA-Z0-9_-][a-zA-Z0-9._-]*\/[a-zA-Z0-9_-][a-zA-Z0-9._-]*$/.test(value)
}
try {
const url = new URL(value)
return (url.protocol === 'https:' || url.protocol === 'http:') && !url.username && !url.password
} catch {
return false
}
}
export const tripoModelSchema = z.enum(['Tripo/Tripo-H3.1', 'Tripo/Tripo-P1.0'])
export const generationModeSchema = z.enum(['text', 'image', 'multiImage'])
@ -23,7 +35,7 @@ export const createTripoTaskSchema = z
prompt: z.string().trim().max(1024).optional(),
image: z.string().trim().url().optional().or(z.literal('')),
images: z.array(multiImageSlotSchema).length(4).optional(),
parameters: tripoParametersSchema.default({}),
parameters: tripoParametersSchema.default({ texture_quality: 'standard', geometry_quality: 'standard', pbr: true, texture: true }),
})
.superRefine((data, ctx) => {
if (data.mode === 'text' && !data.prompt) {
@ -67,7 +79,7 @@ export const createFlexTaskSchema = z.object({
sourceTaskId: z.string().trim().optional(),
sourceName: z.string().trim().max(160).optional(),
sourcePrompt: z.string().trim().max(1024).optional(),
sourceModelUrl: z.string().trim().url().optional().or(z.literal('')),
sourceModelUrl: z.string().trim().refine(isAssetUrl, 'Expected a local asset path or HTTP(S) URL.').optional().or(z.literal('')),
behaviorPrompt: z.string().trim().max(1024).optional(),
model: tripoModelSchema.default('Tripo/Tripo-H3.1'),
parameters: tripoParametersSchema.default({
@ -82,3 +94,8 @@ export const createFlexTaskSchema = z.object({
export type CreateTripoTaskInput = z.infer<typeof createTripoTaskSchema>
export type CreateFlexTaskInput = z.infer<typeof createFlexTaskSchema>
export type SimulationProfile = z.infer<typeof simulationProfileSchema>
export const exportGenesisSchema = z.object({
sourceTaskId: z.string().regex(/^[a-zA-Z0-9_-][a-zA-Z0-9._-]{0,159}$/),
simulation: simulationProfileSchema,
})

View File

@ -3,12 +3,12 @@ import path from 'node:path'
import type { CreateFlexTaskInput, CreateTripoTaskInput } from './schemas.js'
import type { DashScopeTaskOutput, TaskStatus } from './tripoClient.js'
const dataDir = path.join(process.cwd(), '.data')
const dataDir = path.resolve(process.env.REAL2SIM_DATA_ROOT || path.join(process.cwd(), '.data'))
const storePath = path.join(dataDir, 'tasks.json')
export type StoredTask = {
taskId: string
kind: 'generation' | 'flex'
kind: 'generation' | 'flex' | 'sample'
title: string
status: TaskStatus
requestId?: string

View File

@ -44,6 +44,7 @@ export type TripoPayload = {
const apiBase = process.env.DASHSCOPE_API_BASE || 'https://dashscope.aliyuncs.com/api/v1'
function apiKey() {
if (process.env.REAL2SIM_DEMO === '1') throw new Error('Generation is disabled in offline demo mode.')
const key = process.env.DASHSCOPE_API_KEY
if (!key) {
throw new Error('DASHSCOPE_API_KEY is not configured.')
@ -79,7 +80,7 @@ export function toTripoPayload(input: CreateTripoTaskInput): TripoPayload {
}
if (input.mode === 'multiImage') {
payload.input.images = input.images?.map((item) =>
payload.input.images = input.images?.map((item): Record<string, string> =>
item.file_token ? { type: item.type, file_token: item.file_token } : {},
)
}

View File

@ -1,11 +1,7 @@
.app-shell {
min-height: 100vh;
color: #18211f;
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.82), rgba(247, 250, 248, 0.96)),
radial-gradient(circle at 18% 0%, rgba(94, 234, 212, 0.14), transparent 31%),
radial-gradient(circle at 100% 4%, rgba(249, 115, 115, 0.12), transparent 28%),
#f5f8f6;
background: #f7f9fa;
}
.topbar {
@ -13,7 +9,7 @@
top: 0;
z-index: 10;
display: grid;
grid-template-columns: minmax(260px, 1fr) auto minmax(230px, 1fr);
grid-template-columns: minmax(210px, 1fr) auto auto;
gap: 22px;
align-items: center;
padding: 18px 28px;
@ -158,7 +154,7 @@
border: 1px solid rgba(31, 41, 38, 0.12);
border-radius: 8px;
background: rgba(255, 255, 255, 0.78);
box-shadow: 0 18px 42px rgba(15, 23, 42, 0.08);
box-shadow: none;
}
.panel-form,
@ -436,14 +432,27 @@ output {
position: relative;
min-height: 0;
flex: 1;
background:
linear-gradient(90deg, rgba(15, 23, 42, 0.05) 1px, transparent 1px),
linear-gradient(0deg, rgba(15, 23, 42, 0.05) 1px, transparent 1px),
radial-gradient(circle at center, rgba(94, 234, 212, 0.16), transparent 42%),
#f7faf8;
background-size: 34px 34px, 34px 34px, auto, auto;
background: #f1f5f4;
}
.icon-button {
display: grid;
place-items: center;
width: 34px;
height: 34px;
padding: 0;
color: #176c60;
background: #fff;
border: 1px solid #cbd9d5;
border-radius: 6px;
cursor: pointer;
}
.icon-button[aria-pressed='true'] { background: #d4f8ee; }
.viewport-toolbar > div { margin-right: auto; }
button:disabled { cursor: not-allowed; opacity: 0.55; }
.button-grid > * { min-width: 0; padding: 8px; font-size: 13px; }
.canvas-fallback {
display: grid;
width: 100%;
@ -702,6 +711,7 @@ output {
.top-actions {
justify-content: flex-start;
flex-wrap: wrap;
}
.workbench {
@ -712,6 +722,9 @@ output {
.result-panel {
min-height: 500px;
}
.viewport-panel { grid-row: 1; height: 500px; }
.topbar { position: static; gap: 12px; }
}
@media (max-width: 680px) {

View File

@ -28,6 +28,8 @@ import {
health,
isLiveStatus,
listTasks,
importExample,
exportGenesis,
refreshTask,
type FlexRequest,
type GenerationRequest,
@ -272,7 +274,7 @@ function Timeline({
className={task.taskId === currentTaskId ? 'timeline-item selected' : 'timeline-item'}
onClick={() => onSelect(task)}
>
<span>{task.kind === 'flex' ? t.tabs.flex : t.tabs.generation}</span>
<span>{task.kind === 'sample' ? t.app.demoMode : task.kind === 'flex' ? t.tabs.flex : t.tabs.generation}</span>
<strong>{task.title}</strong>
<StatusPill status={task.status} t={t} />
</button>
@ -290,6 +292,7 @@ function App() {
const t = messages[locale]
const [activeTab, setActiveTab] = useState<TabId>('generation')
const [apiConfigured, setApiConfigured] = useState(false)
const [demoMode, setDemoMode] = useState(false)
const [tasks, setTasks] = useState<StoredTask[]>([])
const [currentTask, setCurrentTask] = useState<StoredTask | null>(null)
const [error, setError] = useState('')
@ -313,7 +316,6 @@ function App() {
const [sourceModelUrl, setSourceModelUrl] = useState('')
const [behaviorPrompt, setBehaviorPrompt] = useState('保持原始结构比例,转化为适合抓取、挤压和碰撞仿真的柔性版本')
const [simulation, setSimulation] = useState<SimulationProfile>(defaultSimulation)
const [lastSimulationExport, setLastSimulationExport] = useState<Record<string, unknown> | null>(null)
const successfulTasks = useMemo(() => tasks.filter((task) => getModelUrl(task)), [tasks])
@ -328,7 +330,7 @@ function App() {
useEffect(() => {
health()
.then((response) => setApiConfigured(response.apiConfigured))
.then((response) => { setApiConfigured(response.apiConfigured); setDemoMode(response.demo) })
.catch(() => setApiConfigured(false))
loadTasks().catch((loadError) => setError(loadError instanceof Error ? loadError.message : t.errors.generic))
}, [loadTasks, t.errors.generic])
@ -359,13 +361,43 @@ function App() {
}
function selectTaskForFlex(task: StoredTask) {
setCurrentTask(task)
setSourceTaskId(task.taskId)
setSourceName(task.title)
setSourcePrompt(getSourcePrompt(task))
setSourceModelUrl(getModelUrl(task))
if (task.request.simulation && typeof task.request.simulation === 'object') {
setSimulation(task.request.simulation as SimulationProfile)
}
setActiveTab('flex')
}
async function loadExample() {
setBusy(true)
setError('')
try {
const { task } = await importExample()
selectTaskForFlex(task)
await loadTasks()
} catch (e) { setError(e instanceof Error ? e.message : t.errors.generic) }
finally { setBusy(false) }
}
async function downloadGenesis() {
setBusy(true)
setError('')
try {
const id = sourceTaskId || currentTask?.taskId
if (!id) return
const result = await exportGenesis(id, simulation)
const link = document.createElement('a')
link.href = result.archiveUrl
link.download = ''
link.click()
} catch (e) { setError(e instanceof Error ? e.message : t.errors.generic) }
finally { setBusy(false) }
}
async function handleGenerationSubmit(event: React.FormEvent) {
event.preventDefault()
setBusy(true)
@ -415,7 +447,6 @@ function App() {
try {
const response = await createFlexTask(payload)
setCurrentTask(response.task)
setLastSimulationExport(response.simulationExport)
await loadTasks()
} catch (submitError) {
setError(submitError instanceof Error ? submitError.message : t.errors.generic)
@ -439,9 +470,7 @@ function App() {
}
function exportSimulation() {
const profile =
lastSimulationExport ||
({
const profile = {
schema: 'real2sim.flexProfile.v1',
source: {
taskId: sourceTaskId || undefined,
@ -457,7 +486,13 @@ function App() {
maxDeformation: simulation.deformation / 100,
gravity: simulation.gravity,
},
} satisfies Record<string, unknown>)
solver: {
type: simulation.material === 'cloth' ? 'pbd_cloth' : 'fem_elastic',
substeps: 4,
parametersArePresets: true,
},
generatedAt: new Date().toISOString(),
} satisfies Record<string, unknown>
downloadJson('real2sim-flex-profile.json', profile)
}
@ -488,9 +523,12 @@ function App() {
</button>
</nav>
<div className="top-actions">
<button type="button" className="secondary-button" onClick={loadExample} disabled={busy}>
<Box size={16} />{t.app.loadExample}
</button>
<span className={apiConfigured ? 'api-status ready' : 'api-status missing'}>
{apiConfigured ? <CheckCircle2 size={15} /> : <AlertTriangle size={15} />}
{apiConfigured ? t.app.apiReady : t.app.apiMissing}
{demoMode ? t.app.demoMode : apiConfigured ? t.app.apiReady : t.app.apiMissing}
</span>
<button type="button" className="language-button" onClick={() => setLocale(locale === 'zh' ? 'en' : 'zh')}>
<Languages size={16} />
@ -603,7 +641,7 @@ function App() {
/>
</div>
<button type="submit" className="submit-button" disabled={busy}>
<button type="submit" className="submit-button" disabled={busy || !apiConfigured}>
<Sparkles size={17} />
{busy ? t.generation.generating : t.generation.generate}
</button>
@ -637,17 +675,17 @@ function App() {
</select>
</Field>
<Field label={t.flex.sourcePrompt}>
{apiConfigured ? <Field label={t.flex.sourcePrompt}>
<textarea value={sourcePrompt} onChange={(event) => setSourcePrompt(event.target.value)} />
</Field>
</Field> : null}
<Field label={t.flex.sourceModelUrl}>
<input value={sourceModelUrl} onChange={(event) => setSourceModelUrl(event.target.value)} />
</Field>
<Field label={t.flex.behavior}>
{apiConfigured ? <Field label={t.flex.behavior}>
<textarea value={behaviorPrompt} onChange={(event) => setBehaviorPrompt(event.target.value)} placeholder={t.flex.behaviorPlaceholder} />
</Field>
</Field> : null}
<Field label={t.flex.material}>
<Segmented<SimulationProfile['material']>
@ -673,6 +711,7 @@ function App() {
<Field label={label as string} inline key={key as string}>
<input
type="range"
aria-label={label as string}
min={min as number}
max={max as number}
step={step as number}
@ -690,10 +729,13 @@ function App() {
</div>
<div className="button-grid">
<button type="submit" className="submit-button" disabled={busy}>
<button type="button" className="submit-button" onClick={downloadGenesis} disabled={busy || !sourceTaskId}>
<Package size={17} />{t.task.exportGenesis}
</button>
{apiConfigured ? <button type="submit" className="secondary-button" disabled={busy}>
<Waves size={17} />
{busy ? t.generation.generating : t.flex.generate}
</button>
</button> : null}
<button type="button" className="secondary-button" onClick={exportSimulation}>
<SlidersHorizontal size={17} />
{t.flex.export}

View File

@ -1,7 +1,8 @@
import { Bounds, Center, ContactShadows, Float, MeshDistortMaterial, OrbitControls, useGLTF } from '@react-three/drei'
import { Canvas, useFrame } from '@react-three/fiber'
import { Component, Suspense, useMemo, useRef, type ErrorInfo, type ReactNode } from 'react'
import type { Mesh, Object3D } from 'three'
import { Bounds, Center, ContactShadows, OrbitControls, useGLTF } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { Component, Suspense, useMemo, useState, type ErrorInfo, type ReactNode } from 'react'
import { Box3, Vector3, type Mesh, type Object3D } from 'three'
import { RotateCw } from 'lucide-react'
import type { SimulationProfile, StoredTask } from '../lib/api'
import { getModelUrl, getPreviewImageUrl } from '../lib/api'
import type { Messages } from '../lib/i18n'
@ -36,41 +37,12 @@ class CanvasErrorBoundary extends Component<{ children: ReactNode }, { failed: b
}
}
function SoftBody({ simulation }: { simulation: SimulationProfile }) {
const meshRef = useRef<Mesh>(null)
const distort = 0.14 + simulation.deformation / 220
const speed = 0.75 + simulation.elasticity / 80
useFrame(({ clock }) => {
const mesh = meshRef.current
if (!mesh) return
mesh.rotation.y = clock.elapsedTime * 0.28
mesh.rotation.x = Math.sin(clock.elapsedTime * 0.45) * 0.08
const squeeze = 1 + Math.sin(clock.elapsedTime * speed) * (simulation.elasticity / 900)
mesh.scale.set(1.25 / squeeze, 0.92 * squeeze, 1.05)
})
const colorByMaterial = {
silicone: '#5eead4',
cloth: '#f97373',
inflatable: '#a3e635',
softRobot: '#7dd3fc',
}[simulation.material]
function Placeholder() {
return (
<Float speed={1.15} rotationIntensity={0.35} floatIntensity={0.32}>
<mesh ref={meshRef} castShadow receiveShadow position={[0, 0.25, 0]}>
<icosahedronGeometry args={[1.45, 18]} />
<MeshDistortMaterial
color={colorByMaterial}
distort={distort}
speed={speed}
roughness={0.46}
metalness={0.03}
clearcoat={0.36}
/>
</mesh>
</Float>
<mesh>
<boxGeometry args={[1.5, 1.5, 1.5]} />
<meshStandardMaterial color="#9aaba7" wireframe />
</mesh>
)
}
@ -100,6 +72,8 @@ function AssetModel({ url }: { url: string }) {
const { scene } = useGLTF(url)
const previewScene = useMemo(() => {
const clonedScene = scene.clone(true)
const size = new Box3().setFromObject(clonedScene).getSize(new Vector3())
clonedScene.scale.multiplyScalar(2.2 / Math.max(size.x, size.y, size.z, 0.0001))
enableShadows(clonedScene)
return clonedScene
}, [scene])
@ -113,7 +87,8 @@ function AssetModel({ url }: { url: string }) {
)
}
export function Viewport3D({ task, simulation, labels }: Viewport3DProps) {
export function Viewport3D({ task, labels }: Viewport3DProps) {
const [rotating, setRotating] = useState(false)
const modelUrl = getModelUrl(task)
const previewImageUrl = getPreviewImageUrl(task)
const status = modelUrl ? labels.modelReady : previewImageUrl ? labels.imageReady : task ? labels.preview : labels.idle
@ -125,6 +100,7 @@ export function Viewport3D({ task, simulation, labels }: Viewport3DProps) {
<span className="panel-label">{labels.label}</span>
<strong>{status}</strong>
</div>
{modelUrl ? <button type="button" className="icon-button" aria-label="Rotate model" title="Rotate model" aria-pressed={rotating} onClick={() => setRotating(!rotating)}><RotateCw size={18} /></button> : null}
{modelUrl ? (
<a href={modelUrl} target="_blank" rel="noreferrer" className="mini-link">
GLB
@ -133,18 +109,18 @@ export function Viewport3D({ task, simulation, labels }: Viewport3DProps) {
</div>
<div className="canvas-wrap">
{modelUrl || !previewImageUrl ? (
<CanvasErrorBoundary>
<CanvasErrorBoundary key={modelUrl}>
<Canvas camera={{ position: [3.7, 2.5, 4.2], fov: 42 }} gl={{ preserveDrawingBuffer: true, antialias: true }} shadows>
<ambientLight intensity={1} />
<hemisphereLight args={['#ffffff', '#9ca3af', 1.25]} />
<directionalLight position={[4, 5, 3]} intensity={2.4} castShadow />
<pointLight position={[-3, 2.2, -2]} intensity={0.65} color="#5eead4" />
<Suspense fallback={<SoftBody simulation={simulation} />}>
{modelUrl ? <AssetModel url={modelUrl} /> : <SoftBody simulation={simulation} />}
<Suspense fallback={<Placeholder />}>
{modelUrl ? <AssetModel url={modelUrl} /> : <Placeholder />}
</Suspense>
<StageGrid />
<ContactShadows position={[0, -1.08, 0]} opacity={0.28} scale={5} blur={2.4} far={3} />
<OrbitControls enablePan={false} minDistance={3.4} maxDistance={8} />
<OrbitControls enablePan={false} minDistance={3.4} maxDistance={8} autoRotate={rotating} autoRotateSpeed={1.5} />
</Canvas>
</CanvasErrorBoundary>
) : (

View File

@ -23,7 +23,7 @@ export type TripoTaskOutput = {
export type StoredTask = {
taskId: string
kind: 'generation' | 'flex'
kind: 'generation' | 'flex' | 'sample'
title: string
status: TaskStatus
requestId?: string
@ -85,7 +85,17 @@ async function requestJson<T>(url: string, options?: RequestInit): Promise<T> {
}
export function health() {
return requestJson<{ ok: boolean; apiConfigured: boolean; defaultModel: string }>('/api/health')
return requestJson<{ ok: boolean; apiConfigured: boolean; demo: boolean; defaultModel: string }>('/api/health')
}
export function importExample() {
return requestJson<{ task: StoredTask }>('/api/examples/soft-puck', { method: 'POST' })
}
export function exportGenesis(sourceTaskId: string, simulation: SimulationProfile) {
return requestJson<{ archiveUrl: string }>('/api/genesis/packages', {
method: 'POST', body: JSON.stringify({ sourceTaskId, simulation }),
})
}
export function listTasks() {

View File

@ -3,8 +3,10 @@ export type Locale = 'zh' | 'en'
const dictionaries = {
zh: {
app: {
name: 'real2sim 平台',
subtitle: 'Tripo H3.1 资产生成与柔性仿真工作台',
name: 'Real2Sim Studio',
subtitle: '资产 · 材料 · Genesis',
loadExample: '加载样例',
demoMode: '离线样例',
apiReady: 'API 已配置',
apiMissing: 'API 未配置',
language: '语言',
@ -42,7 +44,7 @@ const dictionaries = {
noTexture: '无贴图',
},
flex: {
title: '柔性物体生成',
title: '材料与仿真导出',
source: '源资产',
sourceModelUrl: '源 GLB',
sourcePrompt: '源描述',
@ -53,10 +55,10 @@ const dictionaries = {
cloth: '织物',
inflatable: '充气膜',
softRobot: '软体机器人',
elasticity: '弹性',
elasticity: '刚度预设',
damping: '阻尼',
thickness: '厚度 mm',
deformation: '最大形变',
deformation: '柔软度预设',
gravity: '重力',
generate: '生成柔性版本',
export: '导出仿真配置',
@ -91,9 +93,9 @@ const dictionaries = {
},
viewport: {
label: '3D 预览',
modelReady: '模型链接已生成',
modelReady: '几何预览',
imageReady: '渲染图已生成',
preview: '软体预览',
preview: '等待资产',
idle: '等待生成',
},
timeline: {
@ -107,8 +109,10 @@ const dictionaries = {
},
en: {
app: {
name: 'real2sim Platform',
subtitle: 'Tripo H3.1 asset generation and soft simulation workbench',
name: 'Real2Sim Studio',
subtitle: 'Assets · Materials · Genesis',
loadExample: 'Load example',
demoMode: 'Offline example',
apiReady: 'API configured',
apiMissing: 'API missing',
language: 'Language',
@ -146,7 +150,7 @@ const dictionaries = {
noTexture: 'Untextured',
},
flex: {
title: 'Flexible Object Generation',
title: 'Material & Simulation Export',
source: 'Source Asset',
sourceModelUrl: 'Source GLB',
sourcePrompt: 'Source Description',
@ -157,10 +161,10 @@ const dictionaries = {
cloth: 'Cloth',
inflatable: 'Inflatable',
softRobot: 'Soft Robot',
elasticity: 'Elasticity',
elasticity: 'Stiffness preset',
damping: 'Damping',
thickness: 'Thickness mm',
deformation: 'Max Deformation',
deformation: 'Softness preset',
gravity: 'Gravity',
generate: 'Generate Soft Variant',
export: 'Export Simulation Profile',
@ -195,9 +199,9 @@ const dictionaries = {
},
viewport: {
label: '3D Preview',
modelReady: 'Model link generated',
modelReady: 'Geometry preview',
imageReady: 'Rendered image generated',
preview: 'Soft preview',
preview: 'Waiting for asset',
idle: 'Waiting',
},
timeline: {

12
tsconfig.server.json Normal file
View File

@ -0,0 +1,12 @@
{
"compilerOptions": {
"target": "ES2023",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"noEmit": true,
"strict": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["server/**/*.ts"]
}

View File

@ -6,8 +6,8 @@ export default defineConfig({
plugins: [react()],
server: {
proxy: {
'/api': 'http://localhost:8787',
'/assets': 'http://localhost:8787',
'/api': `http://127.0.0.1:${process.env.PORT || 8789}`,
'/assets': `http://127.0.0.1:${process.env.PORT || 8789}`,
},
},
})