✨ add local assets and genesis export workflow
This commit is contained in:
parent
9dc5064ad8
commit
3e4c2f444e
25
.githooks/commit-msg
Executable file
25
.githooks/commit-msg
Executable file
@ -0,0 +1,25 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
node --input-type=module - "$1" <<'NODE'
|
||||
import { readFileSync } from 'node:fs';
|
||||
|
||||
const messagePath = process.argv[2];
|
||||
const message = readFileSync(messagePath, 'utf8');
|
||||
const subject = message
|
||||
.split(/\r?\n/)
|
||||
.find((line) => line.trim() && !line.trimStart().startsWith('#'))
|
||||
?.trim();
|
||||
|
||||
if (!subject) {
|
||||
console.error('commit-msg: commit subject is empty.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const startsWithEmoji = /^(?:\p{Emoji_Presentation}|\p{Extended_Pictographic}\uFE0F?)/u.test(subject);
|
||||
|
||||
if (!startsWithEmoji) {
|
||||
console.error('commit-msg: commit subject must start with an emoji, for example "✨ add asset cache".');
|
||||
process.exit(1);
|
||||
}
|
||||
NODE
|
||||
11
.githooks/pre-commit
Executable file
11
.githooks/pre-commit
Executable file
@ -0,0 +1,11 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
echo "pre-commit: running unit tests"
|
||||
npm run test:run
|
||||
|
||||
echo "pre-commit: running lint"
|
||||
npm run lint
|
||||
|
||||
echo "pre-commit: running production build"
|
||||
npm run build
|
||||
95
README.md
95
README.md
@ -7,9 +7,10 @@
|
||||
- 模型生成:支持文生 3D、单图生 3D、多图生 3D。
|
||||
- Tripo 参数:支持 `Tripo-H3.1` / `Tripo-P1.0`、贴图质量、几何精度、PBR、无贴图生成。
|
||||
- 异步任务:后端创建 DashScope 任务,本地保存任务记录,前端支持轮询与手动刷新。
|
||||
- 柔性物体:从资产库选择源 GLB,生成柔性版本 prompt,提交 H3.1 派生任务。
|
||||
- 柔性物体:从本地资产库选择源 GLB,生成柔性版本 prompt,提交 H3.1 派生任务。
|
||||
- 仿真配置:按材料、弹性、阻尼、厚度、最大形变、重力导出 `real2sim.flexProfile.v1` JSON。
|
||||
- 资产库:展示历史任务、预览图、GLB 链接,并可一键作为柔性源。
|
||||
- 本地资产库:DashScope 返回的预览图和 GLB 会缓存到 `.data/assets/<taskId>/`,前端预览、下载和柔性源选择优先使用本地 `/assets/...` URL。
|
||||
- Genesis 导出:柔性任务可导出 `.tar.gz` 包,包含本地 GLB、`flex-profile.json`、`manifest.json`、`genesis_scene.py` 和说明文档。
|
||||
|
||||
## 技术栈
|
||||
|
||||
@ -37,6 +38,49 @@ PORT=8787
|
||||
|
||||
后端地址:`http://localhost:8787`
|
||||
|
||||
## 仓库与提交规范
|
||||
|
||||
当前 GitHub 远端:
|
||||
|
||||
- `origin`:`https://github.com/eust-w/real2sim-platform.git`
|
||||
- GitHub 仓库可见性要求:private。
|
||||
|
||||
Gitea 远端要求:
|
||||
|
||||
- 远端名建议使用 `gitea`。
|
||||
- Gitea 仓库可见性要求:public。
|
||||
- 配置方式:`git remote add gitea <gitea-public-repo-url>`。
|
||||
|
||||
本仓库使用仓库级 Git hooks:
|
||||
|
||||
```bash
|
||||
git config core.hooksPath .githooks
|
||||
```
|
||||
|
||||
提交前 `.githooks/pre-commit` 会依次执行:
|
||||
|
||||
```bash
|
||||
npm run test:run
|
||||
npm run lint
|
||||
npm run build
|
||||
```
|
||||
|
||||
提交信息由 `.githooks/commit-msg` 校验,标题必须以 emoji 开头,例如:
|
||||
|
||||
```bash
|
||||
git commit -m "✨ add local asset cache"
|
||||
```
|
||||
|
||||
每次代码变更后的标准流程:
|
||||
|
||||
```bash
|
||||
git status --short
|
||||
git add <changed-files>
|
||||
git commit -m "✨ describe the change"
|
||||
git push origin main
|
||||
git push gitea main
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
- `GET /api/health`
|
||||
@ -44,6 +88,53 @@ PORT=8787
|
||||
- `POST /api/tripo/tasks`
|
||||
- `GET /api/tripo/tasks/:taskId`
|
||||
- `POST /api/flex/tasks`
|
||||
- `GET /api/genesis/packages/:taskId`
|
||||
- `GET /api/genesis/packages/:taskId/download`
|
||||
|
||||
## 柔性物体实现原理
|
||||
|
||||
当前柔性流程不是本地软体物理求解器,也不会把已有 GLB 直接做网格形变再导出。
|
||||
|
||||
普通版本和柔性版本的区别:
|
||||
|
||||
- 普通版本:直接按用户 prompt / 图片生成 3D 资产,几何和贴图不带仿真语义。
|
||||
- 柔性版本:以源资产描述为基础,加入软硅胶、织物、充气膜或软体机器人等材料提示,并绑定一份 `real2sim.flexProfile.v1` 物理配置。
|
||||
- 两者的 GLB 都只是视觉/几何资产;真正进入仿真器后是否柔性,取决于 Genesis 脚本是否给它分配 FEM / PBD 等柔体材料。
|
||||
|
||||
实际流程是:
|
||||
|
||||
1. 前端选择源资产、材料类型和弹性/阻尼/厚度/最大形变/重力等参数。
|
||||
2. 后端把这些配置转换成材料感知 prompt,例如软硅胶、织物、充气膜或软体机器人结构。
|
||||
3. 后端用这个 prompt 调用 DashScope `Tripo/Tripo-H3.1` 创建一个新的 3D 生成任务。
|
||||
4. 平台同时导出 `real2sim.flexProfile.v1`,把物理参数归一化为仿真配置,供后续 real-to-sim 管线或求解器使用。
|
||||
5. 任务成功后,后端把远程渲染图和 GLB 缓存到本地资产库,前端 3D 视口从本地 `/assets/...` 加载 GLB 预览。
|
||||
|
||||
## Genesis 导出包
|
||||
|
||||
柔性任务结果面板会显示“导出 Genesis 包”。导出包位于 `.data/genesis-packages/<taskId>.tar.gz`,内容包括:
|
||||
|
||||
- `assets/model.glb`:本地缓存的柔性版本 GLB。
|
||||
- `assets/preview.*`:本地渲染预览图,如果任务有预览图。
|
||||
- `flex-profile.json`:UI 仿真滑块导出的物理配置。
|
||||
- `manifest.json`:Genesis solver 和材料参数映射。
|
||||
- `genesis_scene.py`:可直接运行的 Genesis 场景脚本。
|
||||
- `README.md`:包内运行说明和网格限制说明。
|
||||
|
||||
默认映射规则:
|
||||
|
||||
- `silicone` / `inflatable` / `softRobot`:导出为 Genesis FEM Elastic,脚本使用 `gs.materials.FEM.Elastic(...)`。
|
||||
- `cloth`:导出为 Genesis PBD Cloth,脚本使用 `gs.materials.PBD.Cloth(...)`。
|
||||
|
||||
运行方式:
|
||||
|
||||
```bash
|
||||
tar -xzf <taskId>-genesis-flex.tar.gz
|
||||
cd <taskId>
|
||||
pip install genesis-world
|
||||
python genesis_scene.py
|
||||
```
|
||||
|
||||
注意:GLB 本身不携带柔体物理属性。它在 Genesis 中变成柔体,是因为导出包里的 `genesis_scene.py` 给它分配了 Genesis 柔体 solver 和 material。如果 Genesis 无法对 GLB 做 FEM 四面体化,需要先把模型转换成干净、封闭的 OBJ,再更新脚本里的 `MODEL_PATH`。
|
||||
|
||||
## Tripo 接口说明
|
||||
|
||||
|
||||
118
server/assetStore.test.ts
Normal file
118
server/assetStore.test.ts
Normal file
@ -0,0 +1,118 @@
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import type { StoredTask } from './taskStore.js'
|
||||
|
||||
const originalAssetRoot = process.env.REAL2SIM_ASSET_ROOT
|
||||
let temporaryAssetRoot = ''
|
||||
|
||||
function createStoredTask(): StoredTask {
|
||||
return {
|
||||
taskId: 'task/one?',
|
||||
kind: 'generation',
|
||||
title: 'local cache test',
|
||||
status: 'SUCCEEDED',
|
||||
request: {
|
||||
mode: 'text',
|
||||
model: 'Tripo/Tripo-H3.1',
|
||||
prompt: 'a soft robotic gripper',
|
||||
parameters: {
|
||||
texture_quality: 'detailed',
|
||||
geometry_quality: 'standard',
|
||||
pbr: true,
|
||||
texture: true,
|
||||
},
|
||||
},
|
||||
output: {
|
||||
task_id: 'task/one?',
|
||||
task_status: 'SUCCEEDED',
|
||||
results: [
|
||||
{
|
||||
rendered_image_url: 'https://example.test/render.png',
|
||||
pbr_model_url: 'https://example.test/model.glb',
|
||||
},
|
||||
],
|
||||
},
|
||||
createdAt: '2026-06-03T00:00:00.000Z',
|
||||
updatedAt: '2026-06-03T00:00:00.000Z',
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
temporaryAssetRoot = await mkdtemp(path.join(os.tmpdir(), 'real2sim-assets-'))
|
||||
process.env.REAL2SIM_ASSET_ROOT = temporaryAssetRoot
|
||||
vi.resetModules()
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(temporaryAssetRoot, { recursive: true, force: true })
|
||||
vi.unstubAllGlobals()
|
||||
vi.resetModules()
|
||||
if (originalAssetRoot === undefined) {
|
||||
delete process.env.REAL2SIM_ASSET_ROOT
|
||||
} else {
|
||||
process.env.REAL2SIM_ASSET_ROOT = originalAssetRoot
|
||||
}
|
||||
})
|
||||
|
||||
describe('asset store', () => {
|
||||
it('caches task result assets under local /assets URLs', async () => {
|
||||
const fetchMock = vi.fn(async () => new Response(new Uint8Array([1, 2, 3]), { status: 200 }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
const { cacheTaskAssets } = await import('./assetStore.js')
|
||||
const cachedTask = await cacheTaskAssets(createStoredTask())
|
||||
const result = cachedTask.output?.results?.[0]
|
||||
|
||||
expect(result?.local_rendered_image_url).toBe('/assets/task_one_/render.png')
|
||||
expect(result?.local_pbr_model_url).toBe('/assets/task_one_/pbr.glb')
|
||||
expect(await readFile(path.join(temporaryAssetRoot, 'task_one_', 'render.png'))).toEqual(Buffer.from([1, 2, 3]))
|
||||
expect(await readFile(path.join(temporaryAssetRoot, 'task_one_', 'pbr.glb'))).toEqual(Buffer.from([1, 2, 3]))
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2)
|
||||
|
||||
await cacheTaskAssets(cachedTask)
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('caches generic model_url results returned by non-PBR Tripo tasks', async () => {
|
||||
const fetchMock = vi.fn(async () => new Response(new Uint8Array([4, 5, 6]), { status: 200 }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
const task = createStoredTask()
|
||||
task.output!.results = [
|
||||
{
|
||||
rendered_image_url: 'https://example.test/render.webp',
|
||||
model_url: 'https://example.test/model.glb',
|
||||
},
|
||||
]
|
||||
|
||||
const { cacheTaskAssets } = await import('./assetStore.js')
|
||||
const cachedTask = await cacheTaskAssets(task)
|
||||
const result = cachedTask.output?.results?.[0]
|
||||
|
||||
expect(result?.local_rendered_image_url).toBe('/assets/task_one_/render.webp')
|
||||
expect(result?.local_model_url).toBe('/assets/task_one_/model.glb')
|
||||
expect(await readFile(path.join(temporaryAssetRoot, 'task_one_', 'model.glb'))).toEqual(Buffer.from([4, 5, 6]))
|
||||
})
|
||||
|
||||
it('restores local URLs from existing files when signed remote URLs are no longer fetchable', async () => {
|
||||
await mkdir(path.join(temporaryAssetRoot, 'task_one_'), { recursive: true })
|
||||
await writeFile(path.join(temporaryAssetRoot, 'task_one_', 'render.webp'), new Uint8Array([7, 8, 9]))
|
||||
const fetchMock = vi.fn(async () => new Response('expired', { status: 403 }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
const task = createStoredTask()
|
||||
task.output!.results = [
|
||||
{
|
||||
rendered_image_url: 'https://example.test/render.webp?auth_key=expired',
|
||||
},
|
||||
]
|
||||
|
||||
const { cacheTaskAssets } = await import('./assetStore.js')
|
||||
const cachedTask = await cacheTaskAssets(task)
|
||||
|
||||
expect(cachedTask.output?.results?.[0]?.local_rendered_image_url).toBe('/assets/task_one_/render.webp')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
162
server/assetStore.ts
Normal file
162
server/assetStore.ts
Normal file
@ -0,0 +1,162 @@
|
||||
import { access, mkdir, writeFile } from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import type { StoredTask } from './taskStore.js'
|
||||
|
||||
type AssetSpec = {
|
||||
remoteKey: 'rendered_image_url' | 'pbr_model_url' | 'base_model_url' | 'model_url'
|
||||
localKey: 'local_rendered_image_url' | 'local_pbr_model_url' | 'local_base_model_url' | 'local_model_url'
|
||||
baseName: string
|
||||
fallbackExtension: string
|
||||
}
|
||||
|
||||
const assetSpecs = [
|
||||
{
|
||||
remoteKey: 'rendered_image_url',
|
||||
localKey: 'local_rendered_image_url',
|
||||
baseName: 'render',
|
||||
fallbackExtension: '.png',
|
||||
},
|
||||
{
|
||||
remoteKey: 'pbr_model_url',
|
||||
localKey: 'local_pbr_model_url',
|
||||
baseName: 'pbr',
|
||||
fallbackExtension: '.glb',
|
||||
},
|
||||
{
|
||||
remoteKey: 'base_model_url',
|
||||
localKey: 'local_base_model_url',
|
||||
baseName: 'base',
|
||||
fallbackExtension: '.glb',
|
||||
},
|
||||
{
|
||||
remoteKey: 'model_url',
|
||||
localKey: 'local_model_url',
|
||||
baseName: 'model',
|
||||
fallbackExtension: '.glb',
|
||||
},
|
||||
] as const satisfies readonly AssetSpec[]
|
||||
|
||||
export const assetRoot = process.env.REAL2SIM_ASSET_ROOT
|
||||
? path.resolve(process.env.REAL2SIM_ASSET_ROOT)
|
||||
: path.join(process.cwd(), '.data', 'assets')
|
||||
|
||||
function safePathSegment(value: string) {
|
||||
return value.replace(/[^a-zA-Z0-9._-]/g, '_')
|
||||
}
|
||||
|
||||
function extensionFromUrl(url: string) {
|
||||
try {
|
||||
const extension = path.extname(new URL(url).pathname).toLowerCase()
|
||||
return extension || undefined
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
function extensionFromContentType(contentType: string | null) {
|
||||
if (!contentType) return undefined
|
||||
if (contentType.includes('model/gltf-binary')) return '.glb'
|
||||
if (contentType.includes('image/png')) return '.png'
|
||||
if (contentType.includes('image/jpeg')) return '.jpg'
|
||||
if (contentType.includes('image/webp')) return '.webp'
|
||||
return undefined
|
||||
}
|
||||
|
||||
async function fileExists(filePath: string) {
|
||||
try {
|
||||
await access(filePath)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function localAssetUrl(taskId: string, fileName: string) {
|
||||
return `/assets/${encodeURIComponent(safePathSegment(taskId))}/${encodeURIComponent(fileName)}`
|
||||
}
|
||||
|
||||
async function cacheRemoteAsset(taskId: string, remoteUrl: string, baseName: string, fallbackExtension: string) {
|
||||
const parsed = new URL(remoteUrl)
|
||||
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
|
||||
return undefined
|
||||
}
|
||||
|
||||
const extension = extensionFromUrl(remoteUrl) || fallbackExtension
|
||||
const fileName = `${baseName}${extension}`
|
||||
const assetDir = path.join(assetRoot, safePathSegment(taskId))
|
||||
const filePath = path.join(assetDir, fileName)
|
||||
|
||||
if (await fileExists(filePath)) {
|
||||
return localAssetUrl(taskId, fileName)
|
||||
}
|
||||
|
||||
const response = await fetch(remoteUrl)
|
||||
if (!response.ok) {
|
||||
throw new Error(`Asset download failed with HTTP ${response.status}`)
|
||||
}
|
||||
|
||||
const contentTypeExtension = extensionFromContentType(response.headers.get('content-type'))
|
||||
const resolvedFileName = contentTypeExtension && extension === fallbackExtension ? `${baseName}${contentTypeExtension}` : fileName
|
||||
const resolvedFilePath = path.join(assetDir, resolvedFileName)
|
||||
|
||||
await mkdir(assetDir, { recursive: true })
|
||||
const buffer = Buffer.from(await response.arrayBuffer())
|
||||
await writeFile(resolvedFilePath, buffer)
|
||||
|
||||
return localAssetUrl(taskId, resolvedFileName)
|
||||
}
|
||||
|
||||
export async function cacheTaskAssets(task: StoredTask): Promise<StoredTask> {
|
||||
const output = task.output
|
||||
if (!output?.results?.length) return task
|
||||
|
||||
let changed = false
|
||||
const results: NonNullable<NonNullable<StoredTask['output']>['results']> = []
|
||||
|
||||
for (const result of output.results) {
|
||||
const nextResult = { ...result }
|
||||
|
||||
for (const spec of assetSpecs) {
|
||||
const remoteUrl = result[spec.remoteKey]
|
||||
if (!remoteUrl) continue
|
||||
if (result[spec.localKey]?.startsWith('/assets/')) continue
|
||||
|
||||
try {
|
||||
const localUrl = await cacheRemoteAsset(task.taskId, remoteUrl, spec.baseName, spec.fallbackExtension)
|
||||
if (localUrl && nextResult[spec.localKey] !== localUrl) {
|
||||
nextResult[spec.localKey] = localUrl
|
||||
changed = true
|
||||
}
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : 'Unknown asset download error'
|
||||
console.warn(`Could not cache ${spec.remoteKey} for task ${task.taskId}: ${message}`)
|
||||
}
|
||||
}
|
||||
|
||||
results.push(nextResult)
|
||||
}
|
||||
|
||||
if (!changed) return task
|
||||
|
||||
return {
|
||||
...task,
|
||||
output: {
|
||||
...output,
|
||||
results,
|
||||
},
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
export async function cacheTasksAssets(tasks: StoredTask[]) {
|
||||
const nextTasks: StoredTask[] = []
|
||||
let changed = false
|
||||
|
||||
for (const task of tasks) {
|
||||
const nextTask = await cacheTaskAssets(task)
|
||||
nextTasks.push(nextTask)
|
||||
if (nextTask !== task) changed = true
|
||||
}
|
||||
|
||||
return { tasks: nextTasks, changed }
|
||||
}
|
||||
116
server/genesisPackage.test.ts
Normal file
116
server/genesisPackage.test.ts
Normal file
@ -0,0 +1,116 @@
|
||||
import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import type { StoredTask } from './taskStore.js'
|
||||
|
||||
const originalAssetRoot = process.env.REAL2SIM_ASSET_ROOT
|
||||
const originalPackageRoot = process.env.REAL2SIM_GENESIS_PACKAGE_ROOT
|
||||
let temporaryAssetRoot = ''
|
||||
let temporaryPackageRoot = ''
|
||||
|
||||
function createFlexTask(material: 'silicone' | 'cloth' = 'silicone'): StoredTask {
|
||||
return {
|
||||
taskId: `task-${material}`,
|
||||
kind: 'flex',
|
||||
title: `${material} flex object`,
|
||||
status: 'SUCCEEDED',
|
||||
request: {
|
||||
sourceTaskId: 'source-task',
|
||||
sourceName: 'source object',
|
||||
sourceModelUrl: '/assets/source-task/pbr.glb',
|
||||
behaviorPrompt: 'make it deformable',
|
||||
model: 'Tripo/Tripo-H3.1',
|
||||
parameters: {
|
||||
texture_quality: 'detailed',
|
||||
geometry_quality: 'standard',
|
||||
pbr: true,
|
||||
texture: true,
|
||||
},
|
||||
simulation: {
|
||||
material,
|
||||
elasticity: 72,
|
||||
damping: 42,
|
||||
thickness: 3.2,
|
||||
deformation: 64,
|
||||
gravity: 9.8,
|
||||
},
|
||||
},
|
||||
output: {
|
||||
task_id: `task-${material}`,
|
||||
task_status: 'SUCCEEDED',
|
||||
results: [
|
||||
{
|
||||
local_pbr_model_url: `/assets/task-${material}/pbr.glb`,
|
||||
local_rendered_image_url: `/assets/task-${material}/render.webp`,
|
||||
},
|
||||
],
|
||||
},
|
||||
createdAt: '2026-06-03T00:00:00.000Z',
|
||||
updatedAt: '2026-06-03T00:00:00.000Z',
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
temporaryAssetRoot = await mkdtemp(path.join(os.tmpdir(), 'real2sim-assets-'))
|
||||
temporaryPackageRoot = await mkdtemp(path.join(os.tmpdir(), 'real2sim-genesis-'))
|
||||
process.env.REAL2SIM_ASSET_ROOT = temporaryAssetRoot
|
||||
process.env.REAL2SIM_GENESIS_PACKAGE_ROOT = temporaryPackageRoot
|
||||
vi.resetModules()
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(temporaryAssetRoot, { recursive: true, force: true })
|
||||
await rm(temporaryPackageRoot, { recursive: true, force: true })
|
||||
vi.resetModules()
|
||||
if (originalAssetRoot === undefined) {
|
||||
delete process.env.REAL2SIM_ASSET_ROOT
|
||||
} else {
|
||||
process.env.REAL2SIM_ASSET_ROOT = originalAssetRoot
|
||||
}
|
||||
if (originalPackageRoot === undefined) {
|
||||
delete process.env.REAL2SIM_GENESIS_PACKAGE_ROOT
|
||||
} else {
|
||||
process.env.REAL2SIM_GENESIS_PACKAGE_ROOT = originalPackageRoot
|
||||
}
|
||||
})
|
||||
|
||||
describe('Genesis package export', () => {
|
||||
it('creates a runnable FEM package for volumetric flexible materials', async () => {
|
||||
await mkdir(path.join(temporaryAssetRoot, 'task-silicone'), { recursive: true })
|
||||
await writeFile(path.join(temporaryAssetRoot, 'task-silicone', 'pbr.glb'), new Uint8Array([1, 2, 3]))
|
||||
await writeFile(path.join(temporaryAssetRoot, 'task-silicone', 'render.webp'), new Uint8Array([4, 5, 6]))
|
||||
|
||||
const { createGenesisPackage } = await import('./genesisPackage.js')
|
||||
const exported = await createGenesisPackage(createFlexTask('silicone'))
|
||||
|
||||
expect(exported.manifest.genesis.solver).toBe('fem_elastic')
|
||||
expect(exported.manifest.assets.model).toBe('assets/model.glb')
|
||||
expect(exported.manifest.assets.preview).toBe('assets/preview.webp')
|
||||
expect((await stat(exported.archivePath)).size).toBeGreaterThan(0)
|
||||
expect(await readFile(path.join(exported.packageDir, 'assets', 'model.glb'))).toEqual(Buffer.from([1, 2, 3]))
|
||||
|
||||
const script = await readFile(path.join(exported.packageDir, 'genesis_scene.py'), 'utf8')
|
||||
expect(script).toContain('gs.materials.FEM.Elastic')
|
||||
expect(script).toContain('force_retet=True')
|
||||
expect(script).toContain('E=70400')
|
||||
|
||||
const profile = JSON.parse(await readFile(path.join(exported.packageDir, 'flex-profile.json'), 'utf8'))
|
||||
expect(profile.schema).toBe('real2sim.flexProfile.v1')
|
||||
expect(profile.material).toBe('silicone')
|
||||
})
|
||||
|
||||
it('creates a PBD cloth package for cloth materials', async () => {
|
||||
await mkdir(path.join(temporaryAssetRoot, 'task-cloth'), { recursive: true })
|
||||
await writeFile(path.join(temporaryAssetRoot, 'task-cloth', 'pbr.glb'), new Uint8Array([7, 8, 9]))
|
||||
|
||||
const { createGenesisPackage } = await import('./genesisPackage.js')
|
||||
const exported = await createGenesisPackage(createFlexTask('cloth'))
|
||||
|
||||
expect(exported.manifest.genesis.solver).toBe('pbd_cloth')
|
||||
|
||||
const script = await readFile(path.join(exported.packageDir, 'genesis_scene.py'), 'utf8')
|
||||
expect(script).toContain('gs.materials.PBD.Cloth')
|
||||
expect(script).toContain('stretch_stiffness=0.754')
|
||||
})
|
||||
})
|
||||
381
server/genesisPackage.ts
Normal file
381
server/genesisPackage.ts
Normal file
@ -0,0 +1,381 @@
|
||||
import { access, copyFile, mkdir, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { gzipSync } from 'node:zlib'
|
||||
import { assetRoot } from './assetStore.js'
|
||||
import { createSimulationExport } from './flex.js'
|
||||
import type { CreateFlexTaskInput, SimulationProfile } from './schemas.js'
|
||||
import type { StoredTask } from './taskStore.js'
|
||||
|
||||
type GenesisSolver = 'fem_elastic' | 'pbd_cloth'
|
||||
|
||||
type GenesisParameters = {
|
||||
E?: number
|
||||
nu?: number
|
||||
rho?: number
|
||||
damping: number
|
||||
stretchStiffness?: number
|
||||
bendStiffness?: number
|
||||
maxVolume?: number
|
||||
}
|
||||
|
||||
export type GenesisPackageManifest = {
|
||||
schema: 'real2sim.genesisPackage.v1'
|
||||
taskId: string
|
||||
title: string
|
||||
assets: {
|
||||
model: string
|
||||
preview?: string
|
||||
}
|
||||
genesis: {
|
||||
script: string
|
||||
solver: GenesisSolver
|
||||
material: SimulationProfile['material']
|
||||
parameters: GenesisParameters
|
||||
}
|
||||
notes: string[]
|
||||
generatedAt: string
|
||||
}
|
||||
|
||||
export type GenesisPackageResult = {
|
||||
packageDir: string
|
||||
archivePath: string
|
||||
manifest: GenesisPackageManifest
|
||||
}
|
||||
|
||||
export const genesisPackageRoot = process.env.REAL2SIM_GENESIS_PACKAGE_ROOT
|
||||
? path.resolve(process.env.REAL2SIM_GENESIS_PACKAGE_ROOT)
|
||||
: path.join(process.cwd(), '.data', 'genesis-packages')
|
||||
|
||||
function safePathSegment(value: string) {
|
||||
return value.replace(/[^a-zA-Z0-9._-]/g, '_')
|
||||
}
|
||||
|
||||
function isFlexTask(task: StoredTask): task is StoredTask & { request: CreateFlexTaskInput } {
|
||||
return task.kind === 'flex' && typeof (task.request as Partial<CreateFlexTaskInput>).simulation === 'object'
|
||||
}
|
||||
|
||||
function round(value: number, digits = 3) {
|
||||
return Number(value.toFixed(digits))
|
||||
}
|
||||
|
||||
function clamp(value: number, min: number, max: number) {
|
||||
return Math.min(max, Math.max(min, value))
|
||||
}
|
||||
|
||||
function localAssetPath(localUrl?: string) {
|
||||
if (!localUrl?.startsWith('/assets/')) return undefined
|
||||
|
||||
const relativeParts = localUrl
|
||||
.slice('/assets/'.length)
|
||||
.split('/')
|
||||
.filter(Boolean)
|
||||
.map((part) => decodeURIComponent(part))
|
||||
const resolved = path.resolve(assetRoot, ...relativeParts)
|
||||
const root = path.resolve(assetRoot)
|
||||
|
||||
if (resolved !== root && !resolved.startsWith(`${root}${path.sep}`)) {
|
||||
throw new Error('Asset path escapes the local asset root.')
|
||||
}
|
||||
|
||||
return resolved
|
||||
}
|
||||
|
||||
function chooseModelAsset(task: StoredTask) {
|
||||
const result = task.output?.results?.[0]
|
||||
return localAssetPath(result?.local_pbr_model_url) || localAssetPath(result?.local_base_model_url)
|
||||
}
|
||||
|
||||
function choosePreviewAsset(task: StoredTask) {
|
||||
return localAssetPath(task.output?.results?.[0]?.local_rendered_image_url)
|
||||
}
|
||||
|
||||
async function fileExists(filePath: string) {
|
||||
try {
|
||||
await access(filePath)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function materialParameters(simulation: SimulationProfile): { solver: GenesisSolver; parameters: GenesisParameters } {
|
||||
if (simulation.material === 'cloth') {
|
||||
return {
|
||||
solver: 'pbd_cloth',
|
||||
parameters: {
|
||||
damping: round(0.85 + (simulation.damping / 100) * 0.14),
|
||||
stretchStiffness: round(clamp(0.25 + simulation.elasticity * 0.007, 0.25, 0.98)),
|
||||
bendStiffness: round(clamp(0.04 + simulation.thickness * 0.012 + simulation.damping * 0.001, 0.04, 0.45)),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
const materialDefaults = {
|
||||
silicone: { baseE: 20000, nu: 0.47, rho: 1100 },
|
||||
inflatable: { baseE: 12000, nu: 0.42, rho: 260 },
|
||||
softRobot: { baseE: 36000, nu: 0.4, rho: 1050 },
|
||||
}[simulation.material]
|
||||
|
||||
return {
|
||||
solver: 'fem_elastic',
|
||||
parameters: {
|
||||
E: Math.round(clamp(materialDefaults.baseE + simulation.elasticity * 800 - simulation.deformation * 112.5, 12000, 180000)),
|
||||
nu: materialDefaults.nu,
|
||||
rho: materialDefaults.rho,
|
||||
damping: round(0.02 + (simulation.damping / 100) * 0.48),
|
||||
maxVolume: round(clamp((simulation.thickness / 1000) ** 3, 1e-7, 5e-5), 7),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function pythonString(value: string) {
|
||||
return JSON.stringify(value)
|
||||
}
|
||||
|
||||
function genesisScript(manifest: GenesisPackageManifest) {
|
||||
const gravity = manifest.genesis.material === 'cloth' ? -9.8 : -9.8
|
||||
const params = manifest.genesis.parameters
|
||||
|
||||
if (manifest.genesis.solver === 'pbd_cloth') {
|
||||
return `from pathlib import Path
|
||||
|
||||
import genesis as gs
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
MODEL_PATH = ROOT / "assets" / ${pythonString(path.basename(manifest.assets.model))}
|
||||
|
||||
gs.init()
|
||||
|
||||
scene = gs.Scene(
|
||||
sim_options=gs.options.SimOptions(
|
||||
dt=0.01,
|
||||
substeps=4,
|
||||
gravity=(0, 0, ${gravity}),
|
||||
),
|
||||
pbd_options=gs.options.PBDOptions(
|
||||
iterations=10,
|
||||
damping=${params.damping},
|
||||
),
|
||||
)
|
||||
|
||||
scene.add_entity(gs.morphs.Plane())
|
||||
|
||||
soft_body = scene.add_entity(
|
||||
gs.morphs.Mesh(
|
||||
file=str(MODEL_PATH),
|
||||
pos=(0, 0, 0.5),
|
||||
scale=1.0,
|
||||
parse_glb_with_zup=True,
|
||||
),
|
||||
material=gs.materials.PBD.Cloth(
|
||||
stretch_stiffness=${params.stretchStiffness},
|
||||
bend_stiffness=${params.bendStiffness},
|
||||
),
|
||||
)
|
||||
|
||||
scene.build()
|
||||
|
||||
for _ in range(1000):
|
||||
scene.step()
|
||||
`
|
||||
}
|
||||
|
||||
return `from pathlib import Path
|
||||
|
||||
import genesis as gs
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
MODEL_PATH = ROOT / "assets" / ${pythonString(path.basename(manifest.assets.model))}
|
||||
|
||||
gs.init()
|
||||
|
||||
scene = gs.Scene(
|
||||
sim_options=gs.options.SimOptions(
|
||||
dt=0.001,
|
||||
substeps=4,
|
||||
gravity=(0, 0, ${gravity}),
|
||||
),
|
||||
fem_options=gs.options.FEMOptions(
|
||||
dt=1e-3,
|
||||
damping=${params.damping},
|
||||
),
|
||||
)
|
||||
|
||||
scene.add_entity(gs.morphs.Plane())
|
||||
|
||||
soft_body = scene.add_entity(
|
||||
gs.morphs.Mesh(
|
||||
file=str(MODEL_PATH),
|
||||
pos=(0, 0, 0.5),
|
||||
scale=1.0,
|
||||
parse_glb_with_zup=True,
|
||||
force_retet=True,
|
||||
maxvolume=${params.maxVolume},
|
||||
),
|
||||
material=gs.materials.FEM.Elastic(
|
||||
E=${params.E},
|
||||
nu=${params.nu},
|
||||
rho=${params.rho},
|
||||
),
|
||||
)
|
||||
|
||||
scene.build()
|
||||
|
||||
for _ in range(1000):
|
||||
scene.step()
|
||||
`
|
||||
}
|
||||
|
||||
function packageReadme(manifest: GenesisPackageManifest) {
|
||||
return `# Genesis flexible object package
|
||||
|
||||
Task: ${manifest.title}
|
||||
|
||||
## Run
|
||||
|
||||
\`\`\`bash
|
||||
pip install genesis-world
|
||||
python genesis_scene.py
|
||||
\`\`\`
|
||||
|
||||
## Contents
|
||||
|
||||
- \`assets/model${path.extname(manifest.assets.model)}\`: local generated 3D asset.
|
||||
- \`flex-profile.json\`: real2sim physical profile exported from the UI sliders.
|
||||
- \`manifest.json\`: package metadata and Genesis parameter mapping.
|
||||
- \`genesis_scene.py\`: runnable Genesis scene using \`${manifest.genesis.solver}\`.
|
||||
|
||||
## Important
|
||||
|
||||
The GLB stores geometry and visual material only. It becomes deformable in Genesis because \`genesis_scene.py\` loads it with Genesis deformable material settings. If Genesis cannot tetrahedralize the GLB for FEM, convert the model to a clean watertight OBJ and update \`MODEL_PATH\` in the script.
|
||||
`
|
||||
}
|
||||
|
||||
async function collectFiles(dir: string, prefix = ''): Promise<Array<{ absolutePath: string; archivePath: string }>> {
|
||||
const entries = await readdir(dir, { withFileTypes: true })
|
||||
const files = []
|
||||
|
||||
for (const entry of entries) {
|
||||
const absolutePath = path.join(dir, entry.name)
|
||||
const archivePath = prefix ? `${prefix}/${entry.name}` : entry.name
|
||||
if (entry.isDirectory()) {
|
||||
files.push(...(await collectFiles(absolutePath, archivePath)))
|
||||
} else if (entry.isFile()) {
|
||||
files.push({ absolutePath, archivePath })
|
||||
}
|
||||
}
|
||||
|
||||
return files
|
||||
}
|
||||
|
||||
function writeTarString(buffer: Buffer, offset: number, length: number, value: string) {
|
||||
buffer.write(value.slice(0, length), offset, length, 'utf8')
|
||||
}
|
||||
|
||||
function octal(value: number, length: number) {
|
||||
return value.toString(8).padStart(length - 1, '0').slice(-(length - 1)) + '\0'
|
||||
}
|
||||
|
||||
function tarHeader(name: string, size: number, mode = 0o644) {
|
||||
const header = Buffer.alloc(512, 0)
|
||||
writeTarString(header, 0, 100, name)
|
||||
writeTarString(header, 100, 8, octal(mode, 8))
|
||||
writeTarString(header, 108, 8, octal(0, 8))
|
||||
writeTarString(header, 116, 8, octal(0, 8))
|
||||
writeTarString(header, 124, 12, octal(size, 12))
|
||||
writeTarString(header, 136, 12, octal(Math.floor(Date.now() / 1000), 12))
|
||||
header.fill(' ', 148, 156)
|
||||
header.write('0', 156)
|
||||
writeTarString(header, 257, 6, 'ustar')
|
||||
writeTarString(header, 263, 2, '00')
|
||||
|
||||
let checksum = 0
|
||||
for (const byte of header) checksum += byte
|
||||
writeTarString(header, 148, 8, checksum.toString(8).padStart(6, '0') + '\0 ')
|
||||
|
||||
return header
|
||||
}
|
||||
|
||||
async function createTarGz(sourceDir: string, archivePath: string) {
|
||||
const files = await collectFiles(sourceDir)
|
||||
const chunks: Buffer[] = []
|
||||
|
||||
for (const file of files) {
|
||||
const content = await readFile(file.absolutePath)
|
||||
chunks.push(tarHeader(file.archivePath, content.length), content)
|
||||
const remainder = content.length % 512
|
||||
if (remainder) {
|
||||
chunks.push(Buffer.alloc(512 - remainder, 0))
|
||||
}
|
||||
}
|
||||
|
||||
chunks.push(Buffer.alloc(1024, 0))
|
||||
await writeFile(archivePath, gzipSync(Buffer.concat(chunks)))
|
||||
}
|
||||
|
||||
export async function createGenesisPackage(task: StoredTask): Promise<GenesisPackageResult> {
|
||||
if (!isFlexTask(task)) {
|
||||
throw new Error('Genesis export requires a flex task.')
|
||||
}
|
||||
|
||||
const modelAssetPath = chooseModelAsset(task)
|
||||
if (!modelAssetPath) {
|
||||
throw new Error('Genesis export requires a cached local model asset.')
|
||||
}
|
||||
|
||||
const simulation = task.request.simulation
|
||||
const genesis = materialParameters(simulation)
|
||||
const taskDirName = safePathSegment(task.taskId)
|
||||
const packageDir = path.join(genesisPackageRoot, taskDirName)
|
||||
const assetsDir = path.join(packageDir, 'assets')
|
||||
const modelExtension = path.extname(modelAssetPath) || '.glb'
|
||||
const modelRelativePath = `assets/model${modelExtension}`
|
||||
const archivePath = path.join(genesisPackageRoot, `${taskDirName}.tar.gz`)
|
||||
const previewAssetPath = choosePreviewAsset(task)
|
||||
let previewRelativePath = previewAssetPath ? `assets/preview${path.extname(previewAssetPath) || '.webp'}` : undefined
|
||||
|
||||
await rm(packageDir, { recursive: true, force: true })
|
||||
await mkdir(assetsDir, { recursive: true })
|
||||
await copyFile(modelAssetPath, path.join(packageDir, modelRelativePath))
|
||||
if (previewAssetPath && previewRelativePath && (await fileExists(previewAssetPath))) {
|
||||
await copyFile(previewAssetPath, path.join(packageDir, previewRelativePath))
|
||||
} else {
|
||||
previewRelativePath = undefined
|
||||
}
|
||||
|
||||
const manifest: GenesisPackageManifest = {
|
||||
schema: 'real2sim.genesisPackage.v1',
|
||||
taskId: task.taskId,
|
||||
title: task.title,
|
||||
assets: {
|
||||
model: modelRelativePath,
|
||||
...(previewRelativePath ? { preview: previewRelativePath } : {}),
|
||||
},
|
||||
genesis: {
|
||||
script: 'genesis_scene.py',
|
||||
solver: genesis.solver,
|
||||
material: simulation.material,
|
||||
parameters: genesis.parameters,
|
||||
},
|
||||
notes: [
|
||||
'The generated GLB is a visual mesh; Genesis deformability comes from the solver and material assigned in genesis_scene.py.',
|
||||
'For FEM, Genesis may need a clean watertight mesh for tetrahedralization. Convert to OBJ if the GLB cannot be retetrahedralized.',
|
||||
],
|
||||
generatedAt: new Date().toISOString(),
|
||||
}
|
||||
|
||||
await writeFile(path.join(packageDir, 'manifest.json'), JSON.stringify(manifest, null, 2))
|
||||
await writeFile(path.join(packageDir, 'flex-profile.json'), JSON.stringify(createSimulationExport(task.request), null, 2))
|
||||
await writeFile(path.join(packageDir, 'genesis_scene.py'), genesisScript(manifest))
|
||||
await writeFile(path.join(packageDir, 'README.md'), packageReadme(manifest))
|
||||
|
||||
await mkdir(genesisPackageRoot, { recursive: true })
|
||||
await createTarGz(packageDir, archivePath)
|
||||
await stat(archivePath)
|
||||
|
||||
return {
|
||||
packageDir,
|
||||
archivePath,
|
||||
manifest,
|
||||
}
|
||||
}
|
||||
@ -1,10 +1,15 @@
|
||||
import cors from 'cors'
|
||||
import dotenv from 'dotenv'
|
||||
import express from 'express'
|
||||
import { createReadStream } from 'node:fs'
|
||||
import { stat } from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { ZodError } from 'zod'
|
||||
import { assetRoot, cacheTaskAssets, cacheTasksAssets } from './assetStore.js'
|
||||
import { buildFlexiblePrompt, createSimulationExport } from './flex.js'
|
||||
import { createGenesisPackage } from './genesisPackage.js'
|
||||
import { createFlexTaskSchema, createTripoTaskSchema } from './schemas.js'
|
||||
import { getTask, mergeTaskOutput, readTasks, upsertTask, type StoredTask } from './taskStore.js'
|
||||
import { getTask, mergeTaskOutput, readTasks, upsertTask, writeTasks, type StoredTask } from './taskStore.js'
|
||||
import { createTripoTask, queryTripoTask } from './tripoClient.js'
|
||||
|
||||
dotenv.config({ path: '.env.local' })
|
||||
@ -15,6 +20,22 @@ const port = Number(process.env.PORT || 8787)
|
||||
|
||||
app.use(cors())
|
||||
app.use(express.json({ limit: '1mb' }))
|
||||
app.use('/assets', express.static(assetRoot))
|
||||
|
||||
async function readTasksWithLocalAssets() {
|
||||
const tasks = await readTasks()
|
||||
const cached = await cacheTasksAssets(tasks)
|
||||
if (cached.changed) {
|
||||
await writeTasks(cached.tasks)
|
||||
}
|
||||
return cached.tasks
|
||||
}
|
||||
|
||||
async function upsertTaskWithLocalAssets(task: StoredTask) {
|
||||
const cachedTask = await cacheTaskAssets(task)
|
||||
await upsertTask(cachedTask)
|
||||
return cachedTask
|
||||
}
|
||||
|
||||
app.get('/api/health', (_req, res) => {
|
||||
res.json({
|
||||
@ -27,7 +48,7 @@ app.get('/api/health', (_req, res) => {
|
||||
|
||||
app.get('/api/tripo/tasks', async (_req, res, next) => {
|
||||
try {
|
||||
res.json({ tasks: await readTasks() })
|
||||
res.json({ tasks: await readTasksWithLocalAssets() })
|
||||
} catch (error) {
|
||||
next(error)
|
||||
}
|
||||
@ -58,8 +79,8 @@ app.post('/api/tripo/tasks', async (req, res, next) => {
|
||||
updatedAt: now,
|
||||
}
|
||||
|
||||
await upsertTask(task)
|
||||
res.status(201).json({ task })
|
||||
const savedTask = await upsertTaskWithLocalAssets(task)
|
||||
res.status(201).json({ task: savedTask })
|
||||
} catch (error) {
|
||||
next(error)
|
||||
}
|
||||
@ -98,8 +119,8 @@ app.get('/api/tripo/tasks/:taskId', async (req, res, next) => {
|
||||
updatedAt: new Date().toISOString(),
|
||||
} satisfies StoredTask)
|
||||
|
||||
await upsertTask(updated)
|
||||
res.json({ task: updated })
|
||||
const savedTask = await upsertTaskWithLocalAssets(updated)
|
||||
res.json({ task: savedTask })
|
||||
} catch (error) {
|
||||
next(error)
|
||||
}
|
||||
@ -135,9 +156,9 @@ app.post('/api/flex/tasks', async (req, res, next) => {
|
||||
updatedAt: now,
|
||||
}
|
||||
|
||||
await upsertTask(task)
|
||||
const savedTask = await upsertTaskWithLocalAssets(task)
|
||||
res.status(201).json({
|
||||
task,
|
||||
task: savedTask,
|
||||
prompt,
|
||||
simulationExport: createSimulationExport(input),
|
||||
})
|
||||
@ -146,6 +167,46 @@ app.post('/api/flex/tasks', async (req, res, next) => {
|
||||
}
|
||||
})
|
||||
|
||||
app.get('/api/genesis/packages/:taskId', async (req, res, next) => {
|
||||
try {
|
||||
const task = await getTask(req.params.taskId)
|
||||
if (!task) {
|
||||
res.status(404).json({ error: 'Task not found.' })
|
||||
return
|
||||
}
|
||||
|
||||
const savedTask = await upsertTaskWithLocalAssets(task)
|
||||
const exported = await createGenesisPackage(savedTask)
|
||||
res.json({
|
||||
manifest: exported.manifest,
|
||||
archiveUrl: `/api/genesis/packages/${encodeURIComponent(savedTask.taskId)}/download`,
|
||||
})
|
||||
} catch (error) {
|
||||
next(error)
|
||||
}
|
||||
})
|
||||
|
||||
app.get('/api/genesis/packages/:taskId/download', async (req, res, next) => {
|
||||
try {
|
||||
const task = await getTask(req.params.taskId)
|
||||
if (!task) {
|
||||
res.status(404).json({ error: 'Task not found.' })
|
||||
return
|
||||
}
|
||||
|
||||
const savedTask = await upsertTaskWithLocalAssets(task)
|
||||
const exported = await createGenesisPackage(savedTask)
|
||||
const archiveStat = await stat(exported.archivePath)
|
||||
const filename = `${savedTask.taskId}-genesis-flex.tar.gz`
|
||||
res.setHeader('Content-Type', 'application/gzip')
|
||||
res.setHeader('Content-Length', archiveStat.size.toString())
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${filename}"`)
|
||||
createReadStream(exported.archivePath).on('error', next).pipe(res)
|
||||
} catch (error) {
|
||||
next(error)
|
||||
}
|
||||
})
|
||||
|
||||
app.use((error: unknown, _req: express.Request, res: express.Response, _next: express.NextFunction) => {
|
||||
void _next
|
||||
if (error instanceof ZodError) {
|
||||
@ -158,5 +219,6 @@ app.use((error: unknown, _req: express.Request, res: express.Response, _next: ex
|
||||
})
|
||||
|
||||
app.listen(port, () => {
|
||||
console.log(`real2sim assets served from ${path.relative(process.cwd(), assetRoot) || assetRoot}`)
|
||||
console.log(`real2sim API listening on http://localhost:${port}`)
|
||||
})
|
||||
|
||||
@ -3,9 +3,14 @@ import type { CreateTripoTaskInput } from './schemas.js'
|
||||
export type TaskStatus = 'PENDING' | 'RUNNING' | 'SUCCEEDED' | 'FAILED' | 'CANCELED' | 'UNKNOWN'
|
||||
|
||||
export type TripoTaskResult = {
|
||||
model_url?: string
|
||||
pbr_model_url?: string
|
||||
base_model_url?: string
|
||||
rendered_image_url?: string
|
||||
local_model_url?: string
|
||||
local_pbr_model_url?: string
|
||||
local_base_model_url?: string
|
||||
local_rendered_image_url?: string
|
||||
orig_prompt?: string
|
||||
}
|
||||
|
||||
|
||||
18
src/App.css
18
src/App.css
@ -454,6 +454,24 @@ output {
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.viewport-image-preview {
|
||||
display: grid;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
min-height: 420px;
|
||||
place-items: center;
|
||||
padding: 20px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.viewport-image-preview img {
|
||||
width: min(100%, 980px);
|
||||
max-height: 100%;
|
||||
object-fit: contain;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 18px 36px rgba(15, 23, 42, 0.12);
|
||||
}
|
||||
|
||||
.result-panel {
|
||||
min-height: 640px;
|
||||
}
|
||||
|
||||
35
src/App.tsx
35
src/App.tsx
@ -9,6 +9,7 @@ import {
|
||||
Image,
|
||||
Languages,
|
||||
Layers,
|
||||
Package,
|
||||
RefreshCw,
|
||||
SlidersHorizontal,
|
||||
Sparkles,
|
||||
@ -20,7 +21,9 @@ import {
|
||||
createFlexTask,
|
||||
createGenerationTask,
|
||||
getModelUrl,
|
||||
getPrimaryResult,
|
||||
getBaseModelUrl,
|
||||
getPreviewImageUrl,
|
||||
getPbrModelUrl,
|
||||
getSourcePrompt,
|
||||
health,
|
||||
isLiveStatus,
|
||||
@ -159,8 +162,10 @@ function TaskPanel({
|
||||
onUseForFlex: (task: StoredTask) => void
|
||||
t: Messages
|
||||
}) {
|
||||
const result = getPrimaryResult(task)
|
||||
const modelUrl = getModelUrl(task)
|
||||
const previewImageUrl = getPreviewImageUrl(task)
|
||||
const pbrModelUrl = getPbrModelUrl(task)
|
||||
const baseModelUrl = getBaseModelUrl(task)
|
||||
|
||||
return (
|
||||
<aside className="result-panel">
|
||||
@ -174,9 +179,9 @@ function TaskPanel({
|
||||
|
||||
{task ? (
|
||||
<>
|
||||
{result?.rendered_image_url ? (
|
||||
<a href={result.rendered_image_url} target="_blank" rel="noreferrer" className="render-preview">
|
||||
<img src={result.rendered_image_url} alt={t.task.rendered} />
|
||||
{previewImageUrl ? (
|
||||
<a href={previewImageUrl} target="_blank" rel="noreferrer" className="render-preview">
|
||||
<img src={previewImageUrl} alt={t.task.rendered} />
|
||||
</a>
|
||||
) : (
|
||||
<div className="render-empty">
|
||||
@ -206,18 +211,24 @@ function TaskPanel({
|
||||
<RefreshCw size={16} />
|
||||
{t.task.poll}
|
||||
</button>
|
||||
{result?.pbr_model_url ? (
|
||||
<a href={result.pbr_model_url} className="primary-link" target="_blank" rel="noreferrer">
|
||||
{pbrModelUrl ? (
|
||||
<a href={pbrModelUrl} className="primary-link" target="_blank" rel="noreferrer">
|
||||
<Download size={16} />
|
||||
{t.task.downloadPbr}
|
||||
</a>
|
||||
) : null}
|
||||
{result?.base_model_url ? (
|
||||
<a href={result.base_model_url} className="secondary-link" target="_blank" rel="noreferrer">
|
||||
{baseModelUrl ? (
|
||||
<a href={baseModelUrl} className="secondary-link" target="_blank" rel="noreferrer">
|
||||
<Download size={16} />
|
||||
{t.task.downloadBase}
|
||||
</a>
|
||||
) : null}
|
||||
{task.kind === 'flex' && modelUrl ? (
|
||||
<a href={`/api/genesis/packages/${encodeURIComponent(task.taskId)}/download`} className="secondary-link">
|
||||
<Package size={16} />
|
||||
{t.task.exportGenesis}
|
||||
</a>
|
||||
) : null}
|
||||
{modelUrl ? (
|
||||
<button type="button" className="secondary-button" onClick={() => onUseForFlex(task)}>
|
||||
<Waves size={16} />
|
||||
@ -703,12 +714,12 @@ function App() {
|
||||
<div className="asset-list">
|
||||
{tasks.length ? (
|
||||
tasks.map((task) => {
|
||||
const result = getPrimaryResult(task)
|
||||
const modelUrl = getModelUrl(task)
|
||||
const previewImageUrl = getPreviewImageUrl(task)
|
||||
return (
|
||||
<article className="asset-row" key={task.taskId}>
|
||||
{result?.rendered_image_url ? (
|
||||
<img src={result.rendered_image_url} alt="" />
|
||||
{previewImageUrl ? (
|
||||
<img src={previewImageUrl} alt="" />
|
||||
) : (
|
||||
<div className="asset-thumb">
|
||||
<Box size={18} />
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
import { Float, MeshDistortMaterial, OrbitControls, ContactShadows } from '@react-three/drei'
|
||||
import { Bounds, Center, ContactShadows, Float, MeshDistortMaterial, OrbitControls, useGLTF } from '@react-three/drei'
|
||||
import { Canvas, useFrame } from '@react-three/fiber'
|
||||
import { Component, useRef, type ErrorInfo, type ReactNode } from 'react'
|
||||
import type { Mesh } from 'three'
|
||||
import { Component, Suspense, useMemo, useRef, type ErrorInfo, type ReactNode } from 'react'
|
||||
import type { Mesh, Object3D } from 'three'
|
||||
import type { SimulationProfile, StoredTask } from '../lib/api'
|
||||
import { getModelUrl } from '../lib/api'
|
||||
import { getModelUrl, getPreviewImageUrl } from '../lib/api'
|
||||
import type { Messages } from '../lib/i18n'
|
||||
|
||||
type Viewport3DProps = {
|
||||
@ -86,9 +86,37 @@ function StageGrid() {
|
||||
)
|
||||
}
|
||||
|
||||
function enableShadows(object: Object3D) {
|
||||
object.traverse((child) => {
|
||||
const mesh = child as Mesh
|
||||
if (mesh.isMesh) {
|
||||
mesh.castShadow = true
|
||||
mesh.receiveShadow = true
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
function AssetModel({ url }: { url: string }) {
|
||||
const { scene } = useGLTF(url)
|
||||
const previewScene = useMemo(() => {
|
||||
const clonedScene = scene.clone(true)
|
||||
enableShadows(clonedScene)
|
||||
return clonedScene
|
||||
}, [scene])
|
||||
|
||||
return (
|
||||
<Bounds fit clip observe margin={1.25}>
|
||||
<Center>
|
||||
<primitive object={previewScene} />
|
||||
</Center>
|
||||
</Bounds>
|
||||
)
|
||||
}
|
||||
|
||||
export function Viewport3D({ task, simulation, labels }: Viewport3DProps) {
|
||||
const modelUrl = getModelUrl(task)
|
||||
const status = modelUrl ? labels.modelReady : task ? labels.preview : labels.idle
|
||||
const previewImageUrl = getPreviewImageUrl(task)
|
||||
const status = modelUrl ? labels.modelReady : previewImageUrl ? labels.imageReady : task ? labels.preview : labels.idle
|
||||
|
||||
return (
|
||||
<section className="viewport-panel" aria-label={labels.label}>
|
||||
@ -104,18 +132,26 @@ export function Viewport3D({ task, simulation, labels }: Viewport3DProps) {
|
||||
) : null}
|
||||
</div>
|
||||
<div className="canvas-wrap">
|
||||
{modelUrl || !previewImageUrl ? (
|
||||
<CanvasErrorBoundary>
|
||||
<Canvas camera={{ position: [3.7, 2.5, 4.2], fov: 42 }} gl={{ preserveDrawingBuffer: true, antialias: true }} shadows>
|
||||
<ambientLight intensity={1} />
|
||||
<hemisphereLight args={['#ffffff', '#9ca3af', 1.25]} />
|
||||
<directionalLight position={[4, 5, 3]} intensity={2.4} castShadow />
|
||||
<pointLight position={[-3, 2.2, -2]} intensity={0.65} color="#5eead4" />
|
||||
<SoftBody simulation={simulation} />
|
||||
<Suspense fallback={<SoftBody simulation={simulation} />}>
|
||||
{modelUrl ? <AssetModel url={modelUrl} /> : <SoftBody simulation={simulation} />}
|
||||
</Suspense>
|
||||
<StageGrid />
|
||||
<ContactShadows position={[0, -1.08, 0]} opacity={0.28} scale={5} blur={2.4} far={3} />
|
||||
<OrbitControls enablePan={false} minDistance={3.4} maxDistance={8} />
|
||||
</Canvas>
|
||||
</CanvasErrorBoundary>
|
||||
) : (
|
||||
<a href={previewImageUrl} target="_blank" rel="noreferrer" className="viewport-image-preview">
|
||||
<img src={previewImageUrl} alt={labels.imageReady} />
|
||||
</a>
|
||||
)}
|
||||
</div>
|
||||
</section>
|
||||
)
|
||||
|
||||
@ -1,9 +1,14 @@
|
||||
export type TaskStatus = 'PENDING' | 'RUNNING' | 'SUCCEEDED' | 'FAILED' | 'CANCELED' | 'UNKNOWN'
|
||||
|
||||
export type TripoTaskResult = {
|
||||
model_url?: string
|
||||
pbr_model_url?: string
|
||||
base_model_url?: string
|
||||
rendered_image_url?: string
|
||||
local_model_url?: string
|
||||
local_pbr_model_url?: string
|
||||
local_base_model_url?: string
|
||||
local_rendered_image_url?: string
|
||||
orig_prompt?: string
|
||||
}
|
||||
|
||||
@ -109,9 +114,23 @@ export function getPrimaryResult(task?: StoredTask | null) {
|
||||
return task?.output?.results?.[0]
|
||||
}
|
||||
|
||||
export function getModelUrl(task?: StoredTask | null) {
|
||||
export function getPreviewImageUrl(task?: StoredTask | null) {
|
||||
const result = getPrimaryResult(task)
|
||||
return result?.pbr_model_url || result?.base_model_url || ''
|
||||
return result?.local_rendered_image_url || ''
|
||||
}
|
||||
|
||||
export function getPbrModelUrl(task?: StoredTask | null) {
|
||||
const result = getPrimaryResult(task)
|
||||
return result?.local_pbr_model_url || ''
|
||||
}
|
||||
|
||||
export function getBaseModelUrl(task?: StoredTask | null) {
|
||||
const result = getPrimaryResult(task)
|
||||
return result?.local_base_model_url || result?.local_model_url || ''
|
||||
}
|
||||
|
||||
export function getModelUrl(task?: StoredTask | null) {
|
||||
return getPbrModelUrl(task) || getBaseModelUrl(task)
|
||||
}
|
||||
|
||||
export function getSourcePrompt(task?: StoredTask | null) {
|
||||
|
||||
@ -80,6 +80,7 @@ const dictionaries = {
|
||||
poll: '查询状态',
|
||||
downloadPbr: '下载 PBR GLB',
|
||||
downloadBase: '下载基础 GLB',
|
||||
exportGenesis: '导出 Genesis 包',
|
||||
rendered: '渲染预览',
|
||||
pending: '排队中',
|
||||
running: '处理中',
|
||||
@ -91,6 +92,7 @@ const dictionaries = {
|
||||
viewport: {
|
||||
label: '3D 预览',
|
||||
modelReady: '模型链接已生成',
|
||||
imageReady: '渲染图已生成',
|
||||
preview: '软体预览',
|
||||
idle: '等待生成',
|
||||
},
|
||||
@ -182,6 +184,7 @@ const dictionaries = {
|
||||
poll: 'Query Status',
|
||||
downloadPbr: 'Download PBR GLB',
|
||||
downloadBase: 'Download Base GLB',
|
||||
exportGenesis: 'Export Genesis Package',
|
||||
rendered: 'Rendered Preview',
|
||||
pending: 'Pending',
|
||||
running: 'Running',
|
||||
@ -193,6 +196,7 @@ const dictionaries = {
|
||||
viewport: {
|
||||
label: '3D Preview',
|
||||
modelReady: 'Model link generated',
|
||||
imageReady: 'Rendered image generated',
|
||||
preview: 'Soft preview',
|
||||
idle: 'Waiting',
|
||||
},
|
||||
|
||||
@ -7,6 +7,7 @@ export default defineConfig({
|
||||
server: {
|
||||
proxy: {
|
||||
'/api': 'http://localhost:8787',
|
||||
'/assets': 'http://localhost:8787',
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user