gs_playground/docs/real2sim-debug-optimization-report.html

422 lines
15 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Real2Sim 到 MotrixSim 调试与优化复盘</title>
<style>
:root {
color-scheme: light;
--bg: #f5f7fa;
--paper: #ffffff;
--ink: #1f2933;
--muted: #5d6875;
--line: #d8dee8;
--accent: #c6243f;
--accent-2: #0f766e;
--code: #172033;
}
* { box-sizing: border-box; }
body {
margin: 0;
background: var(--bg);
color: var(--ink);
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
line-height: 1.65;
}
main {
max-width: 1120px;
margin: 0 auto;
padding: 40px 24px 72px;
}
header {
background: linear-gradient(135deg, #252b36, #3a2028);
color: white;
border-radius: 16px;
padding: 36px;
margin-bottom: 24px;
box-shadow: 0 18px 40px rgba(20, 27, 38, 0.18);
}
header h1 {
margin: 0 0 12px;
font-size: clamp(30px, 5vw, 48px);
line-height: 1.1;
letter-spacing: 0;
}
header p {
margin: 0;
color: rgba(255, 255, 255, 0.78);
max-width: 780px;
font-size: 17px;
}
section {
background: var(--paper);
border: 1px solid var(--line);
border-radius: 14px;
padding: 26px;
margin: 18px 0;
box-shadow: 0 10px 30px rgba(31, 41, 51, 0.06);
}
h2 {
margin: 0 0 14px;
font-size: 26px;
letter-spacing: 0;
}
h3 {
margin: 24px 0 10px;
font-size: 20px;
letter-spacing: 0;
}
p { margin: 10px 0; }
ul { padding-left: 20px; }
.meta {
display: flex;
flex-wrap: wrap;
gap: 10px;
margin-top: 20px;
}
.pill {
background: rgba(255, 255, 255, 0.12);
border: 1px solid rgba(255, 255, 255, 0.22);
color: white;
border-radius: 999px;
padding: 6px 12px;
font-size: 13px;
}
.grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
gap: 18px;
margin-top: 16px;
}
figure {
margin: 0;
border: 1px solid var(--line);
border-radius: 12px;
overflow: hidden;
background: #111827;
}
figure img {
width: 100%;
display: block;
}
figcaption {
padding: 10px 12px;
background: #ffffff;
color: var(--muted);
font-size: 14px;
}
.hero-image {
margin-top: 18px;
}
.hero-image img {
border-radius: 12px;
border: 1px solid var(--line);
width: 100%;
display: block;
}
table {
width: 100%;
border-collapse: collapse;
margin: 16px 0;
overflow: hidden;
border-radius: 10px;
border: 1px solid var(--line);
}
th, td {
text-align: left;
padding: 11px 12px;
border-bottom: 1px solid var(--line);
vertical-align: top;
}
th {
background: #eef2f7;
font-weight: 700;
}
tr:last-child td { border-bottom: 0; }
code {
background: #eef2f7;
border-radius: 5px;
padding: 2px 5px;
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
font-size: 0.92em;
}
pre {
background: var(--code);
color: #e5edf7;
border-radius: 12px;
overflow-x: auto;
padding: 16px;
font-size: 13px;
}
pre code {
background: transparent;
padding: 0;
color: inherit;
}
.timeline {
border-left: 3px solid var(--accent);
padding-left: 18px;
}
.step {
margin: 20px 0;
padding-left: 6px;
}
.step h3 {
margin-top: 0;
color: var(--accent);
}
.note {
border-left: 4px solid var(--accent-2);
background: #ecfdf5;
padding: 12px 14px;
border-radius: 8px;
color: #164e45;
}
footer {
color: var(--muted);
text-align: center;
padding: 26px 0 0;
font-size: 13px;
}
</style>
</head>
<body>
<main>
<header>
<h1>Real2Sim 到 MotrixSim 调试与优化复盘</h1>
<p>从单张 RGB / 合成多视角重建,到 SAM-3D + Gaussian 资产导入 MotrixSim 的完整调试过程:材质、光照、LOD、法线、清理规则和仿真预览验证。</p>
<div class="meta">
<span class="pill">日期:2026-06-12</span>
<span class="pill">分支:real2sim</span>
<span class="pill">目标:sim-ready assets</span>
<span class="pill">环境:48G 无影云桌面</span>
</div>
</header>
<section>
<h2>最终效果</h2>
<p>最终 MotrixSim 包已切换到高 LOD、带顶点法线、保留瓶盖/提手、带贴图材质的版本。</p>
<pre><code>/opt/gs_playground_real2sim_48q/outputs/office_bottle_gourd_synth_multiview_cap_preserved_20260612_142348/mjcf/scene.xml</code></pre>
<div class="hero-image">
<img src="assets/real2sim-debug-optimization/08-final-cap-preserved-preview-crop.png" alt="最终 MotrixSim 预览效果">
</div>
<table>
<tr><th>项目</th><th>结果</th></tr>
<tr><td>水瓶 visual mesh</td><td><code>239999</code> faces, <code>165529</code> vertex normals, <code>proxy=False</code></td></tr>
<tr><td>葫芦 visual mesh</td><td><code>239999</code> faces, <code>159363</code> vertex normals, <code>proxy=False</code></td></tr>
<tr><td>水瓶 highres mesh</td><td><code>280843</code> faces,瓶盖/提手组件已保留</td></tr>
<tr><td>顶部红色组件</td><td>最终 LOD 检出 <code>27</code> 个顶部高饱和红色组件</td></tr>
<tr><td>测试</td><td>本地 <code>93 passed</code>,远端 <code>93 passed</code></td></tr>
</table>
</section>
<section>
<h2>输入与调试目标</h2>
<p>输入图包含透明红盖水瓶和黑色葫芦瓶。调试目标不是只生成一个可打开文件,而是在 MotrixSim 中稳定查看:瓶盖、提手、透明瓶身、标签、黑色反光物体都要尽量保留。</p>
<figure>
<img src="assets/real2sim-debug-optimization/00-reference-input.png" alt="参考输入图">
<figcaption>参考输入图:水瓶和葫芦瓶是主要重建对象。</figcaption>
</figure>
</section>
<section>
<h2>完整调试链路</h2>
<pre><code>输入图 / 合成多视角
-> GroundingDINO + SAM2 分割
-> SAM-3D object reconstruction
-> mesh / texture / 3DGS assets
-> Real2Sim packaging
-> MJCF + MotrixSim preview
-> 发现问题
-> 定位到 packaging / mesh LOD / material / cleanup 规则
-> 修复并重新 package</code></pre>
<p class="note">这次优化的重点是逐层验证资产链路,而不是只调最终截图:Motrix 实际加载了哪个 mesh、有没有材质贴图、有没有 UV 和法线、清理器是否误删组件,都会直接影响视觉结果。</p>
</section>
<section class="timeline">
<div class="step">
<h3>1. 初始导入:灰白、无材质</h3>
<p>最早的 MotrixSim 预览像灰白模型,标签、透明瓶身、葫芦反光都没有正确表达。</p>
<figure>
<img src="assets/real2sim-debug-optimization/01-initial-untextured-preview.png" alt="初始灰白预览">
<figcaption>问题:材质和贴图没有稳定进入 MotrixSim。</figcaption>
</figure>
<p>修复包括 OBJ 材质 sidecar namespacing、MJCF texture/material 引用、场景灯光和暗贴图自动曝光。</p>
</div>
<div class="step">
<h3>2. 顶部白色漂浮片</h3>
<p>材质变好后,水瓶顶部出现了 SAM-3D 对透明高光区域误重建出的白色片状 artifact。</p>
<figure>
<img src="assets/real2sim-debug-optimization/02-floating-top-artifact.png" alt="顶部漂浮片">
<figcaption>问题:顶部 disconnected sheet 被打包进 MotrixSim。</figcaption>
</figure>
<p>修复是增加组件级顶部漂浮片清理,通过位置、厚度、宽度、贴图亮度和饱和度判定异常组件。</p>
</div>
<div class="step">
<h3>3. Proxy 网格导致形状不稳定</h3>
<p>为了性能,早期使用旋转 proxy mesh。它只有约 14k 面,不是真实 SAM-3D 几何,因此不同视角会看起来变形。</p>
<pre><code>旧 Motrix OBJ: object_000_det_000_motrix_proxy
faces: 14016
verts: 7083
真实 highres OBJ:
faces: 268360
verts: 178835</code></pre>
<p>修复是改为 <code>REAL2SIM_MOTRIX_PREVIEW_MODE=lod</code>,从 highres OBJ 做 texture-preserving decimation。</p>
</div>
<div class="step">
<h3>4. 稀疏、卡顿和模糊</h3>
<div class="grid">
<figure>
<img src="assets/real2sim-debug-optimization/03-sparse-slow-preview.png" alt="稀疏低 FPS">
<figcaption>稀疏且慢的中间版本。</figcaption>
</figure>
<figure>
<img src="assets/real2sim-debug-optimization/04-lowres-blurry-preview.png" alt="低 LOD 模糊">
<figcaption>低 LOD 模糊,瓶身和标签细节不足。</figcaption>
</figure>
</div>
<p>进一步排查发现 OBJ 没有 <code>vn</code> 顶点法线,Motrix 光照下表面显得碎。修复后 face token 从 <code>v/vt</code> 升级为 <code>v/vt/vn</code>,并把 LOD 提升到 180k/240k。</p>
</div>
<div class="step">
<h3>5. 深色材质过暗</h3>
<figure>
<img src="assets/real2sim-debug-optimization/05-dark-material-preview.png" alt="深色材质过暗">
<figcaption>问题:葫芦瓶 baked texture luminance 太低。</figcaption>
</figure>
<pre><code>修复前 object_001_det_001_material_0.png mean ~= 18.6 / 255
修复后 object_001_det_001_material_0.png mean ~= 108.4 / 255</code></pre>
<p>通过低亮度贴图自动曝光和 Motrix 场景灯光解决,同时避免正常亮度的水瓶标签过曝。</p>
</div>
<div class="step">
<h3>6. 高 LOD 后瓶盖缺失</h3>
<p>清晰度提升后,瓶盖/提手局部缺失。排查发现这不是源模型缺失,也不是单纯降面问题,而是顶部 artifact 清理器误删了高饱和红色组件。</p>
<figure>
<img src="assets/real2sim-debug-optimization/07-cap-missing-regression.png" alt="瓶盖缺失">
<figcaption>问题:红色瓶盖/提手组件被清理规则误删。</figcaption>
</figure>
<pre><code>原始水瓶 mesh: 285912 faces
旧清理后 highres: 268360 faces
被误删: 17552 faces, 22 components
最大被删组件 mean RGB: [213.7, 27.7, 48.6]</code></pre>
<p>修复是保留高饱和顶部组件,继续删除低饱和灰白漂浮片。修复后 highres 回升到 <code>280843</code> faces。</p>
</div>
</section>
<section>
<h2>问题与修复对照</h2>
<table>
<tr><th>问题现象</th><th>根因</th><th>修复</th></tr>
<tr><td>灰白无材质</td><td>OBJ/MTL/texture 未稳定接入 Motrix</td><td>sidecar namespacing + MJCF texture/material</td></tr>
<tr><td>顶部白色漂浮片</td><td>SAM-3D 透明物体误重建 disconnected sheets</td><td>组件级顶部漂浮片清理</td></tr>
<tr><td>形状随角度变化</td><td>Motrix 加载的是旋转 proxy</td><td>改用 highres texture-preserving LOD</td></tr>
<tr><td>模糊、颗粒</td><td>LOD 面数偏低且无顶点法线</td><td>240k LOD + smooth vertex normals</td></tr>
<tr><td>葫芦过暗</td><td>baked texture luminance 太低</td><td>暗贴图自动曝光 + 场景灯光</td></tr>
<tr><td>瓶盖/提手缺失</td><td>清理器误删高饱和顶部组件</td><td>保护 saturated top parts</td></tr>
</table>
</section>
<section>
<h2>关键命令</h2>
<h3>本地测试</h3>
<pre><code>python3 -m pytest tests</code></pre>
<h3>远端测试</h3>
<pre><code>cd /opt/gs_playground_real2sim_48q
.venv/bin/python -m unittest tests.test_real2sim_packaging</code></pre>
<h3>无影云桌面预览</h3>
<pre><code>cd /opt/gs_playground_real2sim_48q
scripts/real2sim_wuying_viewer.sh \
--package /opt/gs_playground_real2sim_48q/outputs/office_bottle_gourd_synth_multiview_cap_preserved_20260612_142348 \
--native</code></pre>
</section>
<section>
<h2>关键提交</h2>
<pre><code>15a709f 🧢 fix(real2sim): preserve saturated bottle caps
6543c0d ✨ feat(real2sim): add smooth normals to preview meshes
979a6a5 🧭 fix(real2sim): brighten baked mesh textures
76640e0 💡 fix(real2sim): brighten Motrix preview materials
c29e7ba ✨ feat(real2sim): increase Motrix proxy preview fidelity
dfce2ee 🎨 feat(real2sim): add part material atlas for Motrix proxies</code></pre>
</section>
<section>
<h2>截图索引</h2>
<div class="grid">
<figure>
<img src="assets/real2sim-debug-optimization/00-reference-input.png" alt="参考输入">
<figcaption>参考输入</figcaption>
</figure>
<figure>
<img src="assets/real2sim-debug-optimization/01-initial-untextured-preview.png" alt="灰白预览">
<figcaption>初始材质缺失</figcaption>
</figure>
<figure>
<img src="assets/real2sim-debug-optimization/02-floating-top-artifact.png" alt="顶部漂浮片">
<figcaption>顶部 artifact</figcaption>
</figure>
<figure>
<img src="assets/real2sim-debug-optimization/06-highlod-before-cap-fix.png" alt="高 LOD 前">
<figcaption>高 LOD 前的中间状态</figcaption>
</figure>
<figure>
<img src="assets/real2sim-debug-optimization/07-cap-missing-regression.png" alt="瓶盖缺失">
<figcaption>瓶盖缺失回归</figcaption>
</figure>
<figure>
<img src="assets/real2sim-debug-optimization/08-final-cap-preserved-preview-crop.png" alt="最终效果">
<figcaption>最终效果</figcaption>
</figure>
</div>
</section>
<section>
<h2>经验总结</h2>
<p>这次调试的关键是不要只看最终截图,而要逐层检查资产链路:Motrix 实际加载的 mesh 是 proxy 还是真实 OBJ,OBJ 是否有 MTL、texture、UV、vertex normals,清理和降面是否改变了原始组件数量。</p>
<p>最终修复不是单点调参,而是把材质、光照、LOD、法线、清理规则和 Motrix 加载验证一起闭环。</p>
<p>仍然存在的限制是:透明瓶身、反光和水体细节仍受 SAM-3D 单物体重建质量限制。继续提升时,应优先使用更多真实多视角照片,而不是只依赖合成视角。</p>
</section>
<footer>
Generated for GS-Playground Real2Sim debug sharing. Markdown version: real2sim-debug-optimization-report.md
</footer>
</main>
</body>
</html>