feat: 新增 GPU 节点画像与 3D 拓扑工具

- 新增只读 GPU 节点采集器,以统一 JSON 描述系统、CPU/NUMA、GPU、NVLink、PCIe、RDMA、网络与功耗信息
- 统一 Bond、独立 RoCE 和 InfiniBand 端点模型,支持 GPU 到 RDMA 端点的全局最优唯一匹配
- 亲和关系严格记录 nvidia-smi topo -m 证据,未暴露的 IB 设备不推测 GPU 连线
- 新增原生 HTML 运维画像和 Canvas 3D CT,可按 NUMA、PCIe 分组及 Bond 逐层查看物理成员端口
- 展示 CPU 亲和、P-State、当前功耗与 Power Limit,并提供 RDMA、Bond、IB、管理网络及整机健康检查
- 排除 CNI/veth 等虚拟接口对管理网络画像的干扰,普通管理 Bond 不计入 RDMA Bond 验收
- 提供脱敏 B300 真机示例、离线自包含页面、使用文档及 17 项解析和渲染回归测试
- 修正生成物忽略规则,确保两个 HTML 源模板能够随工具提交
This commit is contained in:
hongshuai.dong 2026-07-25 13:47:36 +08:00
parent 219065eea1
commit 6ea2acf2d6
11 changed files with 12939 additions and 0 deletions

View File

@ -19,6 +19,7 @@
- [配置说明](#配置说明) - [配置说明](#配置说明)
- [测试 SOP(标准操作流程)](#测试-sop标准操作流程) - [测试 SOP(标准操作流程)](#测试-sop标准操作流程)
- [报告输出](#报告输出) - [报告输出](#报告输出)
- [GPU 节点画像工具](#gpu-节点画像工具)
- [故障排除](#故障排除) - [故障排除](#故障排除)
--- ---
@ -42,6 +43,11 @@ servertest/
│ ├── training_sim.py # 训练模拟 │ ├── training_sim.py # 训练模拟
│ └── report.py # 报告生成 │ └── report.py # 报告生成
├── scripts/ ├── scripts/
│ ├── gpu_node_profile/ # JSON 驱动的运维画像与 3D CT
│ │ ├── gpu_node_profile.py # 只读采集与渲染入口
│ │ ├── templates/ # 两个独立 HTML 源模板
│ │ ├── examples/ # 脱敏真机 JSON 与自包含示例页面
│ │ └── tests/ # 解析与渲染回归测试
│ └── multinode_nccl/ # 独立 shell 脚本版 (历史备用) │ └── multinode_nccl/ # 独立 shell 脚本版 (历史备用)
│ ├── nccl_env.sh # 环境变量 (MPI路径/IB网卡/bond0) │ ├── nccl_env.sh # 环境变量 (MPI路径/IB网卡/bond0)
│ ├── hosts_all # 全量 34 台节点列表 │ ├── hosts_all # 全量 34 台节点列表
@ -677,6 +683,27 @@ python3 gpu_tester.py --test all --format html --output report.html
--- ---
## GPU 节点画像工具
`scripts/gpu_node_profile/` 提供独立、只读的 GPU 节点画像工具。采集器可在
一次执行中生成结构化 JSON,并由同一份 JSON 分别渲染运维画像和可逐层钻取
的 3D CT 页面。两个 HTML 源模板位于 `templates/`;`examples/` 包含脱敏
真机 JSON 和由其生成的两张自包含示例页面。支持 NUMA、PCIe、GPU、
NVLink/NVSwitch、RoCE Bond、无 Bond 的独立 RoCE/InfiniBand 端口、
物理网卡、CPU 亲和性和功耗信息。GPU 到 RDMA 端点的亲和关系来自
`nvidia-smi topo -m`,页面不会为矩阵未暴露的设备推测连线;详细命令与
采集边界见该目录的 `README.md`。
```bash
cd scripts/gpu_node_profile
python3 gpu_node_profile.py \
--snapshot-out gpu_node_snapshot.json \
--output gpu_node_profile.html \
--topology-output gpu_topology_3d.html
```
---
## 故障排除 ## 故障排除
| 问题 | 原因 | 解决方案 | | 问题 | 原因 | 解决方案 |

5
scripts/gpu_node_profile/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
/gpu_node_snapshot*.json
/gpu_node_profile*.html
/gpu_topology_3d*.html
/b300_*_live.json
/b300_*_live.html

View File

@ -0,0 +1,155 @@
# GPU Node Profile
`gpu_node_profile.py` 通过只读命令采集 GPU、NVLink、PCIe/NUMA、RoCE
Bond、RDMA 和 InfiniBand 信息,生成通用结构化 JSON,并使用 `templates/`
中的两个独立 HTML 源模板,从同一份 JSON 渲染完全离线、无 Mermaid、
无 CDN 依赖的运维画像与 3D 拓扑 CT。
页面顶部按“系统与软件 / GPU / CPU与内存”展示整机概览;中部的
GPU-RDMA 拓扑支持“总览 / NUMA 分览”即时切换,并继续展示 InfiniBand、
OS 可见的带内管理网卡和可读取的 BMC 带外信息;最底部横向展示健康检查
与验证结论。
3D CT 页面使用原生 Canvas 对结构化 JSON 动态布局,包含 NUMA 半透明地域、
CPU Socket、PCIe Root Complex、PCIe Switch 分组、GPU、NVSwitch、RoCE、
InfiniBand 和管理网卡。NUMA 与 PCIe 均来自数据,不包含 B300 专用坐标,
因此同一渲染器也可用于 A800 等其他 GPU 服务器。
页面提供递归内部镜头,可按“整机 → NUMA → PCIe 分组 → Bond → 物理网卡”
逐层钻取。进入任一层后会隐藏外层与无关的同级设备,只保留当前壳体和后代
节点;顶部面包屑说明当前位置,“上一层”按原路径返回。双击壳体继续深入,
双击普通设备时也会沿该设备的父级路径每次深入一层,例如连续双击 GPU 会
依次进入整机、NUMA 和对应 PCIe 分组;到达最深层后才执行设备聚焦。
`Shift + 拖动`可平移,`Esc`返回整机外观。
Canvas 标签使用高对比浅色底与黑色粗体。每个 RoCE Bond 会作为透明聚合
容器展示,并在内部继续拆出物理成员网卡、速率和 BDF。NVLink/NVSwitch
连线在设备盒子上层高亮绘制,GPU 端点带编号;该 Fabric 只连接 GPU,
不会连接网卡。
网络模型同时支持两种形态:存在 Bond 时渲染
`GPU → RoCE Bond → 物理成员端口`;不存在 Bond 时根据
`nvidia-smi topo -m`、RDMA 名称和 PCIe 路径渲染
`GPU → 独立 RoCE/InfiniBand 端点`。多张 GPU 与多个等距端点会进行
全局最优、确定性的唯一分配,避免顺序贪心导致所有 GPU 错误指向同一张
网卡;端点少于 GPU 时才保留可共享的最近路径语义。亲和关系只接受
`nvidia-smi topo -m` 暴露的 `PIX/PXB/PHB/NODE/SYS` 证据;若独立 IB
未出现在矩阵中,会保留设备和 PCIe/NUMA 信息并标记 `not-exposed`,但不会
推测一条 GPU 连线。
JSON 中的 `network_endpoints` 是两张页面共用的标准网络模型:
```text
network_endpoints[]
id / type(bond|standalone) / protocol(RoCE|InfiniBand)
rdma / numa / bdf / state / topology_visibility
members[] # Bond 为多个物理口,直连端点为一个物理口
gpu_links[] # GPU、PIX/PXB/...、用途及证据来源
gpus[].primary_network_id # 指向主要计算 RDMA 端点
gpus[].network_links[] # 保留主要端点和额外最近端点
```
CPU 与内存概览、NUMA 标尺和每张 GPU 卡片会同时展示 `nvidia-smi topo -m`
解析出的推荐 CPU 范围及内存 NUMA。该信息描述硬件拓扑的近端路径,不表示
训练或推理进程已经通过 `taskset`、cgroup、Slurm 或 Kubernetes 完成实际绑核。
GPU 概览和单卡画像会展示采集时刻的 `power.draw`、当前 `power.limit` 与
P-State。Power Limit 表示当前生效的 GPU 功耗上限,不等于整机额定功耗;
P0 表示最高性能状态,也不单独等价于 GPU 持续满载。
## 在 GPU 节点执行
```bash
python3 gpu_node_profile.py \
--output gpu_node_profile.html \
--topology-output gpu_topology_3d.html \
--snapshot-out gpu_node_snapshot.json \
--expected-gpus 8 \
--expected-bonds 8 \
--expected-ib 4
```
脚本不会修改网卡、驱动、Bond 或系统配置。唯一写操作是生成用户指定的
HTML/JSON 文件。`--expected-bonds` 校验的是 RDMA/RoCE Bond;普通管理
Bond 会保留在原始网卡信息中,但不会计入加速网络端点数量。
三个产物的关系:
```text
只读采集 → gpu_node_snapshot.json
├─→ gpu_node_profile.html
└─→ gpu_topology_3d.html
```
源码与演示数据的关系:
```text
gpu_node_profile.py # 只读采集、校验和渲染入口
templates/gpu_node_profile.html # 运维画像源模板
templates/gpu_topology_3d.html # 3D CT 源模板
examples/b300_demo.json # 由真机快照脱敏得到的演示数据
examples/b300_profile_demo.html # demo.json 生成的自包含示例
examples/b300_topology_3d_demo.html
```
最终生成的两张 HTML 会把 JSON 嵌入页面,因此可以直接双击、复制到其他机器
或离线归档。它们不会在打开时读取旁边的 `demo.json`;修改或替换 JSON 后,
需要重新运行 Python 渲染:
```bash
python3 gpu_node_profile.py \
--snapshot-in examples/b300_demo.json \
--output examples/b300_profile_demo.html \
--topology-output examples/b300_topology_3d_demo.html \
--expected-gpus 8 \
--expected-bonds 8 \
--expected-ib 4
```
## 使用已有快照离线渲染
```bash
python3 gpu_node_profile.py \
--snapshot-in examples/b300_demo.json \
--output b300_profile_demo.html \
--topology-output b300_topology_3d_demo.html \
--expected-gpus 8 \
--expected-bonds 8 \
--expected-ib 4
```
只采集 JSON 并写到标准输出,适合通过 SSH 在远端只读采集、在本地渲染:
```bash
python3 gpu_node_profile.py --snapshot-stdout
```
## 测试
```bash
python3 -m unittest discover -s tests -v
```
## 采集边界
- `nvcc` 表示当前 PATH 指向的 CUDA Toolkit,不代表容器或应用实际加载的
CUDA Runtime。
- LACP Bond 展示的是成员链路聚合容量,不承诺单连接可达到聚合带宽。
- 虚拟机中的 PCIe/NUMA 数据表示 Guest 可见拓扑,不能代替宿主机物理布线
和交换机侧验证。
- `pci_fabric` 保存 Guest 可见的 PCIe 祖先路径、桥和端点。3D 页面中的空间
坐标由这些逻辑关系动态生成,不代表设备在机箱中的厘米级真实位置。
- `PIX/PXB/PHB/NODE/SYS` 表示拓扑距离。`PIX` 可作为 GPUDirect RDMA 的
良好候选路径,但只有完成驱动能力和实际 RDMA 传输验证后才能标记为
`GDR Verified`。
- CPU 亲和表示 NVIDIA 拓扑给出的推荐 CPU/内存范围,不是运行中进程的
`Cpus_allowed_list`。评估实际 H2D 性能还需结合进程绑核、内存落点、
pinned memory 和带宽实测。
- GPU 功耗为报告生成时的一次瞬时采样;周期性训练负载的平均功耗和峰值
需要连续采样。整机功耗还需结合 BMC/DCMI 或机柜 PDU,不能简单等同于
所有 GPU Power Limit 之和。
- 独立BMC口通常不进入宿主OS网络命名空间。脚本会尝试从常见LAN通道只读
执行 `ipmitool lan print`;读取不到时会在画像中明确标记“OS不可见”。
- 管理网络区域只渲染物理 PCI 网卡、Bond 或默认路由接口;CNI、veth 等
容器虚拟接口仍可保留在原始 JSON 的 `nics` 中,但不会污染整机画像。

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,246 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="color-scheme" content="dark">
<title>@@TITLE@@</title>
<style>
:root {
--bg:#07100e; --surface:#0d1916; --surface-2:#12231e; --line:#284238;
--text:#edf8f3; --muted:#8da79b; --green:#76b900; --green-2:#a5e34b;
--cyan:#39d5d0; --amber:#ffbd4a; --red:#ff6b6b; --shadow:0 24px 60px rgba(0,0,0,.28);
}
* { box-sizing:border-box; }
html { background:var(--bg); }
body {
margin:0; color:var(--text); background:
radial-gradient(circle at 18% -5%,rgba(118,185,0,.14),transparent 29rem),
radial-gradient(circle at 92% 8%,rgba(57,213,208,.08),transparent 24rem), var(--bg);
font:14px/1.5 Inter,ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;
}
body::before {
content:""; position:fixed; inset:0; pointer-events:none; opacity:.26;
background-image:linear-gradient(rgba(255,255,255,.025) 1px,transparent 1px),
linear-gradient(90deg,rgba(255,255,255,.025) 1px,transparent 1px);
background-size:32px 32px;
}
.hero,main,footer { width:min(1680px,calc(100% - 48px)); margin-inline:auto; position:relative; }
.hero { padding:44px 0 28px; display:flex; align-items:flex-end; justify-content:space-between; gap:24px; }
.kicker,.eyebrow { color:var(--green-2); font-size:11px; font-weight:800; letter-spacing:.18em; }
h1 { margin:6px 0 2px; font-size:clamp(34px,4vw,58px); letter-spacing:-.045em; line-height:1; }
.hero p,.gpu-card p,.nic-card p { color:var(--muted); margin:8px 0 0; }
.hero-actions { display:flex; align-items:center; gap:12px; }
button {
border:1px solid var(--line); background:var(--surface-2); color:var(--text);
border-radius:12px; padding:10px 14px; font-weight:750; cursor:pointer;
}
button:hover { border-color:var(--green); }
.overall { display:flex; gap:9px; align-items:center; font-weight:800; padding:10px 14px; border-radius:999px; border:1px solid var(--line); }
.overall i { width:9px; height:9px; border-radius:50%; box-shadow:0 0 18px currentColor; }
.overall.healthy { color:var(--green-2); }.overall.healthy i{background:var(--green-2)}
.overall.attention { color:var(--amber); }.overall.attention i{background:var(--amber)}
.overall.critical { color:var(--red); }.overall.critical i{background:var(--red)}
.overview-grid { display:grid; grid-template-columns:1.15fr .9fr 1fr; gap:12px; margin-bottom:14px; }
.overview-card,.panel {
background:linear-gradient(145deg,rgba(18,35,30,.94),rgba(10,21,18,.96));
border:1px solid var(--line); box-shadow:var(--shadow);
}
.overview-card { min-height:290px; border-radius:18px; padding:20px; overflow:hidden; }
.overview-head { display:flex; align-items:flex-start; justify-content:space-between; gap:14px; }
.overview-lead { color:var(--muted); margin:12px 0 0; min-height:42px; }
.overview-primary { display:flex; align-items:baseline; gap:9px; margin:8px 0 17px; }
.overview-primary b { font-size:52px; line-height:1; letter-spacing:-.055em; }
.overview-primary span { color:var(--muted); font-weight:750; }
.gpu-power-summary { margin:-5px 0 12px; }
.gpu-power-summary>div:first-child { display:flex; justify-content:space-between; align-items:baseline; gap:10px; font-size:10px; color:var(--muted); }
.gpu-power-summary>div:first-child b { color:var(--text); font-size:12px; }
.gpu-power-summary>small { display:block; margin-top:4px; color:var(--muted); font-size:8px; }
.overview-metrics { display:grid; grid-template-columns:repeat(2,1fr); gap:1px; background:var(--line); border:1px solid var(--line); border-radius:10px; overflow:hidden; }
.overview-metrics span { background:rgba(7,16,14,.72); padding:10px; min-width:0; }
.overview-metrics b,.overview-metrics small { display:block; }
.overview-metrics b { font-size:17px; }.overview-metrics small { color:var(--muted); margin-top:2px; font-size:9px; }
.overview-facts { display:grid; grid-template-columns:repeat(2,1fr); gap:1px; background:var(--line); border:1px solid var(--line); border-radius:10px; overflow:hidden; margin-top:12px; }
.overview-facts div { background:rgba(7,16,14,.72); padding:9px 10px; min-width:0; }
.overview-facts dt { font-size:9px; letter-spacing:.08em; text-transform:uppercase; }
.overview-facts dd { text-align:left; margin-top:3px; font-size:11px; font-weight:700; overflow-wrap:anywhere; }
.overview-facts small { color:var(--muted); }
.compact { padding:7px 9px; font-size:11px; }
.memory-row { display:flex; justify-content:space-between; gap:10px; color:var(--muted); font-size:10px; }
.memory-row b { color:var(--text); }
.memory-meter { height:6px; background:#172821; border-radius:10px; margin:7px 0 14px; overflow:hidden; }
.memory-meter i { display:block; height:100%; background:linear-gradient(90deg,var(--cyan),var(--green-2)); }
.cpu-metrics { grid-template-columns:repeat(3,1fr); }
.cpu-affinity-summary { display:grid; grid-template-columns:repeat(2,minmax(0,1fr)); gap:6px; margin-top:9px; }
.cpu-affinity-summary span { border:1px solid var(--line); background:rgba(7,16,14,.55); border-radius:8px; padding:7px 8px; min-width:0; }
.cpu-affinity-summary b,.cpu-affinity-summary small { display:block; }
.cpu-affinity-summary b { color:var(--green-2); font-size:10px; }.cpu-affinity-summary small { color:var(--muted); font-size:8px; overflow-wrap:anywhere; }
.panel { border-radius:20px; padding:22px; margin-bottom:14px; overflow:hidden; }
.section-head { display:flex; justify-content:space-between; align-items:flex-start; gap:20px; margin-bottom:20px; }
h2 { margin:3px 0 0; font-size:24px; letter-spacing:-.025em; }
.legend { display:flex; flex-wrap:wrap; gap:7px; justify-content:flex-end; }
.badge { display:inline-flex; align-items:center; width:max-content; border:1px solid var(--line); color:var(--muted); border-radius:999px; padding:3px 8px; font-size:10px; font-weight:850; letter-spacing:.05em; white-space:nowrap; }
.badge.ok { color:var(--green-2); border-color:rgba(165,227,75,.34); background:rgba(118,185,0,.09); }
.badge.warn { color:var(--amber); border-color:rgba(255,189,74,.35); background:rgba(255,189,74,.08); }
.badge.bad { color:var(--red); border-color:rgba(255,107,107,.35); background:rgba(255,107,107,.08); }
.topology-viewbar { display:flex; align-items:center; justify-content:space-between; gap:14px; margin:-5px 0 14px; padding:10px 12px; border:1px solid var(--line); border-radius:12px; background:rgba(7,16,14,.48); }
.topology-viewbar b,.topology-viewbar small { display:block; }
.topology-viewbar b { font-size:11px; }.topology-viewbar small { color:var(--muted); font-size:9px; margin-top:2px; }
.numa-switcher { display:flex; align-items:center; gap:4px; padding:3px; border:1px solid var(--line); border-radius:10px; background:#07100e; }
.numa-switch { min-width:72px; padding:6px 10px; border:0; border-radius:7px; background:transparent; color:var(--muted); font-size:10px; }
.numa-switch:hover { border-color:transparent; color:var(--text); }
.numa-switch.active { color:#07100e; background:var(--green-2); box-shadow:0 0 18px rgba(165,227,75,.12); }
.numa-ruler { display:grid; grid-template-columns:1fr 1fr; margin:0 0 8px; gap:2px; }
.numa-ruler span { text-align:center; padding:5px; color:var(--muted); font-size:11px; font-weight:800; letter-spacing:.14em; border-bottom:1px solid var(--line); }
.numa-ruler small { display:block; margin-top:2px; color:var(--green-2); font-size:9px; letter-spacing:.04em; }
.affinity-note { color:var(--muted); text-align:center; font-size:9px; margin:3px 0 10px; }
.nvlink-fabric {
border:1px solid rgba(118,185,0,.5); background:linear-gradient(90deg,rgba(118,185,0,.18),rgba(57,213,208,.08),rgba(118,185,0,.18));
border-radius:13px; padding:11px 16px; display:flex; align-items:center; justify-content:center; gap:18px; color:var(--green-2); margin-bottom:16px;
}
.nvlink-fabric span { font-size:11px; letter-spacing:.12em; font-weight:900; }
.nvlink-fabric small { color:var(--muted); }
.rails { display:grid; gap:10px; }
.topology-panel { overflow-x:auto; }
.rail { position:relative; display:grid; grid-template-rows:auto 54px 1fr; }
.rail[hidden],.numa-ruler [hidden] { display:none!important; }
.rail::before { content:""; position:absolute; z-index:0; left:50%; top:150px; bottom:150px; width:2px; background:linear-gradient(var(--green),var(--cyan)); opacity:.6; }
.gpu-card,.nic-card,.ib-card { position:relative; z-index:1; border:1px solid var(--line); border-radius:14px; background:rgba(7,16,14,.92); }
.gpu-card { padding:13px; border-top:3px solid var(--green); min-height:278px; }
.card-head { display:flex; justify-content:space-between; align-items:center; gap:8px; }
.gpu-card h3,.nic-card h4,.ib-card h4 { margin:10px 0 0; line-height:1.14; }
.gpu-card h3 { min-height:33px; font-size:15px; }
.mono { font:10px/1.4 ui-monospace,SFMono-Regular,Menlo,monospace; }
.gpu-affinity { display:grid; grid-template-columns:1.4fr .6fr; gap:5px; margin-top:8px; }
.gpu-affinity span { min-width:0; background:var(--surface); border:1px solid var(--line); border-radius:7px; padding:5px 6px; }
.gpu-affinity small,.gpu-affinity b { display:block; }
.gpu-affinity small { color:var(--muted); font-size:7px; letter-spacing:.08em; }
.gpu-affinity b { color:var(--green-2); margin-top:1px; font:9px/1.25 ui-monospace,monospace; overflow-wrap:anywhere; }
.gpu-stats { display:grid; grid-template-columns:1.05fr .75fr .9fr; gap:3px; margin-top:15px; }
.gpu-stats span { min-width:0; }.gpu-stats b,.gpu-stats small { display:block; }
.gpu-stats b { font-size:12px; }.gpu-stats small { color:var(--muted); font-size:9px; }
.meter { height:4px; background:#172821; border-radius:10px; margin-top:11px; overflow:hidden; }
.meter i { display:block; height:100%; background:linear-gradient(90deg,var(--green),var(--green-2)); }
.gpu-power { margin-top:10px; padding-top:8px; border-top:1px solid var(--line); }
.gpu-power>div:first-child { display:flex; align-items:flex-start; justify-content:space-between; gap:5px; }
.gpu-power span,.gpu-power b,.gpu-power small { display:block; min-width:0; }
.gpu-power b { font-size:10px; }.gpu-power small { color:var(--muted); font-size:7px; margin-top:1px; }
.gpu-power em { flex:0 0 auto; color:var(--cyan); border:1px solid rgba(57,213,208,.3); border-radius:999px; padding:2px 5px; font:normal 800 8px/1.2 ui-monospace,monospace; }
.power-meter { height:5px; margin-top:6px; overflow:hidden; border-radius:999px; background:#172821; }
.power-meter i { display:block; height:100%; background:linear-gradient(90deg,var(--cyan),var(--green-2),var(--amber)); }
.relation { position:relative; z-index:2; align-self:center; justify-self:center; display:flex; align-items:center; flex-direction:column; gap:3px; color:var(--muted); font-size:9px; }
.nic-card { padding:13px; border-top:3px solid var(--cyan); min-height:234px; }
.nic-card h4,.ib-card h4 { font-size:18px; }.nic-card p { font:10px ui-monospace,monospace; }
.bond-capacity { display:flex; align-items:baseline; gap:6px; margin:14px 0 10px; }
.bond-capacity b { font-size:24px; }.bond-capacity span { color:var(--muted); font-size:9px; }
.ports { display:grid; gap:6px; }
.port { display:flex; align-items:center; gap:8px; background:var(--surface); border:1px solid var(--line); border-radius:8px; padding:7px; }
.port-dot { width:7px; height:7px; flex:0 0 auto; border-radius:50%; background:var(--muted); }
.port.up .port-dot { background:var(--green-2); box-shadow:0 0 9px rgba(165,227,75,.6); }
.port.down .port-dot { background:var(--red); }
.port b,.port small { display:block; }.port b { font-size:11px; }.port small { color:var(--muted); font:8px ui-monospace,monospace; margin-top:2px; }
.capacity-note { color:var(--muted); margin:16px 0 0; font-size:11px; }
.network-grid { display:grid; grid-template-columns:1fr 1fr; gap:14px; }
.ib-grid { display:grid; grid-template-columns:repeat(4,minmax(0,1fr)); gap:10px; }
.ib-card { min-width:0; padding:14px; border-top:3px solid var(--amber); overflow:hidden; }
.ib-card .card-head { align-items:flex-start; flex-wrap:wrap; row-gap:7px; }
.ib-card .card-head .eyebrow { min-width:0; overflow-wrap:anywhere; font-size:9px; letter-spacing:.13em; }
.ib-card .card-head .badge { flex:0 0 auto; margin-left:auto; max-width:100%; padding:3px 6px; font-size:8px; letter-spacing:.03em; }
.ib-rate { color:var(--amber); font-size:30px; font-weight:850; margin:13px 0; }
.ib-rate small { font-size:11px; margin-left:3px; color:var(--muted); }
dl { margin:0; }.ib-card dl { display:grid; gap:5px; }
.ib-card dl div { display:flex; justify-content:space-between; gap:8px; font-size:10px; }
.ib-card dd { min-width:0; overflow-wrap:anywhere; }
dt { color:var(--muted); } dd { margin:0; text-align:right; }
.management-grid { display:grid; grid-template-columns:repeat(2,minmax(0,1fr)); gap:10px; }
.management-card { border:1px solid var(--line); border-top:3px solid var(--cyan); border-radius:14px; background:rgba(7,16,14,.92); padding:14px; min-height:205px; }
.management-card.bmc-card { border-top-color:#8da79b; }
.management-card.unavailable { border-top-color:var(--line); }
.management-title { display:flex; justify-content:space-between; align-items:baseline; gap:10px; margin-top:12px; }
.management-title h4 { margin:0; font-size:19px; }.management-title b { color:var(--cyan); font-size:18px; }
.management-card .address { color:var(--text); background:var(--surface); border:1px solid var(--line); border-radius:8px; padding:8px; min-height:38px; font:10px/1.5 ui-monospace,monospace; }
.management-card dl { display:grid; gap:4px; }
.management-card dl div { display:flex; justify-content:space-between; gap:8px; font-size:9px; }
.management-card .card-note { color:var(--muted); font-size:10px; margin-top:12px; }
.health-panel { margin-bottom:14px; }
.checks { list-style:none; padding:0; margin:0; display:grid; grid-template-columns:repeat(auto-fit,minmax(175px,1fr)); gap:8px; }
.check { display:flex; align-items:center; gap:11px; padding:12px; min-height:68px; border:1px solid var(--line); border-radius:10px; background:rgba(7,16,14,.55); }
.check-icon { width:25px; height:25px; display:grid; place-items:center; border-radius:50%; font-weight:900; flex:0 0 auto; }
.check.ok .check-icon { color:var(--green-2); background:rgba(118,185,0,.13); }
.check.warning .check-icon { color:var(--amber); background:rgba(255,189,74,.12); }
.check.critical .check-icon { color:var(--red); background:rgba(255,107,107,.12); }
.check b,.check small { display:block; }.check small { color:var(--muted); }
.ghost { background:transparent; }
pre { white-space:pre-wrap; word-break:break-word; background:#050b09; color:#b8cdc3; border:1px solid var(--line); border-radius:12px; padding:16px; max-height:480px; overflow:auto; font:11px/1.6 ui-monospace,monospace; }
dialog { width:min(1100px,calc(100% - 32px)); max-height:86vh; color:var(--text); background:var(--surface); border:1px solid var(--line); border-radius:18px; padding:20px; box-shadow:0 35px 100px rgba(0,0,0,.65); }
dialog::backdrop { background:rgba(0,0,0,.72); backdrop-filter:blur(5px); }
.dialog-head { display:flex; align-items:flex-start; justify-content:space-between; gap:20px; }
dialog pre { max-height:68vh; }
.empty-state { color:var(--muted); border:1px dashed var(--line); border-radius:12px; padding:30px; text-align:center; grid-column:1/-1; }
footer { padding:8px 0 30px; color:var(--muted); display:flex; justify-content:space-between; font-size:11px; }
@media (max-width:1100px) {
.overview-grid { grid-template-columns:1fr 1fr; }
.system-overview { grid-column:1/-1; }
.network-grid { grid-template-columns:1fr; }
}
@media (max-width:640px) {
.hero,main,footer { width:min(100% - 24px,1680px); }
.hero { align-items:flex-start; flex-direction:column; padding-top:28px; }
.overview-grid { grid-template-columns:1fr; }.system-overview { grid-column:auto; }
.panel { padding:15px; }.section-head { flex-direction:column; }
.ib-grid,.management-grid { grid-template-columns:1fr; }
.topology-viewbar { align-items:flex-start; flex-direction:column; }
.numa-switcher { width:100%; }.numa-switch { flex:1; min-width:0; }
.nvlink-fabric { align-items:flex-start; flex-direction:column; gap:2px; }
}
@media print {
:root { --bg:#fff; --surface:#fff; --surface-2:#f3f6f4; --line:#cbd6d0; --text:#102018; --muted:#52675d; --shadow:none; }
body { background:#fff; }.hero,main,footer { width:100%; }.hero { padding-top:10px; }
button,dialog { display:none!important; }.panel,.overview-card { break-inside:avoid; }
.topology-panel { overflow:visible; zoom:.72; }.rails { min-width:0; }
}
</style>
</head>
<body>
@@BODY@@
<script>
const rawDialog=document.getElementById('raw-dialog');
document.querySelector('[data-open-raw]')?.addEventListener('click',()=>rawDialog?.showModal());
document.querySelector('[data-close-raw]')?.addEventListener('click',()=>rawDialog?.close());
rawDialog?.addEventListener('click',event=>{if(event.target===rawDialog)rawDialog.close();});
const topologyRails=document.querySelector('.rails');
const numaRuler=document.querySelector('.numa-ruler');
const numaSwitches=[...document.querySelectorAll('[data-numa-filter]')];
function setNumaView(value){
const rails=[...document.querySelectorAll('.rail[data-numa]')];
const rulers=[...document.querySelectorAll('.numa-ruler [data-numa]')];
let visibleRails=0;
rails.forEach(rail=>{
const visible=value==='all'||rail.dataset.numa===value;
rail.hidden=!visible;
if(visible)visibleRails+=1;
});
const visibleRulers=[];
rulers.forEach(ruler=>{
const visible=value==='all'||ruler.dataset.numa===value;
ruler.hidden=!visible;
if(visible)visibleRulers.push(ruler);
});
if(topologyRails){
const count=Math.max(1,visibleRails);
topologyRails.style.gridTemplateColumns=`repeat(${count},minmax(132px,1fr))`;
topologyRails.style.minWidth=`${count*132+Math.max(0,count-1)*10}px`;
}
if(numaRuler){
numaRuler.style.gridTemplateColumns=visibleRulers.map(item=>`${item.dataset.count||1}fr`).join(' ')||'1fr';
}
numaSwitches.forEach(button=>{
const active=button.dataset.numaFilter===value;
button.classList.toggle('active',active);
button.setAttribute('aria-pressed',String(active));
});
}
numaSwitches.forEach(button=>button.addEventListener('click',()=>setNumaView(button.dataset.numaFilter)));
</script>
</body>
</html>

View File

@ -0,0 +1,980 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="color-scheme" content="dark">
<title>@@TITLE@@</title>
<style>
:root {
--bg:#050c0a; --surface:#0b1713; --surface-2:#10241d; --line:#28493d;
--text:#eef9f4; --muted:#8fa99e; --green:#a5e34b; --cyan:#51d9d2;
--amber:#ffbd4a; --purple:#b592ff; --red:#ff6b6b;
}
* { box-sizing:border-box; }
html,body { margin:0; min-height:100%; background:var(--bg); color:var(--text); }
body {
font:14px/1.45 Inter,ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;
background:
radial-gradient(circle at 16% 0,rgba(118,185,0,.15),transparent 30rem),
radial-gradient(circle at 90% 8%,rgba(81,217,210,.08),transparent 28rem),
var(--bg);
}
button { font:inherit; }
.shell { min-height:100vh; display:grid; grid-template-rows:auto 1fr auto; }
.topbar {
min-width:0; display:flex; align-items:flex-end; justify-content:space-between;
gap:24px; padding:22px 28px 16px; border-bottom:1px solid var(--line);
}
.kicker { color:var(--green); font-size:10px; font-weight:850; letter-spacing:.19em; }
h1 { margin:4px 0 0; font-size:clamp(25px,3vw,42px); letter-spacing:-.045em; line-height:1; }
.subtitle { margin:7px 0 0; color:var(--muted); }
.top-stats { display:flex; flex-wrap:wrap; justify-content:flex-end; gap:8px; }
.stat {
min-width:96px; padding:8px 11px; border:1px solid var(--line); border-radius:10px;
background:rgba(11,23,19,.76);
}
.stat b,.stat small { display:block; }.stat b { font-size:15px; }.stat small { color:var(--muted); font-size:9px; }
.workspace { min-height:0; display:grid; grid-template-columns:minmax(0,1fr) 330px; gap:0; }
.stage { min-width:0; min-height:720px; position:relative; overflow:hidden; }
#topology-canvas { display:block; width:100%; height:100%; min-height:720px; outline:none; cursor:grab; }
#topology-canvas:active { cursor:grabbing; }
#topology-canvas:focus-visible { box-shadow:inset 0 0 0 2px var(--green); }
.toolbar {
position:absolute; z-index:4; left:18px; right:18px; top:16px;
display:flex; align-items:flex-start; justify-content:space-between; gap:12px;
pointer-events:none;
}
.toolbar-group {
pointer-events:auto; display:flex; flex-wrap:wrap; align-items:center; gap:5px;
padding:5px; border:1px solid var(--line); border-radius:11px;
background:rgba(5,12,10,.86); backdrop-filter:blur(10px);
}
.toolbar-label { padding:0 7px; color:var(--muted); font-size:9px; font-weight:800; letter-spacing:.12em; }
.tool-button {
border:0; border-radius:8px; padding:7px 10px; color:var(--muted);
background:transparent; cursor:pointer; font-size:10px; font-weight:800;
}
.tool-button:hover { color:var(--text); background:rgba(255,255,255,.05); }
.tool-button[aria-pressed="true"] { color:#07100e; background:var(--green); }
.tool-button:disabled { opacity:.35; cursor:not-allowed; }
.tool-button.reset { color:var(--text); border:1px solid var(--line); }
.camera-status {
position:absolute; z-index:3; left:50%; top:72px; transform:translateX(-50%);
padding:6px 10px; border:1px solid rgba(165,227,75,.32); border-radius:999px;
color:var(--green); background:rgba(5,12,10,.76); backdrop-filter:blur(8px);
font:800 9px ui-monospace,monospace; letter-spacing:.1em; pointer-events:none;
}
.canvas-note {
position:absolute; z-index:3; left:22px; bottom:18px; max-width:min(630px,calc(100% - 44px));
padding:8px 11px; color:var(--muted); font-size:10px; border:1px solid var(--line);
border-radius:9px; background:rgba(5,12,10,.8); backdrop-filter:blur(8px);
}
.legend {
position:absolute; z-index:3; right:20px; bottom:18px; display:flex; flex-wrap:wrap;
justify-content:flex-end; gap:9px 12px; max-width:52%; padding:8px 10px;
border:1px solid var(--line); border-radius:9px; background:rgba(5,12,10,.8);
}
.legend span { display:flex; align-items:center; gap:5px; color:var(--muted); font-size:9px; }
.legend i { width:18px; height:3px; border-radius:9px; background:var(--line-color); }
.inspector {
min-width:0; padding:20px; border-left:1px solid var(--line);
background:linear-gradient(180deg,rgba(13,28,23,.97),rgba(7,16,13,.97));
overflow:auto;
}
.inspector h2 { margin:4px 0 5px; font-size:21px; }.inspector>p { color:var(--muted); margin:0 0 18px; }
.scope {
margin-bottom:16px; padding:10px; color:var(--muted); border:1px solid var(--line);
border-left:3px solid var(--amber); border-radius:9px; background:rgba(255,189,74,.04); font-size:10px;
}
.mini-grid { display:grid; grid-template-columns:1fr 1fr; gap:7px; margin-bottom:18px; }
.mini-grid div { min-width:0; padding:9px; border:1px solid var(--line); border-radius:8px; background:rgba(5,12,10,.5); }
.mini-grid b,.mini-grid small { display:block; }.mini-grid b { font-size:16px; }.mini-grid small { color:var(--muted); font-size:9px; }
.detail {
padding-top:15px; border-top:1px solid var(--line);
}
.detail-head { display:flex; justify-content:space-between; gap:12px; align-items:flex-start; }
.detail h3 { margin:3px 0 0; font-size:19px; }.detail-kind { color:var(--green); font-size:9px; font-weight:850; letter-spacing:.15em; }
.detail-status { color:var(--cyan); font:800 10px ui-monospace,monospace; }
.detail dl { display:grid; gap:1px; margin:14px 0 0; border:1px solid var(--line); border-radius:9px; overflow:hidden; }
.detail dl div { display:grid; grid-template-columns:92px minmax(0,1fr); gap:8px; padding:8px 9px; background:rgba(5,12,10,.52); border-bottom:1px solid var(--line); }
.detail dl div:last-child { border-bottom:0; }
.detail dt { color:var(--muted); font-size:9px; }.detail dd { margin:0; text-align:right; overflow-wrap:anywhere; font:10px/1.4 ui-monospace,monospace; }
.path-guide { margin-top:18px; }.path-guide h3 { margin:0 0 9px; font-size:13px; }
.path-guide ul { margin:0; padding-left:17px; color:var(--muted); font-size:10px; }
.path-guide li+li { margin-top:5px; }
footer { display:flex; justify-content:space-between; gap:16px; padding:10px 28px 16px; color:var(--muted); font-size:9px; border-top:1px solid var(--line); }
@media (max-width:980px) {
.workspace { grid-template-columns:1fr; }
.inspector { border-left:0; border-top:1px solid var(--line); }
.stage,#topology-canvas { min-height:620px; }
.mini-grid { grid-template-columns:repeat(4,minmax(0,1fr)); }
}
@media (max-width:640px) {
.topbar { align-items:flex-start; flex-direction:column; padding-inline:16px; }
.top-stats { justify-content:flex-start; }.workspace { display:block; }
.stage,#topology-canvas { min-height:560px; }
.toolbar { left:10px; right:10px; flex-direction:column; }
.toolbar-group { width:100%; }.tool-button { flex:1; }
.legend { display:none; }.canvas-note { left:10px; bottom:10px; max-width:calc(100% - 20px); }
.mini-grid { grid-template-columns:1fr 1fr; }
footer { padding-inline:16px; flex-direction:column; }
}
</style>
</head>
<body>
<div class="shell">
<header class="topbar">
<div>
<span class="kicker">MACHINE CT · GUEST-VISIBLE TOPOLOGY</span>
<h1 id="machine-title">GPU Machine 3D CT</h1>
<p class="subtitle" id="machine-subtitle"></p>
</div>
<div class="top-stats" id="top-stats"></div>
</header>
<main class="workspace">
<section class="stage" aria-label="交互式 3D 硬件拓扑">
<div class="toolbar">
<div class="toolbar-group" id="view-controls"><span class="toolbar-label">视图</span></div>
<div class="toolbar-group" id="camera-controls">
<span class="toolbar-label">镜头</span>
<button class="tool-button" type="button" data-camera="outside" aria-pressed="true">外观</button>
<button class="tool-button" type="button" data-camera="inside" aria-pressed="false">进入内部</button>
<button class="tool-button" type="button" id="camera-back" disabled>上一层</button>
<button class="tool-button" type="button" id="focus-selected" disabled>聚焦所选</button>
</div>
<div class="toolbar-group" id="layer-controls">
<span class="toolbar-label">图层</span>
<button class="tool-button" type="button" data-layer="cpu" aria-pressed="true">CPU</button>
<button class="tool-button" type="button" data-layer="pcie" aria-pressed="true">PCIe</button>
<button class="tool-button" type="button" data-layer="gpu" aria-pressed="true">GPU</button>
<button class="tool-button" type="button" data-layer="fabric" aria-pressed="true">NVSwitch</button>
<button class="tool-button" type="button" data-layer="network" aria-pressed="true">网络</button>
<button class="tool-button reset" type="button" id="reset-view">复位</button>
</div>
</div>
<canvas id="topology-canvas" tabindex="0" aria-label="可旋转的 GPU、CPU、PCIe、NVSwitch 和网卡三维拓扑"></canvas>
<div class="camera-status" id="camera-status">整机外观 · 双击壳体逐层进入</div>
<div class="canvas-note">双击壳体逐层进入 · 上一层返回 · 拖动旋转 · Shift+拖动平移 · Esc 返回整机</div>
<div class="legend" aria-label="连线图例">
<span><i style="--line-color:#a5e34b"></i>NVLink(仅连 GPU)</span>
<span><i style="--line-color:#51d9d2"></i>GPUDirect 候选</span>
<span><i style="--line-color:#ffbd4a"></i>PCIe</span>
<span><i style="--line-color:#b592ff"></i>CPU 亲和</span>
<span><i style="--line-color:#ff6b6b"></i>跨 NUMA / SYS</span>
</div>
</section>
<aside class="inspector">
<span class="kicker">TOPOLOGY INSPECTOR</span>
<h2>逻辑拓扑切片</h2>
<p>NUMA 是地域,PCIe 是道路,GPUDirect 是设备间数据路径。</p>
<div class="scope">当前视图来自 Guest 可见的 PCIe/NUMA 拓扑。直通或虚拟 PCIe 可能隐藏宿主机真实插槽几何位置。</div>
<div class="mini-grid" id="machine-metrics"></div>
<section class="detail" id="selection-detail" aria-live="polite"></section>
<section class="path-guide">
<h3>距离语义</h3>
<ul>
<li>PIX:最多经过一个 PCIe Switch,通常是最优 GPU–NIC 路径。</li>
<li>PXB/PHB:经过更多 Switch 或 Host Bridge。</li>
<li>NODE:同 NUMA 内跨 Host Bridge。</li>
<li>SYS:跨 NUMA/CPU 互联,路径最长。</li>
</ul>
</section>
</aside>
</main>
<footer><span>GPU Machine CT v@@VERSION@@ · 原生 Canvas · 无外部依赖</span><span id="generated-at"></span></footer>
</div>
<script type="application/json" id="machine-data">@@MACHINE_DATA@@</script>
<script>
(() => {
'use strict';
const data = JSON.parse(document.getElementById('machine-data').textContent);
const canvas = document.getElementById('topology-canvas');
const ctx = canvas.getContext('2d');
const system = data.system || {};
const gpus = data.gpus || [];
const networkEndpoints = data.network_endpoints || [];
const allNics = data.nics || [];
const endpointById = new Map(networkEndpoints.map(endpoint => [endpoint.id,endpoint]));
const networkByGpu = new Map(gpus.map(gpu => [Number(gpu.index),endpointById.get(gpu.primary_network_id)]).filter(([,endpoint]) => endpoint));
const numaIds = [...new Set(gpus.map(gpu => gpu.numa).filter(value => value !== null && value !== undefined))];
if (!numaIds.length) numaIds.push(0);
document.getElementById('machine-title').textContent = `${system.hostname || 'GPU Node'} · 3D Hardware CT`;
document.getElementById('machine-subtitle').textContent = `${system.manufacturer || ''} ${system.product || ''} · ${system.os || ''}`.trim();
document.getElementById('generated-at').textContent = `采集时间 ${data.generated_at || '—'} · ${data.pci_fabric?.scope || 'guest-visible'}`;
const totalPower = gpus.reduce((sum,gpu) => sum + (Number(gpu.power_w) || 0),0);
const totalLimit = gpus.reduce((sum,gpu) => sum + (Number(gpu.power_limit_w) || 0),0);
const metrics = [
['GPU', gpus.length],
['CPU', system.cpu_physical_cores || system.cpu_logical || '—'],
['NUMA', numaIds.length],
['GPU Power', totalPower ? `${(totalPower/1000).toFixed(2)} kW` : '—']
];
document.getElementById('top-stats').innerHTML = metrics.map(([label,value]) => `<span class="stat"><b>${escapeHtml(value)}</b><small>${escapeHtml(label)}</small></span>`).join('');
document.getElementById('machine-metrics').innerHTML = [
['CPU Sockets', system.cpu_sockets || '—'],
['PCI Bridges', data.pci_fabric?.bridges?.length || '—'],
['NVLinks', data.nvlink?.active_links || '—'],
['Power Limit', totalLimit ? `${(totalLimit/1000).toFixed(2)} kW` : '—']
].map(([label,value]) => `<div><b>${escapeHtml(value)}</b><small>${escapeHtml(label)}</small></div>`).join('');
const state = {
yaw:-0.64, pitch:0.52, zoom:1, focus:'all', selected:null,
cameraMode:'outside', target:{x:0,y:3.4,z:0}, scopeStack:[],
layers:new Set(['cpu','pcie','gpu','fabric','network'])
};
let scene = {boxes:[],edges:[]};
let hitTargets = [];
let pointer = null;
let cameraAnimation = 0;
function escapeHtml(value) {
return String(value ?? '').replace(/[&<>"']/g, char => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[char]));
}
function deepestCommonPath(paths) {
const valid = paths.filter(path => Array.isArray(path) && path.length);
if (valid.length < 2) return '';
const shortest = Math.min(...valid.map(path => path.length));
let common = '';
for (let index=0; index<shortest; index+=1) {
const value = valid[0][index];
if (valid.every(path => path[index] === value)) common = value;
else break;
}
return common;
}
function addBox(box) {
scene.boxes.push({
selectable:true, opacity:1, ...box,
size:box.size || {x:2,y:1,z:1}
});
return box.id;
}
function addEdge(from,to,type,label,layer,meta={}) {
scene.edges.push({from,to,type,label,layer,meta});
}
function buildScene() {
scene = {boxes:[],edges:[]};
const visibleNumas = state.focus === 'all' ? numaIds : numaIds.filter(id => String(id) === String(state.focus));
const domainGap = 18;
const totalWidth = Math.max(18,visibleNumas.length * domainGap);
addBox({
id:'chassis', kind:'chassis', layer:'pcie', selectable:true,
label:`${system.virtualization || 'HOST'} · GUEST VISIBLE`,
position:{x:0,y:3.4,z:0}, size:{x:totalWidth,y:10,z:17},
color:'#56756a', opacity:.08, wire:true
});
const gpuNodeIds = [];
visibleNumas.forEach((numa,domainIndex) => {
const centerX = (domainIndex - (visibleNumas.length-1)/2) * domainGap;
const domainGpus = gpus.filter(gpu => String(gpu.numa) === String(numa));
const cpuAffinity = domainGpus.find(gpu => gpu.cpu_affinity)?.cpu_affinity || '未解析';
const domainColor = numaIds.indexOf(numa) % 2 ? '#4fc6e8' : '#76b900';
addBox({
id:`numa-${numa}`, kind:'numa', layer:'cpu', selectable:true,
parentId:'chassis',
label:`NUMA ${numa}`, sublabel:`CPU ${cpuAffinity}`,
position:{x:centerX,y:3.2,z:0}, size:{x:16,y:8,z:15},
color:domainColor, opacity:.11, wire:true,
meta:{'CPU Affinity':cpuAffinity,'Memory NUMA':numa,'GPU Count':domainGpus.length}
});
addBox({
id:`cpu-${numa}`, kind:'cpu', layer:'cpu', label:`CPU / NUMA ${numa}`,
parentId:`numa-${numa}`,
sublabel:`Cores ${cpuAffinity}`, position:{x:centerX,y:1.4,z:4.7},
size:{x:5.8,y:1.3,z:2.3}, color:'#b592ff',
meta:{NUMA:numa,'CPU Affinity':cpuAffinity,Model:system.cpu_model || '—'}
});
addBox({
id:`root-${numa}`, kind:'pcie-root', layer:'pcie', label:'PCIe Root Complex',
parentId:`numa-${numa}`,
sublabel:`NUMA ${numa}`, position:{x:centerX,y:2.6,z:2.2},
size:{x:6.8,y:.8,z:1.6}, color:'#ffbd4a',
meta:{NUMA:numa,Scope:'Guest-visible host bridge'}
});
addEdge(`cpu-${numa}`,`root-${numa}`,'control','CPU / Host Bridge','pcie');
const columns = Math.max(1,Math.min(4,domainGpus.length));
domainGpus.forEach((gpu,index) => {
const row = Math.floor(index/columns);
const column = index%columns;
const x = centerX + (column-(columns-1)/2)*3.45;
const z = -.7-row*4.4;
const gpuId = `gpu-${gpu.index}`;
const endpoint = networkByGpu.get(Number(gpu.index));
const nicId = endpoint ? `network-${endpoint.id}` : '';
const primaryOwners = (endpoint?.gpu_links || [])
.filter(link => link.role === 'primary')
.map(link => Number(link.gpu_index))
.sort((left,right) => left-right);
const endpointOwner = primaryOwners.length > 0
&& primaryOwners[0] === Number(gpu.index)
&& (endpoint?.type === 'bond' || primaryOwners.length === 1);
const memberPaths = (endpoint?.members || []).map(member => member.pci_path);
const sharedBridge = deepestCommonPath([gpu.pci_path,...memberPaths]);
const relation = gpu.network_relation || '未解析';
addBox({
id:`pair-${gpu.index}`, kind:'pcie-group', layer:'pcie', selectable:true,
parentId:`numa-${numa}`,
label:sharedBridge ? `PCIe Switch ${sharedBridge}` : `PCIe Group · ${relation}`,
sublabel:`GPU${gpu.index} ↔ ${endpoint?.name || 'RDMA ?'}`,
position:{x,y:1.55,z:z-1.05}, size:{x:3.15,y:3.4,z:4.35},
color:relation === 'PIX' ? '#51d9d2' : '#ffbd4a', opacity:.08, wire:true,
meta:{Relation:relation,'Shared Bridge':sharedBridge || '未暴露','GPU BDF':gpu.bdf || '—'}
});
addBox({
id:gpuId, kind:'gpu', layer:'gpu', label:`GPU ${gpu.index}`,
parentId:`pair-${gpu.index}`,
sublabel:gpu.name || 'NVIDIA GPU', position:{x,y:2.35,z},
size:{x:2.55,y:1.45,z:1.7}, color:'#76b900',
meta:{
NUMA:gpu.numa,'CPU Affinity':gpu.cpu_affinity || '—',BDF:gpu.bdf || '—',
'PCI Path':(gpu.pci_path || []).join(' → ') || '未采集',
Power:gpu.power_w ? `${Number(gpu.power_w).toFixed(0)} W / ${Number(gpu.power_limit_w || 0).toFixed(0)} W` : '—',
PState:gpu.pstate || '—',Utilization:gpu.utilization_pct !== null ? `${gpu.utilization_pct}%` : '—'
}
});
gpuNodeIds.push(gpuId);
addEdge(`root-${numa}`,gpuId,'pcie',relation,'pcie',{relation});
addEdge(`cpu-${numa}`,gpuId,'affinity',`CPU ${cpuAffinity}`,'cpu');
if (endpoint?.type === 'bond' && endpointOwner) {
const slaves=endpoint.members || [];
const capacity=Number(endpoint.capacity_mbps)||0;
addBox({
id:nicId, kind:'bond', layer:'network', label:endpoint.name,
parentId:`pair-${gpu.index}`,
sublabel:`${slaves.length} ports · ${endpoint.rdma || 'RoCE / RDMA'}`,
position:{x,y:.62,z:z-2.15}, size:{x:2.85,y:1.45,z:1.65},
color:'#51d9d2',opacity:.1,wire:true,labelOffset:.85,
meta:{
NUMA:slaves[0]?.numa ?? numa,Relation:relation,
RDMA:endpoint.rdma || '—',Capacity:capacity ? `${capacity/1000} Gb/s` : '—',
Members:slaves.map(slave => `${slave.name} ${slave.bdf || ''}`).join(', ') || '—',
'Shared Bridge':sharedBridge || '未暴露'
}
});
slaves.forEach((slave,slaveIndex) => {
const portId=`nic-${endpoint.name}-${slave.name}`;
const offset=(slaveIndex-(slaves.length-1)/2)*1.18;
const speed=Number(slave.speed_mbps) ? `${Number(slave.speed_mbps)/1000}G` : '—';
addBox({
id:portId,kind:'roce-port',layer:'network',label:slave.name,
parentId:nicId,
sublabel:speed,position:{x:x+offset,y:.68,z:z-2.15},
size:{x:1.02,y:.78,z:1.08},color:'#79f4ee',
meta:{
Bond:endpoint.name,RDMA:endpoint.rdma || '—',NUMA:slave.numa ?? numa,
BDF:slave.bdf || '—',Speed:speed,State:slave.status || '—',
Driver:slave.driver || '—','PCI Path':(slave.pci_path || []).join(' → ') || '未采集'
}
});
addEdge(portId,nicId,'bond-member','Bond member','network');
addEdge(`root-${numa}`,portId,'pcie',relation,'pcie',{relation});
});
}
if (endpoint?.type === 'bond') {
(endpoint.members || []).forEach(slave => {
addEdge(
gpuId,
`nic-${endpoint.name}-${slave.name}`,
'gdr',
`${relation} · GDR candidate`,
'network',
{relation,port:slave.name}
);
});
} else if (endpoint && endpointOwner) {
const member=(endpoint.members || [])[0] || endpoint;
const speed=Number(endpoint.capacity_mbps) ? `${Number(endpoint.capacity_mbps)/1000}G` : '—';
const kind=endpoint.protocol === 'InfiniBand' ? 'infiniband' : 'roce-port';
addBox({
id:nicId,kind,layer:'network',label:endpoint.name,
parentId:`pair-${gpu.index}`,
sublabel:`${endpoint.protocol || 'RDMA'} · ${speed}`,
position:{x,y:.68,z:z-2.15},size:{x:2.35,y:1.05,z:1.35},
color:endpoint.protocol === 'InfiniBand' ? '#ff9f43' : '#79f4ee',
meta:{
Mode:'Standalone RDMA',RDMA:endpoint.rdma || '—',
NUMA:endpoint.numa ?? member.numa ?? numa,Relation:relation,
Evidence:endpoint.topology_visibility === 'nvidia-smi-topo'
? 'nvidia-smi topo -m' : 'Topology relation not exposed',
BDF:endpoint.bdf || member.bdf || '—',Speed:speed,
State:endpoint.state || member.status || '—',
'PCI Path':(endpoint.pci_path || member.pci_path || []).join(' → ') || '未采集'
}
});
addEdge(`root-${numa}`,nicId,'pcie',relation,'pcie',{relation});
}
if (endpoint?.type !== 'bond' && endpoint) {
addEdge(gpuId,nicId,'gdr',`${relation} · GDR candidate`,'network',{relation,port:endpoint.name});
}
});
const unpairedEndpoints = networkEndpoints.filter(endpoint =>
endpoint.type === 'standalone'
&& String(endpoint.numa) === String(numa)
&& (endpoint.gpu_links || []).filter(link => link.role === 'primary').length !== 1
);
unpairedEndpoints.forEach((endpoint,index) => {
const x = centerX - 5.4 + index*2.8;
const id = `network-${endpoint.id}`;
const isIb = endpoint.protocol === 'InfiniBand';
const speed = Number(endpoint.capacity_mbps)
? `${Number(endpoint.capacity_mbps)/1000} Gb/s` : '—';
addBox({
id,kind:isIb ? 'infiniband' : 'roce-port',
layer:'network',label:endpoint.name,
parentId:`numa-${numa}`,
sublabel:`${endpoint.protocol || 'RDMA'} ${speed} · GPU relation ${endpoint.topology_visibility === 'nvidia-smi-topo' ? 'exposed' : 'not exposed'}`,
position:{x,y:.75,z:6.35},
size:{x:2.35,y:1.05,z:1.4},color:isIb ? '#ff9f43' : '#79f4ee',
meta:{
NUMA:endpoint.numa,BDF:endpoint.bdf || '—',
RDMA:endpoint.rdma || '—',State:endpoint.state || '—',
'GPU Topology':endpoint.topology_visibility === 'nvidia-smi-topo'
? 'nvidia-smi topo -m' : 'Not exposed; no affinity inferred',
'PCI Path':(endpoint.pci_path || []).join(' → ') || '未采集'
}
});
addEdge(`root-${numa}`,id,'pcie',isIb ? 'Storage / IB' : 'Standalone RoCE','pcie');
(endpoint?.gpu_links || []).filter(link => link.role==='nearest').forEach(link => {
const gpuId=`gpu-${link.gpu_index}`;
if (scene.boxes.some(box => box.id===gpuId)) {
addEdge(gpuId,id,'gdr',`${link.relation} · nearest ${endpoint.protocol}`,'network',{relation:link.relation,port:endpoint.name});
}
});
});
const acceleratorMembers = new Set(networkEndpoints.flatMap(endpoint => (endpoint.members || []).map(member => member.name)));
const acceleratorGroups = new Set(networkEndpoints.filter(endpoint => endpoint.type==='bond').map(endpoint => endpoint.name));
const management = allNics.filter(nic =>
String(nic.numa) === String(numa)
&& !acceleratorMembers.has(nic.name)
&& !acceleratorGroups.has(nic.name)
&& (nic.bdf || nic.default_route || nic.is_bond)
);
management.slice(0,2).forEach((nic,index) => {
const id = `mgmt-${nic.name}`;
addBox({
id,kind:'management',layer:'network',label:nic.name,
parentId:`numa-${numa}`,
sublabel:nic.default_route ? 'In-band management' : 'OS network',
position:{x:centerX+5.4-index*2.7,y:.65,z:6.35},size:{x:2.25,y:1,z:1.35},color:'#7d9b90',
meta:{NUMA:nic.numa,BDF:nic.bdf || '—',Driver:nic.driver || '—',Address:(nic.addresses || []).join(', ') || '—'}
});
addEdge(`root-${numa}`,id,'pcie','Management','pcie');
});
});
if (gpuNodeIds.length > 1 && data.nvlink?.active_links) {
addBox({
id:'nvswitch',kind:'nvswitch',layer:'fabric',label:data.nvlink.fabric_type || 'NVLink Fabric',
parentId:'chassis',
sublabel:`${gpuNodeIds.length}/${gpus.length} GPU · GPU endpoints only`,
position:{x:0,y:8.15,z:-1.2},size:{x:Math.max(9,totalWidth-3),y:.8,z:2.1},color:'#a5e34b',
meta:{
Connection:'GPU endpoints only · not NIC',
'Active Links':data.nvlink.active_links,
'Links / GPU':data.nvlink.links_per_gpu || '—',
'Speed / Link':`${data.nvlink.speed_gbs || '—'} GB/s`
}
});
gpuNodeIds.forEach(id => {
const gpuIndex=Number(id.replace('gpu-',''));
addEdge(id,'nvswitch','nvlink',`GPU ${gpuIndex} ↔ NVSwitch`,'fabric',{gpuIndex});
});
}
renderDetail();
}
const viewControls = document.getElementById('view-controls');
function buildViewControls() {
const views = [['all','总览'],...numaIds.map(id => [String(id),`NUMA ${id}`])];
viewControls.querySelectorAll('button').forEach(button => button.remove());
views.forEach(([value,label]) => {
const button = document.createElement('button');
button.type='button'; button.className='tool-button'; button.textContent=label;
button.dataset.view=value; button.setAttribute('aria-pressed',String(state.focus === value));
button.addEventListener('click',() => {
state.focus=value; state.selected=null;
state.scopeStack=state.cameraMode==='inside' ? ['chassis'] : [];
buildViewControls(); buildScene();
setCameraMode(state.cameraMode,null,false);
});
viewControls.appendChild(button);
});
}
function cameraPreset(mode,box=null) {
if (mode === 'inside') {
const zoomByKind={chassis:1.48,numa:1.72,'pcie-group':2.5,bond:3.25};
return {
yaw:-.2,pitch:.27,zoom:box ? (zoomByKind[box.kind] || 2.15) : 1.48,
target:box ? {...box.position} : {x:0,y:2.25,z:-1.2}
};
}
return {yaw:-.64,pitch:.52,zoom:1,target:{x:0,y:3.4,z:0}};
}
function animateCamera(next,duration=380) {
const animation=++cameraAnimation;
const start={
yaw:state.yaw,pitch:state.pitch,zoom:state.zoom,target:{...state.target}
};
const started=performance.now();
function frame(now) {
if (animation!==cameraAnimation) return;
const raw=Math.min(1,(now-started)/duration);
const progress=1-Math.pow(1-raw,3);
state.yaw=start.yaw+(next.yaw-start.yaw)*progress;
state.pitch=start.pitch+(next.pitch-start.pitch)*progress;
state.zoom=start.zoom+(next.zoom-start.zoom)*progress;
state.target={
x:start.target.x+(next.target.x-start.target.x)*progress,
y:start.target.y+(next.target.y-start.target.y)*progress,
z:start.target.z+(next.target.z-start.target.z)*progress
};
draw();
if (raw<1) requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
}
function updateCameraControls() {
document.querySelectorAll('[data-camera]').forEach(button => {
button.setAttribute('aria-pressed',String(button.dataset.camera===state.cameraMode));
});
const selectedBox=scene.boxes.find(box => box.id===state.selected && box.selectable);
const scopeBoxes=state.scopeStack.map(id => scene.boxes.find(box => box.id===id)).filter(Boolean);
const breadcrumb=['整机',...scopeBoxes.filter(box => box.kind!=='chassis').map(box => box.label)].join(' › ');
document.getElementById('focus-selected').disabled=!selectedBox;
document.getElementById('camera-back').disabled=state.cameraMode!=='inside';
document.getElementById('camera-status').textContent=state.cameraMode==='inside'
? `${breadcrumb} · 当前层之外已隐藏`
: '整机外观 · 双击壳体逐层进入';
}
function setCameraMode(mode,box=null,animate=true) {
state.cameraMode=mode;
if (mode==='outside') state.scopeStack=[];
const next=cameraPreset(mode,box);
updateCameraControls();
if (animate) animateCamera(next);
else {
cameraAnimation+=1;
state.yaw=next.yaw;state.pitch=next.pitch;state.zoom=next.zoom;
state.target={...next.target};draw();
}
}
const shellKinds=new Set(['chassis','numa','pcie-group','bond']);
function isShell(box) {
return Boolean(box && shellKinds.has(box.kind));
}
function activeScopeId() {
return state.scopeStack[state.scopeStack.length-1] || null;
}
function isDescendant(boxId,scopeId) {
let current=scene.boxes.find(box => box.id===boxId);
const visited=new Set();
while (current?.parentId && !visited.has(current.id)) {
if (current.parentId===scopeId) return true;
visited.add(current.id);
current=scene.boxes.find(box => box.id===current.parentId);
}
return false;
}
function scopePath(box) {
const path=[];
let current=box;
const visited=new Set();
while (current && !visited.has(current.id)) {
visited.add(current.id);
if (isShell(current)) path.unshift(current.id);
current=current.parentId ? scene.boxes.find(item => item.id===current.parentId) : null;
}
return path;
}
function nextScopeToward(box) {
const path=scopePath(box);
if (!path.length) return null;
const current=activeScopeId();
if (!current) return scene.boxes.find(item => item.id===path[0]) || null;
const currentIndex=path.indexOf(current);
const nextId=currentIndex>=0 ? path[currentIndex+1] : null;
return nextId ? scene.boxes.find(item => item.id===nextId) || null : null;
}
function enterScope(box,animate=true) {
const scope=isShell(box) ? box : scene.boxes.find(item => item.id==='chassis');
if (!scope) return;
const current=activeScopeId();
const targetPath=scopePath(scope);
if (state.cameraMode!=='inside' || !current) {
state.scopeStack=targetPath;
} else if (scope.id!==current && isDescendant(scope.id,current)) {
const currentIndex=targetPath.indexOf(current);
state.scopeStack=[
...state.scopeStack,
...targetPath.slice(Math.max(0,currentIndex+1)).filter(id => !state.scopeStack.includes(id))
];
}
state.cameraMode='inside';
state.selected=scope.kind==='chassis' ? null : scope.id;
renderDetail();setCameraMode('inside',scope,animate);
}
function leaveScope() {
if (state.cameraMode!=='inside') return;
state.scopeStack.pop();
const parentId=activeScopeId();
if (!parentId) {
state.selected=null;renderDetail();setCameraMode('outside');
return;
}
const parent=scene.boxes.find(box => box.id===parentId);
state.selected=parent?.kind==='chassis' ? null : parent?.id || null;
renderDetail();setCameraMode('inside',parent);
}
document.querySelectorAll('[data-camera]').forEach(button => button.addEventListener('click',() => {
const selectedBox=scene.boxes.find(box => box.id===state.selected && box.selectable);
const nextScope=selectedBox ? nextScopeToward(selectedBox) : null;
if (button.dataset.camera==='outside') {
state.selected=null;renderDetail();setCameraMode('outside');
} else if (nextScope) {
enterScope(nextScope);
} else if (isShell(selectedBox)) {
enterScope(selectedBox);
} else if (state.cameraMode!=='inside') {
enterScope(scene.boxes.find(box => box.id==='chassis'));
} else if (selectedBox) {
setCameraMode('inside',selectedBox);
}
}));
document.getElementById('camera-back').addEventListener('click',leaveScope);
document.getElementById('focus-selected').addEventListener('click',() => {
const selectedBox=scene.boxes.find(box => box.id===state.selected && box.selectable);
if (selectedBox) setCameraMode('inside',selectedBox);
});
document.querySelectorAll('[data-layer]').forEach(button => button.addEventListener('click',() => {
const layer = button.dataset.layer;
if (state.layers.has(layer)) state.layers.delete(layer); else state.layers.add(layer);
button.setAttribute('aria-pressed',String(state.layers.has(layer)));
if (state.selected && !visibleBox(scene.boxes.find(box => box.id === state.selected))) state.selected=null;
renderDetail(); draw();
}));
document.getElementById('reset-view').addEventListener('click',() => {
state.selected=null;setCameraMode('outside');renderDetail();
});
function visibleBox(box) {
if (!box) return false;
if (state.cameraMode==='outside') {
return box.kind==='chassis' || box.kind==='numa' || state.layers.has(box.layer);
}
const scopeId=activeScopeId() || 'chassis';
if (box.id===scopeId) return box.kind!=='chassis';
if (!isDescendant(box.id,scopeId)) return false;
return box.kind==='numa' || state.layers.has(box.layer);
}
function visibleEdge(edge) {
const from=scene.boxes.find(box => box.id===edge.from);
const to=scene.boxes.find(box => box.id===edge.to);
return state.layers.has(edge.layer) && visibleBox(from) && visibleBox(to);
}
function cameraPoint(point) {
const cy=Math.cos(state.yaw), sy=Math.sin(state.yaw);
const cp=Math.cos(state.pitch), sp=Math.sin(state.pitch);
const translated={
x:point.x-state.target.x,y:point.y-state.target.y,z:point.z-state.target.z
};
const x=translated.x*cy-translated.z*sy;
const z=translated.x*sy+translated.z*cy;
return {x,y:translated.y*cp-z*sp,z:translated.y*sp+z*cp};
}
function project(point,scale) {
const p=cameraPoint(point);
return {x:canvas.clientWidth/2+p.x*scale,y:canvas.clientHeight*.58-p.y*scale,z:p.z};
}
function corners(box) {
const {position:p,size:s}=box;
const points=[];
[-1,1].forEach(dx => [-1,1].forEach(dy => [-1,1].forEach(dz => points.push({x:p.x+dx*s.x/2,y:p.y+dy*s.y/2,z:p.z+dz*s.z/2}))));
return points;
}
const faceIndexes=[[0,1,3,2],[4,6,7,5],[0,4,5,1],[2,3,7,6],[0,2,6,4],[1,5,7,3]];
function hexToRgba(hex,alpha) {
const value=hex.replace('#','');
const number=parseInt(value.length===3?value.split('').map(x=>x+x).join(''):value,16);
return `rgba(${(number>>16)&255},${(number>>8)&255},${number&255},${alpha})`;
}
function boxCenter(id) {
const box=scene.boxes.find(item => item.id===id);
return box?.position || null;
}
function relationColor(edge) {
if (edge.meta?.relation === 'SYS') return '#ff6b6b';
return {
nvlink:'#b8ff56',gdr:'#51d9d2',pcie:'#ffbd4a',affinity:'#b592ff',
control:'#8fa99e','bond-member':'#79f4ee'
}[edge.type] || '#8fa99e';
}
function roundedRectPath(x,y,width,height,radius) {
const r=Math.min(radius,width/2,height/2);
ctx.beginPath();
ctx.moveTo(x+r,y);ctx.lineTo(x+width-r,y);ctx.quadraticCurveTo(x+width,y,x+width,y+r);
ctx.lineTo(x+width,y+height-r);ctx.quadraticCurveTo(x+width,y+height,x+width-r,y+height);
ctx.lineTo(x+r,y+height);ctx.quadraticCurveTo(x,y+height,x,y+height-r);
ctx.lineTo(x,y+r);ctx.quadraticCurveTo(x,y,x+r,y);ctx.closePath();
}
function drawLabelChip(box,point,dimmed) {
const isDomain=box.kind==='numa';
const mainFont=isDomain?'900 12px ui-sans-serif,sans-serif':'900 10px ui-sans-serif,sans-serif';
const subFont='800 8px ui-monospace,monospace';
ctx.save();ctx.globalAlpha=dimmed ? .34 : 1;ctx.textAlign='center';
ctx.font=mainFont;
const mainWidth=ctx.measureText(box.label).width;
ctx.font=subFont;
const subWidth=box.sublabel ? ctx.measureText(box.sublabel).width : 0;
const width=Math.max(mainWidth,subWidth)+14;
const height=box.sublabel ? 29 : 18;
const left=point.x-width/2,top=point.y-13;
roundedRectPath(left,top,width,height,5);
ctx.fillStyle=isDomain ? hexToRgba(box.color,.96) : 'rgba(238,249,244,.96)';
ctx.fill();
ctx.strokeStyle=hexToRgba(box.color,.9);ctx.lineWidth=1.2;ctx.stroke();
ctx.fillStyle='#06100c';ctx.font=mainFont;
ctx.fillText(box.label,point.x,top+12);
if (box.sublabel) {
ctx.fillStyle='#17251f';ctx.font=subFont;
ctx.fillText(box.sublabel,point.x,top+23);
}
ctx.restore();
}
function drawEdgeTag(text,x,y,color) {
ctx.save();ctx.font='900 8px ui-monospace,monospace';ctx.textAlign='center';
const width=ctx.measureText(text).width+9;
roundedRectPath(x-width/2,y-8,width,14,4);
ctx.fillStyle=color;ctx.fill();
ctx.fillStyle='#06100c';ctx.fillText(text,x,y+2);
ctx.restore();
}
function drawEdge(edge,scale,selected,overlay=false) {
const from=boxCenter(edge.from),to=boxCenter(edge.to);
if (!from || !to) return;
const a=project(from,scale),b=project(to,scale);
const related=!selected || edge.from===selected || edge.to===selected;
const color=relationColor(edge);
ctx.save();
ctx.strokeStyle=hexToRgba(color,related ? (overlay ? 1 : .85) : (overlay ? .24 : .13));
ctx.lineWidth=related
? (overlay ? 4.8 : edge.type==='gdr' ? 3 : edge.type==='bond-member' ? 2 : 1.3)
: (overlay ? 2 : 1);
if (overlay) {
ctx.shadowColor=color;ctx.shadowBlur=related ? 12 : 3;
ctx.lineCap='round';
}
if (edge.type==='affinity') ctx.setLineDash([5,5]);
if (edge.meta?.relation==='SYS') ctx.setLineDash([8,4]);
ctx.beginPath();ctx.moveTo(a.x,a.y);ctx.lineTo(b.x,b.y);ctx.stroke();
ctx.shadowBlur=0;
if (overlay) {
[a,b].forEach(point => {
ctx.beginPath();ctx.arc(point.x,point.y,related ? 4.5 : 3,0,Math.PI*2);
ctx.fillStyle=hexToRgba(color,related ? 1 : .3);ctx.fill();
ctx.strokeStyle='#06100c';ctx.lineWidth=1.5;ctx.stroke();
});
if (related) {
const markerX=a.x+(b.x-a.x)*.18,markerY=a.y+(b.y-a.y)*.18;
drawEdgeTag(`GPU ${edge.meta?.gpuIndex ?? '?'}`,markerX,markerY,color);
}
} else if (related && selected && Math.hypot(a.x-b.x,a.y-b.y)>70) {
drawEdgeTag(edge.label,(a.x+b.x)/2,(a.y+b.y)/2-4,color);
}
ctx.restore();
}
function drawGrid(scale) {
ctx.save(); ctx.strokeStyle='rgba(122,164,147,.08)'; ctx.lineWidth=1;
for (let value=-24; value<=24; value+=2) {
const a=project({x:value,y:0,z:-12},scale),b=project({x:value,y:0,z:12},scale);
const c=project({x:-24,y:0,z:value},scale),d=project({x:24,y:0,z:value},scale);
ctx.beginPath();ctx.moveTo(a.x,a.y);ctx.lineTo(b.x,b.y);ctx.stroke();
ctx.beginPath();ctx.moveTo(c.x,c.y);ctx.lineTo(d.x,d.y);ctx.stroke();
}
ctx.restore();
}
function draw() {
const width=canvas.clientWidth,height=canvas.clientHeight;
const dpr=Math.min(2,window.devicePixelRatio || 1);
if (canvas.width!==Math.round(width*dpr) || canvas.height!==Math.round(height*dpr)) {
canvas.width=Math.round(width*dpr);canvas.height=Math.round(height*dpr);
}
ctx.setTransform(dpr,0,0,dpr,0,0);
ctx.clearRect(0,0,width,height);
const domainCount=state.focus==='all'?Math.max(1,numaIds.length):1;
const scale=Math.max(18,Math.min(width/(domainCount*22),height/23))*state.zoom;
drawGrid(scale);
const selected=state.selected;
scene.edges.filter(edge => visibleEdge(edge) && edge.layer!=='fabric').forEach(edge => {
drawEdge(edge,scale,selected);
});
const faces=[];
const labels=[];
hitTargets=[];
scene.boxes.filter(visibleBox).forEach(box => {
const projected=corners(box).map(point => project(point,scale));
const alpha = box.opacity ?? 1;
faceIndexes.forEach((indexes,faceIndex) => {
const points=indexes.map(index => projected[index]);
faces.push({box,points,depth:points.reduce((sum,p)=>sum+p.z,0)/points.length,faceIndex,alpha});
});
const xs=projected.map(p=>p.x),ys=projected.map(p=>p.y);
hitTargets.push({box,rect:{left:Math.min(...xs),right:Math.max(...xs),top:Math.min(...ys),bottom:Math.max(...ys)},depth:cameraPoint(box.position).z});
labels.push({
box,
point:project({
x:box.position.x,
y:box.position.y+box.size.y/2+(box.labelOffset ?? .18),
z:box.position.z
},scale)
});
});
faces.sort((a,b) => a.depth-b.depth).forEach(face => {
const isSelected=face.box.id===selected;
const dimmed=selected && !isSelected && face.box.kind!=='numa' && face.box.kind!=='chassis';
const internalBoundary=state.cameraMode==='inside' && (
face.box.kind==='numa' || face.box.kind==='pcie-group' || face.box.kind==='bond'
);
const baseAlpha=internalBoundary
? face.alpha*.22
: face.box.wire ? face.alpha : Math.min(.86,face.alpha*(face.faceIndex===4?1:.72));
ctx.save();
ctx.fillStyle=hexToRgba(face.box.color,dimmed?baseAlpha*.16:baseAlpha);
ctx.strokeStyle=hexToRgba(face.box.color,isSelected ? 1 : internalBoundary ? .26 : face.box.wire ? .55 : .75);
ctx.lineWidth=isSelected ? 2.8 : face.box.wire ? 1.1 : 1;
if (face.box.wire) ctx.setLineDash([7,5]);
ctx.beginPath();face.points.forEach((point,index) => index?ctx.lineTo(point.x,point.y):ctx.moveTo(point.x,point.y));ctx.closePath();
ctx.fill();ctx.stroke();ctx.restore();
});
scene.edges.filter(edge => visibleEdge(edge) && edge.layer==='fabric').forEach(edge => {
drawEdge(edge,scale,selected,true);
});
labels.sort((a,b)=>a.point.z-b.point.z).forEach(({box,point}) => {
if (box.kind==='chassis') return;
if (state.cameraMode==='inside' && box.kind==='numa') return;
const dimmed=selected && box.id!==selected && box.kind!=='numa';
drawLabelChip(box,point,dimmed);
});
}
function renderDetail() {
const root=document.getElementById('selection-detail');
const box=scene.boxes.find(item => item.id===state.selected);
if (!box) {
root.innerHTML=`<span class="detail-kind">MACHINE OVERVIEW</span><h3>${escapeHtml(system.hostname || 'GPU Node')}</h3><dl>
<div><dt>CPU Model</dt><dd>${escapeHtml(system.cpu_model || '—')}</dd></div>
<div><dt>CPU Topology</dt><dd>${escapeHtml(`${system.cpu_sockets || '—'} sockets · ${system.cpu_logical || '—'} logical`)}</dd></div>
<div><dt>GPU</dt><dd>${escapeHtml(`${gpus.length} × ${gpus[0]?.name || '—'}`)}</dd></div>
<div><dt>Virtualization</dt><dd>${escapeHtml(system.virtualization || 'bare metal / unknown')}</dd></div>
<div><dt>PCI Scope</dt><dd>${escapeHtml(data.pci_fabric?.scope || 'guest-visible')}</dd></div>
</dl>`;
return;
}
const meta=box.meta || {};
const fields=Object.entries(meta).map(([key,value]) => `<div><dt>${escapeHtml(key)}</dt><dd>${escapeHtml(value)}</dd></div>`).join('');
root.innerHTML=`<div class="detail-head"><div><span class="detail-kind">${escapeHtml(box.kind.toUpperCase())}</span><h3>${escapeHtml(box.label)}</h3></div><span class="detail-status">${escapeHtml(box.sublabel || '')}</span></div><dl>${fields || '<div><dt>Info</dt><dd>无附加字段</dd></div>'}</dl>`;
}
function pickTarget(x,y) {
return hitTargets
.filter(hit => hit.box.selectable && x>=hit.rect.left && x<=hit.rect.right && y>=hit.rect.top && y<=hit.rect.bottom)
.sort((a,b) => {
const areaA=(a.rect.right-a.rect.left)*(a.rect.bottom-a.rect.top);
const areaB=(b.rect.right-b.rect.left)*(b.rect.bottom-b.rect.top);
return areaA-areaB || b.depth-a.depth;
})[0];
}
canvas.addEventListener('pointerdown',event => {
cameraAnimation+=1;
canvas.setPointerCapture(event.pointerId);
pointer={
id:event.pointerId,x:event.clientX,y:event.clientY,startX:event.clientX,startY:event.clientY,
yaw:state.yaw,pitch:state.pitch,target:{...state.target},pan:event.shiftKey || event.button===1
};
});
canvas.addEventListener('pointermove',event => {
if (!pointer || pointer.id!==event.pointerId) return;
if (pointer.pan) {
const factor=.022/state.zoom;
state.target.x=pointer.target.x-(event.clientX-pointer.x)*factor;
state.target.y=pointer.target.y+(event.clientY-pointer.y)*factor;
} else {
state.yaw=pointer.yaw+(event.clientX-pointer.x)*.008;
state.pitch=Math.max(-.15,Math.min(1.1,pointer.pitch+(event.clientY-pointer.y)*.006));
}
draw();
});
canvas.addEventListener('pointerup',event => {
if (!pointer || pointer.id!==event.pointerId) return;
const moved=Math.hypot(event.clientX-pointer.startX,event.clientY-pointer.startY);
if (moved<6) {
const rect=canvas.getBoundingClientRect(),x=event.clientX-rect.left,y=event.clientY-rect.top;
const target=pickTarget(x,y);
state.selected=target?.box.id || null;renderDetail();updateCameraControls();draw();
}
pointer=null;
});
canvas.addEventListener('dblclick',event => {
const rect=canvas.getBoundingClientRect(),x=event.clientX-rect.left,y=event.clientY-rect.top;
const target=pickTarget(x,y);
if (!target) return;
state.selected=target.box.id;
const nextScope=nextScopeToward(target.box);
if (nextScope) enterScope(nextScope);
else if (isShell(target.box)) enterScope(target.box);
else { renderDetail();setCameraMode('inside',target.box); }
});
canvas.addEventListener('wheel',event => {
event.preventDefault();
cameraAnimation+=1;
const limits=state.cameraMode==='inside' ? [.72,3.8] : [.55,2.3];
state.zoom=Math.max(limits[0],Math.min(limits[1],state.zoom*(event.deltaY>0 ? .92 : 1.08)));draw();
},{passive:false});
canvas.addEventListener('keydown',event => {
const key=event.key;
if (['ArrowLeft','ArrowRight','ArrowUp','ArrowDown'].includes(key)) {
event.preventDefault();
cameraAnimation+=1;
if (key==='ArrowLeft') state.yaw-=.08;if (key==='ArrowRight') state.yaw+=.08;
if (key==='ArrowUp') state.pitch=Math.min(1.1,state.pitch+.06);if (key==='ArrowDown') state.pitch=Math.max(-.15,state.pitch-.06);
draw();
}
if (key==='Escape') {
event.preventDefault();state.selected=null;renderDetail();setCameraMode('outside');
}
});
new ResizeObserver(draw).observe(canvas);
buildViewControls();buildScene();updateCameraControls();draw();
})();
</script>
</body>
</html>

View File

@ -0,0 +1,133 @@
{
"schema_version": 1,
"generated_at": "2026-07-25T09:45:54+08:00",
"collector_version": "1.0.0",
"system": {
"hostname": "root-2",
"os": "Ubuntu 24.04.1 LTS",
"kernel": "6.8.0-71-generic",
"architecture": "x86_64",
"manufacturer": "Baidu Cloud",
"product": "Baidu Cloud BCC",
"product_version": "6.10.0",
"virtualization": "kvm",
"cpu_model": "Intel(R) Xeon(R) 6787P",
"numa_nodes": 2,
"cpu_logical": 332,
"cpu_sockets": 2,
"cpu_cores_per_socket": 83,
"cpu_physical_cores": 166,
"cpu_threads_per_core": 2,
"memory_total_kib": 4227858432,
"memory_available_kib": 3549581076,
"memory_used_kib": 678277356
},
"software": {
"nvidia_driver": "595.71.05",
"cuda_toolkit": "12.8",
"cuda_note": "Toolkit selected by nvcc; application/container runtime may differ."
},
"gpus": [
{"index": 0, "name": "NVIDIA B30Z", "uuid": "GPU-0", "bdf": "63:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 34720, "temperature_c": 48, "utilization_pct": 12, "pstate": "P0", "power_w": 312, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 0},
{"index": 1, "name": "NVIDIA B30Z", "uuid": "GPU-1", "bdf": "67:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 50812, "temperature_c": 51, "utilization_pct": 18, "pstate": "P0", "power_w": 326, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 0},
{"index": 2, "name": "NVIDIA B30Z", "uuid": "GPU-2", "bdf": "6b:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 19236, "temperature_c": 47, "utilization_pct": 8, "pstate": "P0", "power_w": 296, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 0},
{"index": 3, "name": "NVIDIA B30Z", "uuid": "GPU-3", "bdf": "6f:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 62840, "temperature_c": 54, "utilization_pct": 23, "pstate": "P0", "power_w": 341, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 0},
{"index": 4, "name": "NVIDIA B30Z", "uuid": "GPU-4", "bdf": "a3:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 41560, "temperature_c": 50, "utilization_pct": 15, "pstate": "P0", "power_w": 318, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 1},
{"index": 5, "name": "NVIDIA B30Z", "uuid": "GPU-5", "bdf": "a7:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 28410, "temperature_c": 49, "utilization_pct": 10, "pstate": "P0", "power_w": 307, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 1},
{"index": 6, "name": "NVIDIA B30Z", "uuid": "GPU-6", "bdf": "ab:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 73320, "temperature_c": 56, "utilization_pct": 29, "pstate": "P0", "power_w": 359, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 1},
{"index": 7, "name": "NVIDIA B30Z", "uuid": "GPU-7", "bdf": "af:00.0", "driver": "595.71.05", "memory_total_mib": 285184, "memory_used_mib": 22350, "temperature_c": 48, "utilization_pct": 9, "pstate": "P0", "power_w": 301, "power_limit_w": 1100, "power_default_limit_w": 1100, "power_min_limit_w": 200, "power_max_limit_w": 1100, "numa": 1}
],
"topology": {
"rows": {},
"nic_legend": {
"NIC1": "mlx5_bond_0", "NIC2": "mlx5_bond_1", "NIC3": "mlx5_bond_2", "NIC4": "mlx5_bond_3",
"NIC5": "mlx5_bond_4", "NIC6": "mlx5_bond_5", "NIC7": "mlx5_bond_6", "NIC8": "mlx5_bond_7"
},
"gpu_nic": {
"GPU0": [{"slot": "NIC1", "rdma": "mlx5_bond_0", "relation": "PIX"}],
"GPU1": [{"slot": "NIC2", "rdma": "mlx5_bond_1", "relation": "PIX"}],
"GPU2": [{"slot": "NIC3", "rdma": "mlx5_bond_2", "relation": "PIX"}],
"GPU3": [{"slot": "NIC4", "rdma": "mlx5_bond_3", "relation": "PIX"}],
"GPU4": [{"slot": "NIC5", "rdma": "mlx5_bond_4", "relation": "PIX"}],
"GPU5": [{"slot": "NIC6", "rdma": "mlx5_bond_5", "relation": "PIX"}],
"GPU6": [{"slot": "NIC7", "rdma": "mlx5_bond_6", "relation": "PIX"}],
"GPU7": [{"slot": "NIC8", "rdma": "mlx5_bond_7", "relation": "PIX"}]
},
"gpu_affinity": {
"GPU0": {"cpu_affinity": "0-165", "numa_affinity": "0", "gpu_numa_id": "N/A"},
"GPU1": {"cpu_affinity": "0-165", "numa_affinity": "0", "gpu_numa_id": "N/A"},
"GPU2": {"cpu_affinity": "0-165", "numa_affinity": "0", "gpu_numa_id": "N/A"},
"GPU3": {"cpu_affinity": "0-165", "numa_affinity": "0", "gpu_numa_id": "N/A"},
"GPU4": {"cpu_affinity": "166-331", "numa_affinity": "1", "gpu_numa_id": "N/A"},
"GPU5": {"cpu_affinity": "166-331", "numa_affinity": "1", "gpu_numa_id": "N/A"},
"GPU6": {"cpu_affinity": "166-331", "numa_affinity": "1", "gpu_numa_id": "N/A"},
"GPU7": {"cpu_affinity": "166-331", "numa_affinity": "1", "gpu_numa_id": "N/A"}
}
},
"pci_fabric": {
"scope": "guest-visible",
"virtualization": "passthrough-or-virtual-pci",
"bridges": [
{"bdf": "61:00.0", "parent_bdf": "60:00.0", "numa": 0, "description": "PCI bridge", "children": ["63:00.0", "64:00.0"]},
{"bdf": "a1:00.0", "parent_bdf": "a0:00.0", "numa": 1, "description": "PCI bridge", "children": ["a3:00.0", "a4:00.0"]}
],
"endpoints": [
{"kind": "gpu", "label": "GPU0", "bdf": "63:00.0", "numa": 0, "path": ["60:00.0", "61:00.0", "63:00.0"], "description": "NVIDIA GPU"},
{"kind": "nic", "label": "eth1", "bdf": "64:00.0", "numa": 0, "path": ["60:00.0", "61:00.0", "64:00.0"], "description": "Ethernet controller"},
{"kind": "gpu", "label": "GPU4", "bdf": "a3:00.0", "numa": 1, "path": ["a0:00.0", "a1:00.0", "a3:00.0"], "description": "NVIDIA GPU"},
{"kind": "nic", "label": "eth9", "bdf": "a4:00.0", "numa": 1, "path": ["a0:00.0", "a1:00.0", "a4:00.0"], "description": "Ethernet controller"}
]
},
"nvlink": {
"fabric_type": "NVSwitch Fabric",
"per_gpu": {
"GPU0": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU1": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU2": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU3": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU4": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU5": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU6": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125],
"GPU7": [53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125, 53.125]
},
"active_links": 144,
"links_per_gpu": 18,
"speed_gbs": 53.125
},
"nics": [
{
"name": "eth0",
"driver": "mlx5_core",
"bdf": "02:00.0",
"numa": 0,
"speed_mbps": 200000,
"state": "up",
"carrier": 1,
"mac": "fa:20:20:2e:4a:3f",
"addresses": ["192.0.2.10/24"],
"default_route": true
}
],
"bonds": [
{"name": "bond0", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_0", "slaves": [{"name": "eth1", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "64:00.0", "numa": 0}, {"name": "eth2", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "64:00.1", "numa": 0}]},
{"name": "bond1", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_1", "slaves": [{"name": "eth3", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "68:00.0", "numa": 0}, {"name": "eth4", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "68:00.1", "numa": 0}]},
{"name": "bond2", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_2", "slaves": [{"name": "eth5", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "6c:00.0", "numa": 0}, {"name": "eth6", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "6c:00.1", "numa": 0}]},
{"name": "bond3", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_3", "slaves": [{"name": "eth7", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "70:00.0", "numa": 0}, {"name": "eth8", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "70:00.1", "numa": 0}]},
{"name": "bond4", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_4", "slaves": [{"name": "eth9", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "a4:00.0", "numa": 1}, {"name": "eth10", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "a4:00.1", "numa": 1}]},
{"name": "bond5", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_5", "slaves": [{"name": "eth11", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "a8:00.0", "numa": 1}, {"name": "eth12", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "a8:00.1", "numa": 1}]},
{"name": "bond6", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_6", "slaves": [{"name": "eth13", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "ac:00.0", "numa": 1}, {"name": "eth14", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "ac:00.1", "numa": 1}]},
{"name": "bond7", "mode": "IEEE 802.3ad Dynamic link aggregation", "hash_policy": "layer3+4 (1)", "status": "up", "rdma": "mlx5_bond_7", "slaves": [{"name": "eth15", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "b0:00.0", "numa": 1}, {"name": "eth16", "status": "up", "speed_mbps": 400000, "failures": 0, "bdf": "b0:00.1", "numa": 1}]}
],
"ib_devices": [
{"name": "mlx5_17", "ca_type": "MT4129", "firmware": "28.46.5500", "state": "Active", "physical_state": "LinkUp", "rate_gbps": 100, "lid": "1", "link_layer": "InfiniBand", "bdf": "b3:00.0", "numa": 1},
{"name": "mlx5_18", "ca_type": "MT4129", "firmware": "28.46.5500", "state": "Active", "physical_state": "LinkUp", "rate_gbps": 100, "lid": "4", "link_layer": "InfiniBand", "bdf": "b4:00.0", "numa": 1},
{"name": "mlx5_19", "ca_type": "MT4129", "firmware": "28.46.5500", "state": "Active", "physical_state": "LinkUp", "rate_gbps": 100, "lid": "5", "link_layer": "InfiniBand", "bdf": "b5:00.0", "numa": 1},
{"name": "mlx5_20", "ca_type": "MT4129", "firmware": "28.46.5500", "state": "Active", "physical_state": "LinkUp", "rate_gbps": 100, "lid": "6", "link_layer": "InfiniBand", "bdf": "b6:00.0", "numa": 1}
],
"bmc": {
"detected": false,
"note": "独立 BMC 带外口未向当前 OS/IPMI 通道暴露"
},
"rdma_map": {},
"notes": []
}

View File

@ -0,0 +1,430 @@
import io
import json
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import gpu_node_profile as profile
ROOT = Path(__file__).resolve().parents[1]
FIXTURE = ROOT / "tests" / "fixtures" / "b300_snapshot.json"
class ParserTests(unittest.TestCase):
def test_normalize_bdf_uses_endpoint_from_sysfs_path(self):
path = (
"/sys/devices/pci0000:a0/0000:a0:00.4/0000:b1:00.0/"
"0000:b2:00.0/0000:b3:00.0"
)
self.assertEqual(profile.normalize_bdf(path), "b3:00.0")
def test_cuda_build_version_is_concise(self):
self.assertEqual(profile.concise_version("12.8.20250303"), "12.8")
def test_parse_topology_and_mapping(self):
text = """
GPU0 GPU1 NIC0 NIC1 CPU Affinity NUMA Affinity
GPU0 X NV18 PIX SYS 0-165 0
GPU1 NV18 X SYS PIX 166-331 1
NIC Legend:
NIC0: mlx5_bond_0
NIC1: mlx5_bond_1
"""
parsed = profile.parse_topology(text)
self.assertEqual(parsed["nic_legend"]["NIC0"], "mlx5_bond_0")
self.assertEqual(parsed["gpu_nic"]["GPU0"][0]["relation"], "PIX")
self.assertEqual(
parsed["gpu_affinity"]["GPU0"]["cpu_affinity"], "0-165"
)
self.assertEqual(
parsed["gpu_affinity"]["GPU1"]["numa_affinity"], "1"
)
self.assertNotIn("CPU", parsed["rows"]["GPU0"])
def test_parse_bond(self):
text = """
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
Slave Interface: eth1
MII Status: up
Speed: 400000 Mbps
Link Failure Count: 0
"""
bond = profile.parse_bond_text("bond0", text)
self.assertEqual(bond["status"], "up")
self.assertEqual(bond["slaves"][0]["speed_mbps"], 400000)
def test_nvlink_fabric_type_does_not_assume_nvswitch(self):
direct = {"rows": {"GPU0": {"GPU1": "NV4"}, "GPU1": {"GPU0": "NV4"}}}
self.assertEqual(
profile.infer_nvlink_fabric_type(direct, 2), "NVLink Fabric"
)
switched = {
"rows": {
f"GPU{source}": {
f"GPU{target}": "NV18"
for target in range(4)
if target != source
}
for source in range(4)
}
}
self.assertEqual(
profile.infer_nvlink_fabric_type(switched, 4), "NVSwitch Fabric"
)
def test_render_is_offline_native_html(self):
snapshot = json.loads(FIXTURE.read_text(encoding="utf-8"))
profile.attach_gpu_bond_mapping(snapshot)
report = profile.render_html(
snapshot, {"gpus": 8, "bonds": 8, "ib": 4}
)
self.assertIn("GPU ↔ RDMA 亲和画像", report)
self.assertIn("系统与软件", report)
self.assertIn("CPU 与内存", report)
self.assertIn("带内与带外管理网络", report)
self.assertIn("eth0", report)
self.assertIn("带外 · BMC", report)
self.assertIn("HEALTH 与验证结果", report)
self.assertIn("CPU / NUMA 亲和", report)
self.assertIn("GPU 功耗遥测", report)
self.assertIn("GPU 瞬时功耗", report)
self.assertIn("2.56 kW / 8.80 kW", report)
self.assertIn("瞬时 / 1.10 kW 上限", report)
self.assertIn(">P0<", report)
self.assertIn("GPU0–3", report)
self.assertIn("推荐 CPU 0-165", report)
self.assertIn("不代表当前训练进程已经绑核", report)
self.assertIn('data-numa-filter="all"', report)
self.assertIn('data-numa-filter="0"', report)
self.assertIn('data-numa-filter="1"', report)
self.assertIn("function setNumaView(value)", report)
self.assertIn(".rail[hidden],.numa-ruler [hidden]", report)
self.assertIn(".ib-card .card-head { align-items:flex-start; flex-wrap:wrap;", report)
self.assertIn("eth0 承载默认路由且链路 Up", report)
self.assertIn("mlx5_bond_7", report)
self.assertIn("NVLINK / NVSWITCH FABRIC", report)
self.assertLess(report.index("系统与软件"), report.index("GPU ↔ RDMA 亲和画像"))
self.assertLess(report.index("带内与带外管理网络"), report.index("HEALTH 与验证结果"))
self.assertNotIn("mermaid", report.lower())
self.assertNotIn("https://", report.lower())
self.assertNotIn("http://", report.lower())
def test_render_3d_topology_is_offline_and_json_driven(self):
snapshot = json.loads(FIXTURE.read_text(encoding="utf-8"))
profile.attach_gpu_bond_mapping(snapshot)
report = profile.render_topology_3d_html(snapshot)
self.assertIn("GPU Machine 3D CT", report)
self.assertIn('id="machine-data"', report)
self.assertIn('id="topology-canvas"', report)
self.assertIn("function buildScene()", report)
self.assertIn("function deepestCommonPath(paths)", report)
self.assertIn('data-camera="inside"', report)
self.assertIn("function setCameraMode(mode,box=null,animate=true)", report)
self.assertIn("双击壳体逐层进入", report)
self.assertIn("state.cameraMode==='inside'", report)
self.assertIn("kind:'bond'", report)
self.assertIn("kind:'roce-port'", report)
self.assertIn("GPU endpoints only · not NIC", report)
self.assertIn("function drawEdge(edge,scale,selected,overlay=false)", report)
self.assertIn("function scopePath(box)", report)
self.assertIn("function nextScopeToward(box)", report)
self.assertIn("function enterScope(box,animate=true)", report)
self.assertIn('id="camera-back"', report)
self.assertIn("parentId:`pair-${gpu.index}`", report)
self.assertIn("PCIe Root Complex", report)
self.assertIn("GDR candidate", report)
self.assertIn('"scope": "guest-visible"', report)
self.assertNotIn("https://", report.lower())
self.assertNotIn("http://", report.lower())
def test_html_templates_are_separate_source_files(self):
operations = profile.load_html_template("gpu_node_profile.html")
topology = profile.load_html_template("gpu_topology_3d.html")
self.assertIn("@@BODY@@", operations)
self.assertIn("@@MACHINE_DATA@@", topology)
self.assertNotIn("https://", operations.lower())
self.assertNotIn("https://", topology.lower())
def test_build_pci_fabric_preserves_bridge_hierarchy(self):
gpus = [
{
"index": 0,
"bdf": "63:00.0",
"numa": 0,
"pci_path": ["60:00.0", "61:00.0", "63:00.0"],
}
]
nics = [
{
"name": "eth1",
"bdf": "64:00.0",
"numa": 0,
"pci_path": ["60:00.0", "61:00.0", "64:00.0"],
}
]
with patch.object(profile, "pci_numa", return_value=0):
fabric = profile.build_pci_fabric(gpus, nics, [], {})
self.assertEqual(fabric["scope"], "guest-visible")
bridge = next(item for item in fabric["bridges"] if item["bdf"] == "61:00.0")
self.assertEqual(bridge["children"], ["63:00.0", "64:00.0"])
def test_standalone_roce_endpoints_map_uniquely_to_gpus(self):
nics = [
{
"name": "eth0",
"bdf": "0b:00.0",
"numa": 0,
"pci_path": ["08:00.0", "0b:00.0"],
"speed_mbps": 100000,
"state": "up",
"carrier": 1,
"default_route": True,
"is_bond": False,
},
{
"name": "eth1",
"bdf": "5a:00.0",
"numa": 0,
"pci_path": ["4c:00.0", "54:00.0", "5a:00.0"],
"speed_mbps": 100000,
"state": "up",
"carrier": 1,
"default_route": False,
"is_bond": False,
},
{
"name": "eth2",
"bdf": "5a:00.1",
"numa": 0,
"pci_path": ["4c:00.0", "54:00.0", "5a:00.1"],
"speed_mbps": 100000,
"state": "up",
"carrier": 1,
"default_route": False,
"is_bond": False,
},
]
rdma_map = {"mlx5_0": "eth0", "mlx5_1": "eth1", "mlx5_2": "eth2"}
endpoints = profile.build_network_endpoints(
nics,
[],
[],
rdma_map,
{
"mlx5_1": {"state": "ACTIVE"},
"mlx5_2": {"state": "ACTIVE"},
},
)
self.assertEqual([item["name"] for item in endpoints], ["eth1", "eth2"])
self.assertTrue(all(item["type"] == "standalone" for item in endpoints))
snapshot = {
"gpus": [
{"index": 0, "pci_path": ["4c:00.0", "53:00.0"], "numa": 0},
{"index": 1, "pci_path": ["4c:00.0", "58:00.0"], "numa": 0},
],
"nics": nics,
"bonds": [],
"ib_devices": [],
"rdma_map": rdma_map,
"network_endpoints": endpoints,
"topology": {
"gpu_affinity": {},
"gpu_nic": {
"GPU0": [
{"rdma": "mlx5_1", "relation": "PXB"},
{"rdma": "mlx5_2", "relation": "PXB"},
],
"GPU1": [
{"rdma": "mlx5_1", "relation": "PXB"},
{"rdma": "mlx5_2", "relation": "PXB"},
],
},
},
}
profile.attach_gpu_network_mapping(snapshot)
primary_ids = {
gpu.get("primary_network_id") for gpu in snapshot["gpus"]
}
self.assertEqual(primary_ids, {"rdma:mlx5_1", "rdma:mlx5_2"})
self.assertTrue(
all(gpu.get("network_relation") == "PXB" for gpu in snapshot["gpus"])
)
health = profile.evaluate(
snapshot, {"gpus": 2, "bonds": None, "ib": None}
)
affinity = next(
item for item in health["checks"] if item["title"] == "GPU ↔ RDMA 亲和"
)
self.assertEqual(affinity["level"], "ok")
self.assertIn("PXB 2", affinity["detail"])
operations = profile.render_html(
snapshot, {"gpus": 2, "bonds": None, "ib": None}
)
topology = profile.render_topology_3d_html(snapshot)
self.assertEqual(operations.count("ROCE · DIRECT"), 2)
self.assertNotIn("未发现对应网络路径", operations)
self.assertIn("Mode:'Standalone RDMA'", topology)
self.assertIn("endpoint.topology_visibility", topology)
def test_unique_mapping_uses_global_minimum_not_greedy_first_match(self):
endpoints = [
{
"id": "rdma:mlx5_1",
"type": "standalone",
"protocol": "RoCE",
"name": "eth1",
"rdma": "mlx5_1",
"role": "compute",
"members": [{"name": "eth1", "pci_path": []}],
},
{
"id": "rdma:mlx5_2",
"type": "standalone",
"protocol": "RoCE",
"name": "eth2",
"rdma": "mlx5_2",
"role": "compute",
"members": [{"name": "eth2", "pci_path": []}],
},
]
snapshot = {
"gpus": [
{"index": 0, "pci_path": [], "numa": 0},
{"index": 1, "pci_path": [], "numa": 0},
],
"bonds": [],
"network_endpoints": endpoints,
"topology": {
"gpu_affinity": {},
"gpu_nic": {
"GPU0": [
{"rdma": "mlx5_1", "relation": "PIX"},
{"rdma": "mlx5_2", "relation": "PIX"},
],
"GPU1": [
{"rdma": "mlx5_1", "relation": "PIX"},
{"rdma": "mlx5_2", "relation": "PHB"},
],
},
},
}
profile.attach_gpu_network_mapping(snapshot)
self.assertEqual(snapshot["gpus"][0]["network"], "eth2")
self.assertEqual(snapshot["gpus"][1]["network"], "eth1")
self.assertTrue(
all(gpu["network_relation"] == "PIX" for gpu in snapshot["gpus"])
)
def test_non_rdma_bond_is_not_an_accelerator_endpoint(self):
management_bond = {
"name": "bond-mgmt",
"rdma": "",
"status": "up",
"slaves": [{"name": "eno1", "status": "up"}],
}
endpoints = profile.build_network_endpoints(
[],
[management_bond],
[],
{},
)
self.assertEqual(endpoints, [])
health = profile.evaluate(
{
"gpus": [],
"bonds": [management_bond],
"ib_devices": [],
"network_endpoints": [],
"topology": {},
},
{"gpus": None, "bonds": 0, "ib": None},
)
self.assertFalse(
any(item["title"] == "Bond 数量不符合预期" for item in health["checks"])
)
def test_power_telemetry_health_does_not_flag_p0_as_an_error(self):
snapshot = json.loads(FIXTURE.read_text(encoding="utf-8"))
profile.attach_gpu_bond_mapping(snapshot)
health = profile.evaluate(snapshot, {"gpus": 8, "bonds": 8, "ib": 4})
power_check = next(
item for item in health["checks"] if item["title"] == "GPU 功耗遥测"
)
self.assertEqual(power_check["level"], "ok")
self.assertIn("P0 8/8", power_check["detail"])
def test_collect_gpus_keeps_power_when_optional_range_is_unsupported(self):
core = (
"0, NVIDIA Test, GPU-0, 00000000:63:00.0, 595.71.05, "
"275040, 1024, 42, 75"
)
power = "0, P0, 875.5, 1100"
with (
patch.object(profile, "run", side_effect=[core, power, ""]),
patch.object(profile, "pci_numa", return_value=0),
):
gpus = profile.collect_gpus()
self.assertEqual(len(gpus), 1)
self.assertEqual(gpus[0]["pstate"], "P0")
self.assertEqual(gpus[0]["power_w"], 875.5)
self.assertEqual(gpus[0]["power_limit_w"], 1100)
self.assertNotIn("power_max_limit_w", gpus[0])
def test_incomplete_power_telemetry_is_a_warning(self):
snapshot = json.loads(FIXTURE.read_text(encoding="utf-8"))
snapshot["gpus"][0].pop("power_limit_w")
profile.attach_gpu_bond_mapping(snapshot)
health = profile.evaluate(snapshot, {"gpus": 8, "bonds": 8, "ib": 4})
power_check = next(
item for item in health["checks"] if item["title"] == "GPU功耗信息不完整"
)
self.assertEqual(power_check["level"], "warning")
report = profile.render_html(snapshot, {"gpus": 8, "bonds": 8, "ib": 4})
self.assertIn("遥测不完整", report)
def test_cli_renders_snapshot(self):
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "profile.html"
topology_output = Path(directory) / "topology-3d.html"
result = profile.main(
[
"--snapshot-in",
str(FIXTURE),
"--output",
str(output),
"--topology-output",
str(topology_output),
"--expected-gpus",
"8",
"--expected-bonds",
"8",
"--expected-ib",
"4",
]
)
self.assertEqual(result, 0)
self.assertTrue(output.exists())
self.assertGreater(output.stat().st_size, 30_000)
self.assertTrue(topology_output.exists())
self.assertIn(
'id="topology-canvas"',
topology_output.read_text(encoding="utf-8"),
)
def test_cli_can_emit_structured_snapshot_only(self):
buffer = io.StringIO()
with patch.object(profile.sys, "stdout", buffer):
result = profile.main(["--snapshot-in", str(FIXTURE), "--snapshot-stdout"])
self.assertEqual(result, 0)
snapshot = json.loads(buffer.getvalue())
self.assertEqual(snapshot["schema_version"], 1)
self.assertEqual(snapshot["pci_fabric"]["scope"], "guest-visible")
if __name__ == "__main__":
unittest.main()