🔥 Run WishForge from the cluster only: drop the Mac deployment and its copies

- The cluster's settings come only from the operator inputs kept in
  .wishforge/cluster-poc (carried keys, signing key, plans, profiles,
  host keys); the Mac's .env.local and service files are no longer read
- restore puts back one of the node's snapshots inside the cluster: the
  files are checked against the manifest before anything is emptied, the
  backup container restores the database, row counts decide
- Snapshots also keep Fanyu's Runner records
- Remove migrate, the Mac backup copy and its launchd job, and the Mac
  public entry (frp gateway) code and manifests
- Mark the Mac operating guides as retired

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
eust-w 2026-11-02 02:31:17 +08:00
parent aa151d16f9
commit fb67484dd1
18 changed files with 220 additions and 1792 deletions

View File

@ -1,8 +1,8 @@
# WishForge 部署在 POC 集群
把原本跑在 Mac 上的 WishForge(数据库、编排、对象存储、控制面、网页、执行器、发布 Runner)整体迁到 DroboticFlow 所在的百度云 POC 集群(`https://120.48.181.105:6443`)。执行器仍然使用 Claude CLI,模型改走公司地瓜网关(Anthropic 协议,`https://ai-api.d-robotics.cc`,客户端自动拼接 `/v1/messages`),默认 `deepseek-pro@latest`,小模型槽位 `deepseek-flash@latest`。
WishForge(数据库、编排、对象存储、控制面、网页、执行器、发布 Runner、备份)只运行在 DroboticFlow 所在的百度云 POC 集群(`https://120.48.181.105:6443`)。它在 2026-11-01 从 Mac 迁入,Mac 上的部署和数据已于 2026-11-02 删除;操作者的电脑只保存集群设置的输入(见「数据与密钥」),不运行任何服务。执行器仍然使用 Claude CLI,模型改走公司地瓜网关(Anthropic 协议,`https://ai-api.d-robotics.cc`,客户端自动拼接 `/v1/messages`),默认 `deepseek-pro@latest`,小模型槽位 `deepseek-flash@latest`。
## 当前进度(2026-11-01)
## 当前进度(2026-11-02)
| 步骤 | 状态 |
|---|---|
@ -10,14 +10,13 @@
| 命名空间、节点数据目录、本地卷、访问规则、准入策略 | 完成,权限已逐项核对 |
| Gitea 部署密钥 | 已添加到 `longtao.wu/droboticflow`(写权限),集群内拉取已验证 |
| Secret 与 ConfigMap | 已创建,网关 Key 已写入 |
| 演练迁移(`migrate --rehearsal`) | 完成:逐表行数一致;Runner 保持空闲,Mac 照常服务 |
| 演练验收 | 隔离标签与日志脱敏修复已部署并在新运行中验证;发布后重跑愿望优化、幂等提交、实现、暂停/继续、独立检查、审核门禁和取消,通过并完成产物独立验收,见[部署后验收](cluster-poc-postdeploy-2026-11-01.md)。该轮暴露的取消期间检查 Pod 回收竞态已修复并复现验证通过,见[检查 Pod 清理验收](cluster-poc-checkpod-cleanup-2026-11-01.md) |
| 公网入口(独立负载均衡 + 公网 IP + 域名) | **`https://wishforge.d-robotics.cc`** 已启用正式证书,校验通过;负载均衡 `180.76.227.59`,HTTP 跳转 HTTPS。本机解析正常,其他内网设备需确认公司 DNS 的记录,见下文 |
| 正式切换(`migrate --final`) | 2026-11-01 完成:Mac 服务已停,数据逐表一致导入,集群 Runner 启动;之后 Mac 只作回退 |
| 从 Mac 迁入 | 2026-11-01 演练后正式切换,数据逐表一致导入;2026-11-02 删除 Mac 部署、数据和备份副本,迁移命令也已移除 |
| 项目 | 2026-11-01 旧项目全部删除,只保留两个:「DroboticFlow POC」(`poc` 分支 → POC 集群)和「DroboticFlow Fanyu · dev」(`dev` 分支 → Fanyu 集群),见下文「两个项目」 |
| 每日备份 | 2026-11-01 已部署:节点上每日快照、Mac 每日 10:17 复制(launchd 已安装并试跑成功),在演练副本上完成一次恢复演练,见下文「备份与恢复」 |
| 每日备份 | 节点上每日快照,恢复在集群内完成,见下文「备份与恢复」 |
演练验收:
迁入前后的验收:
| 项目 | 结果 |
|---|---|
@ -34,8 +33,6 @@
| 合入、构建、发布 | 正式切换后由第一个真实愿望验收(演练副本不向 `poc` 分支推送、不发布) |
| Pod 重启后恢复:运行停在代码审核时删除 Pod,约 25 秒恢复就绪;数据行数、运行状态、待审核记录和 Temporal 工作流不变,之后取消正常完成 | 通过 |
Mac 上的服务仍在运行,切换前的临时公网链接由 Cloudflare 隧道提供(地址以维护者给出的为准),切换完成后停用。
公网入口是 WishForge 专属负载均衡的公网 IP `180.76.227.59`,以及公司自己的域名 `wishforge.d-robotics.cc`(公网 DNS 在阿里云,A 记录指向这个 IP;不使用 sslip.io、Cloudflare 等第三方域名)。HTTPS 时域名和 IP 各有一张 Let's Encrypt 证书,由 Caddy 自动续签,HTTP 的 GET/HEAD 跳转到 HTTPS,其他方法拒绝;控制台以域名为主地址。也可以用 `public --http` 改成纯 HTTP,这时密码和会话明文传输。
2026-10-30 查询时,公司内网 DNS(`10.9.0.10`、`10.48.6.10`,hobot.cc 的 AD 域控)自己托管的 `d-robotics.cc` 区域没有这条记录,返回 NXDOMAIN;仅在公网 DNS 加记录不一定能让内网用户访问。2026-11-01 本机已能正常解析并通过域名访问。其他内网设备若仍解析失败,需要 IT 在内网区域补同样的 A 记录 `wishforge` → `180.76.227.59`;在那之前可用 IP 访问。`public --domain` 只要求公网解析正确,本机 DNS 解析不到时会给出提醒。
@ -49,7 +46,7 @@ Mac 上的服务仍在运行,切换前的临时公网链接由 Cloudflare 隧
- 两个项目的环境都不预授权:**每次发布都要有人批准**。愿望一律用 Claude CLI 执行,从目标分支开始;受保护检查都在 POC 集群的检查 Pod 里跑。
- 两个项目沿用旧 POC 项目的「后端代码」划分(除 `src/`、`tests/frontend/`、`**/*.md` 外都算后端),前端账号只能改前端。POC 项目的成员沿用旧项目(管理员和 `qianduan`);Fanyu 项目先只有管理员,要给其他账号,在账号管理页面勾选项目。
- 每个环境有固定的 profile:POC 的在 Mac 的 `~/.config/wishforge/poc-profile.json`,Fanyu 的在 `.wishforge/cluster-poc/fanyu-profile.json`(由 `fanyu-access` 生成)。profile 钉死分支、集群地址、命名空间及其 UID、各 Deployment 的 UID、镜像仓库和访问地址;任何一项对不上,构建和发布都会拒绝。
- 每个环境有固定的 profile:POC 的在 `.wishforge/cluster-poc/poc-profile.json`,Fanyu 的在 `.wishforge/cluster-poc/fanyu-profile.json`(由 `fanyu-access` 生成)。profile 钉死分支、集群地址、命名空间及其 UID、各 Deployment 的 UID、镜像仓库和访问地址;任何一项对不上,构建和发布都会拒绝。
- Fanyu 的 api 镜像同时跑在 api、annotation-processor、annotation-validator 三个 Deployment 上:一次发布把三者都换成新 api 镜像(顺序 coordinator → worker → api → processor → validator),逐个核对 rollout 和 Pod 摘要;回滚时每个 Deployment 恢复成它自己原来的镜像。headpose、wilor 不在发布范围内。
- Fanyu 集群里 WishForge 的身份([fanyu-access.yaml](../infra/k8s/cluster-poc/fanyu-access.yaml)):Service Account `droboticflow-fanyu/wishforge`,只能读取并更新那 5 个 Deployment 的镜像、查看 rollout、在 `droboticflow-fanyu` 里创建构建 Pod,读不到 Secret 和 ConfigMap;准入策略只允许它创建固定 Kaniko 镜像、只挂临时目录和镜像仓库凭据、不带令牌的构建 Pod。它的长期令牌只在 WishForge 集群的 Secret `wishforge-fanyu-kube` 里,只挂进 executor(构建)和 runner-fanyu(发布)。删除 Fanyu 集群里的 Secret `wishforge-token` 即可吊销。
@ -65,11 +62,11 @@ node scripts/cluster-poc.mjs fanyu-access --kubeconfig ~/workSpace/droboticflow/
node scripts/cluster-poc.mjs projects --replace
```
它删除每个项目及其愿望、运行、发布、环境、Runner、证据和审核记录,以及运行事件、对象存储里的产物和托管的 Git 仓库;保留账号、成员、用量账本和审计日志。两个 Runner 的注册信息保存在 Mac 的 `.wishforge/cluster-poc/projects.json`(0600),之后的 `secrets`/`apply` 从这里生成两个 Runner 的设置。
它删除每个项目及其愿望、运行、发布、环境、Runner、证据和审核记录,以及运行事件、对象存储里的产物和托管的 Git 仓库;保留账号、成员、用量账本和审计日志。两个 Runner 的注册信息保存在操作者电脑的 `.wishforge/cluster-poc/projects.json`(0600),之后的 `secrets`/`apply` 从这里生成两个 Runner 的设置。
## 架构
命名空间 `wishforge` 中一个单副本 StatefulSet([platform.yaml](../infra/k8s/cluster-poc/platform.yaml)),固定在 CPU 节点 192.168.80.249(由本地卷的节点亲和性决定)。Pod 里的容器共享回环网络,结构与 Mac 相同:
命名空间 `wishforge` 中一个单副本 StatefulSet([platform.yaml](../infra/k8s/cluster-poc/platform.yaml)),固定在 CPU 节点 192.168.80.249(由本地卷的节点亲和性决定)。Pod 里的容器共享回环网络:
| 容器 | 内容 | 监听 |
|---|---|---|
@ -92,11 +89,11 @@ node scripts/cluster-poc.mjs projects --replace
- 验证器每次重建工作区目录后会换一个新的检查 Pod;命令超时或被取消时连同 Pod 一起删除;Pod 自身最长存活 2 小时;
- 起不来或 exec 连不上时,运行停在「基础设施」等待,不会当作检查失败。
与 Mac 的差异:Mac 上检查在 Seatbelt 中断网运行;集群里检查 Pod 可以访问外网和集群内网(集群不执行网络策略)。这是选择「不放宽集群安全」换来的代价,证据中记录为 `checkSandbox: pod`。
注意:检查 Pod 可以访问外网和集群内网(集群不执行网络策略)。这是选择「不放宽集群安全」换来的代价,证据中记录为 `checkSandbox: pod`。
### 身份与权限([rbac.yaml](../infra/k8s/cluster-poc/rbac.yaml)、[admission.yaml](../infra/k8s/cluster-poc/admission.yaml))
WishForge 在集群里只有一个身份 `wishforge/wishforge`,它的短期令牌只挂进 executor 和 runner 两个容器,取代 Mac 上的集群管理员 kubeconfig:
WishForge 在集群里只有一个身份 `wishforge/wishforge`,它的短期令牌只挂进 executor 和 runner 两个容器:
- `wishforge-checks`:创建、删除检查 Pod 并 exec;
- `droboticflow`:只能读取和更新 `droboticflow-api`、`droboticflow-worker`、`drosync-coordinator` 三个 Deployment,创建 Kaniko 构建 Pod;读不到任何 Secret 或 ConfigMap;
@ -108,10 +105,9 @@ WishForge 在集群里只有一个身份 `wishforge/wishforge`,它的短期令
- 节点 192.168.80.249 的本地盘 `/home/cce/wishforge`([storage.yaml](../infra/k8s/cluster-poc/storage.yaml)),回收策略 Retain,各容器只挂自己的子目录。检查 Pod 另有一个只指向其中 `exec/` 目录的本地卷。
- 每个容器只读自己的设置文件(`wishforge-env` 中的 `platform.env`、`executor.env`、`runner.env`),executor 拿不到数据库和对象存储的密码。
- 从 Mac 原样带过去的只有持久数据依赖的密钥和令牌(`CARRIED`,见 [cluster-poc.mjs](../scripts/cluster-poc.mjs));数据库和对象存储使用新的随机密码,只保存在 Mac 私有的 `.wishforge/cluster-poc/generated.json`(0600)。
- 网关 Key 只写进集群 Secret `wishforge-claude`,不在 Mac 上另存。
- 集群版生成新的本机管理员密码;个人账号随数据库迁移,沿用原密码。会话密钥更新,所有人需要重新登录一次。
- 项目设置里的受保护安装、检查命令是 `poc-project-check.mjs` 的绝对路径。从 Mac 迁来的项目写的是 Mac 上的 Node 和仓库路径,platform 容器每次启动、控制面开始服务之前把它们改成镜像里的 `/usr/local/bin/node /opt/wishforge/scripts/poc-project-check.mjs install|verify`([cluster-poc-project.ts](../scripts/cluster-poc-project.ts));别的命令和已经指向镜像的命令不动。检查命令变化会让旧的检查证据不能再用于发布,迁移前已要求没有待发布的版本,所以没有影响。
- 集群设置的输入只在操作者电脑的 `.wishforge/cluster-poc/`(0700,文件 0600):持久数据依赖的密钥和令牌 `carried.env`(`CARRIED`,见 [cluster-poc.mjs](../scripts/cluster-poc.mjs),其中 `WF_VAULT_KEY` 解密数据库里的凭据)、Runner 签名密钥 `runner-ed25519.pem`、套餐 `plans.json`、POC 与 Fanyu 的 profile、Gitea 主机密钥 `known_hosts`、数据库与对象存储密码 `generated.json`、两个 Runner 的注册 `projects.json`。`secrets`/`apply` 只读这些文件和 POC 集群的 kubeconfig。**这个目录要另外妥善备份**:快照里没有它,丢了就无法解密快照里的凭据,也无法重建 Pod 设置。
- 网关 Key 只写进集群 Secret `wishforge-claude`,不在本地另存。
- 项目的受保护安装、检查命令是镜像里的 `/usr/local/bin/node /opt/wishforge/scripts/poc-project-check.mjs install|verify`;platform 容器每次启动时把仍指向别处的这类命令改回镜像里的路径([cluster-poc-project.ts](../scripts/cluster-poc-project.ts))。
### 实现要点
@ -124,7 +120,7 @@ WishForge 在集群里只有一个身份 `wishforge/wishforge`,它的短期令
- SWC 的原生模块缓存拒绝父目录对其他用户可写的位置,platform 容器的 `HOME` 放在持久卷里的私有目录。
- 网页启动器只透传 `WF_LOCAL_API_TOKEN`,不透传 `WF_DEV_AUTH_TOKEN`。集群设置必须把前者设成控制面所接受的后者;否则管理员密码校验和会话都成功,项目接口仍返回 401。凭据只在 platform 容器内使用,不下发到 executor 或 runner。
- OpenSSH 按用户数据库里的家目录找 `~/.ssh`,不看 `$HOME`,所以 Git 的 SSH 目录挂在镜像用户的家目录 `/home/wishforge`。
- 受保护检查的工具链除了 Node、npm、Go、Git,还有 python3(带 NumPy、OpenCV、PyYAML)和 ffmpeg:DroboticFlow 的 Go 测试(`internal/worker`)用它们运行算子 runner `backend/operator-runner/main.py` 的视频和数据集算子。Mac 上检查用的是 `/opt/homebrew` 里的 python3 和 ffmpeg;镜像里装 Debian 的 `python3`、`ffmpeg`,Python 包按 [check-python-requirements.txt](../infra/images/check-python-requirements.txt) 固定版本和哈希安装。缺了它们,这些测试会失败或等到超时。
- 受保护检查的工具链除了 Node、npm、Go、Git,还有 python3(带 NumPy、OpenCV、PyYAML)和 ffmpeg:DroboticFlow 的 Go 测试(`internal/worker`)用它们运行算子 runner `backend/operator-runner/main.py` 的视频和数据集算子。镜像里装 Debian 的 `python3`、`ffmpeg`,Python 包按 [check-python-requirements.txt](../infra/images/check-python-requirements.txt) 固定版本和哈希安装。缺了它们,这些测试会失败或等到超时。
## 需要操作者完成
@ -146,7 +142,7 @@ WishForge 在集群里只有一个身份 `wishforge/wishforge`,它的短期令
kubectl --kubeconfig ~/.config/droboticflow/kubeconfigs/poc.config -n wishforge delete pod wishforge-0
```
3. **授权启动与迁移**:Claude Code 的自动权限检查可能拦截 `apply` 和 `migrate`,这时在交互式 `claude` 终端中为这些命令加权限规则,或者自己在终端里执行下面的命令。
3. **授权启动**:Claude Code 的自动权限检查可能拦截 `apply`,这时在交互式 `claude` 终端中为这些命令加权限规则,或者自己在终端里执行下面的命令。
## 命令顺序
@ -162,12 +158,6 @@ node scripts/cluster-poc.mjs secrets
容器只在启动时读取设置文件(其中也写着检查 Pod 使用的镜像)。`apply` 每次都会先刷新这些 Secret 和 ConfigMap,内容有变化时 Pod 模板上的摘要随之改变,Pod 自动按新设置重建;重新构建镜像后运行 `apply` 即可,平台和检查 Pod 会用同一个镜像。
演练迁移:复制数据库快照、Git、POC 构建记录和对象存储,Temporal 为空,Runner 保持空闲,Mac 照常服务:
```bash
node scripts/cluster-poc.mjs migrate --rehearsal
```
```bash
node scripts/cluster-poc.mjs status
```
@ -178,7 +168,7 @@ node scripts/cluster-poc.mjs status
kubectl --kubeconfig ~/.config/droboticflow/kubeconfigs/poc.config -n wishforge port-forward pod/wishforge-0 13000:3000
```
保持它运行,在浏览器打开 `http://127.0.0.1:13000`。网页只接受登记过的来源地址,本机端口必须是 13000(Mac 上自己的控制台占着 3000)。演练副本只用来验收登录和浏览:不要在上面提愿望(有直接合入权限的账号会推送到 `poc`,而 Mac 也在服务),正式迁移时它的数据会被 Mac 的数据覆盖。
保持它运行,在浏览器打开 `http://127.0.0.1:13000`。网页只接受登记过的来源地址,本机端口必须是 13000。
集群版管理员密码在 Pod 的私有文件里,复制到剪贴板(不打印):
@ -210,10 +200,7 @@ node scripts/cluster-poc.mjs public --http
## 备份与恢复
集群里没有可跨节点挂载的存储(没有 StorageClass),备份分两层:
- **节点上**:Pod 里的 `backup` 容器每天 03:20(北京时间)取一份快照,写到数据盘的 `backups/<UTC 时间>/`,保留最近 7 份。Pod 启动时若最新快照超过一天,会在约一分钟后补一份;失败一小时后重试。
- **Mac 上**:`backup` 把最新快照复制到 `.wishforge/cluster-poc/backups/<UTC 时间>/`(0700,保留 14 份),逐个文件核对大小和 SHA-256。`backup --schedule` 安装 launchd 任务,每天 10:17 自动执行(Mac 睡眠时在唤醒后补跑),日志在 `.wishforge/cluster-poc/backup.log`;Pod 最近一次快照失败或最新快照超过 36 小时,命令在复制后报错退出;由 launchd 运行时,任何失败都会弹出 macOS 通知「WishForge 集群备份失败」。
Pod 里的 `backup` 容器每天 03:20(北京时间)取一份快照,写到数据盘的 `backups/<UTC 时间>/`,保留最近 7 份。Pod 启动时若最新快照超过一天,会在约一分钟后补一份;失败一小时后重试。`status` 显示节点上的快照、最新时间和最近一次结果。
一份快照包括:
@ -221,77 +208,38 @@ node scripts/cluster-poc.mjs public --http
|---|---|
| `postgres.dump` | `pg_dump -Fc`。与逐表行数取自同一个数据库快照(`pg_export_snapshot`),行数记在 `manifest.json`,恢复后据此核对 |
| `temporal.db` | Temporal 的 SQLite,经 SQLite 在线备份复制并通过 `integrity_check` |
| `git.tgz`、`minio.tgz`、`poc-state.tgz`、`runner-state.tgz`、`platform-state.tgz` | 对应数据目录;`platform-state` 不含可重建的 `home` 缓存,`minio` 不含 MinIO 启动时会清理的 `.minio.sys/tmp` |
| `git.tgz`、`minio.tgz`、`poc-state.tgz`、`runner-state.tgz`、`runner-state-fanyu.tgz`、`platform-state.tgz` | 对应数据目录;`platform-state` 不含可重建的 `home` 缓存,`minio` 不含 MinIO 启动时会清理的 `.minio.sys/tmp` |
| `manifest.json` | 时间、镜像、行数,以及每个文件的大小和 SHA-256 |
不包括运行工作区和检查缓存(`exec/`)、Claude 的家目录、Caddy 的证书(会自动重新签发)。快照里有加密保存的凭据和管理员密码文件;解密这些凭据要用 Mac 上 `.env.local` 里的 `WF_VAULT_KEY` 等密钥,恢复也依赖 `.wishforge/cluster-poc/generated.json`,这些不在快照里,要和快照一起保管好。
不包括运行工作区和检查缓存(`exec/`)、Claude 的家目录、Caddy 的证书(会自动重新签发)。解密快照里的凭据需要 `carried.env` 里的 `WF_VAULT_KEY`,见「数据与密钥」。
`backup` 容器只读挂载它要保存的目录,只写 `backups/`;通过回环连接数据库(需要数据库密码),没有集群令牌、设置文件或密钥。镜像带有与数据库同版本的 `pg_dump`、`pg_restore`(从固定的 postgres 镜像复制)。
立即取一份快照并复制到 Mac:
恢复:用节点上的一份快照替换集群里的全部数据(数据库、Temporal、Git、对象存储,以及平台、POC 和两个 Runner 的记录),全程在集群内完成:
1. 停掉 WishForge Pod,起一个只挂数据卷的恢复 Pod;
2. 先按 `manifest.json` 逐个核对快照文件的大小和 SHA-256,核对不过就什么都不动、把 Pod 启动回来;
3. 清空并解包各数据目录,放回 Temporal 的数据库;
4. 维护模式启动 Pod(只有数据库、Temporal、对象存储在跑),由 `backup` 容器经回环把 `postgres.dump` 导入全新的 PostgreSQL;
5. 逐表行数与快照记录一致才正常启动。节点上的快照保留不动。
```bash
node scripts/cluster-poc.mjs backup --now
node scripts/cluster-poc.mjs restore <UTC 时间,例如 20261102T192000Z> --runner
```
每天自动复制(取消用 `--unschedule`):
```bash
node scripts/cluster-poc.mjs backup --schedule
```
`status` 会显示节点上的快照数、最新时间和最近一次结果。
恢复:用 Mac 上的一份快照替换集群里的全部数据(数据库、Temporal、Git、对象存储,以及平台、POC 和 Runner 的记录),流程与 `migrate` 相同——停 Pod、清空并解包目录、维护模式下导入数据库、逐表行数与快照记录一致才启动。节点上已有的快照保留。正式切换后要让 Runner 工作,加 `--runner`:
```bash
node scripts/cluster-poc.mjs restore .wishforge/cluster-poc/backups/<UTC 时间>
```
2026-11-01 在演练副本上的验收(集群空闲、快照后无新写入):
| 项目 | 结果 |
|---|---|
| 部署后 `backup` 容器约一分钟内自动取第一份快照(29.2 MB);`backup --now` 再取一份,均复制到 Mac 并逐文件核对通过 | 通过 |
| launchd 任务手动触发一次,退出码 0 | 通过 |
| `restore` 最新快照:约 3 分钟,逐表行数一致后启动;Git、POC 记录、Runner 记录、管理员凭据文件逐字节一致,MinIO 对象一致;登录页 200,未登录 API 401 | 通过 |
| 首次 `restore` 在上传时遇到集群连接超时 | 当时 Pod 停在 0 副本,已手动恢复,数据未动;随后改为每次上传核对 SHA-256 并重试、上传失败时自动重启 Pod |
这台 Mac 到集群的连接会丢掉 `kubectl exec` 输出流的末尾,下载速度约 150 kB/s。所以 `backup` 逐个文件下载:容器端发完文件后等 Mac 关闭输入才退出,Mac 收齐全部字节后才关闭。一份快照大约要 4 分钟。
节点故障时,在新节点上执行 `bootstrap`(先把 [storage.yaml](../infra/k8s/cluster-poc/storage.yaml) 里的节点改成新节点)、`apply`,再 `restore`。
## 正式切换
切换窗口约 30 分钟,期间停止写入:
1. Mac 上确认没有排队、执行中的运行,没有就绪或部署中的发布(`migrate` 会检查,不满足时拒绝执行)。
2. 停止 Mac 服务,确认 `stopCompleted=true`:
```bash
node scripts/local-poc-service.mjs stop
```
3. 正式迁移:会短暂停止 Mac 上 WishForge 的 MinIO 和 Temporal 容器以取得一致副本(之后重新启动作为回退);导入后逐表比对行数,一致才启动;Runner 开始工作:
```bash
node scripts/cluster-poc.mjs migrate --final
```
4. 用个人账号登录新地址验收,然后停用 Cloudflare 临时链接。Mac 上的数据保留不动。
不加 `--runner` 时两个 Runner 保持空闲,不会部署任何东西。
## 回退
- 切换前任何一步失败:Mac 照常服务,集群里的副本可以随时用 `migrate` 重新覆盖。
- 切换后、集群尚无新写入:`kubectl -n wishforge scale statefulset wishforge --replicas=0`,再在 Mac 上 `node scripts/local-poc-service.mjs start`。
- 集群已有新写入:先缩容冻结集群,把数据反向导回 Mac 后再启动 Mac 服务。导回的项目检查命令是镜像里的路径,启动前在 Mac 上运行 `node --import tsx scripts/setup-local-poc.ts`,把它们恢复成 Mac 的路径。
- 数据出错:用出错前的快照 `restore`。
- 新镜像有问题:把 `.wishforge/cluster-poc/image.json` 换回上一个镜像的记录后 `apply`。
- 删除 `wishforge-public` 会回收公网 IP,删除后在百度云控制台核对 BLB 和 EIP 已经释放。
## 已知限制
- IDE(Theia、OCT)没有迁移,集群版网页上的 IDE 入口不可用。
- 检查 Pod 有网络(见上文)。
- 数据在单节点本地盘上。节点上的快照和数据在同一块盘上,节点损坏时只能用 Mac 上的副本恢复;Mac 每天复制一次,所以最多丢失约一天的数据,且 Mac 长期关机期间没有异地副本。
- **数据和快照都在同一个节点的同一块本地盘上**。快照能防误删、误改和软件故障,但节点或磁盘损坏时数据和快照会一起丢失;集群里没有可跨节点挂载的存储(没有 StorageClass),目前没有异地副本。
- Temporal 使用开发服务,Web UI 已关闭。
- 模型名取自网关的 `/v1/models`(2026-10-30):`deepseek-pro@latest`、`deepseek-flash@latest` 是网关的别名,都已用 Anthropic 协议实测可用;`[1m]` 后缀让 Claude CLI 按 1M 上下文处理。网关文档里推荐的 `deepseek-v4-flash` 不在列表中。

View File

@ -1,5 +1,7 @@
# 本机工作空间的协作者访问
> **已停用(2026-11-02)**:WishForge 只运行在 POC 集群,Mac 本机部署及其数据已删除。现行部署、备份与恢复见 [cluster-poc-deployment.md](cluster-poc-deployment.md);本文仅作历史记录。
本机模式面向获准的可信操作者。网页可监听局域网地址;控制 API、数据库、Temporal、Provisioner 等服务仍监听本机回环地址。平台在服务端验证本机管理员或个人账号登录后,以对应身份调用控制 API;个人账号还携带账号 ID 和密码版本,由控制面按当前权限授权。浏览器收不到控制 API 凭据。
## 配置与启动
@ -37,7 +39,7 @@ npm run local:network -- --address 192.168.1.20 --apply
## 登录与退出
协作者通过局域网或经批准的 HTTPS 入口访问,使用管理员在「成员与权限」页创建的个人账号登录,并只授予所需项目与操作权限。不要共享本机管理员账号。登录会话为加密、HttpOnly、SameSite=Strict 的 cookie,1 小时到期;HTTPS 入口额外设置 Secure,退出会清除会话 cookie。公网入口的配置、限制与验收见 [local-public-entry.md](local-public-entry.md)。
协作者通过局域网或经批准的 HTTPS 入口访问,使用管理员在「成员与权限」页创建的个人账号登录,并只授予所需项目与操作权限。不要共享本机管理员账号。登录会话为加密、HttpOnly、SameSite=Strict 的 cookie,1 小时到期;HTTPS 入口额外设置 Secure,退出会清除会话 cookie。
本机管理员密码是 256 位随机值;个人账号使用各自的随机初始密码。登录限流用于限制 scrypt 校验开销。15 分钟内错误超过 8 次后,密码校验在全局范围内每 2 秒最多一次,而不是锁定 15 分钟,局域网中其他设备输错密码不会把管理员锁在外面。限流不信任客户端提供的转发 IP,状态存在当前 Web 进程内,进程重启会清空。更换凭据文件(尤其 passwordHash 或 sessionSecret)会使旧会话失效。所有写入请求必须携带与访问入口完全一致的 Origin;未在白名单的 Host 或跨站请求被拒绝。

View File

@ -1,8 +1,8 @@
# 本机许愿服务运行说明
本机运行 WishForge 网页、控制面、Claude CLI 和发布 Runner。用户通过网页登录后提交愿望;项目固定接入 Gitea 的 `longtao.wu/droboticflow` 仓库 `poc` 分支,目标环境为既有 POC。Claude 修改独立工作区,受保护检查通过后由交付流程处理 Git 提交和发布。
> **已停用(2026-11-02)**:WishForge 只运行在 POC 集群,Mac 本机部署及其数据已删除。现行部署、备份与恢复见 [cluster-poc-deployment.md](cluster-poc-deployment.md);本文仅作历史记录。
迁到 POC 集群运行的部署、迁移与回退见 [cluster-poc-deployment.md](cluster-poc-deployment.md);切换完成前,本文描述的 Mac 服务仍是正式入口,也是迁移的数据来源。
本机运行 WishForge 网页、控制面、Claude CLI 和发布 Runner。用户通过网页登录后提交愿望;项目固定接入 Gitea 的 `longtao.wu/droboticflow` 仓库 `poc` 分支,目标环境为既有 POC。Claude 修改独立工作区,受保护检查通过后由交付流程处理 Git 提交和发布。
## 后台运行
@ -26,7 +26,7 @@ node scripts/local-poc-service.mjs start
## 入口与登录
本机入口为 `http://localhost:3000`;同一可达局域网的其他设备使用 `http://本机局域网IP:3000`。是否跨网可达由现有路由和防火墙决定。当前部署另有经批准的 Cloudflare 临时 HTTPS 隧道,地址以维护者提供的链接和私有入口白名单为准;它只代理 Web,不公开内部 API 或 IDE。百度云独立公网 IP 入口的实现与上线闸门见 [local-public-entry.md](local-public-entry.md),不能把清单已生成当成公网入口已开通。
本机入口为 `http://localhost:3000`;同一可达局域网的其他设备使用 `http://本机局域网IP:3000`。是否跨网可达由现有路由和防火墙决定。当前部署另有经批准的 Cloudflare 临时 HTTPS 隧道,地址以维护者提供的链接和私有入口白名单为准;它只代理 Web,不公开内部 API 或 IDE。
本机管理员的登录凭据保存在 `.wishforge/local-access.json`,只允许本机当前用户读取(0600)。文件含用户名、密码和会话签名材料,不应上传到 Gitea、放入网页静态目录或复制到日志。文件不存在时可运行 `node scripts/setup-local-access.mjs` 创建;重复运行保留已有凭据。`.env.local` 的 `WF_LOCAL_ACCESS_FILE` 指向该文件的绝对路径。

View File

@ -1,151 +0,0 @@
# 本机 WishForge 的公网入口
## 当前状态(2026-10-29)
本机已有 Cloudflare 临时 HTTPS 隧道,最终地址以维护者提供的链接及私有 `.env.local` 中的 `WF_LOCAL_ALLOWED_ORIGINS` 为准。它只转发 Web,不直接公开 API、数据库或 IDE。隧道停止、Mac 休眠或断网会中断访问。
**百度云独立公网 IP 入口尚未开通。** 本次完成了脚本、配置、镜像及真实二进制校验,但在目标集群的 NetworkPolicy 验收处停止:
- 指定的测试端口可以正常访问,正向控制通过。
- 另一个本应被默认拒绝的入站端口仍可访问;等待策略传播并重复检查 8 次仍为 HTTP 200。
- 删除临时出站放行策略后,出站连接仍可访问;重复检查 8 次仍为 HTTP 200。
- 因此不能把“已创建 NetworkPolicy 对象”当作网络隔离已生效。
没有创建新的 BLB/EIP,没有改动原 `droboticflow-api-public`,没有追加新 origin,没有安装或启动本机 Caddy/frpc LaunchAgent。网关 Deployment、Pod、ConfigMap、专用隧道 Secret 和网络策略已撤回。`wishforge-access` 命名空间保留:CCE 自动同步了镜像拉取凭据,清理程序没有擅自删除这些自动管理对象;没有运行中的工作负载或公网 Service。
后续需要先由运维确认并启用目标 CCE 网络模式支持的 NetworkPolicy 执行能力,然后重新验收;或者选择具备独立网络边界的中转环境。**不能为完成公网访问而跳过此闸门,或擅自修改整个 POC 的网络组件。**
## 设计边界
平台和数据继续留在 Mac。集群只运行单副本 FRP 服务端和回环代理校验进程;浏览器 HTTPS 经 TCP 隧道直达 Mac 的 Caddy,再反代 `127.0.0.1:3000`。
| 公网端口 | 网关端口 | Mac 回环端口 | 用途 |
|---|---|---|---|
| 80/TCP | 18080 | 18080 | ACME HTTP-01;普通 GET/HEAD 跳转到标准 HTTPS 端口,其他方法拒绝 |
| 443/TCP | 18443 | 18443 | 用户 HTTPS |
| 7000/TCP | 7000 | frpc 主动连接 | 双向 TLS 加随机令牌的隧道控制及工作连接 |
不使用 sslip.io,不新增云主机、磁盘或数据库,不把 4000、4300 等内部端口加入隧道。只有在下述验收通过后,才可以申请独立的 BLB/EIP。
### 隧道与权限
- FRP 固定 v0.71.1,容器镜像引用固定摘要。
- 使用专用私有 CA 双向 TLS,客户端明确校验服务器名称 `wishforge-entry.internal`;这个名称仅是私有传输证书的身份,不需要公网 DNS。
- 服务端只允许 18080/18443。回环 `NewProxy` 钩子进一步限定两个固定名称的 TCP 代理,拒绝 STCP、UDP、其他端口及分组代理;钩子不可用时注册失败。
- 一个副本,Deployment 使用 `Recreate`,避免两个 FRP 实例分担同一个客户端的控制连接与用户连接。
- 就绪检查针对 7000 和策略进程,不依赖客户端注册后才出现的代理端口,避免启动死锁。
- 非 root、禁止提权、只读根文件系统、无 ServiceAccount 令牌、无宿主机挂载、无 Docker socket。
- 命名空间默认拒绝出入网络;服务端不需要主动访问业务 Pod、数据库或 Kubernetes API。
## 文件与本机状态
- `infra/k8s/local-entry/kustomization.yaml`:非公网网关资源;**不包含**收费的公网 Service。
- `infra/k8s/local-entry/public-service.yaml`:单独的新 LoadBalancer 清单,不能指向旧业务的 LB ID 或 selector。
- `scripts/configure-local-entry.mjs`:精确 origin 的预览、追加、按回执撤销,以及固定 Caddy/frpc 配置生成。
- `scripts/local-entry-service.mjs`:仅管理自己的 Caddy/frpc 进程,不改变原 platform/runner 服务组。
私有状态位于已被 Git 忽略的 `.wishforge/local-entry/`:目录 0700,秘密文件 0600。
- `bin/`:核对官方摘要后的 Caddy、frpc。
- `mtls/`:专用 CA、传输证书及令牌。CA 私钥和客户端私钥不上传集群;只把服务端需要的材料放到专用 Secret。
- `caddy-staging/`、`caddy-production/`:独立的 ACME 账户与证书状态,不得混用。
- `receipts/`:环境文件私有备份及本次追加 origin 的回执。
- `logs/`:独立日志,按文件轮转;不启用 HTTP 请求访问日志或调试级认证日志。
- `network-verification.json`:本次集群网络隔离实测结果。`false` 不能当作可发布状态。
所有启动脚本在 import 时都不会启动服务;ConfigMap 挂载脚本需按真实路径识别入口,不能用未解析的 `argv` 路径直接比较。
## 重新实施前必须满足
1. 集群网络隔离重新实测通过:有正向对照,禁止的入站端口被拦截,撤销允许规则后出站也被拦截。仅检查 API 对象或网络插件名称不够。
2. 入口对象名和命名空间 UID 经检查仍属于本任务;恢复时不能覆盖他人后来创建的对象。
3. 新资源仅引用该入口镜像、Secret 与 selector;不修改 DroboticFlow 或其他业务资源。
4. 用户已经确认新增 LB/EIP 费用,且未改变申请范围。
之后才能单独创建公网 Service。其配置显式选择 `small1`、标准 BGP、按流量计费及 5 Mbps 初始峰值上限,避免默认的 100 Mbps。**5 Mbps 不是账单封顶;Mac 离线时 LB/EIP 仍可能计费。**
## 新 IP 分配后的配置流程
以下命令只在前置条件通过、维护者取得本次专属新 IP 后执行;占位符不是实际可用地址。
先生成 staging 配置,不修改 Web origin:
```bash
node scripts/configure-local-entry.mjs --config <新公网IPv4> --staging
```
验证所需私有证书文件及 Caddy/frpc 配置:
```bash
node scripts/local-entry-service.mjs validate
```
安装独立用户服务(需本机登录的 macOS 用户):
```bash
node scripts/local-entry-service.mjs install
```
staging 仅用于确认 ACME 验证路径,不能作为浏览器信任验收。验证通过后,停止入口服务、确认 `stopCompleted=true`,重新生成不带 `--staging` 的配置,再启动并检查生产证书:
```bash
node scripts/local-entry-service.mjs stop
```
```bash
node scripts/configure-local-entry.mjs --config <新公网IPv4>
```
```bash
node scripts/local-entry-service.mjs start
```
Caddy 显式使用 Let's Encrypt `shortlived` profile,不使用内部 CA。公网 IP 证书需要自动续签,状态目录必须保留。`default_sni` 使没有 SNI 的 IP 访问也能选中证书;不得用跳过 TLS 校验的请求代替验收。首次签发成功不等于已经观察到自然续签成功。
只在可信 HTTPS 已通过验证后,预览并添加该 origin:
```bash
node scripts/configure-local-entry.mjs --address <新公网IPv4>
```
```bash
node scripts/configure-local-entry.mjs --address <新公网IPv4> --apply
```
工具只追加 `https://<新IP>`,保留 localhost、LAN、Cloudflare 及其他原有入口;不打印环境文件内容、不改凭据。随后在所有愿望、构建与发布空闲且 Runner 为新鲜 idle 时,按原运维流程重启本机服务,使新 origin 生效。`doctor.ready` 或 `/usage.activeRuns=0` 单独不足以证明安全空闲。
## 验收与运维
```bash
node scripts/local-entry-service.mjs status
```
检查可信 IP 证书、证书到期时间、HTTPS `/login`、未登录 API 拒绝、错误 Host/Origin 拒绝、HTTP 重定向不泄露本机端口,以及原 POC `/api/health` 和备用入口。
登录后的 Cookie、写入、SSE、WebSocket 检查必须使用已获授权的真实会话;不得读取管理员密码、伪造会话或擅自创建测试账号。没有验证的项目须明确标注,不得把匿名登录页可访问描述成完整业务验收。
服务由独立 LaunchAgent `cc.d-robotics.wishforge.local-entry` 管理,只随当前用户登录运行。不改变系统休眠设置;合盖、休眠和网络中断仍会影响访问和续签。
停止入口不会停止 WishForge:
```bash
node scripts/local-entry-service.mjs stop
```
按回执撤销本次新增的 origin,不整份覆盖旧环境文件:
```bash
node scripts/configure-local-entry.mjs --revert <回执文件> --apply
```
删除本次公网 Service 前必须确认它的 UID 和关联的 LB/EIP 属于本任务。删除会使地址失效,且地址可能无法恢复;需核对云端实际资源回收情况,不能仅凭 Kubernetes 对象消失就认定停止计费。命名空间中出现其他人或系统管理的额外对象时,不直接整体删除。
## 离线检查
```bash
node --test scripts/configure-local-entry.test.mjs scripts/local-entry-service.test.mjs
```
```bash
kubectl kustomize infra/k8s/local-entry
```

View File

@ -226,9 +226,9 @@ spec:
- { name: data, mountPath: /var/lib/wishforge/runner-state, subPath: runner-state-fanyu }
- { name: data, mountPath: /var/lib/wishforge/poc-state, subPath: poc-state }
- { name: runner-fanyu-tmp, mountPath: /tmp }
# Daily snapshots of the data onto this node's disk (scripts/cluster-poc-backup.mjs); the
# operator Mac copies them off the node (cluster-poc.mjs backup). It reads the data it saves
# read-only and the database through loopback; it has no token, settings or keys.
# Daily snapshots of the data onto this node's disk (scripts/cluster-poc-backup.mjs), which
# cluster-poc.mjs restore puts back. It reads the data it saves read-only and the database
# through loopback; it has no token, settings or keys.
- name: backup
image: ${WISHFORGE_IMAGE}
args: [backup]
@ -247,6 +247,7 @@ spec:
- { name: data, mountPath: /mnt/data/platform-state, subPath: platform-state, readOnly: true }
- { name: data, mountPath: /mnt/data/poc-state, subPath: poc-state, readOnly: true }
- { name: data, mountPath: /mnt/data/runner-state, subPath: runner-state, readOnly: true }
- { name: data, mountPath: /mnt/data/runner-state-fanyu, subPath: runner-state-fanyu, readOnly: true }
- { name: data, mountPath: /mnt/data/temporal, subPath: temporal, readOnly: true }
- { name: backup-tmp, mountPath: /tmp }
- name: caddy

View File

@ -1,21 +0,0 @@
bindAddr = "0.0.0.0"
bindPort = 7000
proxyBindAddr = "0.0.0.0"
allowPorts = [{ single = 18080 }, { single = 18443 }]
maxPortsPerClient = 2
transport.tls.force = true
transport.tls.certFile = "/run/entry-secret/server.crt"
transport.tls.keyFile = "/run/entry-secret/server.key"
transport.tls.trustedCaFile = "/run/entry-secret/ca.crt"
auth.method = "token"
auth.additionalScopes = ["HeartBeats", "NewWorkConns"]
auth.tokenSource.type = "file"
auth.tokenSource.file.path = "/run/entry-secret/token"
log.to = "console"
log.level = "info"
log.disablePrintColor = true
[[httpPlugins]]
name = "wishforge-web-only"
addr = "127.0.0.1:19000"
path = "/validate"
ops = ["NewProxy"]

View File

@ -1,123 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: wishforge-access
labels:
app.kubernetes.io/part-of: wishforge-local-entry
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: v1.34
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: local-entry
namespace: wishforge-access
labels:
app.kubernetes.io/part-of: wishforge-local-entry
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: wishforge-local-entry
template:
metadata:
labels:
app.kubernetes.io/name: wishforge-local-entry
app.kubernetes.io/part-of: wishforge-local-entry
spec:
automountServiceAccountToken: false
terminationGracePeriodSeconds: 20
nodeSelector:
kubernetes.io/os: linux
kubernetes.io/arch: amd64
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: frps
image: docker.io/fatedier/frps:v0.71.1@sha256:48720c81f048e842f7f0b8053753c1a8514053d59d2dee8a953f67995de73a18
args: ['-c', '/etc/entry/frps.toml']
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
resources:
requests: { cpu: 100m, memory: 64Mi }
limits: { cpu: 500m, memory: 256Mi }
ports:
- { name: tunnel, containerPort: 7000 }
- { name: web-http, containerPort: 18080 }
- { name: web-https, containerPort: 18443 }
readinessProbe:
tcpSocket: { port: tunnel }
periodSeconds: 5
livenessProbe:
tcpSocket: { port: tunnel }
initialDelaySeconds: 15
periodSeconds: 15
volumeMounts:
- { name: config, mountPath: /etc/entry, readOnly: true }
- { name: credentials, mountPath: /run/entry-secret, readOnly: true }
- name: proxy-policy
image: docker.io/library/node:24-alpine@sha256:8a32c8cf649a49915a66e81e7a609223c708d0adabf288ee3f4077d765542f85
command: ['node', '/etc/entry/proxy-policy.mjs']
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
resources:
requests: { cpu: 25m, memory: 32Mi }
limits: { cpu: 100m, memory: 128Mi }
readinessProbe:
exec:
command: ['node', '/etc/entry/proxy-policy.mjs', '--check']
timeoutSeconds: 3
periodSeconds: 10
livenessProbe:
exec:
command: ['node', '/etc/entry/proxy-policy.mjs', '--check']
timeoutSeconds: 3
initialDelaySeconds: 15
periodSeconds: 20
volumeMounts:
- { name: config, mountPath: /etc/entry, readOnly: true }
volumes:
- name: config
configMap: { name: entry-config }
- name: credentials
secret:
secretName: entry-mtls
defaultMode: 0440
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: wishforge-access
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: entry-ports
namespace: wishforge-access
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: wishforge-local-entry
policyTypes: [Ingress]
ingress:
- ports:
- { protocol: TCP, port: 7000 }
- { protocol: TCP, port: 18080 }
- { protocol: TCP, port: 18443 }

View File

@ -1,10 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: wishforge-access
resources:
- gateway.yaml
configMapGenerator:
- name: entry-config
files:
- frps.toml
- proxy-policy.mjs

View File

@ -1,76 +0,0 @@
import http from 'node:http';
import { realpathSync } from 'node:fs';
import { pathToFileURL } from 'node:url';
export const POLICY_PORT = 19000;
const ports = new Map([
['wishforge-http', 18080],
['wishforge-https', 18443],
]);
/** FRP's NewProxy hook: only the two named web transports, never another proxy type. */
export function proxyDecision(request) {
const proxy = request?.content;
const allowed =
request?.op === 'NewProxy' &&
proxy?.proxy_type === 'tcp' &&
ports.has(proxy.proxy_name) &&
ports.get(proxy.proxy_name) === proxy.remote_port &&
!proxy.group &&
!proxy.group_key &&
!proxy.custom_domains?.length &&
!proxy.subdomain &&
!proxy.multiplexer;
return allowed
? { reject: false, unchange: true }
: { reject: true, reject_reason: 'Only the configured WishForge HTTP and HTTPS TCP proxies are allowed' };
}
/** Loopback only; no request bodies or authentication material enter logs. */
export function policyServer() {
return http.createServer({ requestTimeout: 5000, headersTimeout: 5000, maxHeaderSize: 8192 }, async (req, res) => {
const reply = (status, body) => {
res.writeHead(status, { 'Content-Type': 'application/json', 'Cache-Control': 'no-store' });
res.end(JSON.stringify(body));
};
if (req.method === 'GET' && req.url === '/healthz') return reply(200, { ready: true });
if (req.method !== 'POST' || req.url?.split('?')[0] !== '/validate')
return reply(403, { reject: true, reject_reason: 'Unsupported operation' });
try {
const chunks = [];
let size = 0;
for await (const chunk of req) {
size += chunk.length;
if (size > 16384) {
reply(413, { reject: true, reject_reason: 'Request too large' });
req.destroy();
return;
}
chunks.push(chunk);
}
reply(200, proxyDecision(JSON.parse(Buffer.concat(chunks).toString('utf8'))));
} catch {
if (!res.headersSent) reply(400, { reject: true, reject_reason: 'Invalid request' });
}
});
}
/** ConfigMap mounts use symlinks; Node resolves the main module but keeps the original argv path. */
export function isPolicyMain(moduleUrl, script = process.argv[1]) {
try {
return Boolean(script && moduleUrl === pathToFileURL(realpathSync(script)).href);
} catch {
return false;
}
}
if (isPolicyMain(import.meta.url)) {
if (process.argv[2] === '--check') {
const response = await fetch(`http://127.0.0.1:${POLICY_PORT}/healthz`, { signal: AbortSignal.timeout(1500) });
if (!response.ok || !(await response.json()).ready) process.exitCode = 1;
} else {
const server = policyServer();
server.listen(POLICY_PORT, '127.0.0.1');
for (const signal of ['SIGTERM', 'SIGINT']) process.once(signal, () => server.close());
}
}

View File

@ -1,25 +0,0 @@
# Apply separately, only after the isolated gateway and NetworkPolicy checks pass.
# This creates a NEW billable BLB/EIP. It never references an existing business load balancer.
apiVersion: v1
kind: Service
metadata:
name: local-entry-public
namespace: wishforge-access
labels:
app.kubernetes.io/part-of: wishforge-local-entry
annotations:
service.beta.kubernetes.io/cce-load-balancer-lb-name: wishforge-access-local-entry
service.beta.kubernetes.io/cce-load-balancer-internal-vpc: 'false'
service.beta.kubernetes.io/cce-elastic-ip-route-type: BGP
service.beta.kubernetes.io/cce-elastic-ip-billing-method: ByTraffic
service.beta.kubernetes.io/cce-elastic-ip-bandwidth-in-mbps: '5'
service.beta.kubernetes.io/blb-performance-level: small1
spec:
type: LoadBalancer
externalTrafficPolicy: Cluster
selector:
app.kubernetes.io/name: wishforge-local-entry
ports:
- { name: http, protocol: TCP, port: 80, targetPort: web-http }
- { name: https, protocol: TCP, port: 443, targetPort: web-https }
- { name: tunnel, protocol: TCP, port: 7000, targetPort: tunnel }

View File

@ -1,11 +1,12 @@
#!/usr/bin/env node
// Daily snapshots of WishForge's data in the POC cluster (docs/cluster-poc-deployment.md), taken by
// the WishForge Pod's backup container onto the node's disk; `cluster-poc.mjs backup` copies the
// newest one off the node to the operator Mac. Importing it starts nothing.
// the WishForge Pod's backup container onto the node's disk; `cluster-poc.mjs restore` puts one
// back. Importing it starts nothing.
//
// node scripts/cluster-poc-backup.mjs take one snapshot a day, until Kubernetes stops the container
// node scripts/cluster-poc-backup.mjs once take one snapshot now
// node scripts/cluster-poc-backup.mjs latest print the newest complete snapshot's manifest and the last result
// node scripts/cluster-poc-backup.mjs verify DIR check a snapshot's files against its manifest
import { spawn } from 'node:child_process';
import { createHash } from 'node:crypto';
import { createReadStream } from 'node:fs';
@ -20,10 +21,10 @@ export const DATA_ROOT = '/mnt/data';
/** The database beside it on the Pod's loopback; the password comes from PGPASSWORD. */
const DATABASE = { host: '127.0.0.1', port: 5432, user: 'wishforge', database: 'wishforge' };
/** The directories saved as archives, beside the database dump and Temporal's database. */
export const ARCHIVES = ['git', 'minio', 'platform-state', 'poc-state', 'runner-state'];
export const ARCHIVES = ['git', 'minio', 'platform-state', 'poc-state', 'runner-state', 'runner-state-fanyu'];
/** Not saved: the platform container's caches, rebuilt when it starts, and MinIO's scratch space and trash. */
const ARCHIVE_EXCLUDES = { 'platform-state': ['./home'], minio: ['./.minio.sys/tmp'] };
/** Snapshots kept on the node; the operator Mac keeps its own copies longer. */
/** Snapshots kept on the node. */
export const KEEP = 7;
/** 03:20 in Beijing, when nobody is expected to be working. */
export const SCHEDULE = { hourUtc: 19, minute: 20 };
@ -287,11 +288,17 @@ export async function main(argv = process.argv.slice(2)) {
await once(BACKUP_ROOT);
return 0;
}
if (argv[0] === 'verify' && argv.length === 2) {
const manifest = JSON.parse(await readFile(path.join(argv[1], 'manifest.json'), 'utf8'));
await verifyFiles(argv[1], manifest.files);
console.log(`[backup] snapshot ${manifest.name}: every file matches its manifest`);
return 0;
}
if (argv[0] === 'latest' && argv.length === 1) {
process.stdout.write(`${JSON.stringify(await latest())}\n`);
return 0;
}
throw new Error('Usage: cluster-poc-backup.mjs [once|latest]');
throw new Error('Usage: cluster-poc-backup.mjs [once|latest|verify DIR]');
}
if (process.argv[1] && pathToFileURL(path.resolve(process.argv[1])).href === import.meta.url)

View File

@ -1,11 +1,10 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { mkdir, mkdtemp, readdir, rm, stat, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { nextRun, retention, snapshotName, takeSnapshot, verifyFiles, sha256File } from './cluster-poc-backup.mjs';
import { failureNotification, launchAgentPlist } from './cluster-poc.mjs';
import { restoreScript } from './cluster-poc.mjs';
test('snapshots are taken daily at 03:20 Beijing time and named by their UTC time', () => {
assert.equal(nextRun(new Date('2026-11-01T10:00:00Z')).toISOString(), '2026-11-01T19:20:00.000Z');
@ -22,7 +21,7 @@ test('the newest complete snapshots are kept; older ones and leftover partial on
assert.deepEqual(retention(names.slice(0, 1), 7).remove, []);
});
test('a copied snapshot is accepted only when every file has its recorded size and SHA-256', async t => {
test('a snapshot is restored only when every file has its recorded size and SHA-256', async t => {
const directory = await mkdtemp(path.join(os.tmpdir(), 'wf-backup-verify-'));
t.after(() => rm(directory, { recursive: true, force: true }));
await writeFile(path.join(directory, 'git.tgz'), 'archive');
@ -47,36 +46,16 @@ test('a snapshot that fails leaves no partial snapshot or lock behind, and a hel
assert.ok(await stat(path.join(root, '.lock')));
});
test('the Mac copies the newest snapshot every day through a launchd job it can read back', async t => {
const plist = launchAgentPlist({
node: '/opt/homebrew/bin/node',
script: '/repo/scripts/cluster-poc.mjs',
root: '/repo & co',
pathEnv: '/opt/homebrew/bin:/usr/bin',
home: '/Users/someone',
log: '/repo/.wishforge/cluster-poc/backup.log',
});
assert.match(plist, /<key>Hour<\/key><integer>10<\/integer>/);
assert.match(plist, /<string>backup<\/string>/);
assert.match(plist, /\/repo &#38; co/);
if (process.platform !== 'darwin') return;
const directory = await mkdtemp(path.join(os.tmpdir(), 'wf-backup-plist-'));
t.after(() => rm(directory, { recursive: true, force: true }));
const file = path.join(directory, 'job.plist');
await writeFile(file, plist);
const parsed = JSON.parse(execFileSync('plutil', ['-convert', 'json', '-o', '-', file], { encoding: 'utf8' }));
assert.deepEqual(parsed.ProgramArguments, ['/opt/homebrew/bin/node', '/repo/scripts/cluster-poc.mjs', 'backup']);
assert.equal(parsed.WorkingDirectory, '/repo & co');
assert.deepEqual(parsed.StartCalendarInterval, { Hour: 10, Minute: 17 });
assert.equal(parsed.EnvironmentVariables.WF_BACKUP_NOTIFY, '1');
});
test('a failing daily copy is announced in a notification whose text cannot break out of its quotes', () => {
const script = failureNotification('Backup needs attention: "x" \\ end\nsecond line');
assert.equal(
script,
'display notification "Backup needs attention: \\"x\\" \\\\ end second line" with title "WishForge 集群备份失败" sound name "Basso"',
);
if (process.platform === 'darwin')
execFileSync('osacompile', ['-o', path.join(os.tmpdir(), `wf-notify-${process.pid}.scpt`), '-e', script]);
test('a restore checks the snapshot before it empties anything, and unpacks only what the snapshot holds', () => {
const script = restoreScript('20261102T192000Z', ['git', 'minio']).split('\n');
const verify = script.findIndex(line => line.includes('cluster-poc-backup.mjs verify'));
const firstReset = script.findIndex(line => line.includes('-mindepth 1 -delete'));
assert.ok(verify >= 0 && verify < firstReset);
assert.ok(script.indexOf('touch /tmp/verified') < firstReset);
assert.ok(script.some(line => line.startsWith("mkdir -p 'runner-state-fanyu'")));
assert.ok(!script.some(line => line.includes("-C 'runner-state-fanyu'")));
assert.ok(!script.some(line => /backups'? &&/.test(line) && line.includes('-delete')), 'snapshots are never emptied');
assert.equal(script.at(-2), 'touch /tmp/restored');
assert.throws(() => restoreScript('../x'), /Not a snapshot/);
assert.throws(() => restoreScript('20261102T192000Z', ['postgres']), /Unknown archive/);
});

View File

@ -1,6 +1,7 @@
#!/usr/bin/env node
// Operator CLI of WishForge's deployment in the POC cluster (docs/cluster-poc-deployment.md).
// Every command names its cluster explicitly and prints no secret. Importing it starts nothing.
// WishForge runs in the cluster only; this machine keeps the operator's inputs (STATE) and nothing else.
//
// node scripts/cluster-poc.mjs image build and push the image in the cluster (Kaniko)
// node scripts/cluster-poc.mjs bootstrap namespaces, storage, access rules
@ -8,30 +9,38 @@
// node scripts/cluster-poc.mjs gateway-key [--stdin] the model gateway key, from the clipboard or stdin
// node scripts/cluster-poc.mjs secrets Secrets and ConfigMaps from this Mac's settings
// node scripts/cluster-poc.mjs apply [--maintenance] [--runner] the WishForge Pod
// node scripts/cluster-poc.mjs migrate --rehearsal|--final copy this Mac's data into it
// node scripts/cluster-poc.mjs public [--production] the public HTTPS entry (billed load balancer)
// node scripts/cluster-poc.mjs fanyu-access --kubeconfig <Fanyu admin kubeconfig> WishForge's identity in Fanyu's cluster
// node scripts/cluster-poc.mjs projects --replace delete every project; create the POC and Fanyu projects
// node scripts/cluster-poc.mjs backup [--now] copy the newest daily snapshot to this Mac
// node scripts/cluster-poc.mjs backup --schedule do that every day (launchd)
// node scripts/cluster-poc.mjs restore <snapshot> [--runner] replace the cluster's data with a copied snapshot
// node scripts/cluster-poc.mjs restore <snapshot> [--runner] replace the cluster's data with a node snapshot
// node scripts/cluster-poc.mjs status show the deployment's state
import { spawn } from 'node:child_process';
import { createHash, randomBytes } from 'node:crypto';
import { createReadStream, createWriteStream } from 'node:fs';
import { chmod, mkdir, mkdtemp, readFile, readdir, rename, rm, stat, writeFile } from 'node:fs/promises';
import { chmod, mkdir, mkdtemp, readFile, rename, rm, stat, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { Resolver, getServers, resolve4 } from 'node:dns/promises';
import { isIP } from 'node:net';
import { setTimeout as delay } from 'node:timers/promises';
import { publicIPv4 } from './configure-local-entry.mjs';
import { ARCHIVES, COUNTS_SQL, retention, verifyFiles } from './cluster-poc-backup.mjs';
import { ARCHIVES, COUNTS_SQL } from './cluster-poc-backup.mjs';
export const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
export const STATE = path.join(ROOT, '.wishforge/cluster-poc');
export const KUBECONFIG = path.join(os.homedir(), '.config/droboticflow/kubeconfigs/poc.config');
export const POC_PROFILE = path.join(os.homedir(), '.config/wishforge/poc-profile.json');
/**
* The operator's inputs to the cluster's settings, kept private here: the keys and tokens the data
* depends on (carried.env, see CARRIED), the Runners' signing key, the plan catalog, the POC's
* profile and Gitea's pinned host keys.
*/
export const INPUTS = {
carried: path.join(STATE, 'carried.env'),
signingKey: path.join(STATE, 'runner-ed25519.pem'),
plans: path.join(STATE, 'plans.json'),
knownHosts: path.join(STATE, 'known_hosts'),
};
export const POC_PROFILE = path.join(STATE, 'poc-profile.json');
export const CLUSTER = 'https://120.48.181.105:6443';
export const NAMESPACE = 'wishforge';
export const CHECK_NAMESPACE = 'wishforge-checks';
@ -82,31 +91,6 @@ function capture(executable, args, options = {}) {
child.stdin.end(options.input);
});
}
/** A command whose stdin comes from `from` or whose stdout goes to `to`: binary-safe, never in memory. */
function pipeFile(executable, args, { from, to, env }) {
return new Promise((resolve, reject) => {
const child = spawn(executable, args, {
cwd: ROOT,
env,
stdio: [from ? 'pipe' : 'ignore', to ? 'pipe' : 'ignore', 'pipe'],
});
const err = [];
child.stderr.on('data', chunk => err.push(chunk));
child.once('error', reject);
if (from) createReadStream(from).on('error', reject).pipe(child.stdin);
const written = to
? new Promise((done, fail) =>
child.stdout
.pipe(createWriteStream(to, { mode: 0o600 }))
.on('finish', done)
.on('error', fail),
)
: Promise.resolve();
child.once('close', code =>
written.then(() => resolve({ code, stderr: Buffer.concat(err).toString('utf8') }), reject),
);
});
}
export async function kubectl(args, { input, timeoutSeconds = 60, allowFailure = false } = {}) {
const result = await capture(
'kubectl',
@ -384,6 +368,25 @@ const DOMAIN = /^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2
const PUBLIC_ORIGIN = new RegExp(`^https?://(?:\\d{1,3}(?:\\.\\d{1,3}){3}|${DOMAIN.source.slice(1, -1)})$`);
/** Public resolvers that show what the internet, Let's Encrypt included, sees of a domain. */
const PUBLIC_RESOLVERS = ['223.5.5.5', '119.29.29.29'];
/** A publicly routable IPv4 address: never a URL, private, shared or documentation address. */
export function publicIPv4(value) {
if (isIP(value) !== 4) throw new Error('A public IPv4 address is required');
const [a, b, c] = value.split('.').map(Number);
if (
a === 0 ||
a === 10 ||
a === 127 ||
a >= 224 ||
(a === 100 && b >= 64 && b <= 127) ||
(a === 169 && b === 254) ||
(a === 172 && b >= 16 && b <= 31) ||
(a === 192 && (b === 168 || b === 0 || (b === 88 && c === 99))) ||
(a === 198 && (b === 18 || b === 19 || (b === 51 && c === 100))) ||
(a === 203 && b === 0 && c === 113)
)
throw new Error('The entry must use a publicly routable IPv4 address');
return value;
}
/** A domain name for the public entry, as the operator's DNS points it at the load balancer. */
export function publicDomain(name) {
const domain = String(name ?? '')
@ -397,7 +400,7 @@ export const PORT_FORWARD_ORIGIN = 'http://127.0.0.1:13000';
/** The settings files of the platform, executor and Runner containers; see {@link CARRIED}. */
export function clusterEnvironments({ local, runner, runnerFanyu, image, generated, origins = [] }) {
const missing = [
...CARRIED.filter(key => !local[key]).map(key => `.env.local ${key}`),
...CARRIED.filter(key => !local[key]).map(key => `carried.env ${key}`),
...RUNNER_CARRIED.filter(key => !runner[key]).map(key => `runner.env ${key}`),
...(runnerFanyu ? RUNNER_CARRIED.filter(key => !runnerFanyu[key]).map(key => `Fanyu's Runner ${key}`) : []),
];
@ -500,19 +503,6 @@ export function envFile(values) {
return `${lines.join('\n')}\n`;
}
/** The POC files as the cluster's containers use them: the same bindings, at their own paths. */
export function clusterPocFiles({ profile, binding }) {
return {
profile: { ...profile, kubeconfig: `${PATHS.config}/kubeconfig` },
binding: {
...binding,
profileFile: `${PATHS.pocState}/poc-profile.json`,
stateDirectory: `${PATHS.pocState}/poc`,
gitHome: PATHS.gitHome,
},
};
}
/** WishForge's identity at the POC cluster's address, by its short-lived projected token. */
export function clusterKubeconfig(certificateAuthorityData) {
if (!/^[A-Za-z0-9+/=]+$/.test(certificateAuthorityData)) throw new Error('Invalid cluster CA data');
@ -806,28 +796,15 @@ async function gatewayKey(flag) {
console.log(`The gateway key is in the cluster (Secret wishforge-claude)${flag ? '.' : '; clear your clipboard.'}`);
}
/**
* The projects the cluster delivers and their Runners: once `projects` replaced them, the
* cluster's own (STATE/projects.json); before, the Mac's POC project, migrated with its data.
*/
async function clusterProjects(local) {
/** The projects the cluster delivers and their Runners (projects --replace). */
async function clusterProjects() {
const pocProfile = { ...JSON.parse(await readFile(POC_PROFILE, 'utf8')), kubeconfig: `${PATHS.config}/kubeconfig` };
const fanyuProfile = await readFile(FANYU_PROFILE, 'utf8').then(JSON.parse, () => undefined);
const profiles = {
'poc-profile.json': pocProfile,
...(fanyuProfile ? { 'fanyu-profile.json': fanyuProfile } : {}),
};
const replaced = await readFile(PROJECTS_FILE, 'utf8').then(JSON.parse, () => undefined);
if (!replaced) {
const runner = await readEnv(path.join(ROOT, '.wishforge/local-poc/runner.env'));
const binding = JSON.parse(await readFile(local.WF_LOCAL_POC_BINDING_FILE, 'utf8'));
return {
profiles,
bindings: clusterPocFiles({ profile: pocProfile, binding }).binding,
runner,
keys: { 'runner-public.json': await readFile(runner.WF_RUNNER_PUBLIC_KEYS, 'utf8') },
};
}
const replaced = JSON.parse(await readFile(PROJECTS_FILE, 'utf8'));
for (const binding of replaced.bindings)
if (!profiles[path.basename(binding.profileFile)])
throw new Error(`No ${path.basename(binding.profileFile)} for project ${binding.projectId}; run fanyu-access`);
@ -853,8 +830,8 @@ async function clusterProjects(local) {
async function secrets() {
await assertCluster();
const local = await readEnv(path.join(ROOT, '.env.local'));
const projects = await clusterProjects(local);
const local = await readEnv(INPUTS.carried);
const projects = await clusterProjects();
const { runner, runnerFanyu } = projects;
const image = (await imageRecord()).reference;
const generated = await generatedSecrets();
@ -874,10 +851,8 @@ async function secrets() {
'jsonpath={.clusters[0].cluster.certificate-authority-data}',
])
).stdout.trim();
const hosts = (await capture('ssh-keygen', ['-F', 'gitea.d-robotics.cc'])).stdout
.split('\n')
.filter(line => line && !line.startsWith('#'));
if (!hosts.length) throw new Error('No pinned host key for gitea.d-robotics.cc in ~/.ssh/known_hosts');
const hosts = (await readFile(INPUTS.knownHosts, 'utf8')).split('\n').filter(line => line && !line.startsWith('#'));
if (!hosts.length) throw new Error(`No pinned host key for gitea.d-robotics.cc in ${INPUTS.knownHosts}`);
// The Pod mounts the Claude settings; until gateway-key stores the key, they name the gateway only.
if (!(await exists('secret', 'wishforge-claude', NAMESPACE))) {
await applyObjects([
@ -894,9 +869,9 @@ async function secrets() {
'runner.env': envFile(environments.runner),
...(environments.runnerFanyu ? { 'runner-fanyu.env': envFile(environments.runnerFanyu) } : {}),
}),
secret('wishforge-keys', { 'runner-ed25519.pem': await readFile(local.WF_RUNNER_SIGNING_KEY_FILE, 'utf8') }),
secret('wishforge-keys', { 'runner-ed25519.pem': await readFile(INPUTS.signingKey, 'utf8') }),
configMap('wishforge-config', {
'plans.json': await readFile(local.WF_PLAN_CATALOG_FILE, 'utf8'),
'plans.json': await readFile(INPUTS.plans, 'utf8'),
...projects.keys,
...Object.fromEntries(
Object.entries(projects.profiles).map(([name, profile]) => [name, `${JSON.stringify(profile, null, 2)}\n`]),
@ -914,8 +889,9 @@ async function secrets() {
}),
];
await applyObjects(objects);
console.log('Secrets and ConfigMaps are in place: three settings files, keys, POC files and the Caddyfile.');
return createHash('sha256').update(JSON.stringify(objects)).digest('hex').slice(0, 32);
const digest = createHash('sha256').update(JSON.stringify(objects)).digest('hex').slice(0, 32);
console.log(`Secrets and ConfigMaps are in place: settings, keys, POC files and the Caddyfile (settings ${digest}).`);
return digest;
}
async function apply(...flags) {
@ -978,33 +954,13 @@ async function status() {
}
}
// ------------------------------------------------------------------------------ migration
// ------------------------------------------------------------------------------- restore
const MAC = {
postgres: 'wishforge-dev-postgres-1',
minio: 'wishforge-dev-minio-1',
temporal: 'wishforge-dev-temporal-1',
};
const docker = (args, options = {}) =>
capture('docker', ['--context', 'colima', ...args], {
env: { PATH: process.env.PATH, HOME: os.homedir() },
...options,
});
const macSql = async sql => {
const result = await docker(['exec', MAC.postgres, 'psql', '-U', 'wishforge', '-d', 'wishforge', '-At', '-c', sql]);
if (result.code !== 0) throw new Error(`Reading the Mac database failed: ${result.stderr.trim()}`);
return result.stdout.trim();
};
/** What the Mac is still doing: a migration starts only when nothing runs, deploys or is dispatched. */
/** What the cluster is still doing: projects are replaced and data restored only when nothing runs. */
export const UNFINISHED_SQL = `select kind || ' ' || id || ' ' || coalesce(data->>'status', '') from entities where
(kind = 'run' and data->>'status' in ('queued', 'running')) or
(kind = 'runner_task' and data->>'status' in ('pending', 'dispatching', 'unknown')) or
(kind = 'release' and data->>'status' in ('ready', 'deploying', 'unknown'))`;
async function macService() {
const result = await capture(process.execPath, [path.join(ROOT, 'scripts/local-poc-service.mjs'), 'status']);
return JSON.parse(result.stdout);
}
async function waitFor(description, check, minutes = 10) {
for (const until = Date.now() + minutes * 60_000; Date.now() < until; await delay(5000)) if (await check()) return;
throw new Error(`Timed out waiting for ${description}`);
@ -1014,7 +970,7 @@ const podGone = async name =>
await kubectl(['-n', NAMESPACE, 'get', 'pod', name, '--ignore-not-found', '-o', 'name'], { allowFailure: true })
).stdout.trim() === '';
/** A restricted Pod with the data volume, to put files in place while the WishForge Pod is stopped. */
/** A restricted Pod with the data volume, to put a snapshot's files in place while the WishForge Pod is stopped. */
const restorePod = image => ({
apiVersion: 'v1',
kind: 'Pod',
@ -1057,77 +1013,70 @@ const restorePod = image => ({
},
});
const SNAPSHOT = /^\d{8}T\d{6}Z$/;
/**
* Replaces the cluster's data, with the WishForge Pod stopped: each tarball's directory, the
* PostgreSQL and Temporal directories and the run workspaces are emptied first. The database is
* then restored from `dump` into a fresh PostgreSQL started with nothing else running, and the Pod
* starts only if every table's row count is `counts`.
* Puts snapshot `name`'s files in place, in the restore Pod: each file is checked against the
* manifest first, and only then are the directories emptied and unpacked. `archives` are the
* directories the snapshot holds; the others are emptied. /tmp/verified and /tmp/restored mark how
* far it got, since kubectl can lose an exec's status.
*/
async function replaceData({ image, tarballs, dump, counts, runner }) {
// ---- put the files in place while the WishForge Pod is stopped
export function restoreScript(name, archives = ARCHIVES) {
if (!SNAPSHOT.test(name)) throw new Error(`Not a snapshot name: ${name}`);
if (archives.some(archive => !ARCHIVES.includes(archive))) throw new Error('Unknown archive');
const snapshot = `/mnt/data/backups/${name}`;
const reset = ['postgres', 'temporal', 'exec/runs', ...ARCHIVES];
return [
'set -e',
`node /opt/wishforge/scripts/cluster-poc-backup.mjs verify '${snapshot}'`,
'touch /tmp/verified',
'cd /mnt/data',
...reset.map(dir => `mkdir -p '${dir}' && find '${dir}' -mindepth 1 -delete && chmod 0700 '${dir}'`),
...archives.map(dir => `tar -xzf '${snapshot}/${dir}.tgz' -C '${dir}' --no-same-owner --no-same-permissions`),
`cp '${snapshot}/temporal.db' temporal/temporal.db && chmod 0600 temporal/temporal.db`,
'touch /tmp/restored',
'echo restored: $(find minio -type f | wc -l) object storage files, $(find git -type f | wc -l) Git files',
].join('\n');
}
/**
* Replaces the cluster's data with a snapshot on the node, with the WishForge Pod stopped. The
* database is then restored into a fresh PostgreSQL started with nothing else running, by the
* backup container, which reads the snapshot and reaches the database on loopback; the Pod starts
* only if every table's row count is `counts`.
*/
async function replaceData({ image, snapshot, archives, counts, runner }) {
const script = restoreScript(snapshot, archives);
if (await exists('statefulset', 'wishforge', NAMESPACE)) {
await kubectl(['-n', NAMESPACE, 'scale', 'statefulset', 'wishforge', '--replicas=0']);
await waitFor('the WishForge Pod to stop', () => podGone('wishforge-0'));
}
await kubectl(['-n', NAMESPACE, 'delete', 'pod', 'wishforge-restore', '--ignore-not-found', '--wait=true']);
await kubectl(['create', '-f', '-'], { input: JSON.stringify(restorePod(image)) });
const inRestorePod = (script, options) =>
kubectl(['-n', NAMESPACE, 'exec', 'wishforge-restore', '-c', 'restore', '--', '/bin/sh', '-c', script], options);
// This Mac's link to the cluster drops connections: each upload is checked, and tried again.
const upload = async file => {
const target = `/tmp/${path.basename(file)}`;
const sum = await sha256File(file);
for (let attempt = 1; ; attempt++) {
await kubectl(['-n', NAMESPACE, 'cp', file, `wishforge-restore:${target}`, '-c', 'restore'], {
timeoutSeconds: 600,
allowFailure: true,
});
const check = await inRestorePod(`sha256sum '${target}' 2>/dev/null`, { allowFailure: true });
if (check.stdout.startsWith(sum)) return;
if (attempt === 5) throw new Error(`Uploading ${path.basename(file)} failed ${attempt} times`);
await delay(3000 * attempt);
}
};
let changed = false;
const inRestorePod = (command, options) =>
kubectl(['-n', NAMESPACE, 'exec', 'wishforge-restore', '-c', 'restore', '--', '/bin/sh', '-c', command], options);
try {
await kubectl(['-n', NAMESPACE, 'wait', '--for=condition=Ready', 'pod/wishforge-restore', '--timeout=600s'], {
timeoutSeconds: 620,
});
const reset = [...new Set(['postgres', 'temporal', 'exec/runs', ...Object.keys(tarballs)])];
const script = [
'set -e',
'cd /mnt/data',
...reset.map(dir => `mkdir -p '${dir}' && find '${dir}' -mindepth 1 -delete && chmod 0700 '${dir}'`),
...Object.keys(tarballs).map(
dir => `tar -xzf '/tmp/${dir.replaceAll('/', '-')}.tgz' -C '${dir}' --no-same-owner --no-same-permissions`,
),
'touch /tmp/restored',
'echo restored: $(find minio -type f | wc -l) object storage files, $(find git -type f | wc -l) Git files',
].join('\n');
for (const file of Object.values(tarballs)) await upload(file);
changed = true;
// kubectl can lose an exec's status: the script's own mark says whether it finished.
const restored = await inRestorePod(script, { timeoutSeconds: 600, allowFailure: true });
const finished = await inRestorePod('test -f /tmp/restored && echo finished', { allowFailure: true });
if (finished.stdout.trim() !== 'finished')
const restored = await inRestorePod(script, { timeoutSeconds: 1200, allowFailure: true });
const marks = (await inRestorePod('ls /tmp', { allowFailure: true })).stdout;
if (!/\brestored\b/.test(marks)) {
if (/\bverified\b/.test(marks))
throw new Error(
`Unpacking snapshot ${snapshot} did not finish (${restored.stderr.trim().slice(0, 400)}); the Pod stays stopped, run this again`,
);
await kubectl(['-n', NAMESPACE, 'scale', 'statefulset', 'wishforge', '--replicas=1'], { allowFailure: true });
throw new Error(
`Unpacking the data did not finish (${restored.stderr.trim().slice(0, 400)}); the Pod stays stopped, run this again`,
`Snapshot ${snapshot} did not pass its check (${restored.stderr.trim().slice(0, 400)}); nothing was changed and the WishForge Pod is starting again`,
);
console.log(restored.stdout.trim() || 'Data unpacked.');
} catch (error) {
if (changed) throw error;
await kubectl(['-n', NAMESPACE, 'delete', 'pod', 'wishforge-restore', '--ignore-not-found', '--wait=true'], {
allowFailure: true,
});
await kubectl(['-n', NAMESPACE, 'scale', 'statefulset', 'wishforge', '--replicas=1'], { allowFailure: true });
throw new Error(`${error.message}\nNothing was changed; the WishForge Pod is starting again.`);
}
console.log(restored.stdout.trim() || 'Files restored.');
} finally {
await kubectl(['-n', NAMESPACE, 'delete', 'pod', 'wishforge-restore', '--ignore-not-found', '--wait=true'], {
allowFailure: true,
});
}
// ---- the database, into a fresh PostgreSQL started with nothing else running
await apply('--maintenance');
await kubectl(['-n', NAMESPACE, 'scale', 'statefulset', 'wishforge', '--replicas=1']);
await waitFor('PostgreSQL in the cluster', async () => {
@ -1148,26 +1097,16 @@ async function replaceData({ image, tarballs, dump, counts, runner }) {
'-d',
'wishforge',
],
{
allowFailure: true,
},
{ allowFailure: true },
);
return ready.code === 0;
});
const restored = await pipeFile(
'kubectl',
[
'--kubeconfig',
KUBECONFIG,
'-n',
NAMESPACE,
'exec',
'-i',
'wishforge-0',
'-c',
'postgres',
'--',
// kubectl can lose an exec's status: the row counts say whether the restore took.
const restored = await kubectl(
backupExec([
'pg_restore',
'-h',
'127.0.0.1',
'-U',
'wishforge',
'-d',
@ -1175,10 +1114,10 @@ async function replaceData({ image, tarballs, dump, counts, runner }) {
'--no-owner',
'--no-privileges',
'--exit-on-error',
],
{ from: dump, env: { PATH: process.env.PATH, HOME: os.homedir() } },
`/var/lib/wishforge/backups/${snapshot}/postgres.dump`,
]),
{ timeoutSeconds: 1800, allowFailure: true },
);
if (restored.code !== 0) throw new Error(`pg_restore failed: ${restored.stderr.slice(0, 2000)}`);
const clusterCounts = (
await kubectl([
'-n',
@ -1199,104 +1138,13 @@ async function replaceData({ image, tarballs, dump, counts, runner }) {
])
).stdout.trim();
if (clusterCounts !== counts)
throw new Error(`Row counts differ after the restore:\nSource: ${counts}\nCluster: ${clusterCounts}`);
throw new Error(
`Row counts differ after the restore (${restored.stderr.trim().slice(0, 800)}):\nSnapshot: ${counts}\nCluster: ${clusterCounts}`,
);
console.log(`Database restored; row counts match: ${counts}`);
await apply(...(runner ? ['--runner'] : []));
}
/**
* Copies the Mac's WishForge into the cluster. --rehearsal copies a snapshot while the Mac keeps
* serving: the database, Git, POC records and object storage, but an empty Temporal and a Runner
* that stays idle, so the copy can act on nothing. --final copies everything after the Mac's
* service is stopped, including Temporal, and starts the Runner. Either replaces what the cluster
* held; Claude homes and the cluster's administrator credentials are kept.
*/
async function migrate(mode) {
if (!['--rehearsal', '--final'].includes(mode)) throw new Error('Usage: cluster-poc.mjs migrate --rehearsal|--final');
const final = mode === '--final';
await assertCluster();
const image = (await imageRecord()).reference;
const unfinished = await macSql(UNFINISHED_SQL);
if (unfinished) throw new Error(`The Mac is still working; wait until these finish:\n${unfinished}`);
if (final && (await macService()).state !== 'stopped')
throw new Error('Stop the Mac service first (node scripts/local-poc-service.mjs stop)');
const directory = await mkdtemp(path.join(os.tmpdir(), 'wishforge-migrate-'));
await chmod(directory, 0o700);
try {
// ---- export from the Mac
// The Mac may still write during a rehearsal: a dump counts only when no row changed around it.
const dump = path.join(directory, 'wishforge.dump');
let macCounts;
for (let attempt = 1; !macCounts; attempt++) {
const before = await macSql(COUNTS_SQL);
const dumped = await pipeFile(
'docker',
[
'--context',
'colima',
'exec',
MAC.postgres,
'pg_dump',
'-U',
'wishforge',
'-d',
'wishforge',
'-Fc',
'--no-owner',
'--no-privileges',
],
{ to: dump, env: { PATH: process.env.PATH, HOME: os.homedir() } },
);
if (dumped.code !== 0) throw new Error(`pg_dump failed: ${dumped.stderr}`);
if ((await macSql(COUNTS_SQL)) === before) macCounts = before;
else if (attempt === 5)
throw new Error('The Mac database kept changing during the dump; try again when it is quiet');
}
const stopped = [];
try {
for (const service of final ? ['minio', 'temporal'] : []) {
const result = await docker(['stop', MAC[service]]);
if (result.code !== 0) throw new Error(`Stopping ${MAC[service]} failed: ${result.stderr}`);
stopped.push(service);
}
for (const [service, source] of [['minio', '/data'], ...(final ? [['temporal', '/var/lib/temporal']] : [])]) {
const copy = await docker(['cp', `${MAC[service]}:${source}/.`, path.join(directory, service)]);
if (copy.code !== 0) throw new Error(`Copying ${service} data failed: ${copy.stderr}`);
}
} finally {
for (const service of stopped) await docker(['start', MAC[service]]);
}
const archives = {
git: path.join(ROOT, '.wishforge/git'),
'runner-state': path.join(ROOT, '.wishforge/local-poc/runner-state'),
'poc-state/poc': path.join(os.homedir(), '.local/share/wishforge/poc'),
minio: path.join(directory, 'minio'),
...(final ? { temporal: path.join(directory, 'temporal') } : {}),
};
const tarballs = {};
for (const [target, source] of Object.entries(archives)) {
const file = path.join(directory, `${target.replaceAll('/', '-')}.tgz`);
const made = await capture('tar', ['--no-mac-metadata', '-czf', file, '-C', source, '.'], {
env: { PATH: process.env.PATH, COPYFILE_DISABLE: '1' },
});
if (made.code !== 0) throw new Error(`Packing ${target} failed: ${made.stderr}`);
tarballs[target] = file;
}
console.log(
`Exported from the Mac: database (${((await stat(dump)).size / 1e6).toFixed(1)} MB), ${Object.keys(tarballs).join(', ')}.`,
);
await replaceData({ image, tarballs, dump, counts: macCounts, runner: final });
console.log(
final
? 'Migration complete; the cluster now serves WishForge.'
: 'Rehearsal copy is running; its Runner stays idle.',
);
} finally {
await rm(directory, { recursive: true, force: true });
}
}
// ------------------------------------------------------------------------------ public entry
/** The load balancer's public IPv4, once CCE has assigned it; private addresses it also lists are skipped. */
@ -1373,18 +1221,11 @@ async function publicEntry(...args) {
);
}
// ---------------------------------------------------------------------------------- backups
// ------------------------------------------------------------------------------- snapshots
const BACKUPS = path.join(STATE, 'backups');
/** Snapshots this Mac keeps; the node keeps a week (cluster-poc-backup.mjs). */
export const LOCAL_KEEP = 14;
/** A snapshot this much older than the schedule says the Pod's backup is not working. */
const STALE_HOURS = 36;
export const LAUNCH_AGENT = 'cc.d-robotics.wishforge.cluster-backup';
const LAUNCH_AGENT_FILE = path.join(os.homedir(), 'Library/LaunchAgents', `${LAUNCH_AGENT}.plist`);
const backupExec = args => ['-n', NAMESPACE, 'exec', 'wishforge-0', '-c', 'backup', '--', ...args];
/**
* Copies one file out of the backup container. kubectl drops whatever of an exec's output is
* Copies one file out of one of the WishForge Pod's containers. kubectl drops whatever of an exec's output is
* still in flight when the remote command exits, so the remote side sends the file and then waits
* for this end to close its input, which happens only once every byte has arrived.
*/
@ -1440,168 +1281,41 @@ async function download(remote, local, bytes, container = 'backup') {
}
}
const xml = value => String(value).replace(/[<>&'"]/g, character => `&#${character.charCodeAt(0)};`);
/**
* The launchd job that runs `backup` every day at 10:17, after the Pod's 03:20 snapshot; a Mac
* asleep then runs it when it wakes.
* Replaces everything the cluster holds with one of the node's daily snapshots (`status` lists
* them): the database, Temporal, Git, object storage and the platform's, POC's and Runners'
* records. Starts the Runners only with --runner. The node keeps its snapshots.
*/
export function launchAgentPlist({ node, script, root, pathEnv, home, log, hour = 10, minute = 17 }) {
const string = value => `<string>${xml(value)}</string>`;
return `<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>${string(LAUNCH_AGENT)}
<key>ProgramArguments</key>
<array>${string(node)}${string(script)}${string('backup')}</array>
<key>WorkingDirectory</key>${string(root)}
<key>EnvironmentVariables</key>
<dict><key>PATH</key>${string(pathEnv)}<key>HOME</key>${string(home)}<key>WF_BACKUP_NOTIFY</key>${string('1')}</dict>
<key>StartCalendarInterval</key>
<dict><key>Hour</key><integer>${hour}</integer><key>Minute</key><integer>${minute}</integer></dict>
<key>StandardOutPath</key>${string(log)}
<key>StandardErrorPath</key>${string(log)}
</dict>
</plist>
`;
}
async function scheduleBackup(enable) {
const domain = `gui/${process.getuid()}`;
await capture('launchctl', ['bootout', `${domain}/${LAUNCH_AGENT}`]);
if (!enable) {
await rm(LAUNCH_AGENT_FILE, { force: true });
console.log('The daily copy of the cluster backup is no longer scheduled.');
return;
}
await privateState();
const log = path.join(STATE, 'backup.log');
await mkdir(path.dirname(LAUNCH_AGENT_FILE), { recursive: true });
await writeFile(
LAUNCH_AGENT_FILE,
launchAgentPlist({
node: process.execPath,
script: path.join(ROOT, 'scripts/cluster-poc.mjs'),
root: ROOT,
pathEnv: process.env.PATH,
home: os.homedir(),
log,
}),
{ mode: 0o644 },
);
const loaded = await capture('launchctl', ['bootstrap', domain, LAUNCH_AGENT_FILE]);
if (loaded.code !== 0) throw new Error(`launchctl bootstrap failed: ${loaded.stderr.trim()}`);
console.log(`Every day at 10:17 this Mac copies the newest cluster snapshot to ${BACKUPS}; log: ${log}`);
}
/**
* Copies the newest snapshot the Pod's backup container took (with --now, one it takes first) to
* this Mac, checks every file against its manifest, and keeps {@link LOCAL_KEEP}. Fails, after
* copying, when the Pod's last attempt failed or its newest snapshot is stale.
*/
/** A macOS notification, so a failing daily copy run by launchd does not go unnoticed in its log. */
export function failureNotification(message) {
const quote = value => `"${String(value).replace(/\s+/g, ' ').slice(0, 200).replace(/["\\]/g, '\\$&')}"`;
return `display notification ${quote(message)} with title ${quote('WishForge 集群备份失败')} sound name "Basso"`;
}
async function backup(...flags) {
try {
await copyBackup(...flags);
} catch (error) {
if (process.env.WF_BACKUP_NOTIFY === '1') await capture('osascript', ['-e', failureNotification(error.message)]);
throw error;
}
}
async function copyBackup(...flags) {
if (flags.length === 1 && ['--schedule', '--unschedule'].includes(flags[0]))
return scheduleBackup(flags[0] === '--schedule');
if (flags.some(flag => flag !== '--now'))
throw new Error('Usage: cluster-poc.mjs backup [--now|--schedule|--unschedule]');
await assertCluster();
const startedAt = Date.now();
// The snapshot's own record says how it went: kubectl can lose an exec's last output and status.
if (flags.includes('--now'))
await kubectl(backupExec(['node', 'scripts/cluster-poc-backup.mjs', 'once']), {
timeoutSeconds: 3600,
allowFailure: true,
});
const { manifest, last } = JSON.parse(
(await kubectl(backupExec(['node', 'scripts/cluster-poc-backup.mjs', 'latest']))).stdout,
);
if (flags.includes('--now') && !(last?.ok && Date.parse(last.at) >= startedAt))
throw new Error(`The snapshot failed${last && !last.ok ? `: ${last.error}` : ''}`);
if (!manifest)
throw new Error(`The cluster has no snapshot yet${last?.error ? `; the last attempt failed: ${last.error}` : ''}`);
await privateState();
await mkdir(BACKUPS, { recursive: true, mode: 0o700 });
const target = path.join(BACKUPS, manifest.name);
if (await stat(path.join(target, 'manifest.json')).catch(() => undefined))
console.log(`Snapshot ${manifest.name} is already on this Mac.`);
else {
const partial = path.join(BACKUPS, `.partial-${manifest.name}`);
await rm(partial, { recursive: true, force: true });
await mkdir(partial, { mode: 0o700 });
try {
for (const [file, { bytes }] of Object.entries(manifest.files))
await download(`/var/lib/wishforge/backups/${manifest.name}/${file}`, path.join(partial, file), bytes);
await writeFile(path.join(partial, 'manifest.json'), `${JSON.stringify(manifest, null, 2)}\n`, { mode: 0o600 });
await verifyFiles(partial, manifest.files);
await rename(partial, target);
} catch (error) {
await rm(partial, { recursive: true, force: true });
throw error;
}
const bytes = Object.values(manifest.files).reduce((sum, file) => sum + file.bytes, 0);
console.log(`Copied snapshot ${manifest.name} (${(bytes / 1e6).toFixed(1)} MB, verified) to ${target}`);
}
for (const entry of retention(await readdir(BACKUPS), LOCAL_KEEP).remove)
await rm(path.join(BACKUPS, entry), { recursive: true, force: true });
const problems = [];
if (last && !last.ok) problems.push(`the Pod's last snapshot attempt failed at ${last.at}: ${last.error}`);
const hours = (Date.now() - Date.parse(manifest.takenAt)) / 3600_000;
if (hours > STALE_HOURS) problems.push(`the newest snapshot is ${Math.floor(hours)} hours old`);
if (problems.length) throw new Error(`Backup needs attention: ${problems.join('; ')}`);
}
/**
* Replaces everything the cluster holds with a snapshot copied to this Mac (see `backup`): the
* database, Temporal, Git, object storage and the platform's, POC's and Runner's records. Starts
* the Runner only with --runner. The node's own snapshots are kept.
*/
async function restore(source, ...flags) {
if (!source || source.startsWith('--') || flags.some(flag => flag !== '--runner'))
throw new Error('Usage: cluster-poc.mjs restore <snapshot directory> [--runner]');
const directory = path.resolve(source);
const manifest = JSON.parse(await readFile(path.join(directory, 'manifest.json'), 'utf8'));
if (!manifest.database?.counts) throw new Error('The snapshot records no row counts to check the restore against');
for (const file of ['postgres.dump', 'temporal.db', ...ARCHIVES.map(name => `${name}.tgz`)])
if (!manifest.files[file]) throw new Error(`The snapshot has no ${file}`);
await verifyFiles(directory, manifest.files);
async function restore(name, ...flags) {
if (!name || name.startsWith('--') || flags.some(flag => flag !== '--runner'))
throw new Error('Usage: cluster-poc.mjs restore <snapshot, e.g. 20261102T192000Z> [--runner]');
restoreScript(name);
await assertCluster();
const image = (await imageRecord()).reference;
const { snapshots } = JSON.parse(
(await kubectl(backupExec(['node', 'scripts/cluster-poc-backup.mjs', 'latest']))).stdout,
);
if (!snapshots.includes(name))
throw new Error(`The node has no snapshot ${name}; it has ${snapshots.join(', ') || 'none'}`);
const remote = `/var/lib/wishforge/backups/${name}/manifest.json`;
const work = await mkdtemp(path.join(os.tmpdir(), 'wishforge-restore-'));
await chmod(work, 0o700);
try {
const temporal = path.join(work, 'temporal.tgz');
const packed = await capture('tar', ['--no-mac-metadata', '-czf', temporal, '-C', directory, 'temporal.db'], {
env: { PATH: process.env.PATH, COPYFILE_DISABLE: '1' },
});
if (packed.code !== 0) throw new Error(`Packing Temporal's database failed: ${packed.stderr}`);
const tarballs = {
...Object.fromEntries(ARCHIVES.map(name => [name, path.join(directory, `${name}.tgz`)])),
temporal,
};
console.log(`Restoring snapshot ${manifest.name} (taken ${manifest.takenAt}) …`);
const size = (await kubectl(backupExec(['/bin/sh', '-c', `wc -c < ${remote}`]))).stdout.trim();
await download(remote, path.join(work, 'manifest.json'), Number(size));
const manifest = JSON.parse(await readFile(path.join(work, 'manifest.json'), 'utf8'));
if (!manifest.database?.counts) throw new Error('The snapshot records no row counts to check the restore against');
for (const file of ['postgres.dump', 'temporal.db'])
if (!manifest.files[file]) throw new Error(`The snapshot has no ${file}`);
const archives = ARCHIVES.filter(archive => manifest.files[`${archive}.tgz`]);
console.log(`Restoring snapshot ${name} (taken ${manifest.takenAt}) …`);
await replaceData({
image,
tarballs,
dump: path.join(directory, 'postgres.dump'),
snapshot: name,
archives,
counts: manifest.database.counts,
runner: flags.includes('--runner'),
});
console.log(`Restored snapshot ${manifest.name}${flags.includes('--runner') ? '' : '; the Runner stays idle'}.`);
console.log(`Restored snapshot ${name}${flags.includes('--runner') ? '' : '; the Runners stay idle'}.`);
} finally {
await rm(work, { recursive: true, force: true });
}
@ -1831,8 +1545,6 @@ const COMMANDS = {
'gateway-key': gatewayKey,
secrets,
apply,
migrate,
backup,
restore,
'fanyu-access': fanyuAccess,
projects: projectsCommand,

View File

@ -12,7 +12,7 @@ import {
publicService,
clusterEnvironments,
clusterKubeconfig,
clusterPocFiles,
publicIPv4,
envFile,
fanyuKubeconfig,
fanyuProfile,
@ -107,22 +107,23 @@ test('settings files hold one value per line and read back the same', () => {
assert.throws(() => envFile({ A: "it's" }), /single-line/);
});
test('POC files keep their bindings and only move to the containers’ paths', () => {
const files = clusterPocFiles({
profile: { id: 'droboticflow-poc', kubeconfig: '/Users/x/poc.config', namespaceUid: 'u' },
binding: { projectId: 'p', profileFile: '/Users/x/p.json', stateDirectory: '/Users/x/s', gitHome: '/Users/x' },
});
assert.deepEqual(files.profile, {
id: 'droboticflow-poc',
kubeconfig: '/etc/wishforge/config/kubeconfig',
namespaceUid: 'u',
});
assert.equal(files.binding.projectId, 'p');
assert.equal(files.binding.gitHome, '/home/wishforge');
const config = JSON.parse(clusterKubeconfig('Q0E='));
assert.equal(config.clusters[0].cluster.server, 'https://120.48.181.105:6443');
assert.deepEqual(config.users[0].user, { tokenFile: '/var/run/wishforge/kube/token' });
assert.throws(() => clusterKubeconfig('not base64!'), /CA/);
test('the public entry takes a public IPv4 only, never a URL, private or documentation address', () => {
assert.equal(publicIPv4('120.48.164.20'), '120.48.164.20');
for (const value of [
'https://8.8.8.8',
'localhost',
'127.0.0.1',
'10.0.0.1',
'192.168.1.2',
'172.16.2.1',
'100.64.1.2',
'169.254.169.254',
'203.0.113.1',
'198.51.100.1',
'224.0.0.1',
'8.8.8.8\n',
])
assert.throws(() => publicIPv4(value), value);
});
test('the public entry serves a Let’s Encrypt IP certificate and redirects to the standard HTTPS port', () => {
@ -210,7 +211,7 @@ test('the WishForge Pod is restricted everywhere and only Caddy listens beyond l
);
assert.deepEqual(
backup.volumeMounts.filter(mount => mount.readOnly).map(mount => mount.subPath),
['git', 'minio', 'platform-state', 'poc-state', 'runner-state', 'temporal'],
['git', 'minio', 'platform-state', 'poc-state', 'runner-state', 'runner-state-fanyu', 'temporal'],
);
assert.deepEqual([...new Set(mounts('backup'))], ['data', 'backup-tmp']);
assert.equal(backup.env.find(item => item.name === 'WF_IMAGE').value, IMAGE);

View File

@ -1,248 +0,0 @@
#!/usr/bin/env node
import { constants } from 'node:fs';
import { lstat, mkdir, open, readFile, writeFile } from 'node:fs/promises';
import { randomUUID } from 'node:crypto';
import { isIP } from 'node:net';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { parseEnv } from 'node:util';
import { replaceEnvFile, setEnvValues } from './env-file.mjs';
export const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const KEY = 'WF_LOCAL_ALLOWED_ORIGINS';
export const entryRoot = (root = ROOT) => path.join(root, '.wishforge/local-entry');
export function publicIPv4(value) {
if (isIP(value) !== 4) throw new Error('A public IPv4 address is required');
const [a, b, c] = value.split('.').map(Number);
if (
a === 0 ||
a === 10 ||
a === 127 ||
a >= 224 ||
(a === 100 && b >= 64 && b <= 127) ||
(a === 169 && b === 254) ||
(a === 172 && b >= 16 && b <= 31) ||
(a === 192 && (b === 168 || b === 0 || (b === 88 && c === 99))) ||
(a === 198 && (b === 18 || b === 19 || (b === 51 && c === 100))) ||
(a === 203 && b === 0 && c === 113)
)
throw new Error('The entry must use a publicly routable IPv4 address');
return value;
}
export async function privateDirectory(directory) {
await mkdir(directory, { recursive: true, mode: 0o700 });
const stat = await lstat(directory);
if (!stat.isDirectory() || stat.isSymbolicLink() || stat.uid !== process.getuid() || stat.mode & 0o077)
throw new Error('The entry state directory must be operator-owned and private');
}
export async function readPrivate(file) {
const handle = await open(file, constants.O_RDONLY | constants.O_NOFOLLOW);
try {
const stat = await handle.stat();
if (!stat.isFile() || stat.uid !== process.getuid() || stat.mode & 0o077)
throw new Error('Configuration must be an operator-owned private file');
return await handle.readFile('utf8');
} finally {
await handle.close();
}
}
function originsIn(source) {
const env = parseEnv(source);
if (
env.WF_LOCAL_ACCESS !== 'true' ||
env.WF_WEB_SERVE_MODE !== 'production' ||
env.WF_DEPLOYMENT_MODE !== 'development'
)
throw new Error('An authenticated production Web build in local development mode is required');
const origins = (env[KEY] ?? '')
.split(',')
.map(value => value.trim())
.filter(Boolean);
for (const origin of origins) {
const url = new URL(origin);
if (!['http:', 'https:'].includes(url.protocol) || url.origin !== origin || url.username || url.password)
throw new Error('Existing allowed origins must be exact HTTP(S) origins');
}
if (!origins.length) throw new Error('Keep the existing local entry origins configured');
return origins;
}
export function entryPlan(source, address) {
const origin = `https://${publicIPv4(address)}`;
const previous = originsIn(source);
const origins = [...new Set([...previous, origin])];
return {
origin,
previous,
origins,
added: !previous.includes(origin),
source: setEnvValues(source, { [KEY]: origins.join(',') }),
};
}
/** Preview is read-only. Applying only adds this origin; its receipt can remove only that addition. */
export async function configureEntry({ root = ROOT, address, apply = false, revert }) {
const file = path.join(root, '.env.local');
const before = await readPrivate(file);
if (revert) {
const receipts = path.join(entryRoot(root), 'receipts');
const receiptFile = path.resolve(revert);
if (path.dirname(receiptFile) !== receipts) throw new Error('Use a receipt from this entry state directory');
const receipt = JSON.parse(await readPrivate(receiptFile));
if (
receipt.version !== 1 ||
receipt.added !== true ||
receipt.origin !== `https://${publicIPv4(new URL(receipt.origin).hostname)}`
)
throw new Error('The receipt does not describe an origin this tool added');
const previous = originsIn(before);
const origins = previous.filter(value => value !== receipt.origin);
const changed = origins.length !== previous.length;
if (apply && changed) await replaceEnvFile(file, before, setEnvValues(before, { [KEY]: origins.join(',') }));
return {
origin: receipt.origin,
removed: changed,
applied: apply && changed,
origins,
restartRequired: apply && changed,
};
}
const { source: next, ...summary } = entryPlan(before, address);
if (!apply || !summary.added) return { ...summary, applied: false };
const state = entryRoot(root);
await privateDirectory(state);
const receipts = path.join(state, 'receipts');
await privateDirectory(receipts);
const id = randomUUID();
await writeFile(path.join(receipts, `${id}.env`), before, { flag: 'wx', mode: 0o600 });
const receipt = path.join(receipts, `${id}.json`);
await writeFile(receipt, JSON.stringify({ version: 1, origin: summary.origin, added: true }, null, 2), {
flag: 'wx',
mode: 0o600,
});
await replaceEnvFile(file, before, next);
return { ...summary, applied: true, receipt, restartRequired: true };
}
/** All user HTTPS terminates on this Mac; the cluster transports opaque TCP. */
export function entryConfigs(address, state, staging = false) {
const ip = publicIPv4(address);
const ca = `https://acme${staging ? '-staging' : ''}-v02.api.letsencrypt.org/directory`;
const caddy = `{
admin off
skip_install_trust
default_bind 127.0.0.1
http_port 18080
https_port 18443
default_sni ${ip}
auto_https disable_redirects
servers {
protocols h1 h2
}
}
http://${ip}:18080 {
bind 127.0.0.1
@safe method GET HEAD
redir @safe https://${ip}{uri} 308
respond 405
}
https://${ip}:18443 {
bind 127.0.0.1
tls {
issuer acme {
dir ${ca}
test_dir ${ca}
profile shortlived
disable_tlsalpn_challenge
alt_http_port 18080
}
}
reverse_proxy 127.0.0.1:3000 {
header_up X-Forwarded-Proto https
header_up -Forwarded
}
}
`;
const file = name => JSON.stringify(path.join(state, 'mtls', name));
const frpc = `serverAddr = "${ip}"
serverPort = 7000
loginFailExit = false
transport.protocol = "tcp"
transport.tls.enable = true
transport.tls.serverName = "wishforge-entry.internal"
transport.tls.certFile = ${file('client.crt')}
transport.tls.keyFile = ${file('client.key')}
transport.tls.trustedCaFile = ${file('ca.crt')}
auth.method = "token"
auth.additionalScopes = ["HeartBeats", "NewWorkConns"]
auth.tokenSource.type = "file"
auth.tokenSource.file.path = ${file('token')}
log.to = "console"
log.level = "info"
log.disablePrintColor = true
[[proxies]]
name = "wishforge-http"
type = "tcp"
localIP = "127.0.0.1"
localPort = 18080
remotePort = 18080
[[proxies]]
name = "wishforge-https"
type = "tcp"
localIP = "127.0.0.1"
localPort = 18443
remotePort = 18443
`;
return { caddy, frpc };
}
export async function writeEntryConfigs({ root = ROOT, address, staging = false }) {
const state = entryRoot(root);
await privateDirectory(state);
const configs = entryConfigs(address, state, staging);
for (const [name, content] of [
['Caddyfile', configs.caddy],
['frpc.toml', configs.frpc],
]) {
const file = path.join(state, name);
try {
const before = await readPrivate(file);
await replaceEnvFile(file, before, content);
} catch (error) {
if (error.code !== 'ENOENT') throw error;
await writeFile(file, content, { flag: 'wx', mode: 0o600 });
}
}
const settings = path.join(state, 'entry.json');
const content = JSON.stringify({ version: 1, address: publicIPv4(address), staging }, null, 2);
try {
const before = await readPrivate(settings);
await replaceEnvFile(settings, before, content);
} catch (error) {
if (error.code !== 'ENOENT') throw error;
await writeFile(settings, content, { flag: 'wx', mode: 0o600 });
}
return { address, staging, state };
}
if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
const args = process.argv.slice(2);
try {
let result;
if (args[0] === '--address' && args[1] && args.length <= 3 && (!args[2] || args[2] === '--apply'))
result = await configureEntry({ address: args[1], apply: args[2] === '--apply' });
else if (args[0] === '--revert' && args[1] && args.length <= 3 && (!args[2] || args[2] === '--apply'))
result = await configureEntry({ revert: args[1], apply: args[2] === '--apply' });
else if (args[0] === '--config' && args[1] && args.length <= 3 && (!args[2] || args[2] === '--staging'))
result = await writeEntryConfigs({ address: args[1], staging: args[2] === '--staging' });
else
throw new Error(
'Usage: configure-local-entry.mjs --address <public-ip> [--apply] | --revert <receipt> [--apply] | --config <public-ip> [--staging]',
);
console.log(JSON.stringify(result, null, 2));
} catch (error) {
console.error(error.message);
process.exitCode = 1;
}
}

View File

@ -1,115 +0,0 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { chmod, lstat, mkdtemp, readFile, rm, symlink, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import {
configureEntry,
entryConfigs,
entryPlan,
entryRoot,
publicIPv4,
writeEntryConfigs,
} from './configure-local-entry.mjs';
const IP = '120.48.164.20';
const SOURCE = [
'WF_DEPLOYMENT_MODE=development',
'WF_LOCAL_ACCESS=true',
'WF_WEB_SERVE_MODE=production',
'WF_LOCAL_ALLOWED_ORIGINS=http://localhost:3000,http://127.0.0.1:3000,https://previous.trycloudflare.com',
'WF_DEV_AUTH_TOKEN=fixture-private-value',
'',
].join('\n');
async function fixture(t) {
const root = await mkdtemp(path.join(os.tmpdir(), 'wf-entry-config-test-'));
t.after(() => rm(root, { recursive: true, force: true }));
await writeFile(path.join(root, '.env.local'), SOURCE, { mode: 0o600 });
return root;
}
test('only accepts public IPv4, never a URL, private or documentation address', () => {
assert.equal(publicIPv4(IP), IP);
for (const value of [
'https://8.8.8.8',
'localhost',
'127.0.0.1',
'10.0.0.1',
'192.168.1.2',
'172.16.2.1',
'100.64.1.2',
'169.254.169.254',
'203.0.113.1',
'198.51.100.1',
'224.0.0.1',
'::1',
'8.8.8.8\n',
])
assert.throws(() => publicIPv4(value));
});
test('previews without writing, retains old origins and never returns private environment text', async t => {
const root = await fixture(t);
const result = await configureEntry({ root, address: IP });
assert.equal(result.applied, false);
assert.equal(await readFile(path.join(root, '.env.local'), 'utf8'), SOURCE);
assert.ok(result.origins.includes('https://previous.trycloudflare.com'));
assert.ok(!JSON.stringify(result).includes('fixture-private-value'));
await assert.rejects(lstat(entryRoot(root)), { code: 'ENOENT' });
assert.throws(() => entryPlan(SOURCE.replace('WF_LOCAL_ACCESS=true', 'WF_LOCAL_ACCESS=false'), IP));
assert.throws(() => entryPlan(SOURCE.replace('WF_WEB_SERVE_MODE=production', 'WF_WEB_SERVE_MODE=development'), IP));
});
test('applies once and reverts only its addition, preserving later changes', async t => {
const root = await fixture(t);
const result = await configureEntry({ root, address: IP, apply: true });
assert.equal(result.applied, true);
assert.equal((await lstat(result.receipt)).mode & 0o777, 0o600);
const file = path.join(root, '.env.local');
const applied = await readFile(file, 'utf8');
assert.ok(applied.includes('fixture-private-value'));
assert.ok(applied.includes(`https://${IP}`));
assert.equal((await configureEntry({ root, address: IP, apply: true })).applied, false);
await writeFile(file, `${applied.replace(`https://${IP}`, `https://${IP},https://later.example`)}OTHER=new\n`);
const preview = await configureEntry({ root, revert: result.receipt });
assert.equal(preview.applied, false);
const reverted = await configureEntry({ root, revert: result.receipt, apply: true });
assert.equal(reverted.applied, true);
const after = await readFile(file, 'utf8');
assert.ok(!after.includes(IP));
assert.ok(after.includes('https://later.example'));
assert.ok(after.includes('https://previous.trycloudflare.com'));
assert.ok(after.includes('OTHER=new'));
assert.equal((await lstat(file)).mode & 0o777, 0o600);
await assert.rejects(configureEntry({ root, revert: path.join(root, 'elsewhere.json'), apply: true }));
});
test('refuses shared or symbolic-link environment files', async t => {
const root = await fixture(t);
const file = path.join(root, '.env.local');
await chmod(file, 0o644);
await assert.rejects(configureEntry({ root, address: IP, apply: true }));
await rm(file);
await writeFile(path.join(root, 'elsewhere'), SOURCE, { mode: 0o600 });
await symlink(path.join(root, 'elsewhere'), file);
await assert.rejects(configureEntry({ root, address: IP, apply: true }));
});
test('generates fixed loopback targets, explicit IP ACME and distinct staging storage settings', async t => {
const root = await fixture(t);
const result = await writeEntryConfigs({ root, address: IP, staging: true });
const { caddy, frpc } = entryConfigs(IP, result.state, true);
assert.match(caddy, /default_bind 127\.0\.0\.1/);
assert.match(caddy, new RegExp(`default_sni ${IP.replaceAll('.', '\\.')}`));
assert.match(caddy, /acme-staging-v02/);
assert.match(caddy, /profile shortlived/);
assert.match(caddy, /disable_tlsalpn_challenge/);
assert.match(caddy, /auto_https disable_redirects/);
assert.match(caddy, /reverse_proxy 127\.0\.0\.1:3000/);
assert.match(caddy, /header_up X-Forwarded-Proto https/);
assert.ok(!caddy.includes('tls internal'));
assert.ok(!caddy.includes('strict_sni_host'));
assert.match(frpc, /transport\.tls\.trustedCaFile/);
assert.match(frpc, /transport\.tls\.certFile/);
assert.match(frpc, /auth\.tokenSource\.file\.path/);
assert.equal((frpc.match(/\[\[proxies\]\]/g) ?? []).length, 2);
for (const unsafe of ['4000', '4300', 'visitors', 'plugins', 'webServer']) assert.ok(!frpc.includes(unsafe));
await writeEntryConfigs({ root, address: IP, staging: false });
assert.ok(!(await readFile(path.join(result.state, 'Caddyfile'), 'utf8')).includes('acme-staging'));
assert.equal((await lstat(path.join(result.state, 'entry.json'))).mode & 0o777, 0o600);
});

View File

@ -1,305 +0,0 @@
#!/usr/bin/env node
// A separate LaunchAgent for the public entry. Importing this module starts nothing.
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, lstatSync, readFileSync, renameSync, writeFileSync } from 'node:fs';
import { readdir, readFile } from 'node:fs/promises';
import { homedir } from 'node:os';
import { X509Certificate } from 'node:crypto';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { setTimeout as delay } from 'node:timers/promises';
import { ROOT, entryRoot, entryConfigs, privateDirectory, readPrivate, publicIPv4 } from './configure-local-entry.mjs';
import { minimalEnvironment, rotateAppend } from './local-poc-service.mjs';
export const LABEL = 'cc.d-robotics.wishforge.local-entry';
const STATE = entryRoot();
const NODE = '/opt/homebrew/bin/node';
const DOMAIN = `gui/${process.getuid?.()}`;
const SERVICE = `${DOMAIN}/${LABEL}`;
const PLIST = path.join(homedir(), 'Library/LaunchAgents', `${LABEL}.plist`);
const STATE_FILE = path.join(STATE, 'service-state.json');
const log = (name, message) => rotateAppend(path.join(STATE, 'logs', `${name}.log`), message);
export function entryEnvironment(state, staging, network = {}) {
const proxies = Object.fromEntries(
['HTTP_PROXY', 'HTTPS_PROXY', 'NO_PROXY'].filter(key => network[key]).map(key => [key, network[key]]),
);
return {
...minimalEnvironment(state),
TMPDIR: path.join(state, 'tmp'),
XDG_DATA_HOME: path.join(state, staging ? 'caddy-staging' : 'caddy-production'),
XDG_CONFIG_HOME: path.join(state, 'caddy-config'),
...proxies,
};
}
export function entryCommands(state) {
return [
{
name: 'caddy',
binary: path.join(state, 'bin/caddy'),
args: ['run', '--config', path.join(state, 'Caddyfile'), '--adapter', 'caddyfile'],
},
{ name: 'frpc', binary: path.join(state, 'bin/frpc'), args: ['-c', path.join(state, 'frpc.toml')] },
];
}
const xml = value =>
String(value).replace(
/[<>&"']/g,
char => ({ '<': '&lt;', '>': '&gt;', '&': '&amp;', '"': '&quot;', "'": '&apos;' })[char],
);
export function entryPlist(root = ROOT, network = {}) {
const env = entryEnvironment(entryRoot(root), false, network);
return `<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>${LABEL}</string>
<key>ProgramArguments</key><array>${[NODE, path.join(root, 'scripts/local-entry-service.mjs'), 'run'].map(value => `<string>${xml(value)}</string>`).join('')}</array>
<key>WorkingDirectory</key><string>${xml(root)}</string>
<key>EnvironmentVariables</key><dict>${Object.entries(env)
.map(([key, value]) => `<key>${key}</key><string>${xml(value)}</string>`)
.join('')}</dict>
<key>RunAtLoad</key><true/><key>KeepAlive</key><true/>
<key>ThrottleInterval</key><integer>30</integer><key>ExitTimeOut</key><integer>25</integer>
<key>Umask</key><integer>63</integer><key>ProcessType</key><string>Background</string>
<key>StandardOutPath</key><string>/dev/null</string><key>StandardErrorPath</key><string>/dev/null</string>
</dict></plist>\n`;
}
function launchctl(args, checked = true) {
const result = spawnSync('/bin/launchctl', args, { encoding: 'utf8', timeout: 35000, env: minimalEnvironment() });
if (checked && result.status !== 0) throw new Error(`launchctl ${args[0]} failed (exit ${result.status})`);
return result;
}
const registered = () => launchctl(['print', SERVICE], false).status === 0;
async function settings() {
const data = JSON.parse(await readPrivate(path.join(STATE, 'entry.json')));
if (data.version !== 1 || typeof data.staging !== 'boolean') throw new Error('Invalid entry configuration');
publicIPv4(data.address);
return data;
}
async function preflight() {
if (process.platform !== 'darwin') throw new Error('The entry service uses a macOS user LaunchAgent');
await privateDirectory(STATE);
const data = await settings();
const expected = entryConfigs(data.address, STATE, data.staging);
for (const [name, content] of [
['Caddyfile', expected.caddy],
['frpc.toml', expected.frpc],
])
if ((await readPrivate(path.join(STATE, name))) !== content)
throw new Error(`Regenerate the managed ${name} before starting`);
for (const name of ['ca.crt', 'client.crt', 'client.key', 'token']) await readPrivate(path.join(STATE, 'mtls', name));
for (const { binary } of entryCommands(STATE)) {
const stat = lstatSync(binary);
if (!stat.isFile() || stat.isSymbolicLink() || stat.uid !== process.getuid() || stat.mode & 0o022)
throw new Error('Entry binaries must be operator-owned regular files, not writable by others');
}
const env = entryEnvironment(STATE, data.staging, process.env);
for (const directory of [env.TMPDIR, env.XDG_DATA_HOME, env.XDG_CONFIG_HOME, path.join(STATE, 'logs')])
await privateDirectory(directory);
return { data, env };
}
async function validate() {
const { env } = await preflight();
for (const [binary, args] of [
[path.join(STATE, 'bin/caddy'), ['validate', '--config', path.join(STATE, 'Caddyfile'), '--adapter', 'caddyfile']],
[path.join(STATE, 'bin/frpc'), ['verify', '-c', path.join(STATE, 'frpc.toml')]],
]) {
const result = spawnSync(binary, args, { env, cwd: ROOT, timeout: 30000, encoding: 'utf8' });
if (result.status !== 0)
throw new Error(`${path.basename(binary)} configuration validation failed (exit ${result.status})`);
}
return { valid: true };
}
/** Signals only children this supervisor spawned, never PIDs loaded from a stale state file. */
export async function stopChildren(children, kill = process.kill, wait = delay) {
const active = child => child.pid && child.exitCode === null && child.signalCode === null;
const signal = name => {
for (const child of children.filter(active)) {
try {
kill(-child.pid, name);
} catch (error) {
if (error.code !== 'ESRCH') throw error;
}
}
};
signal('SIGTERM');
for (let attempt = 0; attempt < 30 && children.some(active); attempt++) await wait(200);
signal('SIGKILL');
}
async function supervise() {
const { data, env } = await preflight();
const children = [];
let stopping;
const stop = () => (stopping ??= stopChildren(children));
for (const signal of ['SIGTERM', 'SIGINT']) process.once(signal, () => void stop());
const ended = [];
for (const command of entryCommands(STATE)) {
const child = spawn(command.binary, command.args, {
cwd: ROOT,
env,
shell: false,
detached: true,
stdio: ['ignore', 'pipe', 'pipe'],
});
children.push(child);
child.stdout.on('data', bytes => log(command.name, bytes));
child.stderr.on('data', bytes => log(command.name, bytes));
ended.push(
new Promise(resolve => {
child.once('error', () => {
log('supervisor', `${command.name} failed to start\n`);
resolve();
});
child.once('close', code => {
log('supervisor', `${command.name} exited ${code}\n`);
resolve();
});
}),
);
}
writeFileSync(
STATE_FILE,
JSON.stringify({
supervisorPid: process.pid,
children: children.map((child, index) => ({ name: entryCommands(STATE)[index].name, pid: child.pid })),
address: data.address,
staging: data.staging,
startedAt: new Date().toISOString(),
}),
{ mode: 0o600 },
);
await Promise.race(ended);
await stop();
await Promise.all(ended);
}
async function certificates(directory, address) {
const found = [];
async function visit(dir, depth) {
if (depth > 8) return;
for (const entry of await readdir(dir, { withFileTypes: true }).catch(() => [])) {
const file = path.join(dir, entry.name);
if (entry.isDirectory()) await visit(file, depth + 1);
else if (entry.isFile() && entry.name.endsWith('.crt')) {
try {
const cert = new X509Certificate(await readFile(file));
if (cert.checkIP(address))
found.push({
subjectAltName: cert.subjectAltName,
issuer: cert.issuer,
validTo: cert.validTo,
expiresInHours: Math.floor((Date.parse(cert.validTo) - Date.now()) / 3600000),
});
} catch {
/* An unrelated certificate is not an entry certificate. */
}
}
}
}
await visit(directory, 0);
return found;
}
async function status() {
const printed = launchctl(['print', SERVICE], false);
const pid = printed.status === 0 ? Number(printed.stdout.match(/^\s*pid = (\d+)$/m)?.[1]) || null : null;
let data;
try {
data = await settings();
} catch {
/* Not yet configured. */
}
let saved;
try {
saved = JSON.parse(readFileSync(STATE_FILE, 'utf8'));
} catch {
/* No matching supervisor state. */
}
let httpsStatus = null;
if (pid && data && !data.staging) {
try {
httpsStatus = (
await fetch(`https://${data.address}/login`, { redirect: 'manual', signal: AbortSignal.timeout(7000) })
).status;
} catch {
/* Unavailable or an untrusted/expired certificate. */
}
}
return {
label: LABEL,
installed: existsSync(PLIST),
registered: printed.status === 0,
pid,
children: pid && saved?.supervisorPid === pid ? saved.children : [],
address: data?.address,
staging: data?.staging,
httpsStatus,
ready: Boolean(pid && httpsStatus === 200),
certificates: data ? await certificates(entryEnvironment(STATE, data.staging).XDG_DATA_HOME, data.address) : [],
};
}
export function entryAction(args) {
if (!args.length) return 'status';
if (args.length !== 1 || !['install', 'start', 'stop', 'status', 'validate', 'run'].includes(args[0]))
throw new Error('Usage: local-entry-service.mjs install|start|stop|status|validate');
return args[0];
}
export async function main(args = process.argv.slice(2)) {
const action = entryAction(args);
if (action === 'run') return supervise();
if (action === 'status') return status();
if (action === 'validate') return validate();
if (action === 'stop') {
const printed = launchctl(['print', SERVICE], false);
const pid = printed.status === 0 ? Number(printed.stdout.match(/^\s*pid = (\d+)$/m)?.[1]) || null : null;
const pids = pid ? [pid] : [];
try {
const saved = JSON.parse(await readPrivate(STATE_FILE));
if (pid && saved.supervisorPid === pid)
pids.push(
...saved.children
.filter(child => ['caddy', 'frpc'].includes(child.name) && Number.isSafeInteger(child.pid) && child.pid > 1)
.map(child => child.pid),
);
} catch {
/* No current supervisor state to wait for. */
}
const alive = value => {
try {
process.kill(value, 0);
return true;
} catch (error) {
return error.code !== 'ESRCH';
}
};
launchctl(['disable', SERVICE]);
if (registered()) launchctl(['bootout', DOMAIN, PLIST]);
for (let attempt = 0; attempt < 60 && (registered() || pids.some(alive)); attempt++) await delay(250);
return { label: LABEL, stopCompleted: !registered() && !pids.some(alive) };
}
await validate();
if (action === 'install') {
const content = entryPlist(ROOT, process.env);
if (registered()) throw new Error('Stop this entry service before installing its LaunchAgent');
const temporary = `${PLIST}.tmp`;
writeFileSync(temporary, content, { mode: 0o600 });
const result = spawnSync('/usr/bin/plutil', ['-lint', temporary], { encoding: 'utf8', timeout: 10000 });
if (result.status !== 0) throw new Error('Invalid entry LaunchAgent plist');
renameSync(temporary, PLIST);
}
if (!existsSync(PLIST)) throw new Error('Install the entry service first');
launchctl(['enable', SERVICE]);
if (!registered()) launchctl(['bootstrap', DOMAIN, PLIST]);
launchctl(['kickstart', SERVICE]);
return status();
}
if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href)
main()
.then(result => {
if (result) console.log(JSON.stringify(result, null, 2));
if (result?.stopCompleted === false) process.exitCode = 1;
})
.catch(error => {
console.error(error.message);
process.exitCode = 1;
});

View File

@ -1,148 +0,0 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtemp, readFile, rm, symlink } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { once } from 'node:events';
import { parseAllDocuments } from 'yaml';
import {
entryAction,
entryCommands,
entryEnvironment,
entryPlist,
LABEL,
stopChildren,
} from './local-entry-service.mjs';
import { isPolicyMain, policyServer, proxyDecision } from '../infra/k8s/local-entry/proxy-policy.mjs';
test('recognizes the policy entrypoint through a ConfigMap-style symlink without starting on import', async t => {
const directory = await mkdtemp(path.join(os.tmpdir(), 'wf-policy-link-'));
t.after(() => rm(directory, { recursive: true, force: true }));
const moduleUrl = new URL('../infra/k8s/local-entry/proxy-policy.mjs', import.meta.url);
const link = path.join(directory, 'proxy-policy.mjs');
await symlink(fileURLToPath(moduleUrl), link);
assert.equal(isPolicyMain(moduleUrl.href, link), true);
assert.equal(isPolicyMain(import.meta.url, link), false);
assert.equal(isPolicyMain(moduleUrl.href, path.join(directory, 'missing')), false);
});
test('entry commands and environment do not inherit platform credentials or control its services', () => {
const env = entryEnvironment('/private/state', true, {
WF_DEV_AUTH_TOKEN: 'never-copy',
HOME: '/Users/operator',
HTTPS_PROXY: 'http://127.0.0.1:8089',
});
assert.ok(!('WF_DEV_AUTH_TOKEN' in env));
assert.equal(env.HOME, '/private/state');
assert.equal(env.HTTPS_PROXY, 'http://127.0.0.1:8089');
assert.ok(env.XDG_DATA_HOME.endsWith('caddy-staging'));
assert.ok(entryEnvironment('/private/state', false).XDG_DATA_HOME.endsWith('caddy-production'));
const commands = entryCommands('/private/state');
assert.deepEqual(
commands.map(command => command.name),
['caddy', 'frpc'],
);
assert.ok(commands.every(command => command.binary.startsWith('/private/state/bin/')));
const plist = entryPlist('/work/repo&space', { WF_DEV_AUTH_TOKEN: 'never-copy' });
assert.match(plist, /repo&amp;space/);
assert.ok(plist.includes(LABEL));
assert.ok(!plist.includes('never-copy'));
assert.ok(!plist.includes('cc.d-robotics.wishforge.local-poc'));
assert.equal(entryAction([]), 'status');
assert.throws(() => entryAction(['start', '--anything']));
});
test('stops only live children it owns, never already finished processes', async () => {
const children = [
{ pid: 10001, exitCode: null, signalCode: null },
{ pid: 10002, exitCode: 0, signalCode: null },
];
const sent = [];
await stopChildren(
children,
(pid, signal) => {
sent.push([pid, signal]);
children[0].signalCode = signal;
},
async () => {},
);
assert.deepEqual(sent, [[-10001, 'SIGTERM']]);
});
test('FRP registration only accepts the two named TCP ports', () => {
const base = { op: 'NewProxy', content: { proxy_name: 'wishforge-http', proxy_type: 'tcp', remote_port: 18080 } };
assert.deepEqual(proxyDecision(base), { reject: false, unchange: true });
assert.equal(
proxyDecision({ ...base, content: { proxy_name: 'wishforge-https', proxy_type: 'tcp', remote_port: 18443 } })
.reject,
false,
);
for (const change of [
{ proxy_type: 'stcp' },
{ proxy_type: 'udp' },
{ remote_port: 4000 },
{ proxy_name: 'other' },
{ group: 'shared' },
{ custom_domains: ['anything.example'] },
])
assert.equal(proxyDecision({ ...base, content: { ...base.content, ...change } }).reject, true);
for (const request of [null, {}, { ...base, op: 'Login' }]) assert.equal(proxyDecision(request).reject, true);
});
test('policy endpoint rejects malformed input and reports health without logging content', async t => {
const server = policyServer();
server.listen(0, '127.0.0.1');
await once(server, 'listening');
t.after(() => {
server.closeAllConnections();
server.close();
});
const origin = `http://127.0.0.1:${server.address().port}`;
assert.equal((await fetch(`${origin}/healthz`)).status, 200);
assert.equal((await fetch(`${origin}/other`)).status, 403);
const invalid = await fetch(`${origin}/validate`, { method: 'POST', body: '{' });
assert.equal(invalid.status, 400);
assert.equal((await invalid.json()).reject, true);
const allowed = await fetch(`${origin}/validate?op=NewProxy`, {
method: 'POST',
body: JSON.stringify({
op: 'NewProxy',
content: { proxy_name: 'wishforge-https', proxy_type: 'tcp', remote_port: 18443 },
}),
});
assert.equal((await allowed.json()).reject, false);
});
test('cluster manifests are isolated and do not implicitly create the billable Service', async () => {
const directory = new URL('../infra/k8s/local-entry/', import.meta.url);
const raw = await readFile(new URL('gateway.yaml', directory), 'utf8');
const resources = parseAllDocuments(raw).map(document => document.toJS());
const deploy = resources.find(resource => resource.kind === 'Deployment');
assert.equal(deploy.metadata.namespace, 'wishforge-access');
assert.equal(deploy.spec.replicas, 1);
assert.equal(deploy.spec.strategy.type, 'Recreate');
const pod = deploy.spec.template.spec;
assert.equal(pod.automountServiceAccountToken, false);
assert.equal(pod.securityContext.runAsNonRoot, true);
assert.ok(
pod.containers.every(
container =>
container.image.includes('@sha256:') &&
container.securityContext.readOnlyRootFilesystem &&
!container.securityContext.allowPrivilegeEscalation,
),
);
assert.equal(pod.containers[0].readinessProbe.tcpSocket.port, 'tunnel');
assert.ok(!pod.volumes.some(volume => volume.hostPath || volume.persistentVolumeClaim));
const kustomization = await readFile(new URL('kustomization.yaml', directory), 'utf8');
assert.ok(!kustomization.includes('public-service.yaml'));
const service = parseAllDocuments(await readFile(new URL('public-service.yaml', directory), 'utf8'))[0].toJS();
assert.deepEqual(
service.spec.ports.map(port => port.port),
[80, 443, 7000],
);
assert.equal(service.metadata.annotations['service.beta.kubernetes.io/cce-elastic-ip-bandwidth-in-mbps'], '5');
assert.equal(service.metadata.annotations['service.beta.kubernetes.io/cce-elastic-ip-billing-method'], 'ByTraffic');
assert.equal(service.metadata.annotations['service.beta.kubernetes.io/cce-elastic-ip-route-type'], 'BGP');
assert.ok(!('service.beta.kubernetes.io/cce-load-balancer-id' in service.metadata.annotations));
assert.ok(!service.spec.loadBalancerIP);
assert.ok(!raw.includes('droboticflow-api'));
assert.ok(!raw.includes('wishforge.io/tenant'));
});