diff --git a/doc/drosync-Mongo与BOS隔离验收-2026-10-31.md b/doc/drosync-Mongo与BOS隔离验收-2026-10-31.md index f29ce4d5..49d31526 100644 --- a/doc/drosync-Mongo与BOS隔离验收-2026-10-31.md +++ b/doc/drosync-Mongo与BOS隔离验收-2026-10-31.md @@ -8,6 +8,8 @@ 没有部署 API/coordinator、没有升级设备、没有读取业务归档、没有创建生产补录任务,也没有执行生产 clean。公网 1.1.6 安装包和 Rust 实现保持不变。 +> 后续 [POC 上线只读预检](drosync-POC上线只读预检-2026-10-31.md) 发现:线上 Mongo 为 8.0.17-r1、存在超过 64 MiB 的业务回执样本,并有既有暂存清理 NoSuchKey 错误。需要针对这些门槛补充加固/验收;本报告的 Mongo 7、小对象隔离通过结论不扩大为生产全量放行。 + ## 1. 隔离边界与资源清理 | 项目 | 实际范围 | diff --git a/doc/drosync-POC上线只读预检-2026-10-31.md b/doc/drosync-POC上线只读预检-2026-10-31.md new file mode 100644 index 00000000..d4d29a98 --- /dev/null +++ b/doc/drosync-POC上线只读预检-2026-10-31.md @@ -0,0 +1,132 @@ +# drosync POC 上线只读预检与回退准备 + +日期:2026-10-31。候选运行时代码基线:`935f76ce`,分支 `poc`;目标仅为 POC 命名空间 `droboticflow`,公网入口 `120.48.164.19`。 + +## 结论:基础条件具备,但建议暂缓直接放行 + +API、coordinator 和 worker 都为 2/2 就绪;Mongo、审计签名配置和新集合所需权限具备;两份回退镜像清单已通过认证 GET 和摘要核验。 + +**不能把本次预检写成“全绿,可以直接批量补录”。** 上线前建议先处理三件事: + +1. **现有 coordinator 的暂存清理持续出现 `NoSuchKey`。** 候选代码未修复这一幂等性问题,升级不会自动消除它。应补齐缺失对象的安全处理、领取条件及空路径保护,并回归权限/网络错误不能被吞掉。 +2. **业务样本存在大于默认 64 MiB 分段的对象。** 之前真实 BOS 验收只覆盖小对象和显式子范围 GET,首批业务补录前应补一个大于 64 MiB 的合成对象,验证协调器按默认分段跨多个 GET 完整核验。 +3. **线上 Mongo 为 `8.0.17-r1`,前一轮隔离验收为 `7.0.40`。** 这不是已经发现版本不兼容,但不能用 Mongo 7 的写入验收替代同主版本验证;应在独立 Mongo 8.0 环境复跑旧 BSON/CAS/claim 测试,不在生产库试写。 + +此外,两个抽样 Session 的申报大小与归档回执不同,不能承诺只补 MD5 就能让它们成功。应从首批试点中单独区分,后续由数据负责人核对。 + +本轮没有构建或推送镜像、没有滚动更新、没有创建补录作业、没有修改数据库/索引、没有读取或删除业务 BOS 对象内容。只新增本地诊断输出及此报告;公网 1.1.6 客户端包保持不变。 + +## 1. 当前线上基线 + +| 组件 | 实际运行镜像摘要 | 修订 | 就绪 | 镜像内声明的源码 | +|---|---|---:|---:|---| +| `droboticflow-api` | `sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2` | 531 | 2/2 | `91641b523423665257ba30893fc423b7404301da` | +| `drosync-coordinator` | `sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692` | 50 | 2/2 | `789145750b544eeed297e129b4cb46bcdbf01233` | +| `droboticflow-worker` | `sha256:540dc05908745d69c4b0f1488d833b1add18953d21196a75aba31b7b969eb990` | 435 | 2/2 | 本次不升级 | + +镜像仓库均为 `ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/` 下对应组件名称。 + +- API 和 coordinator 四个 Pod 均为 ready,重启次数 0。 +- `/api/health` 报告 API 新镜像及 `status=ok`;两个 coordinator 的 `/health` 与 `/ready` 均正常、`enabled=true`。 +- 两个承载节点 Ready,未报告内存、磁盘或 PID 压力。这里只检查状态,不据此宣称构建/滚动扩容容量已压测。 +- 预检时没有活动 Kaniko Job。 +- 结束时再次核对三个 Deployment:镜像、generation 与预检起点相同,未发生本轮发布变更。 + +### 镜像标签偏差 + +coordinator 的实际镜像是上表的 `339695d5…`,但环境变量 `DROBOTICFLOW_IMAGE` 仍为 `b94f88ec…`。这只是旧的报告标签,不是实际运行镜像的证明。worker 也存在报告标签与实际摘要不同的情况,本次不修改 worker。 + +后续升级 coordinator 时,应同时使其镜像和报告标签指向新摘要。选择回退镜像时必须采用实际 `imageID`;原标签值作为“恢复原配置”的参考单独记录,不能拿它替代实际旧镜像。 + +## 2. 配置、权限和索引 + +- API/coordinator 都显式启用对象同步,使用相同 Mongo URI 和数据库 `poc`;只读 `buildInfo` 返回 Mongo `8.0.17-r1`。 +- Mongo URI、注册表凭据和审计私钥仅在诊断进程内存中使用,不写入报告、命令行参数或仓库。 +- 审计签名 Secret 及 key ID 存在;私钥配置可解码为 32 字节,符合当前启动校验要求。没有输出私钥。 +- 当前 Mongo 主体具有新集合的 createCollection、createIndex、find、insert、update 等有效权限;本轮没有实际执行这些写操作。 +- 现有 head 唯一索引和 Session/item 抽样所需索引存在。 +- **`objectSyncDigestBackfills` 集合尚不存在。** 新版 API/coordinator 启动时的 `EnsureIndexes` 会创建它及四个索引(id、operationId 唯一索引,状态/时间和组织/空间/时间索引)。这是未来获批部署的初始化行为,本轮未执行。 +- 查询时活动同步会话为 0,待运行补录作业为 0;对账/发布作业的 pending/running 查询也为 0。它们不包含下面提到的已提交 item 的后台暂存清理。 +- 权限判定允许创建本命名空间的构建 Job、修改两个目标 Deployment;仅做授权查询,没有创建 Job 或 patch 资源。 + +不能直接执行全量 `deploy-k8s.sh` 作为本次发布捷径:它会重新渲染全套资源,而 POC 默认配置中的对象同步开关可能与线上显式值不同。应仅更新 API 和 coordinator 的镜像/报告标签,保留其他环境、Secret 引用、worker、runner 和副本设置。 + +## 3. 遗留回执的小样本结果 + +从最新 20 个 partial Session 中取 3 个不同 mapping,每个最多抽查 5 个 `deny_archived_hash_change` 条目,共取得 **7 个匹配 head**: + +| 检查 | 观察 | +|---|---| +| MD5 | 7/7 缺少 `md5sum` 字段 | +| 旧 SHA256 | 7/7 有合法 64 字符十六进制 `sha256` | +| 对象身份 | 7/7 有 ETag、Session/item 绑定和平台版本,未标 drift | +| 核验记录 | 7/7 没有 attestation | +| VersionID | 7/7 缺失;本轮没有 HEAD 业务对象,不能据此证明其当前云端版本状态 | +| 声明与回执大小 | 5 个相同,2 个不同:7201 vs 7133、11933 vs 10417 字节 | +| 大对象 | 两个回执大小分别为 96,785,824 和 111,020,451 字节,均超过 64 MiB | +| 客户端版本 | 这些 Session 记录为 1.1.1;不等价于确认设备今天仍安装 1.1.1,试点前需现场核对 | + +上述样本印证了旧字段兼容需求,但**不证明云端字节与 SHA256 锚一致,更不能直接作为删除授权**。其中一个 Session 总计 1991 个 denied,不能将 5 个条目的抽样结果外推到整个 Session。 + +匹配的空间处于 enabled/create_only,连接凭据字段非空,BOS endpoint 为公网 `https://bj.bcebos.com`。连接级 RootPrefix 未设置,空间级前缀存在;首批作业必须明确限定 Session/目录范围和预算,禁止默认 wholeSpace 或 allowUnanchored。生产读取实际走内网还是计费公网,仍需在执行补录前确认,不能根据地域相同就假定免费。 + +抽样只读取字段形态、长度、大小和状态。提交的证据对业务标识做短 SHA256 化,不保存对象路径、内容摘要值、用户名、Mongo URI 或凭据。原始选定标识只保留在本机 Git 忽略的 `output/` 目录,用于后续经授权确定范围。 + +## 4. 发现的 cleanup 故障 + +最初只匹配结构化 error/fatal 日志格式不足以代表健康。扩展普通文本错误检查后,两个 coordinator 都出现: + +```text +[drosync-coordinator] cleanup error: item …: BOS delete object: [Code: NoSuchKey; Message: The specified key does not exist.; …] +``` + +只读抽查日志中的 4 个 item: + +- 状态均为 `committed`; +- `stagingKey` 字段存在且非空,与 `targetKey` 不同; +- `lastError` 包含 NoSuchKey,抽样时 `attemptCount=1`; +- 本次未向 BOS 发起这些对象的 HEAD、GET 或 DELETE。 + +代码路径:[cleanupStaging](../internal/objectsync/coordinator.go:567) 调用 [BOSPublisher.Delete](../internal/objectsync/bos.go:296)。删除返回错误时,在清空暂存字段和退役任务之前直接返回。处理器保存错误、增加尝试次数并更新时间;之后仍可按暂存保留期再次被领取。该路径从当前已部署版本到候选版本未被修复。 + +这是**后台暂存对象清理**,不是本次诊断执行了用户的本地 `drosync clean`,也没有证据表明正式归档被误删。 + +另一个需要覆盖的代码边界是 [Mongo cleanup 领取条件](../internal/store/object_sync.go:1056):`stagingKey: {$ne: ""}` 也可能匹配缺字段的文档。需要补非空字符串和空对象路径的回归保护;本次四个线上错误样本的字段均非空,不能把它当成这四条日志已确认的原因。 + +建议修复时只对**已确认不存在的合法暂存对象**采用幂等完成,不吞掉权限、网络、错误 bucket 等失败,不改业务回执、不手工清空生产字段。 + +## 5. 后续发布顺序——未执行 + +1. 先修复 cleanup 幂等性及相关边界,在独立 Mongo 8.0 上复跑写入/租约回归;补一个大于 64 MiB 的专用合成对象验证默认多段读取。都不得选择业务对象作破坏性测试,不在生产库试写。 +2. 固定通过验证的已提交 `poc` HEAD,再核对相对两份线上源码的生产改动。只构建 `api,drosync-coordinator`,镜像发布到 `dataflow-poc`;不构建算子、不执行 live operator sync,保留 13 个既有 live pin。 +3. 实际部署前重新查询镜像、revision、generation、后台工作和待运行补录作业。本预检是快照,不是锁;若与记录不同,先重新评估。 +4. 建议先滚动 API,确认副本、路由和新增索引正常;再次确认无意外排队补录,再滚动 coordinator。期间不创建业务补录作业。 +5. 新旧镜像及 `DROBOTICFLOW_IMAGE` 用同一实际摘要;逐组件等待 rollout 成功、核对实际 imageID/源码声明,并检查后台阶段错误,而不是只看 readiness。 +6. 镜像部署与首次业务补录分别批准。首次补录显式指定范围、对象数和字节预算;不预期小预算立刻解决一个含 1991 个拒绝项的完整 Session。 +7. 设备上的配置和 1.1.6 包不在本次服务端部署范围内;后续现场同步、clean 预览和实际删除仍分阶段确认。 + +本轮只验证旧镜像的 pull 清单可访问,未通过试推送验证未来的 push 权限,也没有实际构建新镜像。 + +## 6. 精确回退参考 + +[rollback-reference.json](drosync-poc-preflight-evidence-2026-10-31/rollback-reference.json) 保存了两个 Deployment 的容器名、实际旧镜像、原报告标签、修订、副本数与滚动策略。两份旧镜像清单均已从 CCR 认证读取,返回摘要与本地重算清单摘要一致;没有下载镜像层。 + +- API 回退镜像:`ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2`,当前 revision **531**。 +- coordinator 回退镜像:`ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692`,当前 revision **50**。 + +回退前先停止提交新的补录作业,核对在途任务;不要用数据库改状态冒充取消。通常先回退 coordinator,再回退 API。仅恢复本轮修改的容器镜像/报告标签,保护期间其他人的配置变更;若确认没有其他模板改动且历史仍保留,才可使用上述修订的 rollout undo。 + +**不回滚 Mongo 数据、不删除已核验的 MD5/attestation 或新集合。** 老版本不支持新的核验证据来源时,应保留文件,不能为恢复清理能力绕过校验。回退后未结束的新类型作业保留,后续由支持它的版本继续处理。 + +## 证据与只读证明 + +- [预检汇总与待处理项](drosync-poc-preflight-evidence-2026-10-31/summary.json) +- [服务与配置引用快照](drosync-poc-preflight-evidence-2026-10-31/workload-snapshot.json) +- [协调器探针与节点状态](drosync-poc-preflight-evidence-2026-10-31/readiness-and-rollback-probes.json) +- [认证后的回退镜像清单核验](drosync-poc-preflight-evidence-2026-10-31/rollback-registry-verification.json) +- [限量旧回执抽样](drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-snapshot.json)、[集合/任务/权限](drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-extra.json) +- [脱敏 cleanup 日志](drosync-poc-preflight-evidence-2026-10-31/coordinator-cleanup-error.json)、[对应记录形态](drosync-poc-preflight-evidence-2026-10-31/cleanup-readonly-samples.json) +- [结束时 Deployment 无变更检查](drosync-poc-preflight-evidence-2026-10-31/final-service-check.json) +- [只读 Mongo 探针源码](drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-probe.go.txt)、[证据摘要](drosync-poc-preflight-evidence-2026-10-31/evidence-sha256.json) + +探针固定数据库 `poc`,使用主节点读、每条抽样查询 2.5 秒上限及显式字段投影,只执行 find/listIndexes/listCollections/ping/connectionStatus/buildInfo 等读取;不调用项目 Store 初始化、EnsureIndexes、迁移或补录 API。Kubernetes 只读取状态、日志、探针、公共构建环境变量及权限判定。所有结论均受采样范围和时间点限制。 diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/authorization-and-logs.json b/doc/drosync-poc-preflight-evidence-2026-10-31/authorization-and-logs.json new file mode 100644 index 00000000..d850aef2 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/authorization-and-logs.json @@ -0,0 +1,36 @@ +{ + "create jobs.batch": { + "allowed": true, + "requestSucceeded": true + }, + "patch deployments.apps/droboticflow-api": { + "allowed": true, + "requestSucceeded": true + }, + "patch deployments.apps/drosync-coordinator": { + "allowed": true, + "requestSucceeded": true + }, + "podLogSamples": { + "droboticflow-api-797cf8c865-q9wjt": { + "sampledLines": 0, + "readSucceeded": true, + "failureKeywordLines": 0 + }, + "droboticflow-api-797cf8c865-vsd7g": { + "sampledLines": 0, + "readSucceeded": true, + "failureKeywordLines": 0 + }, + "drosync-coordinator-86c44cbb8-5bsgc": { + "sampledLines": 8, + "readSucceeded": true, + "failureKeywordLines": 8 + }, + "drosync-coordinator-86c44cbb8-wc2kj": { + "sampledLines": 9, + "readSucceeded": true, + "failureKeywordLines": 9 + } + } +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/cleanup-readonly-samples.json b/doc/drosync-poc-preflight-evidence-2026-10-31/cleanup-readonly-samples.json new file mode 100644 index 00000000..34d15d7a --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/cleanup-readonly-samples.json @@ -0,0 +1,76 @@ +{ + "capturedAtUTC": "2026-10-31T15:31:14Z", + "cleanupSamples": [ + { + "attemptCount": 1, + "id": "sha256:fcfff07029c9682f", + "lastErrorContainsNoSuchKey": true, + "stageEqualsTarget": false, + "stagingKeyFieldPresent": true, + "stagingKeyNonempty": true, + "status": "committed", + "targetKeyNonempty": true, + "updatedAt": "2026-10-31T14:55:03.301270559Z", + "uploadIdNonempty": false + }, + { + "attemptCount": 1, + "id": "sha256:77416cf175f25a84", + "lastErrorContainsNoSuchKey": true, + "stageEqualsTarget": false, + "stagingKeyFieldPresent": true, + "stagingKeyNonempty": true, + "status": "committed", + "targetKeyNonempty": true, + "updatedAt": "2026-10-31T14:57:30.144776374Z", + "uploadIdNonempty": false + }, + { + "attemptCount": 1, + "id": "sha256:4ed43abdc0f611dc", + "lastErrorContainsNoSuchKey": true, + "stageEqualsTarget": false, + "stagingKeyFieldPresent": true, + "stagingKeyNonempty": true, + "status": "committed", + "targetKeyNonempty": true, + "updatedAt": "2026-10-31T14:55:35.278270163Z", + "uploadIdNonempty": false + }, + { + "attemptCount": 1, + "id": "sha256:75b7adff415255d4", + "lastErrorContainsNoSuchKey": true, + "stageEqualsTarget": false, + "stagingKeyFieldPresent": true, + "stagingKeyNonempty": true, + "status": "committed", + "targetKeyNonempty": true, + "updatedAt": "2026-10-31T14:54:30.006061436Z", + "uploadIdNonempty": false + } + ], + "database": "poc", + "errors": null, + "matchCurrentNonemptyPredicate": [ + { + "id": "sha256:fcfff07029c9682f" + }, + { + "id": "sha256:77416cf175f25a84" + }, + { + "id": "sha256:4ed43abdc0f611dc" + }, + { + "id": "sha256:75b7adff415255d4" + } + ], + "maxQueryMillis": 2500, + "observedCommandCounts": { + "find": 2, + "ping": 1 + }, + "primaryReads": true, + "writesRequested": false +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/coordinator-cleanup-error.json b/doc/drosync-poc-preflight-evidence-2026-10-31/coordinator-cleanup-error.json new file mode 100644 index 00000000..da8dbe42 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/coordinator-cleanup-error.json @@ -0,0 +1,22 @@ +[ + { + "pod": "drosync-coordinator-86c44cbb8-5bsgc", + "sampledLines": 4, + "sanitizedMessage": "[drosync-coordinator] cleanup error: item sha256:4ed43abdc0f611dc: BOS delete object: [Code: NoSuchKey; Message: The specified key does not exist.; RequestId: [identifier redacted]]" + }, + { + "pod": "drosync-coordinator-86c44cbb8-5bsgc", + "sampledLines": 4, + "sanitizedMessage": "[drosync-coordinator] cleanup error: item sha256:77416cf175f25a84: BOS delete object: [Code: NoSuchKey; Message: The specified key does not exist.; RequestId: [identifier redacted]]" + }, + { + "pod": "drosync-coordinator-86c44cbb8-wc2kj", + "sampledLines": 8, + "sanitizedMessage": "[drosync-coordinator] cleanup error: item sha256:75b7adff415255d4: BOS delete object: [Code: NoSuchKey; Message: The specified key does not exist.; RequestId: [identifier redacted]]" + }, + { + "pod": "drosync-coordinator-86c44cbb8-wc2kj", + "sampledLines": 8, + "sanitizedMessage": "[drosync-coordinator] cleanup error: item sha256:fcfff07029c9682f: BOS delete object: [Code: NoSuchKey; Message: The specified key does not exist.; RequestId: [identifier redacted]]" + } +] diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/coordinator-metrics-and-history.json b/doc/drosync-poc-preflight-evidence-2026-10-31/coordinator-metrics-and-history.json new file mode 100644 index 00000000..772a13a6 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/coordinator-metrics-and-history.json @@ -0,0 +1,57 @@ +{ + "checkedAtUTC": "2026-10-31T15:19:07.214998+00:00", + "coordinatorMetrics": { + "drosync-coordinator-86c44cbb8-5bsgc": [ + "drosync_coordinator_up 1", + "drosync_coordinator_enabled 1", + "drosync_coordinator_last_cycle_timestamp_seconds 1793459947", + "drosync_coordinator_last_success_timestamp_seconds 1793459947", + "drosync_coordinator_queue_sample_timestamp_seconds 1793459934", + "drosync_coordinator_queue_age_available{queue=\"commit\"} 0", + "drosync_coordinator_queue_age_available{queue=\"planning\"} 0" + ], + "drosync-coordinator-86c44cbb8-wc2kj": [ + "drosync_coordinator_up 1", + "drosync_coordinator_enabled 1", + "drosync_coordinator_last_cycle_timestamp_seconds 1793459947", + "drosync_coordinator_last_success_timestamp_seconds 1793459947", + "drosync_coordinator_queue_sample_timestamp_seconds 1793459920", + "drosync_coordinator_queue_age_available{queue=\"commit\"} 0", + "drosync_coordinator_queue_age_available{queue=\"planning\"} 0" + ] + }, + "rolloutHistory": { + "droboticflow-api": { + "readSucceeded": true, + "revisions": [ + "521", + "522", + "523", + "524", + "525", + "526", + "527", + "528", + "529", + "530", + "531" + ] + }, + "drosync-coordinator": { + "readSucceeded": true, + "revisions": [ + "40", + "41", + "42", + "43", + "44", + "45", + "46", + "47", + "48", + "49", + "50" + ] + } + } +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/evidence-sha256.json b/doc/drosync-poc-preflight-evidence-2026-10-31/evidence-sha256.json new file mode 100644 index 00000000..6a1ff096 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/evidence-sha256.json @@ -0,0 +1,15 @@ +{ + "authorization-and-logs.json": "9e1f1730e17691a0ace0a137835ecf2ce9b7021b2c63baadfb3c56146cf3f463", + "cleanup-readonly-samples.json": "04769154a1c21c5d784a37f9b75f1a1011d7ade8ba984f7604c21eb52b512cef", + "coordinator-cleanup-error.json": "e9e411a8dd30a3bdc40534badc74b6041620036851520478b617e6a118c38ab3", + "coordinator-metrics-and-history.json": "cdbdd613db3824c0157dd5a4c91a10fe9324d1ca259120f61763c85b8ccec760", + "final-service-check.json": "4e18faf4aeb6530b4aa740124d249d6002b85517a4effa55098093609aeb78fd", + "mongo-readonly-extra.json": "db9f31e8b91d7b6fbeaa0a725357f01bb5f0ccf48cebca3ea88b8bcfd38ef456", + "mongo-readonly-probe.go.txt": "27de25ff2a3440f225f2cd89d32a0aa1fc10589e4ffa315528fb0bccecb93ba9", + "mongo-readonly-snapshot.json": "9bc09d1356f09189eab55c04f4ab06891ab47a47c0260c53528b7e19f780858d", + "readiness-and-rollback-probes.json": "4b8248e7a3a4eb6f35d41d43254d59b40a4acc0f53e529df128918f4d4852e9f", + "rollback-reference.json": "0f4c0d638a3004ed30cb610f2d4d1a712ef50806cce5ec41faa0c4b487151238", + "rollback-registry-verification.json": "26133bf8ae8efcf28b663191f595fb4c71700811f2bb97741693020d1d7191cf", + "summary.json": "a99179aa471d1fcf4dbabb703b9eba0a6af6d73ca4be6f613a2bbd72b2a1ee81", + "workload-snapshot.json": "f7ed02aaf848e83a8350c5da424c687dda9606e6d87f20c2b1abf4e23fcc87b2" +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/final-service-check.json b/doc/drosync-poc-preflight-evidence-2026-10-31/final-service-check.json new file mode 100644 index 00000000..6a12cefb --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/final-service-check.json @@ -0,0 +1,45 @@ +{ + "checkedAtUTC": "2026-10-31T15:36:36.750313+00:00", + "deployments": { + "droboticflow-api": { + "generation": 537, + "revision": "531", + "desired": 2, + "ready": 2, + "images": [ + { + "name": "api", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2" + } + ], + "unchangedSincePreflight": true + }, + "drosync-coordinator": { + "generation": 54, + "revision": "50", + "desired": 2, + "ready": 2, + "images": [ + { + "name": "coordinator", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692" + } + ], + "unchangedSincePreflight": true + }, + "droboticflow-worker": { + "generation": 443, + "revision": "435", + "desired": 2, + "ready": 2, + "images": [ + { + "name": "worker", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-worker@sha256:540dc05908745d69c4b0f1488d833b1add18953d21196a75aba31b7b969eb990" + } + ], + "unchangedSincePreflight": true + } + }, + "mutationsPerformed": false +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-extra.json b/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-extra.json new file mode 100644 index 00000000..af72de21 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-extra.json @@ -0,0 +1,45 @@ +{ + "backgroundJobs": { + "objectSyncDigestBackfills": { + "active": 0, + "collectionAbsent": true + }, + "objectSyncReconciles": { + "active": 0, + "limit": 21, + "mayBeTruncated": false + }, + "objectSyncReleases": { + "active": 0, + "limit": 21, + "mayBeTruncated": false + } + }, + "capturedAtUTC": "2026-10-31T15:44:58Z", + "collectionPresence": { + "objectSyncReconciles": true, + "objectSyncReleases": true + }, + "database": "poc", + "errors": null, + "maxQueryMillis": 2500, + "mongoVersion": "8.0.17-r1", + "newCollectionEffectivePrivileges": { + "createCollection": true, + "createIndex": true, + "find": true, + "insert": true, + "listIndexes": true, + "remove": true, + "update": true + }, + "observedCommandCounts": { + "buildInfo": 1, + "connectionStatus": 1, + "find": 2, + "listCollections": 1, + "ping": 1 + }, + "primaryReads": true, + "writesRequested": false +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-probe.go.txt b/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-probe.go.txt new file mode 100644 index 00000000..065296cc --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-probe.go.txt @@ -0,0 +1,367 @@ +// Read-only preflight: bounded, projected queries against database "poc" only. +// No project store constructor, index creation, migration, update or delete calls. +package main + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "net/url" + "os" + "reflect" + "strings" + "sync" + "time" + + "go.mongodb.org/mongo-driver/bson" + "go.mongodb.org/mongo-driver/event" + "go.mongodb.org/mongo-driver/mongo" + "go.mongodb.org/mongo-driver/mongo/options" + "go.mongodb.org/mongo-driver/mongo/readpref" +) + +type input struct { + URI string `json:"uri"` + Database string `json:"database"` + Extra bool `json:"extra"` + CleanupIDs []string `json:"cleanupIds"` +} + +var failures []map[string]any + +func failed(stage string, err error) { + row := map[string]any{"stage": stage, "type": fmt.Sprintf("%T", err), "timeout": mongo.IsTimeout(err)} + var command mongo.CommandError + if errors.As(err, &command) { + row["code"] = command.Code + } + failures = append(failures, row) +} +func text(m bson.M, key string) string { v, _ := m[key].(string); return v } +func number(value any) int64 { + switch n := value.(type) { + case int64: + return n + case int32: + return int64(n) + case float64: + return int64(n) + case int: + return int64(n) + } + return -1 +} +func shape(m bson.M, key string, want int) map[string]any { + v, exists := m[key] + r := map[string]any{"present": exists} + if !exists { + r["type"] = "missing" + return r + } + if v == nil { + r["type"] = "null" + return r + } + r["type"] = reflect.TypeOf(v).String() + if s, ok := v.(string); ok { + r["length"] = len(s) + b, e := hex.DecodeString(s) + r["validHexDigest"] = e == nil && len(b) == want + } + return r +} +func fingerprint(value string) string { + d := sha256.Sum256([]byte(value)) + return hex.EncodeToString(d[:])[:16] +} + +func main() { + var cfg input + decoder := json.NewDecoder(io.LimitReader(os.Stdin, 32768)) + decoder.DisallowUnknownFields() + if decoder.Decode(&cfg) != nil || cfg.Database != "poc" { + fmt.Fprintln(os.Stderr, "invalid POC read-only configuration") + os.Exit(2) + } + uri, e := url.Parse(cfg.URI) + if e != nil || (uri.Scheme != "mongodb" && uri.Scheme != "mongodb+srv") { + fmt.Fprintln(os.Stderr, "invalid Mongo scheme") + os.Exit(2) + } + ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second) + defer cancel() + var mu sync.Mutex + commands := map[string]int{} + monitor := &event.CommandMonitor{Started: func(_ context.Context, e *event.CommandStartedEvent) { + mu.Lock() + commands[e.CommandName]++ + mu.Unlock() + }} + client, e := mongo.Connect(ctx, options.Client().ApplyURI(cfg.URI).SetRetryWrites(false).SetRetryReads(false).SetServerSelectionTimeout(5*time.Second).SetConnectTimeout(5*time.Second).SetReadPreference(readpref.Primary()).SetMonitor(monitor)) + if e != nil { + fmt.Fprintln(os.Stderr, "Mongo connection initialization failed; details suppressed") + os.Exit(2) + } + defer client.Disconnect(context.Background()) + result := map[string]any{"capturedAtUTC": time.Now().UTC().Format(time.RFC3339), "database": "poc", "primaryReads": true, "maxQueryMillis": 2500, "writesRequested": false} + if e = client.Ping(ctx, readpref.Primary()); e != nil { + failed("ping", e) + result["errors"] = failures + json.NewEncoder(os.Stdout).Encode(result) + os.Exit(1) + } + db := client.Database("poc") + if len(cfg.CleanupIDs) > 0 { + if len(cfg.CleanupIDs) > 10 { + fmt.Fprintln(os.Stderr, "too many cleanup samples") + return + } + for _, id := range cfg.CleanupIDs { + if !strings.HasPrefix(id, "osi_") || len(id) > 64 { + fmt.Fprintln(os.Stderr, "invalid item sample") + return + } + } + opts := options.Find().SetProjection(bson.M{"_id": 0, "id": 1, "status": 1, "stagingKey": 1, "targetKey": 1, "uploadId": 1, "attemptCount": 1, "updatedAt": 1, "lastError": 1}).SetLimit(10).SetMaxTime(2500 * time.Millisecond).SetHint("id_unique") + cursor, err := db.Collection("objectSyncItems").Find(ctx, bson.M{"id": bson.M{"$in": cfg.CleanupIDs}}, opts) + var rows []bson.M + if err != nil { + failed("cleanup-error-samples", err) + } else if err = cursor.All(ctx, &rows); err != nil { + failed("cleanup-error-samples", err) + } + records := []map[string]any{} + for _, row := range rows { + _, present := row["stagingKey"] + records = append(records, map[string]any{"id": row["id"], "status": row["status"], "stagingKeyFieldPresent": present, "stagingKeyNonempty": text(row, "stagingKey") != "", "targetKeyNonempty": text(row, "targetKey") != "", "stageEqualsTarget": text(row, "stagingKey") == text(row, "targetKey"), "uploadIdNonempty": text(row, "uploadId") != "", "attemptCount": number(row["attemptCount"]), "updatedAt": row["updatedAt"], "lastErrorContainsNoSuchKey": strings.Contains(text(row, "lastError"), "NoSuchKey")}) + } + matching, err := db.Collection("objectSyncItems").Find(ctx, bson.M{"id": bson.M{"$in": cfg.CleanupIDs}, "stagingKey": bson.M{"$ne": ""}}, options.Find().SetProjection(bson.M{"_id": 0, "id": 1}).SetLimit(10).SetMaxTime(2500*time.Millisecond).SetHint("id_unique")) + var matched []bson.M + if err != nil { + failed("current-selector-shape", err) + } else if err = matching.All(ctx, &matched); err != nil { + failed("current-selector-shape", err) + } + result["cleanupSamples"] = records + result["matchCurrentNonemptyPredicate"] = matched + result["errors"] = failures + mu.Lock() + copy := map[string]int{} + for k, v := range commands { + copy[k] = v + } + mu.Unlock() + result["observedCommandCounts"] = copy + json.NewEncoder(os.Stdout).Encode(result) + return + } + if cfg.Extra { + var buildInfo bson.M + if err := db.RunCommand(ctx, bson.M{"buildInfo": 1}).Decode(&buildInfo); err != nil { + failed("server-version", err) + } else { + result["mongoVersion"] = text(buildInfo, "version") + } + names, err := db.ListCollectionNames(ctx, bson.M{"name": bson.M{"$in": bson.A{"objectSyncDigestBackfills", "objectSyncReconciles", "objectSyncReleases"}}}) + if err != nil { + failed("collection-presence", err) + } + exists := map[string]bool{} + for _, name := range names { + exists[name] = true + } + result["collectionPresence"] = exists + jobs := map[string]any{} + for _, name := range []string{"objectSyncDigestBackfills", "objectSyncReconciles", "objectSyncReleases"} { + if !exists[name] { + jobs[name] = map[string]any{"collectionAbsent": true, "active": 0} + continue + } + cctx, stop := context.WithTimeout(ctx, 3*time.Second) + cursor, err := db.Collection(name).Find(cctx, bson.M{"status": bson.M{"$in": bson.A{"pending", "running"}}}, options.Find().SetProjection(bson.M{"_id": 0, "id": 1, "status": 1}).SetLimit(21).SetMaxTime(2500*time.Millisecond)) + var rows []bson.M + if err != nil { + failed("background-jobs:"+name, err) + } else if err = cursor.All(cctx, &rows); err != nil { + failed("background-jobs:"+name, err) + } + stop() + jobs[name] = map[string]any{"active": len(rows), "limit": 21, "mayBeTruncated": len(rows) == 21} + } + result["backgroundJobs"] = jobs + var status bson.M + if err := db.RunCommand(ctx, bson.D{{Key: "connectionStatus", Value: 1}, {Key: "showPrivileges", Value: true}}).Decode(&status); err != nil { + failed("own-effective-privileges", err) + } else { + actions := map[string]bool{} + info, _ := status["authInfo"].(bson.M) + privileges, _ := info["authenticatedUserPrivileges"].(bson.A) + for _, value := range privileges { + privilege, ok := value.(bson.M) + if !ok { + continue + } + resource, _ := privilege["resource"].(bson.M) + dbName, dbOK := resource["db"].(string) + collection, _ := resource["collection"].(string) + if resource["anyResource"] != true && !(dbOK && (dbName == "poc" || dbName == "") && collection == "") { + continue + } + granted, _ := privilege["actions"].(bson.A) + for _, action := range granted { + if name, ok := action.(string); ok { + actions[name] = true + } + } + } + required := map[string]bool{} + for _, action := range []string{"createCollection", "createIndex", "find", "insert", "update", "remove", "listIndexes"} { + required[action] = actions[action] || actions["anyAction"] + } + result["newCollectionEffectivePrivileges"] = required + } + result["errors"] = failures + mu.Lock() + copy := map[string]int{} + for k, v := range commands { + copy[k] = v + } + mu.Unlock() + result["observedCommandCounts"] = copy + json.NewEncoder(os.Stdout).Encode(result) + return + } + indexes := map[string]map[string]any{} + for _, name := range []string{"objectSyncHeads", "objectSyncItems", "objectSyncSessions", "objectSyncDigestBackfills", "objectSyncSpaces", "storageConnections"} { + cctx, stop := context.WithTimeout(ctx, 2500*time.Millisecond) + cursor, e := db.Collection(name).Indexes().List(cctx) + names := map[string]any{} + if e != nil { + var command mongo.CommandError + if errors.As(e, &command) && command.Code == 26 { + names["collectionAbsent"] = true + } else { + failed("indexes:"+name, e) + } + } else { + var rows []bson.M + if e = cursor.All(cctx, &rows); e != nil { + failed("indexes:"+name, e) + } else { + for _, row := range rows { + names[text(row, "name")] = map[string]any{"key": row["key"], "unique": row["unique"]} + } + } + } + indexes[name] = names + stop() + } + result["indexes"] = indexes + find := func(name, stage string, filter, projection bson.M, limit int64, sort bson.D, hint string) []bson.M { + cctx, stop := context.WithTimeout(ctx, 3*time.Second) + defer stop() + opts := options.Find().SetLimit(limit).SetBatchSize(int32(limit)).SetProjection(projection).SetMaxTime(2500 * time.Millisecond) + if sort != nil { + opts.SetSort(sort) + } + if hint != "" { + if _, ok := indexes[name][hint]; !ok { + failures = append(failures, map[string]any{"stage": stage, "reason": "required sampling index absent", "index": hint}) + return nil + } + opts.SetHint(hint) + } + cursor, e := db.Collection(name).Find(cctx, filter, opts) + if e != nil { + failed(stage, e) + return nil + } + var rows []bson.M + if e = cursor.All(cctx, &rows); e != nil { + failed(stage, e) + return nil + } + return rows + } + sessionFields := bson.M{"_id": 0, "id": 1, "organizationId": 1, "syncSpaceId": 1, "mappingId": 1, "status": 1, "clientVersion": 1, "counts": 1, "updatedAt": 1, "preview": 1} + active := find("objectSyncSessions", "active-session-sample", bson.M{"status": bson.M{"$in": bson.A{"collecting_manifest", "planning", "planned", "running"}}}, sessionFields, 21, nil, "status_updatedAt") + activeCounts := map[string]int{} + for _, s := range active { + activeCounts[text(s, "status")]++ + } + result["activeSessionSample"] = map[string]any{"returned": len(active), "limit": 21, "mayBeTruncated": len(active) == 21, "statuses": activeCounts} + if indexes["objectSyncDigestBackfills"]["collectionAbsent"] == true { + result["activeBackfills"] = map[string]any{"collectionAbsent": true, "returned": 0} + } else { + jobs := find("objectSyncDigestBackfills", "pending-backfills", bson.M{"status": bson.M{"$in": bson.A{"pending", "running"}}}, bson.M{"_id": 0, "id": 1, "status": 1, "syncSpaceId": 1, "maxObjects": 1, "maxBytes": 1, "wholeSpace": 1, "allowUnanchored": 1}, 21, nil, "") + result["activeBackfills"] = map[string]any{"returned": len(jobs), "limit": 21, "mayBeTruncated": len(jobs) == 21, "jobs": jobs} + } + partials := find("objectSyncSessions", "latest-partial-sessions", bson.M{"status": "partial"}, sessionFields, 20, bson.D{{Key: "updatedAt", Value: -1}}, "status_updatedAt") + samples := []map[string]any{} + scopes := map[string]any{} + seen := map[string]bool{} + for _, session := range partials { + mapping := text(session, "mappingId") + sid := text(session, "id") + org := text(session, "organizationId") + space := text(session, "syncSpaceId") + if seen[mapping] || len(seen) >= 3 || org == "" || space == "" { + continue + } + seen[mapping] = true + items := find("objectSyncItems", "denied-items", bson.M{"sessionId": sid, "status": "denied", "action": "deny_archived_hash_change"}, bson.M{"_id": 0, "id": 1, "targetKey": 1, "sizeBytes": 1, "md5sum": 1, "lastError": 1}, 5, nil, "sessionId_status_action") + itemShapes := []map[string]any{} + for _, item := range items { + key := text(item, "targetKey") + heads := find("objectSyncHeads", "matched-legacy-head", bson.M{"organizationId": org, "syncSpaceId": space, "key": key}, bson.M{"_id": 0, "md5sum": 1, "sha256": 1, "remoteETag": 1, "versionId": 1, "sizeBytes": 1, "platformVersion": 1, "sessionId": 1, "itemId": 1, "md5Attestation": 1, "drifted": 1}, 1, nil, "organizationId_syncSpaceId_key_unique") + row := map[string]any{"itemId": text(item, "id"), "keyFingerprint": fingerprint(key), "declaredSize": number(item["sizeBytes"]), "declaredMD5": shape(item, "md5sum", 16), "denialMentionsMissingMD5": strings.Contains(text(item, "lastError"), "缺少 MD5"), "headFound": len(heads) == 1} + if len(heads) == 1 { + h := heads[0] + row["head"] = map[string]any{"md5sum": shape(h, "md5sum", 16), "sha256": shape(h, "sha256", 32), "etagPresent": text(h, "remoteETag") != "", "versionId": shape(h, "versionId", 0), "sessionIdPresent": text(h, "sessionId") != "", "itemIdPresent": text(h, "itemId") != "", "sizeBytes": number(h["sizeBytes"]), "sizeMatchesDeclared": number(h["sizeBytes"]) == number(item["sizeBytes"]), "platformVersion": number(h["platformVersion"]), "drifted": h["drifted"], "attestationPresent": h["md5Attestation"] != nil} + } + itemShapes = append(itemShapes, row) + } + samples = append(samples, map[string]any{"sessionId": sid, "mappingId": mapping, "spaceId": space, "clientVersion": text(session, "clientVersion"), "updatedAt": session["updatedAt"], "counts": session["counts"], "deniedItemSamples": itemShapes}) + if _, ok := scopes[space]; !ok { + spaces := find("objectSyncSpaces", "matched-space", bson.M{"id": space, "organizationId": org}, bson.M{"_id": 0, "id": 1, "connectionId": 1, "bucket": 1, "rootPrefix": 1, "enabled": 1, "mode": 1, "capability": 1}, 1, nil, "id_unique") + if len(spaces) == 1 { + sp := spaces[0] + scope := map[string]any{"bucket": sp["bucket"], "rootPrefixPresent": text(sp, "rootPrefix") != "", "rootPrefixFingerprint": fingerprint(text(sp, "rootPrefix")), "enabled": sp["enabled"], "mode": sp["mode"], "capabilityState": ""} + if cap, ok := sp["capability"].(bson.M); ok { + scope["capabilityState"] = cap["state"] + } + connid := text(sp, "connectionId") + conns := find("storageConnections", "matched-storage-connection", bson.M{"id": connid, "organizationId": org}, bson.M{"_id": 0, "id": 1, "provider": 1, "endpoint": 1, "bucket": 1, "rootPrefix": 1}, 1, nil, "id_unique") + if len(conns) == 1 { + co := conns[0] + endpoint, _ := url.Parse(text(co, "endpoint")) + scope["connectionId"] = connid + scope["provider"] = co["provider"] + scope["connectionRootPrefixPresent"] = text(co, "rootPrefix") != "" + scope["connectionBucket"] = co["bucket"] + if endpoint != nil { + scope["endpoint"] = endpoint.Scheme + "://" + endpoint.Host + } + } + creds := find("storageConnections", "credential-presence-only", bson.M{"id": connid, "organizationId": org, "accessKeyId": bson.M{"$type": "string", "$ne": ""}, "secretAccessKey": bson.M{"$type": "string", "$ne": ""}}, bson.M{"_id": 0, "id": 1}, 1, nil, "id_unique") + scope["credentialsNonempty"] = len(creds) == 1 + scopes[space] = scope + } + } + } + result["partialSessionSampleLimit"] = 20 + result["mappingSamples"] = samples + result["storageScopes"] = scopes + result["errors"] = failures + mu.Lock() + result["observedCommandCounts"] = commands + mu.Unlock() + json.NewEncoder(os.Stdout).Encode(result) + if len(failures) > 0 { + os.Exit(1) + } +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-snapshot.json b/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-snapshot.json new file mode 100644 index 00000000..725803cc --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/mongo-readonly-snapshot.json @@ -0,0 +1,581 @@ +{ + "activeBackfills": { + "jobs": null, + "limit": 21, + "mayBeTruncated": false, + "returned": 0 + }, + "activeSessionSample": { + "limit": 21, + "mayBeTruncated": false, + "returned": 0, + "statuses": {} + }, + "capturedAtUTC": "2026-10-31T15:04:12Z", + "database": "poc", + "errors": null, + "indexes": { + "objectSyncDigestBackfills": {}, + "objectSyncHeads": { + "_id_": { + "key": { + "_id": 1 + }, + "unique": null + }, + "organizationId_drifted_updatedAt": { + "key": { + "drifted": 1, + "organizationId": 1, + "updatedAt": -1 + }, + "unique": null + }, + "organizationId_syncSpaceId_key_unique": { + "key": { + "key": 1, + "organizationId": 1, + "syncSpaceId": 1 + }, + "unique": true + } + }, + "objectSyncItems": { + "_id_": { + "key": { + "_id": 1 + }, + "unique": null + }, + "id_unique": { + "key": { + "id": 1 + }, + "unique": true + }, + "organizationId_targetKey_createdAt": { + "key": { + "createdAt": -1, + "organizationId": 1, + "targetKey": 1 + }, + "unique": null + }, + "purgeAt_ttl": { + "key": { + "purgeAt": 1 + }, + "unique": null + }, + "sessionId_action_id": { + "key": { + "action": 1, + "id": 1, + "sessionId": 1 + }, + "unique": null + }, + "sessionId_clientItemId_unique": { + "key": { + "clientItemId": 1, + "sessionId": 1 + }, + "unique": true + }, + "sessionId_id": { + "key": { + "id": 1, + "sessionId": 1 + }, + "unique": null + }, + "sessionId_page_status_action": { + "key": { + "action": 1, + "manifestPage": 1, + "sessionId": 1, + "status": 1 + }, + "unique": null + }, + "sessionId_status_action": { + "key": { + "action": 1, + "sessionId": 1, + "status": 1 + }, + "unique": null + }, + "sessionId_status_id": { + "key": { + "id": 1, + "sessionId": 1, + "status": 1 + }, + "unique": null + }, + "session_capture_id": { + "key": { + "id": 1, + "sessionId": 1, + "uploadOrder": 1 + }, + "unique": null + }, + "status_commitRequestedAt": { + "key": { + "commitRequestedAt": 1, + "status": 1 + }, + "unique": null + } + }, + "objectSyncSessions": { + "_id_": { + "key": { + "_id": 1 + }, + "unique": null + }, + "id_unique": { + "key": { + "id": 1 + }, + "unique": true + }, + "organizationId_createdAt": { + "key": { + "createdAt": -1, + "organizationId": 1 + }, + "unique": null + }, + "organization_device_credential_created_id": { + "key": { + "createdAt": -1, + "credentialId": 1, + "deviceId": 1, + "id": -1, + "organizationId": 1 + }, + "unique": null + }, + "purgeAt_ttl": { + "key": { + "purgeAt": 1 + }, + "unique": null + }, + "status_updatedAt": { + "key": { + "status": 1, + "updatedAt": 1 + }, + "unique": null + } + }, + "objectSyncSpaces": { + "_id_": { + "key": { + "_id": 1 + }, + "unique": null + }, + "id_unique": { + "key": { + "id": 1 + }, + "unique": true + }, + "organizationId_connection_bucket_prefix": { + "key": { + "bucket": 1, + "connectionId": 1, + "organizationId": 1, + "rootPrefix": 1 + }, + "unique": null + }, + "organizationId_updatedAt": { + "key": { + "organizationId": 1, + "updatedAt": -1 + }, + "unique": null + } + }, + "storageConnections": { + "_id_": { + "key": { + "_id": 1 + }, + "unique": null + }, + "id_unique": { + "key": { + "id": 1 + }, + "unique": true + }, + "name": { + "key": { + "name": 1 + }, + "unique": null + }, + "organizationId_name": { + "key": { + "name": 1, + "organizationId": 1 + }, + "unique": null + } + } + }, + "mappingSamples": [ + { + "clientVersion": "1.1.1", + "counts": { + "committed": 36, + "create": 36, + "denied": 1, + "failed": 0, + "keptBoth": 0, + "manifest": 37, + "skipped": 0, + "unchanged": 0, + "update": 0 + }, + "deniedItemSamples": [ + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 7201, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 7133, + "sizeMatchesDeclared": false, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:7eb669cb3d380e66", + "keyFingerprint": "066da51e6555ab52" + } + ], + "mappingId": "sha256:8c97d8f1bace40c7", + "sessionId": "sha256:120d70cdd208d0ce", + "spaceId": "sha256:00195d3408caf193", + "updatedAt": "2026-10-30T09:48:11.43862952Z" + }, + { + "clientVersion": "1.1.1", + "counts": { + "committed": 15, + "create": 15, + "denied": 1991, + "failed": 0, + "keptBoth": 0, + "manifest": 2006, + "skipped": 0, + "unchanged": 0, + "update": 0 + }, + "deniedItemSamples": [ + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 6015, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 6015, + "sizeMatchesDeclared": true, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:1f49208a2857dd2a", + "keyFingerprint": "b6e9db6389235804" + }, + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 6015, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 6015, + "sizeMatchesDeclared": true, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:e141a4c43b9d9598", + "keyFingerprint": "4e2c2f65fcfc4a3a" + }, + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 96785824, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 96785824, + "sizeMatchesDeclared": true, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:923021b9ba3594de", + "keyFingerprint": "3e0bfdec491a8d18" + }, + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 111020451, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 111020451, + "sizeMatchesDeclared": true, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:dab4dd72c59fbc23", + "keyFingerprint": "bc01bb568f4c16c7" + }, + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 354, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 354, + "sizeMatchesDeclared": true, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:26de6dd7522ea69a", + "keyFingerprint": "883289a85c3821a2" + } + ], + "mappingId": "sha256:a490bb4299ed4e7c", + "sessionId": "sha256:f9be6b203f90bfde", + "spaceId": "sha256:00195d3408caf193", + "updatedAt": "2026-10-30T09:47:52.450317087Z" + }, + { + "clientVersion": "1.1.1", + "counts": { + "committed": 33, + "create": 33, + "denied": 1, + "failed": 0, + "keptBoth": 0, + "manifest": 34, + "skipped": 0, + "unchanged": 0, + "update": 0 + }, + "deniedItemSamples": [ + { + "declaredMD5": { + "length": 32, + "present": true, + "type": "string", + "validHexDigest": true + }, + "declaredSize": 11933, + "denialMentionsMissingMD5": true, + "head": { + "attestationPresent": false, + "drifted": false, + "etagPresent": true, + "itemIdPresent": true, + "md5sum": { + "present": false, + "type": "missing" + }, + "platformVersion": 1, + "sessionIdPresent": true, + "sha256": { + "length": 64, + "present": true, + "type": "string", + "validHexDigest": true + }, + "sizeBytes": 10417, + "sizeMatchesDeclared": false, + "versionId": { + "present": false, + "type": "missing" + } + }, + "headFound": true, + "itemId": "sha256:c7d6ba56fced209c", + "keyFingerprint": "989534646fa954cd" + } + ], + "mappingId": "sha256:adfe5f0c69eed104", + "sessionId": "sha256:af71b6da61077927", + "spaceId": "sha256:00195d3408caf193", + "updatedAt": "2026-10-30T09:31:59.309131402Z" + } + ], + "maxQueryMillis": 2500, + "observedCommandCounts": { + "find": 16, + "listIndexes": 6, + "ping": 1 + }, + "partialSessionSampleLimit": 20, + "primaryReads": true, + "storageScopes": { + "sha256:00195d3408caf193": { + "bucket": "testxyz", + "capabilityState": "create_only", + "connectionBucket": "testxyz", + "connectionId": "sha256:d759ee26231c8353", + "connectionRootPrefixPresent": false, + "credentialsNonempty": true, + "enabled": true, + "endpoint": "https://bj.bcebos.com", + "mode": "create_only", + "provider": "BOS", + "rootPrefixFingerprint": "e882634a38de0c3e", + "rootPrefixPresent": true + } + }, + "writesRequested": false +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/readiness-and-rollback-probes.json b/doc/drosync-poc-preflight-evidence-2026-10-31/readiness-and-rollback-probes.json new file mode 100644 index 00000000..8ebcf13f --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/readiness-and-rollback-probes.json @@ -0,0 +1,105 @@ +{ + "capturedAtUTC": "2026-10-31T14:57:35.998749+00:00", + "coordinatorProbes": { + "drosync-coordinator-86c44cbb8-5bsgc": { + "health": { + "service": "drosync-coordinator", + "status": "ok", + "enabled": true + }, + "ready": { + "ready": true, + "enabled": true + } + }, + "drosync-coordinator-86c44cbb8-wc2kj": { + "health": { + "service": "drosync-coordinator", + "status": "ok", + "enabled": true + }, + "ready": { + "ready": true, + "enabled": true + } + } + }, + "nodes": { + "192.168.80.249": { + "conditions": { + "KernelDeadlock": "False", + "KubeletUnhealthy": "False", + "FrequentContainerRuntimeRestart": "False", + "GPUUnhealthy": "False", + "PCIEUnhealthy": "False", + "MemoryUnhealthy": "False", + "CustomRecallFault": "False", + "InodesPressure": "False", + "ReadonlyFilesystem": "False", + "FrequentKubeletRestart": "False", + "CorruptDockerOverlay2": "False", + "ContainerRuntimeUnhealthy": "False", + "NICUnhealthy": "False", + "CCESubnetUnavailable": "False", + "NetworkUnavailable": "False", + "CCENetworkUnavailable": "False", + "MemoryPressure": "False", + "DiskPressure": "False", + "PIDPressure": "False", + "Ready": "True" + }, + "unschedulable": false, + "allocatable": { + "cpu": "15600m", + "memory": "28427600Ki", + "ephemeral-storage": "491323269332" + } + }, + "192.168.81.47": { + "conditions": { + "KernelDeadlock": "False", + "ReadonlyFilesystem": "False", + "KubeletUnhealthy": "False", + "InodesPressure": "False", + "CorruptDockerOverlay2": "False", + "FrequentKubeletRestart": "False", + "GPUUnhealthy": "False", + "CustomRecallFault": "False", + "NICUnhealthy": "False", + "MemoryUnhealthy": "False", + "ContainerRuntimeUnhealthy": "False", + "PCIEUnhealthy": "False", + "FrequentContainerRuntimeRestart": "False", + "CCESubnetUnavailable": "False", + "NetworkUnavailable": "False", + "CCENetworkUnavailable": "False", + "MemoryPressure": "False", + "DiskPressure": "False", + "PIDPressure": "False", + "Ready": "True" + }, + "unschedulable": false, + "allocatable": { + "cpu": "15600m", + "memory": "28427588Ki", + "ephemeral-storage": "491323269332" + } + } + }, + "rollbackRegistryHEAD": [ + { + "workload": "droboticflow-api", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "status": 401, + "returnedDigest": null, + "authRequired": true + }, + { + "workload": "drosync-coordinator", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "status": 401, + "returnedDigest": null, + "authRequired": true + } + ] +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/rollback-reference.json b/doc/drosync-poc-preflight-evidence-2026-10-31/rollback-reference.json new file mode 100644 index 00000000..efbc7bd0 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/rollback-reference.json @@ -0,0 +1,37 @@ +{ + "namespace": "droboticflow", + "snapshotOnly": true, + "executionRequiresFreshComparison": true, + "deployments": { + "droboticflow-api": { + "container": "api", + "oldImage": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "oldDroboticflowImageEnv": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "oldRevision": "531", + "observedGeneration": 537, + "replicas": 2, + "strategy": { + "rollingUpdate": { + "maxSurge": 1, + "maxUnavailable": 0 + }, + "type": "RollingUpdate" + } + }, + "drosync-coordinator": { + "container": "coordinator", + "oldImage": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "oldDroboticflowImageEnv": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:b94f88eca2505b784c46ec4190f4ff91c59d11702e735616b4c4a633ec49eab3", + "oldRevision": "50", + "observedGeneration": 54, + "replicas": 2, + "strategy": { + "rollingUpdate": { + "maxSurge": 1, + "maxUnavailable": 0 + }, + "type": "RollingUpdate" + } + } + } +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/rollback-registry-verification.json b/doc/drosync-poc-preflight-evidence-2026-10-31/rollback-registry-verification.json new file mode 100644 index 00000000..05000f12 --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/rollback-registry-verification.json @@ -0,0 +1,23 @@ +{ + "checkedAtUTC": "2026-10-31T15:15:09.694706+00:00", + "onlyGETRequests": true, + "layerDownloads": false, + "images": [ + { + "workload": "droboticflow-api", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "status": 200, + "returnedDigest": "sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "bodyDigest": "sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "verified": true + }, + { + "workload": "drosync-coordinator", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "status": 200, + "returnedDigest": "sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "bodyDigest": "sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "verified": true + } + ] +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/summary.json b/doc/drosync-poc-preflight-evidence-2026-10-31/summary.json new file mode 100644 index 00000000..7846d90d --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/summary.json @@ -0,0 +1,64 @@ +{ + "candidateRuntimeSource": "935f76ce", + "namespace": "droboticflow", + "rolloutRecommendation": "HOLD_PENDING_HARDENING", + "deploymentPerformed": false, + "productionBackfillCreated": false, + "businessBOSReadsOrDeletesPerformed": false, + "databaseProbeWriteCommands": 0, + "checks": { + "apiReady": "2/2", + "coordinatorReady": "2/2", + "auditKeyPresentAndValidLength": true, + "apiCoordinatorSameMongo": true, + "indexCreationPrivilegesAvailable": true, + "rollbackManifestsVerified": true, + "backfillCollectionAbsent": true, + "activeSyncSessionsAtSample": 0, + "pendingBackfillJobsAtSample": 0, + "productionMongoVersion": "8.0.17-r1" + }, + "legacySamples": { + "mappings": 3, + "heads": 7, + "missingMD5WithValidSHA256": 7, + "sizeMismatches": 2, + "largerThan64MiB": 2 + }, + "findings": [ + { + "id": "cleanup-missing-object", + "status": "observed", + "summary": "Both existing coordinator pods report NoSuchKey during staging cleanup. Four sampled committed records retain nonempty staging keys distinct from targets; candidate code still returns the error before retirement." + }, + { + "id": "cleanup-selector-missing-field", + "status": "code-review", + "summary": "Mongo cleanup uses stagingKey $ne empty, which can match missing fields; add nonempty typed-key and empty-path regressions during hardening. This was not the shape of the four live error samples." + }, + { + "id": "mongo-major-version", + "status": "verification-gap", + "summary": "Production buildInfo reports 8.0.17-r1; isolated write/CAS/claim acceptance used 7.0.40. Rerun against an isolated Mongo 8.0 environment; no incompatibility has been established." + }, + { + "id": "large-range-acceptance", + "status": "verification-gap", + "summary": "Two sampled archived sizes exceed 64 MiB (96785824 and 111020451 bytes). Real-storage acceptance so far covered small objects and explicit subranges, not multi-default-range coordinator execution." + }, + { + "id": "different-content", + "status": "observed-receipt-mismatch", + "summary": "Two sampled session declarations differ from stored archive sizes; MD5 backfill cannot make differing payloads equal. No business object content was read." + }, + { + "id": "coordinator-image-label", + "status": "metadata-warning", + "summary": "Coordinator DROBOTICFLOW_IMAGE does not match its actual imageID; use the verified runtime digest for rollback." + } + ], + "privacy": "Opaque business IDs are SHA256-shortened in committed evidence; business object keys/content digests, usernames, database URI and credentials are omitted. Full selected snapshots remain only in ignored local output.", + "checkedAtUTC": "2026-10-31T15:39:31.633200+00:00", + "imageBuildPerformed": false, + "clientReleaseBuildPerformed": false +} diff --git a/doc/drosync-poc-preflight-evidence-2026-10-31/workload-snapshot.json b/doc/drosync-poc-preflight-evidence-2026-10-31/workload-snapshot.json new file mode 100644 index 00000000..cb68aa0e --- /dev/null +++ b/doc/drosync-poc-preflight-evidence-2026-10-31/workload-snapshot.json @@ -0,0 +1,583 @@ +{ + "capturedAtUTC": "2026-10-31T14:56:01.547172+00:00", + "namespace": "droboticflow", + "workloads": { + "droboticflow-api": { + "revision": "531", + "generation": 537, + "resourceVersion": "75078739", + "replicas": 2, + "readyReplicas": 2, + "updatedReplicas": 2, + "availableReplicas": 2, + "strategy": { + "rollingUpdate": { + "maxSurge": 1, + "maxUnavailable": 0 + }, + "type": "RollingUpdate" + }, + "serviceAccount": "droboticflow-api", + "imagePullSecrets": [ + { + "name": "droboticflow-registry" + } + ], + "containers": [ + { + "name": "api", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "imagePullPolicy": "Always", + "resources": { + "limits": { + "cpu": "1", + "memory": "2Gi" + }, + "requests": { + "cpu": "250m", + "memory": "512Mi" + } + }, + "environment": { + "DROBOTICFLOW_IMAGE": { + "reference": { + "source": "literal" + }, + "value": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2" + }, + "DROSYNC_COORDINATOR_PORT": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "DROSYNC_COORDINATOR_PORT", + "present": true, + "optional": false + }, + "value": "3003" + }, + "MONGO_DB": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "MONGO_DB", + "present": true, + "optional": false + }, + "value": "poc" + }, + "DROBOTICFLOW_OBJECT_SYNC_ENABLED": { + "reference": { + "source": "literal" + }, + "value": "true" + }, + "MONGO_URI": { + "reference": { + "source": "secret", + "name": "droboticflow-mongo-secret", + "key": "MONGO_URI", + "present": true, + "optional": false + }, + "nonempty": true + }, + "BOS_ENDPOINT": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "BOS_ENDPOINT", + "present": true, + "optional": false + }, + "value": "https://bj.bcebos.com" + }, + "DROSYNC_SERVER_VERSION": { + "reference": { + "source": "literal" + }, + "value": "1.0.26" + } + }, + "probes": { + "readinessProbe": { + "httpGet": { + "path": "/api/ready", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 1 + }, + "livenessProbe": { + "httpGet": { + "path": "/api/health", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 1 + }, + "startupProbe": { + "httpGet": { + "path": "/api/health", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 3 + } + }, + "envFromPresent": false + } + ] + }, + "drosync-coordinator": { + "revision": "50", + "generation": 54, + "resourceVersion": "74283576", + "replicas": 2, + "readyReplicas": 2, + "updatedReplicas": 2, + "availableReplicas": 2, + "strategy": { + "rollingUpdate": { + "maxSurge": 1, + "maxUnavailable": 0 + }, + "type": "RollingUpdate" + }, + "serviceAccount": "drosync-coordinator", + "imagePullSecrets": [ + { + "name": "droboticflow-registry" + } + ], + "containers": [ + { + "name": "coordinator", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "imagePullPolicy": "Always", + "resources": { + "limits": { + "cpu": "2", + "memory": "2Gi" + }, + "requests": { + "cpu": "200m", + "memory": "256Mi" + } + }, + "environment": { + "DROSYNC_COORDINATOR_PORT": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "DROSYNC_COORDINATOR_PORT", + "present": true, + "optional": false + }, + "value": "3003" + }, + "DROBOTICFLOW_OBJECT_SYNC_ENABLED": { + "reference": { + "source": "literal" + }, + "value": "true" + }, + "WORKER_POLL_INTERVAL_MS": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "WORKER_POLL_INTERVAL_MS", + "present": true, + "optional": false + }, + "value": "2000" + }, + "BOS_ENDPOINT": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "BOS_ENDPOINT", + "present": true, + "optional": false + }, + "value": "https://bj.bcebos.com" + }, + "MONGO_DB": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "MONGO_DB", + "present": true, + "optional": false + }, + "value": "poc" + }, + "MONGO_URI": { + "reference": { + "source": "secret", + "name": "droboticflow-mongo-secret", + "key": "MONGO_URI", + "present": true, + "optional": false + }, + "nonempty": true + }, + "DROSYNC_AUDIT_SIGNING_PRIVATE_KEY": { + "reference": { + "source": "secret", + "name": "drosync-signing-secret", + "key": "DROSYNC_AUDIT_SIGNING_PRIVATE_KEY", + "present": true, + "optional": true + }, + "nonempty": true, + "decodedBytes": 32, + "validLength": true + }, + "DROSYNC_AUDIT_SIGNING_KEY_ID": { + "reference": { + "source": "secret", + "name": "drosync-signing-secret", + "key": "DROSYNC_AUDIT_SIGNING_KEY_ID", + "present": true, + "optional": true + }, + "value": "poc-canary-20260728" + }, + "DROBOTICFLOW_IMAGE": { + "reference": { + "source": "literal" + }, + "value": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:b94f88eca2505b784c46ec4190f4ff91c59d11702e735616b4c4a633ec49eab3" + }, + "DROSYNC_SERVER_VERSION": { + "reference": { + "source": "literal" + }, + "value": "1.0.26" + } + }, + "probes": { + "readinessProbe": { + "httpGet": { + "path": "/ready", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 1 + }, + "livenessProbe": { + "httpGet": { + "path": "/health", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 1 + }, + "startupProbe": { + "httpGet": { + "path": "/health", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 3 + } + }, + "envFromPresent": false + } + ] + }, + "droboticflow-worker": { + "revision": "435", + "generation": 443, + "resourceVersion": "74283922", + "replicas": 2, + "readyReplicas": 2, + "updatedReplicas": 2, + "availableReplicas": 2, + "strategy": { + "rollingUpdate": { + "maxSurge": 1, + "maxUnavailable": 0 + }, + "type": "RollingUpdate" + }, + "serviceAccount": "droboticflow-worker", + "imagePullSecrets": [ + { + "name": "droboticflow-registry" + } + ], + "containers": [ + { + "name": "worker", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-worker@sha256:540dc05908745d69c4b0f1488d833b1add18953d21196a75aba31b7b969eb990", + "imagePullPolicy": "Always", + "resources": { + "limits": { + "cpu": "1", + "memory": "4Gi" + }, + "requests": { + "cpu": "250m", + "memory": "512Mi" + } + }, + "environment": { + "DROBOTICFLOW_IMAGE": { + "reference": { + "source": "literal" + }, + "value": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-worker@sha256:4ab917f8084929f1ba0019195a6a92ea4c0a9f9f3d9c24fa049a4c6d0f19e117" + }, + "DROSYNC_COORDINATOR_PORT": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "DROSYNC_COORDINATOR_PORT", + "present": true, + "optional": false + }, + "value": "3003" + }, + "WORKER_POLL_INTERVAL_MS": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "WORKER_POLL_INTERVAL_MS", + "present": true, + "optional": false + }, + "value": "2000" + }, + "BOS_ENDPOINT": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "BOS_ENDPOINT", + "present": true, + "optional": false + }, + "value": "https://bj.bcebos.com" + }, + "MONGO_DB": { + "reference": { + "source": "configmap", + "name": "droboticflow-config", + "key": "MONGO_DB", + "present": true, + "optional": false + }, + "value": "poc" + }, + "MONGO_URI": { + "reference": { + "source": "secret", + "name": "droboticflow-mongo-secret", + "key": "MONGO_URI", + "present": true, + "optional": false + }, + "nonempty": true + } + }, + "probes": { + "readinessProbe": { + "httpGet": { + "path": "/worker-api/ready", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 1 + }, + "livenessProbe": { + "httpGet": { + "path": "/worker-api/health", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 1 + }, + "startupProbe": { + "httpGet": { + "path": "/worker-api/health", + "port": "http", + "scheme": "HTTP" + }, + "timeoutSeconds": 3 + } + }, + "envFromPresent": false + } + ] + } + }, + "pods": [ + { + "name": "droboticflow-api-797cf8c865-q9wjt", + "app": "droboticflow-api", + "node": "192.168.80.249", + "phase": "Running", + "createdAt": "2026-10-30T07:45:44Z", + "containers": [ + { + "name": "api", + "ready": true, + "restartCount": 0, + "image": "sha256:b61b9f8b6e447868e96093b5e023d05e39151d3910704f4dd931b82f9c266bb2", + "imageID": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "started": true + } + ], + "imageBuildCommit": "91641b523423665257ba30893fc423b7404301da", + "imageBuildTime": "2026-10-30T07:33:02Z", + "logSummary": { + "sampledLines": 0, + "errorLikeLines": 0 + } + }, + { + "name": "droboticflow-api-797cf8c865-vsd7g", + "app": "droboticflow-api", + "node": "192.168.81.47", + "phase": "Running", + "createdAt": "2026-10-30T07:44:58Z", + "containers": [ + { + "name": "api", + "ready": true, + "restartCount": 0, + "image": "sha256:b61b9f8b6e447868e96093b5e023d05e39151d3910704f4dd931b82f9c266bb2", + "imageID": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "started": true + } + ], + "imageBuildCommit": "91641b523423665257ba30893fc423b7404301da", + "imageBuildTime": "2026-10-30T07:33:02Z", + "logSummary": { + "sampledLines": 0, + "errorLikeLines": 0 + } + }, + { + "name": "droboticflow-worker-c64bf84f7-ms7rd", + "app": "droboticflow-worker", + "node": "192.168.81.47", + "phase": "Running", + "createdAt": "2026-10-28T06:31:30Z", + "containers": [ + { + "name": "worker", + "ready": true, + "restartCount": 0, + "image": "sha256:a696761a2453ead74db26c0e89494995102ad216b3aa24cf620a663717050df4", + "imageID": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-worker@sha256:540dc05908745d69c4b0f1488d833b1add18953d21196a75aba31b7b969eb990", + "started": true + } + ] + }, + { + "name": "droboticflow-worker-c64bf84f7-tx7pk", + "app": "droboticflow-worker", + "node": "192.168.80.249", + "phase": "Running", + "createdAt": "2026-10-28T06:30:44Z", + "containers": [ + { + "name": "worker", + "ready": true, + "restartCount": 0, + "image": "sha256:a696761a2453ead74db26c0e89494995102ad216b3aa24cf620a663717050df4", + "imageID": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-worker@sha256:540dc05908745d69c4b0f1488d833b1add18953d21196a75aba31b7b969eb990", + "started": true + } + ] + }, + { + "name": "drosync-coordinator-86c44cbb8-5bsgc", + "app": "drosync-coordinator", + "node": "192.168.80.249", + "phase": "Running", + "createdAt": "2026-10-28T06:30:43Z", + "containers": [ + { + "name": "coordinator", + "ready": true, + "restartCount": 0, + "image": "sha256:8526f6c002abecedcae1e8c9181a5470b4ebd94dc13aa37ed92240b205f44ada", + "imageID": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "started": true + } + ], + "imageBuildCommit": "789145750b544eeed297e129b4cb46bcdbf01233", + "imageBuildTime": "2026-10-28T06:17:16.512Z", + "logSummary": { + "sampledLines": 26, + "errorLikeLines": 0 + } + }, + { + "name": "drosync-coordinator-86c44cbb8-wc2kj", + "app": "drosync-coordinator", + "node": "192.168.81.47", + "phase": "Running", + "createdAt": "2026-10-28T06:30:59Z", + "containers": [ + { + "name": "coordinator", + "ready": true, + "restartCount": 0, + "image": "sha256:8526f6c002abecedcae1e8c9181a5470b4ebd94dc13aa37ed92240b205f44ada", + "imageID": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/drosync-coordinator@sha256:339695d5ac2a20cec9b815f9f947baaf117ca5aee2a1018c7635b7b1ae096692", + "started": true + } + ], + "imageBuildCommit": "789145750b544eeed297e129b4cb46bcdbf01233", + "imageBuildTime": "2026-10-28T06:17:16.512Z", + "logSummary": { + "sampledLines": 27, + "errorLikeLines": 0 + } + } + ], + "apiCoordinatorSameMongoURI": true, + "apiCoordinatorSameDatabase": true, + "activeKanikoJobs": [], + "services": { + "droboticflow-api": { + "selector": { + "app": "droboticflow-api" + }, + "ports": [ + { + "name": "http", + "nodePort": 30080, + "port": 80, + "protocol": "TCP", + "targetPort": 3001 + } + ], + "type": "NodePort" + }, + "drosync-coordinator": { + "selector": { + "app": "drosync-coordinator" + }, + "ports": [ + { + "name": "http", + "port": 3003, + "protocol": "TCP", + "targetPort": 3003 + } + ], + "type": "ClusterIP" + } + }, + "publicAPIHealth": { + "buildCommit": "91641b523423665257ba30893fc423b7404301da", + "buildTime": "2026-10-30T07:33:02Z", + "image": "ccr-29eug8s3-pub.cnc.bj.baidubce.com/dataflow-poc/droboticflow-api@sha256:2cf3ef8e506c9b891632c670a2bfd01d1f8c38dc053be87bfb481b40c578cef2", + "service": "droboticflow-api", + "status": "ok" + } +}