diff --git a/IMPLEMENTATION_STATUS.md b/IMPLEMENTATION_STATUS.md deleted file mode 100644 index 6daaa9f..0000000 --- a/IMPLEMENTATION_STATUS.md +++ /dev/null @@ -1,135 +0,0 @@ -# IMPLEMENTATION_STATUS - -更新时间:2026-04-18 23:44 CST - -## 当前真实状态 - -阶段判断: - -- 方案与控制面能力建设:约 `78%` -- 真正达到“可稳定上线、海外单脑统一接管大陆节点”:约 `58%` - -当前已成立的事实: - -- Ops Center 主控骨架已基本形成 -- Release Hub 已具备主流程判断能力 -- Node Agent 协议、日志回传、runbook / playbook、节点接管视图已具备基础可用性 -- `drive_ops_center.sh` 已补齐: - - `nodes` - - `node-bind-ssh` - -当前最新节点状态: - -- `managed_enabled = 3` -- `agent_ready = 1` -- `ssh_ready = 2` -- `remote_access_ready = 3` -- `pending_bootstrap = 2` - -节点明细: - -- `overseas-control-01` - - `agent_ready` - - `remote_access_ready = true` -- `mainland-controller-01` - - `pending_bootstrap` - - `remote_access_state = ssh_ready` - - `SSH 已录入` - - `node-handover` 已完成 - - `node-bootstrap-plan` 已生成 - - 手工 SSH 登录已验证可达 -- `mainland-worker-01` - - `pending_bootstrap` - - `remote_access_state = ssh_ready` - - `SSH 已录入` - - `node-handover` 已完成 - - `node-bootstrap-plan` 已生成 - -## 代码闭环 - -已完成或基本具备: - -- 统一查看节点接管状态 -- 统一生成与查看 handover / bootstrap 入口 -- 统一查看 go-live summary -- 统一查看日志回传覆盖情况 -- 统一为托管节点录入 SSH 入口的 CLI 能力 -- controller 与 worker 的 bootstrap plan 均可生成 - -当前代码侧已验证的事实: - -- 当前主控链路可以把大陆节点从 `agent_pending` 推到 `ssh_ready` -- `node-handover` 与 `node-bootstrap-plan` 输出一致 -- `go-live-summary` 已看到 `remote_access_ready = 3` - -结论: - -- 当前代码侧已经可以支撑“接管闭环”的执行顺序 -- 当前主阻塞已不再是“还缺哪块控制面功能” -- 但当前 SSH 自动执行链仍默认走密钥模式,和今晚实际提供的密码登录方式不一致 - -## 外部条件闭环 - -当前仍未闭合的条件: - -- 大陆两台虽然已录入 SSH,但远端仓库未具备 node-agent 所需文件 -- 尚未实际触发大陆两台的 bootstrap 执行 -- 尚未建立两台大陆节点的 register / heartbeat 闭环 -- 尚未完成两台大陆节点的 acceptance 闭环 - -结论: - -- 当前最关键的剩余问题不是新代码,而是大陆节点环境与当前控制面能力不匹配 -- 今晚已经不是“SSH 信息缺失”阶段,而是“大陆节点代码/部署内容缺失”阶段 - -## 明确依赖用户提供的信息 - -今晚已经提供并验证: - -- `mainland-controller-01` - - `ssh_host=121.204.244.188` - - `ssh_user=root` - - 手工 SSH 登录成功 -- `mainland-worker-01` - - `ssh_host=121.204.244.248` - - `ssh_user=root` - -当前仍需要你明天确认或操作的,不再是 SSH 信息,而是以下环境修正路径二选一: - -1. 让大陆两台节点仓库更新到包含 node-agent 文件的最新版本 -2. 或把 node-agent 相关文件以正式发布包 / 正式部署方式补齐到大陆两台 - -在上述环境未修正前,以下链路不能真正闭合: - -- `bootstrap execute` -- `register / heartbeat` -- `acceptance` -- `pending_bootstrap -> 已接管` - -## 当前优先级判断 - -最高优先级: - -- 大陆节点接管闭环最小路径 - -当前不应继续优先推进: - -- 控制面横向增强 -- 额外专题设计 -- 与接管闭环无直接关系的发布能力细化 -- 新的页面或模块 - -本轮新增阻塞分类结论: - -- controller 阻塞:环境问题 -- worker 阻塞:环境问题 -- 自动 SSH 执行器限制:权限 / 外部条件问题 - - 现链路仅按 `auth_mode=key` 工作 - - 今晚实际验证可用的是密码 SSH - -## 完成主批次后的预期 - -如果大陆两台环境补齐,并按最小闭环顺序完成: - -- 项目整体可稳定上线进度预计提升到 `60%~65%` -- 一旦 `remote_access_ready=3/3` 成立,后续就从“架构推进期”切换到“发布收口期” diff --git a/TASK_BOARD.md b/TASK_BOARD.md deleted file mode 100644 index af0f5e5..0000000 --- a/TASK_BOARD.md +++ /dev/null @@ -1,148 +0,0 @@ -# TASK_BOARD - -更新时间:2026-04-18 23:44 CST - -## 当前主批次 - -唯一主批次:大陆节点接管闭环最小路径 - -目标: - -- 把当前项目的最高优先级从“继续扩控制面能力”切到“完成大陆节点接管闭环” -- 只围绕这条最短路径推进: - - 录入大陆节点 SSH 入口 - - 生成并复核 bootstrap plan - - 执行 bootstrap - - 建立 register / heartbeat - - 执行 acceptance - - 将 `remote_access_ready` 从 `1/3` 提升到 `3/3` - -主批次成功标准: - -- `mainland-controller-01` 不再是 `pending_bootstrap` -- `mainland-worker-01` 不再是 `pending_bootstrap` -- `ssh_ready >= 2` -- `remote_access_ready = 3/3` -- 大陆两台都具备可由海外控制面统一发起标准动作的条件 - -当前完成度: - -- 已完成 `SSH` 入口录入 -- 已完成 `node-handover` -- 已完成 `node-bootstrap-plan` -- 已确认 `ssh_ready = 2` -- 已确认 `remote_access_ready = 3/3` -- 尚未完成: - - `bootstrap` - - `register / heartbeat` - - `acceptance` - - `pending_bootstrap -> 已接管` - -## 主批次拆解 - -### Batch A1 - -名称:大陆节点接管闭环最小路径 - -边界: - -- 只处理大陆两台: - - `mainland-controller-01` - - `mainland-worker-01` -- 只处理接管闭环,不扩展控制面功能 - -任务: - -- 已完成: - - 录入 `mainland-controller-01` SSH 信息 - - 录入 `mainland-worker-01` SSH 信息 - - 复查 `ops/nodes` - - 逐台执行 `node-handover` - - 逐台生成并复核 `node-bootstrap-plan` - - 复核: - - `go-live-summary` - - `ops/nodes` - - `release-launchpad` -- 待完成: - - 执行 controller 的 `bootstrap -> register -> heartbeat` - - 执行 worker 的 `bootstrap -> register -> heartbeat` - - 对两台执行 `acceptance` - - 让两台都退出 `pending_bootstrap` - -阻塞: - -- controller 与 worker 两台大陆节点都缺少 node-agent 相关运行文件 -- 远端实际缺失: - - `domain-api/app/node_agent.py` - - `domain-api/app/api/routes/ops_agent.py` - - `domain-api/app/services/ops_agent_service.py` - - `domain-api/deploy/systemd/domain-node-agent.service` - - `domain-api/deploy/multi-region/templates/domaincheck-node-agent.env.example` -- 因此 bootstrap 片段虽然已生成,但远端执行时无法完成安装与启动 - -阻塞分类: - -- 主阻塞:环境问题 - - 大陆节点上的仓库内容落后于当前控制面所需的 node-agent 文件集 -- 次阻塞:权限 / 外部条件问题 - - 当前 SSH 自动执行链默认走密钥模式 - - 今晚实际可用的是密码 SSH - -## 候选批次 - -### Candidate B1 - -名称:接管闭环完成后的上线收口 - -前提: - -- 仅在 `remote_access_ready = 3/3` 后进入 - -范围: - -- 重新跑 `go-live-check / stack-diagnosis / release-launchpad` -- 收口发布前证据 -- 判断是否达到正式上线窗口 - -### Candidate B2 - -名称:接管闭环完成后的值班自动化细化 - -前提: - -- 仅在大陆两台已进入稳定接管状态后进入 - -范围: - -- 优化运维体验 -- 优化日常值班命令 -- 优化交接与证据导出 - -当前不进入: - -- `Candidate B1` -- `Candidate B2` - -原因: - -- 当前虽然 `remote_access_ready = 3/3` -- 但大陆两台仍停在 `pending_bootstrap` -- A1 还没有真正闭环 - -## 暂停项 - -以下事项当前不继续推进: - -- 新的控制面专题设计 -- 新页面、新模块、新专题文档扩写 -- 与大陆节点接管闭环无直接关系的 Release Hub 增强 -- 与大陆节点接管闭环无直接关系的 runbook / playbook 扩展 -- 更细粒度日志分析功能 -- 额外的驾驶视图、美化项、信息聚合项 -- 任何不能直接推动 `bootstrap -> acceptance -> remote_access_ready=3/3` 的功能开发 - -## 当前判断 - -当前项目不缺“继续横向扩能力”的理由,缺的是“把最后两台大陆节点真正纳入闭环”的收口动作。 - -因此下一轮只围绕主批次 A1 推进。 diff --git a/HANDOFF_20260418_2321.md b/docs/ops_center_runtime/HANDOFF_20260418_2321.md similarity index 100% rename from HANDOFF_20260418_2321.md rename to docs/ops_center_runtime/HANDOFF_20260418_2321.md diff --git a/HANDOFF_20260418_2344.md b/docs/ops_center_runtime/HANDOFF_20260418_2344.md similarity index 100% rename from HANDOFF_20260418_2344.md rename to docs/ops_center_runtime/HANDOFF_20260418_2344.md diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0007.md b/docs/ops_center_runtime/HANDOFF_20260419_0007.md new file mode 100644 index 0000000..43f1798 --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0007.md @@ -0,0 +1,138 @@ +# HANDOFF 2026-04-19 00:07 + +## 当前最高优先级 + +唯一最高优先级: + +- 完成 `mainland-controller-01` 的公网控制面重接入 + +原因: + +- `mainland-worker-01` 已经接管成功 +- `remote_access_ready = 3/3` 已达成 +- 当前只剩 controller 一台仍处于 `pending_bootstrap` + +## 本轮已完成的步骤 + +已完成: + +1. 复核当前 `nodes / go-live-summary / release-launchpad` +2. 确认两台大陆节点都已拉到最新代码 +3. 重新生成 controller / worker 的 bootstrap plan +4. 在 `mainland-controller-01` 成功执行一次 bootstrap 落地 +5. 在 `mainland-worker-01` 使用公网控制面地址完成: + - bootstrap + - register + - heartbeat +6. 确认 `mainland-worker-01` 已变为: + - `agent_state = online_busy` + - `remote_access_state = hybrid_ready` +7. 为 `mainland-worker-01` 发起接管后验收: + - `playbook_run_code = pbr-137e8b258b` + +## 当前结果 + +当前摘要: + +- `ssh_ready = 2` +- `agent_ready = 2` +- `remote_access_ready = 3/3` +- `pending_bootstrap = 1` + +节点结果: + +- `mainland-worker-01` + - 已接管成功 + - 已开始 acceptance +- `mainland-controller-01` + - 仍未完成接管 + - 当前是唯一剩余阻塞点 + +## 两台节点推进到哪一步 + +### mainland-worker-01 + +已推进到: + +- SSH 已录入 +- `node-handover` 已完成 +- `node-bootstrap-plan` 已生成 +- 使用公网控制面地址完成 bootstrap +- register / heartbeat 已建立 +- 节点已进入: + - `online_busy` + - `hybrid_ready` +- acceptance 已发起 + +当前状态判断: + +- 该节点已经不再属于接管主阻塞 + +### mainland-controller-01 + +已推进到: + +- SSH 已录入 +- `node-handover` 已完成 +- `node-bootstrap-plan` 已生成 +- 首次 bootstrap 已真正落地安装并启动 node-agent + +准确卡点: + +- 第一次写入的 `OPS_CONTROL_PLANE_BASE_URL` 是: + - `http://127.0.0.1:8100` +- 对远端大陆节点来说,这会指向它自己,而不是海外控制面 +- 随后尝试把它改为公网控制面地址时,SSH 握手被远端重置 + +阻塞分类: + +- 主阻塞:外部条件 / 网络或权限问题 +- 不是: + - 新功能缺失 + - 代码目录缺文件 + - Node Agent 主链路不可用 + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做这一件事: + +- 让 `mainland-controller-01` 成功执行“公网控制面地址版本”的 bootstrap 命令块 + +最小动作顺序: + +1. 恢复 controller SSH 可执行窗口 +2. 重新写入: + - `/etc/default/domaincheck-node-agent` +3. 确认其中: + - `OPS_CONTROL_PLANE_BASE_URL=http://152.53.37.118:8100` +4. 重启: + - `domaincheck-node-agent` +5. 回控制面确认 controller 建立 register / heartbeat +6. 对 controller 执行 acceptance + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +当前不建议切低模型的原因: + +- 剩下的是收口型、多条件联动的现场问题 +- 需要稳定地分辨代码、执行参数、SSH 状态和节点真实状态 + +## 现在不要做的事情 + +不要做: + +- 新页面 +- 新模块 +- 控制面增强项 +- 新架构文档扩写 +- Candidate B2 +- 任何与 controller 收口无直接关系的工作 + +## 一句话结论 + +今晚不是“要不要重装系统”的阶段,而是已经把大陆接管闭环收束到只剩一台 controller:`worker-01` 已接管成功,`controller-01` 只差把 node-agent 正确指向海外公网控制面并完成一次成功回连。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0030.md b/docs/ops_center_runtime/HANDOFF_20260419_0030.md new file mode 100644 index 0000000..7eec355 --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0030.md @@ -0,0 +1,151 @@ +# HANDOFF 2026-04-19 00:30 + +## 当前最高优先级 + +唯一最高优先级: + +- 完成 acceptance 与发布前证据收口 + +原因: + +- A1 大陆节点接管闭环已经完成 +- 当前 3 台节点都已在线 +- 当前真正剩余的不是接管问题,而是: + - acceptance 还没全部收口 + - playbook run 仍有排队 / attention + - log sync 仍是 partial + +## 本轮已完成的步骤 + +已完成: + +1. 复跑 `go-live-check` +2. 复跑 `stack-diagnosis` +3. 复跑 `release-launchpad` +4. 复核 `mainland-worker-01` acceptance run: + - `pbr-137e8b258b` +5. 确认 `mainland-controller-01` 已成功进入: + - `online_busy` + - `hybrid_ready` +6. 确认全局摘要: + - `pending_bootstrap = 0` + - `agent_ready = 3` + - `remote_access_ready = 3` + +## 当前结果 + +当前已经成立的事实: + +- 大陆节点接管闭环已完成 +- controller / worker 都已接管成功 +- delivery queue 当前 3/3 健康 +- 发布包与 release gate 已就绪 + +当前仍未收口的事实: + +- `mainland-worker-01` acceptance run `pbr-137e8b258b` + - 仍是 `queued / running` +- `release-launchpad` 当前推荐动作: + - `run_acceptance` + - 推荐目标: + - `mainland-controller-01` +- `log_sync_state = partial_coverage` + - 未覆盖节点: + - `mainland-worker-01` + - `mainland-controller-01` + +## 最新总检摘要 + +### go-live-summary + +- `go_live_status = attention` +- `publish_status = attention` +- `publish_blocking_reasons = []` +- `next_step_action_code = focus_playbook_run` + +说明: + +- 当前不是硬阻断 +- 但仍建议先把 playbook / acceptance 收口完再正式发版 + +### stack-diagnosis + +- `surface_status = healthy` +- `automation_status = attention` +- `stack_status = attention` +- 当前主推荐动作: + - `focus_playbook_run` + +说明: + +- 接口面与 contract 面没有缺口 +- 当前问题集中在运行编排层 + +### release-launchpad + +- latest package 可用 +- latest release `ready` +- default rollout gate `ready` +- 当前 launchpad 仍显示阻断,原因不是 bootstrap,而是: + - `Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。` +- 推荐命令: + - `bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli` + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做这一件事: + +- 对 `mainland-controller-01` 发起 acceptance + +然后依次做: + +1. 观察 `mainland-controller-01` acceptance run +2. 跟完 `mainland-worker-01` acceptance run `pbr-137e8b258b` +3. 复查 playbook runs 是否仍有 `focus_playbook_run` +4. 再复查 `go-live-summary / stack-diagnosis / release-launchpad` +5. 输出上线签收证据摘要 + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +原因: + +- 当前阶段是高耦合收口,不适合切低模型 +- 需要稳定判断 acceptance、playbook、launchpad 和 go-live 之间的关联关系 + +## 现在不要做的事情 + +不要做: + +- 新页面 +- 新模块 +- 控制面增强项 +- 新架构文档扩写 +- 接管之外的自动化扩展 +- 在 acceptance 未收口前提前进入正式 Rollout + +## 上线前证据摘要 + +已经具备的证据: + +- 大陆节点接管完成 +- 3 台节点均在线 +- 发布包可用 +- smoke test 通过 +- final release gate 为 `ready` +- delivery queue 当前 3/3 健康 + +尚缺的证据: + +- controller acceptance 完成结果 +- worker acceptance 完成结果 +- playbook run 队列收口结果 +- log sync partial 覆盖问题的最终结论 + +## 一句话结论 + +项目已经从“接管闭环阶段”进入“上线前证据收口阶段”;当前真正拦在前面的,只剩 acceptance 与 playbook / log sync 这类运行证据,还不是代码能力问题。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0035.md b/docs/ops_center_runtime/HANDOFF_20260419_0035.md new file mode 100644 index 0000000..7ebbf2d --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0035.md @@ -0,0 +1,119 @@ +# HANDOFF 2026-04-19 00:35 + +## 当前最高优先级 + +唯一最高优先级: + +- 跟完 acceptance run,拿到可签收终态 + +原因: + +- acceptance 已全部发起 +- 当前接管不是问题 +- 当前真正卡住的是 acceptance 与 playbook run 收口 + +## 本轮已完成的步骤 + +已完成: + +1. 对 `mainland-controller-01` 发起 acceptance +2. 复核 `mainland-worker-01` acceptance run `pbr-137e8b258b` +3. 复查: + - `go-live-summary` + - `stack-diagnosis` + - `release-launchpad` +4. 输出 acceptance 收口链判断 + +## 本轮执行摘要 + +### controller acceptance 结果 + +- `run_code = pbr-bfed43ea46` +- 当前状态:`running` +- 当前焦点:`health` +- 当前说明: + - `采集健康快照 当前处于排队中,建议先看事件和节点现场输出。` + +### worker acceptance 结果 + +- `run_code = pbr-137e8b258b` +- 当前状态:`running` +- 当前焦点:`health` +- 当前说明: + - `采集健康快照 当前处于排队中,建议先看事件和节点现场输出。` + +### go-live-summary 当前状态 + +- `go_live_status = attention` +- `publish_status = attention` +- `stack_status = attention` +- `launchpad_status = attention` +- `next_step_action_code = focus_playbook_run` + +### stack-diagnosis 当前状态 + +- `surface_status = healthy` +- `automation_status = attention` +- `stack_status = attention` +- `issue_total = 2` +- `blocking_issue_total = 0` +- `next_step_action_code = focus_playbook_run` + +### release-launchpad 当前推荐动作 + +- `status = blocked` +- `recommended_action_code = run_acceptance` +- `recommended_target_node_code = mainland-controller-01` +- 当前推荐摘要: + - `Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。` + +### log_sync_state` + +- 当前仍为:`partial_coverage` +- 未覆盖节点: + - `mainland-worker-01` + - `mainland-controller-01` + +## 当前是否可上线签收 + +当前结论: + +- 还不可上线签收 + +## 若仍不可签收,唯一剩余阻塞是什么 + +唯一剩余阻塞: + +- acceptance playbook run 尚未完成,没有形成可签收终态 + +更具体地说: + +- `pbr-bfed43ea46` 仍在运行 +- `pbr-137e8b258b` 仍在运行 +- 两条 run 都停在排队中的 `health` 步骤 + +当前没有出现明确失败,但也没有成功收口。 + +## 现在不要做的事情 + +不要做: + +- 正式发布动作 +- 新功能开发 +- 新模块 / 新页面 +- 任何与 acceptance 收口无关的实现 + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做: + +- 再次检查 `pbr-bfed43ea46` 与 `pbr-137e8b258b` 是否进入终态 + +如果 acceptance 仍卡在排队: + +- 只记录阻塞点、影响范围和建议下一步 +- 不扩展实现来绕过阻塞 + +## 一句话结论 + +acceptance 收口链已经全部发起,但两条验收编排都还停在队列中;当前离上线签收只差“acceptance 形成终态证据”,不是差功能,不是差接管,也不是差发布包。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0050.md b/docs/ops_center_runtime/HANDOFF_20260419_0050.md new file mode 100644 index 0000000..0ff6593 --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0050.md @@ -0,0 +1,135 @@ +# HANDOFF 2026-04-19 00:50 CST + +## 本轮目标 + +只处理 acceptance 队列不消费问题,不扩功能,不新增页面或专题文档。 + +## 本轮完成 + +### 1. 定位并修复 acceptance 队列卡死 + +定位结果: + +- 两台大陆节点都在持续请求 `/api/v1/ops/agent/pull?limit=1` +- 控制面收到请求,但 acceptance job 长时间停留在 `queued` +- 数据库中存在大量 `idle in transaction` +- 事务卡在: + - `UPDATE ops_job_steps ...` + - 随后又在新连接里 `INSERT INTO ops_job_events ...` + +代码修复: + +- 文件: + - [domain-api/app/services/ops_agent_service.py](/www/wwwroot/getDomain/domain-api/app/services/ops_agent_service.py) +- 修改点: + - `agent_pull_jobs` + - `agent_mark_job_started` + - `agent_complete_job` +- 修复方式: + - 把 `append_ops_job_event(...)` 从事务内部移到 `conn.commit()` 之后执行 + +### 2. 补充最小回归测试 + +- 文件: + - [domain-api/tests/test_ops_agent_service.py](/www/wwwroot/getDomain/domain-api/tests/test_ops_agent_service.py) +- 新增测试: + - `test_agent_pull_jobs_commits_before_appending_event` + - `test_agent_mark_job_started_commits_before_appending_event` + - `test_agent_complete_job_commits_before_appending_event` +- 验证结果: + - `cd /www/wwwroot/getDomain/domain-api && /opt/domaincheck/domainCheck/.venv/bin/python -m unittest tests.test_ops_agent_service -q` + - `Ran 27 tests ... OK` + +### 3. 运行态验证 + +- `domaincheck-api` 已完成强制切换到新版本进程 +- 新进程: + - `MainPID = 614403` +- 运行态日志已确认: + - `job 36 / 41` 开始执行 + - 已出现 `start / events / complete` + +## 当前 acceptance 结果 + +### `mainland-worker-01` + +- run: + - `pbr-137e8b258b` +- 当前状态: + - `attention` +- 结果: + - `4 success + 1 failed` +- 唯一失败步骤: + - `node_agent_logs` +- 失败原因: + - `No journal files were opened due to insufficient permissions` + +### `mainland-controller-01` + +- run: + - `pbr-bfed43ea46` +- 当前状态: + - `attention` +- 结果: + - `4 success + 1 failed` +- 唯一失败步骤: + - `node_agent_logs` +- 失败原因: + - `No journal files were opened due to insufficient permissions` + +## 当前总检状态 + +`go-live-summary`: + +- `go_live_status = attention` +- `publish_status = attention` +- `stack_status = attention` +- `launchpad_status = attention` +- `log_sync_state = full_capture` + +`stack-diagnosis`: + +- `surface_status = healthy` +- `automation_status = attention` +- `stack_status = attention` +- `issue_total = 1` +- `blocking_issue_total = 0` + +## 当前最高优先级 + +唯一最高优先级: + +- 修复远端 Node Agent 读取 `journalctl` 的权限 + +## 当前是否可上线签收 + +结论: + +- 还不可上线签收 + +唯一剩余阻塞: + +- 两条 acceptance run 都因为 `node_agent_logs` 权限不足而未全绿 + +## 下一轮若继续,必须先做什么 + +只做这一件事: + +- 处理 `domaincheck-node-agent` 所在服务账号的 journal 读取权限 + +推荐最小方向: + +- 让运行 Node Agent 的账号具备读取 systemd journal 的权限 +- 然后重跑 acceptance,或只补采失败日志步骤 + +## 现在不要做的事情 + +- 不要回退这次热修复 +- 不要重新 bootstrap +- 不要重装大陆机器 +- 不要扩控制面功能 +- 不要进入正式发布 + +## 本轮一句话结论 + +接管主链已经打通,acceptance 队列事务 bug 已修复;当前只剩远端 `journalctl` 权限问题,解决后即可继续冲刺上线签收。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0101.md b/docs/ops_center_runtime/HANDOFF_20260419_0101.md new file mode 100644 index 0000000..a280627 --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0101.md @@ -0,0 +1,156 @@ +# HANDOFF 2026-04-19 01:01 + +## 当前最高优先级 + +唯一最高优先级: + +- 完成 `B2-Journal Permission Closure` + +原因: + +- 接管链路已经闭合 +- acceptance 队列不消费问题已经修复 +- 当前两条 acceptance run 只剩同一个失败步骤: + - `node_agent_logs` + +## 本轮处理结论 + +本轮没有继续扩展控制面,也没有新增页面或模块。 + +本轮只做了两件事: + +1. 把剩余阻塞进一步收敛为明确的环境落点: + - `domaincheck-node-agent.service` 以 `www:www` 运行 + - 现网 unit 没有给 `www` 补 `journalctl` 所需的读取权限 +2. 在仓库中的 Node Agent systemd 模板加入: + - `SupplementaryGroups=systemd-journal` + +文件变更: + +- `domain-api/deploy/systemd/domain-node-agent.service` + +## 当前状态 + +当前摘要: + +- `pending_bootstrap = 0` +- `agent_ready = 3` +- `remote_access_ready = 3` +- `managed_enabled = 3` +- `log_sync_state = full_capture` + +acceptance 摘要: + +- `mainland-worker-01` + - `run_code = pbr-137e8b258b` + - `status = attention` + - `4 success + 1 failed` + - 唯一失败步骤:`node_agent_logs` +- `mainland-controller-01` + - `run_code = pbr-bfed43ea46` + - `status = attention` + - `4 success + 1 failed` + - 唯一失败步骤:`node_agent_logs` + +## 当前唯一剩余阻塞 + +唯一剩余阻塞: + +- 两台大陆节点尚未刷新新的 Node Agent systemd unit + +准确卡点: + +- 代码仓库里已经补了 `SupplementaryGroups=systemd-journal` +- 但远端节点上的 `/etc/systemd/system/domaincheck-node-agent.service` 还需要重新安装并重启服务 + +阻塞分类: + +- 环境 / 权限问题 + +不是以下问题: + +- 不是 SSH 问题 +- 不是 agent offline +- 不是 acceptance 队列不消费 +- 不是控制面接口 404 + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做这一件事: + +- 在 `mainland-controller-01` 与 `mainland-worker-01` 上重新安装最新 Node Agent unit,并重启 `domaincheck-node-agent` + +然后依次做: + +1. 确认 unit 已带 `SupplementaryGroups=systemd-journal` +2. 重跑两台节点 acceptance +3. 复核: + - `go-live-summary` + - `stack-diagnosis` + - `release-launchpad` +4. 输出上线签收证据摘要 + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +原因: + +- 当前阶段是最后一段 acceptance 收口 +- 需要稳定处理运行证据与状态对账 +- 不需要切到超高推理 + +## 现在不要做的事情 + +不要做: + +- 新功能 +- 新页面 +- 新模块 +- 控制面增强项 +- 新专题文档 +- 正式发布动作 + +## 给用户的下一任务卡片 + +按下面这张任务卡,在两台大陆节点执行: + +```text +任务名:B2-Journal Permission Closure / 节点 Unit 刷新 + +目标: +1. 让 domaincheck-node-agent 具备 journal 读取权限 +2. 为 acceptance 清掉 node_agent_logs 的最后失败点 + +执行节点: +- mainland-controller-01 +- mainland-worker-01 + +执行步骤: +1. 进入最新代码目录 +2. 重新安装最新 unit 文件到 /etc/systemd/system/domaincheck-node-agent.service +3. daemon-reload +4. restart domaincheck-node-agent +5. 输出 unit 关键行与服务状态 + +执行命令: +cd /www/wwwroot/getDomain +install -m 0644 domain-api/deploy/systemd/domain-node-agent.service /etc/systemd/system/domaincheck-node-agent.service +systemctl daemon-reload +systemctl restart domaincheck-node-agent +systemctl cat domaincheck-node-agent | grep -E '^(User|Group|SupplementaryGroups)=' +systemctl status domaincheck-node-agent --no-pager -l +journalctl -u domaincheck-node-agent -n 60 --no-pager + +回传给 Codex 的结果: +1. systemctl cat 中是否出现 SupplementaryGroups=systemd-journal +2. systemctl status 是否 active (running) +3. journalctl 是否还出现 insufficient permissions +``` + +## 一句话结论 + +项目现在不缺接管能力,也不缺控制面能力;真正剩下的只是把两台大陆节点上的 Node Agent unit 刷新到最新,然后重跑 acceptance 把最后一个 `node_agent_logs` 失败点清掉。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0107.md b/docs/ops_center_runtime/HANDOFF_20260419_0107.md new file mode 100644 index 0000000..ad3832a --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0107.md @@ -0,0 +1,126 @@ +# HANDOFF 2026-04-19 01:07 + +## 当前最高优先级 + +唯一最高优先级: + +- 执行 `C1-Acceptance Re-run` + +原因: + +- `B2-Journal Permission Closure` 在运行态上已经具备完成证据 +- 两台节点都已出现: + - `logs.collect ok=True` + - `domaincheck-node-agent` 重启成功 + - `registered: Agent 注册成功` +- 当前只剩旧 acceptance 结果尚未被新运行态覆盖 + +## 本轮确认到的运行态证据 + +`mainland-controller-01`: + +- 旧执行记录中已经出现: + - `action=logs.collect ok=True` +- 之后又完成: + - `domaincheck-node-agent.service` 停止 / 启动 + - `starting node agent` + - `registered: Agent 注册成功` + +`mainland-worker-01`: + +- 旧执行记录中已经出现: + - `action=logs.collect ok=True` +- 之后也完成: + - `domaincheck-node-agent.service` 停止 / 启动 + - `starting node agent` + - `registered: Agent 注册成功` + +## 当前判断 + +当前可以做出的最小结论: + +- 日志权限收口在运行面已经打通 +- Node Agent 当前在线且能重新注册 +- 不需要再回头排查 SSH、bootstrap、heartbeat、队列消费 + +当前还不能直接签收上线的唯一原因: + +- 控制面上仍保留旧 acceptance run 的 `attention` 结果 + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做这一件事: + +- 分别对 `mainland-controller-01` 和 `mainland-worker-01` 重跑 acceptance + +然后依次做: + +1. 记录新的 acceptance `run_code` +2. 确认两条 run 是否全绿 +3. 复核: + - `go-live-summary` + - `stack-diagnosis` + - `release-launchpad` +4. 输出上线签收证据摘要 + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +原因: + +- 当前还是高耦合收口阶段 +- 但已经不需要切超高推理 + +## 现在不要做的事情 + +不要做: + +- 新功能 +- 新页面 +- 新模块 +- 控制面增强项 +- 新专题文档 +- 正式发布动作 + +## 给用户的下一任务卡片 + +按下面这张任务卡继续: + +```text +任务名:C1-Acceptance Re-run / 两台大陆节点验收重跑 + +目标: +1. 用新的运行态覆盖旧 acceptance 失败结果 +2. 确认 mainland-controller-01 与 mainland-worker-01 是否都已通过标准接管验收 +3. 为上线签收准备最终证据 + +执行位置: +- 海外控制面主机 + +执行命令: +cd /www/wwwroot/getDomain +bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli +bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-worker-01 cli + +然后继续执行: +curl -s http://127.0.0.1:8100/api/v1/ops/go-live-summary +echo +curl -s http://127.0.0.1:8100/api/v1/ops/stack-diagnosis +echo +curl -s http://127.0.0.1:8100/api/v1/ops/release-launchpad + +回传给 Codex 的结果: +1. controller 新的 acceptance run_code 和最终状态 +2. worker 新的 acceptance run_code 和最终状态 +3. go-live-summary 当前状态 +4. stack-diagnosis 当前状态 +5. release-launchpad 当前推荐动作 +``` + +## 一句话结论 + +当前项目已经越过“日志权限修复”这一步,下一步不需要再修环境,只需要把两台大陆节点的 acceptance 重跑一遍,用新结果完成上线前签收判断。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0115.md b/docs/ops_center_runtime/HANDOFF_20260419_0115.md new file mode 100644 index 0000000..0bfe54e --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0115.md @@ -0,0 +1,133 @@ +# HANDOFF 2026-04-19 01:15 + +## 当前最高优先级 + +唯一最高优先级: + +- 执行 `D1-Evidence Convergence` + +原因: + +- 大陆两台 acceptance 已经全绿 +- 当前不再是接管链路问题 +- 当前剩余的是“上线签收证据口径”没有完全收敛 + +## 本轮已完成 + +`mainland-controller-01`: + +- acceptance 新 run: + - `pbr-9ce5c85f17` +- 最终结果: + - `status = success` + - `5 success / 0 failed` + +`mainland-worker-01`: + +- acceptance 新 run: + - `pbr-389abd618c` +- 最终结果: + - `status = success` + - `5 success / 0 failed` + +总检现状: + +- `go_live_status = attention` +- `stack_status = attention` +- `publish_status = attention` +- `launchpad_status = attention` +- `log_sync_state = partial_coverage` +- `log_sync_missing_node_codes = [overseas-control-01]` +- `stack_diagnosis.issue_total = 2` +- `stack_diagnosis.blocking_issue_total = 0` + +## 当前判断 + +当前已经可以确认: + +- 大陆节点接管闭环已完成 +- acceptance 已通过 +- Node Agent 日志权限问题已真实解除 + +当前还不能直接签收上线的原因只有一类: + +- 证据口径还没完全收敛 + +具体表现为两处: + +- `overseas-control-01` 还没有补齐日志样本,导致 `log_sync_state = partial_coverage` +- `stack-diagnosis / release-launchpad` 仍保留历史 attention / stale 推荐 + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做这一件事: + +- 给 `overseas-control-01` 补一轮场景日志样本 + +然后依次做: + +1. 复查 `go-live-summary` +2. 复查 `stack-diagnosis` +3. 复查 `release-launchpad` +4. 判断 attention 是否已收敛到可上线签收 + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +原因: + +- 当前还是收口判断阶段 +- 需要稳定地辨别“真实阻塞”和“口径残留” +- 但还不需要切到超高推理 + +## 现在不要做的事情 + +不要做: + +- 新功能 +- 新页面 +- 新模块 +- 控制面增强项 +- 新专题文档 +- 正式发布动作 + +## 下一个任务卡 + +```text +任务名:D1-Evidence Convergence / overseas-control-01 日志补样与总检复核 + +目标: +1. 补齐 overseas-control-01 的场景日志样本 +2. 观察 go-live-summary 是否从 partial_coverage 收敛 +3. 观察 stack-diagnosis 与 release-launchpad 是否仍保留历史残留口径 +4. 判断是否达到可上线签收状态 + +执行位置: +- 海外控制面主机 + +执行命令: +cd /www/wwwroot/getDomain +bash domain-api/deploy/multi-region/drive_ops_center.sh scene-node-log http://127.0.0.1:8100 overseas-control-01 120 full + +然后继续执行: +curl -s http://127.0.0.1:8100/api/v1/ops/go-live-summary +echo +curl -s http://127.0.0.1:8100/api/v1/ops/stack-diagnosis +echo +curl -s http://127.0.0.1:8100/api/v1/ops/releases/launchpad + +回传给 Codex 的结果: +1. overseas-control-01 场景日志是否成功拉到样本 +2. go-live-summary 当前状态 +3. stack-diagnosis 当前 issue_total / blocking_issue_total +4. release-launchpad 当前推荐动作 +5. 当前是否已经达到可上线签收 +``` + +## 一句话结论 + +acceptance 这一关已经过了;下一轮不该再回头修大陆节点,而是只把 `overseas-control-01` 的日志样本和总检残留口径收敛掉。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0121.md b/docs/ops_center_runtime/HANDOFF_20260419_0121.md new file mode 100644 index 0000000..795a16a --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0121.md @@ -0,0 +1,128 @@ +# HANDOFF 2026-04-19 01:21 + +## 当前最高优先级 + +唯一最高优先级: + +- 执行 `E1-Historical Signal Convergence` + +原因: + +- `overseas-control-01` 的日志样本已经补齐 +- `go-live-summary.log_sync_state` 已恢复为 `full_capture` +- 当前不再是样本缺口问题,而是历史 attention 口径未完全收敛 + +## 本轮已完成 + +日志样本补齐: + +- 已执行 `log-sync-recover` +- 已执行 `run_inspection_participating` +- 新巡检 run: + - `pbr-df533c6f4a` +- `overseas-control-01`: + - `scene-log status = full_capture` + - `line_count = 1` + +总检变化: + +- `go-live-summary` + - `log_sync_state: partial_coverage -> full_capture` + - `log_sync_missing_node_codes: [overseas-control-01] -> []` +- `stack-diagnosis` + - `issue_total: 2 -> 1` + - `blocking_issue_total = 0` +- `playbook_runs.problem_runs_total` + - `4 -> 3` + +## 当前判断 + +当前已经可以确认: + +- 大陆节点接管闭环已完成 +- acceptance 已通过 +- participating 节点日志样本覆盖已完成 + +当前还不能直接签收上线的原因只有一类: + +- 历史 playbook / launchpad attention 口径还没完全消化掉 + +具体表现为两处: + +- `stack-diagnosis` 还剩: + - `playbook_runs_need_attention` +- `release-launchpad` 还保留旧推荐: + - `run_acceptance` + - `mainland-controller-01` + +## 下一轮若继续,第一步先做什么 + +下一轮第一步只做这一件事: + +- 聚焦 `focus_playbook_run` + +然后依次做: + +1. 查清 `problem_runs_total = 3` 是不是纯历史旧 run +2. 复查 `stack-diagnosis` +3. 复查 `release-launchpad` +4. 判断当前是否已经达到可上线签收 + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +原因: + +- 当前已经进入最后的收口判断阶段 +- 需要高质量区分“真阻塞”和“历史口径残留” +- 还不需要切超高推理 + +## 现在不要做的事情 + +不要做: + +- 新功能 +- 新页面 +- 新模块 +- 控制面增强项 +- 新专题文档 +- 正式发布动作 + +## 下一个任务卡 + +```text +任务名:E1-Historical Signal Convergence / 历史 playbook attention 收口 + +目标: +1. 确认 stack-diagnosis 剩余的唯一告警是不是纯历史旧 run +2. 判断 release-launchpad 的旧 acceptance 推荐是否只是残留口径 +3. 为上线签收做最后判断 + +执行位置: +- 海外控制面主机 + +执行命令: +cd /www/wwwroot/getDomain +bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run +echo +bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_playbook_run_latest_events +echo +curl -s http://127.0.0.1:8100/api/v1/ops/stack-diagnosis +echo +curl -s http://127.0.0.1:8100/api/v1/ops/releases/launchpad + +回传给 Codex 的结果: +1. focus_playbook_run 指向的具体 run_code +2. 该 run 是否是历史旧 run +3. stack-diagnosis 当前 issue_total +4. release-launchpad 当前推荐动作 +5. 当前是否已经达到可上线签收 +``` + +## 一句话结论 + +日志样本缺口已经补上了;下一轮不该再碰接管和日志链路,而是只把最后那条历史 attention 口径清掉。 diff --git a/docs/ops_center_runtime/HANDOFF_20260419_0225.md b/docs/ops_center_runtime/HANDOFF_20260419_0225.md new file mode 100644 index 0000000..04a43b8 --- /dev/null +++ b/docs/ops_center_runtime/HANDOFF_20260419_0225.md @@ -0,0 +1,129 @@ +# HANDOFF 2026-04-19 02:25 + +## 本轮完成了什么 + +本轮只收口了“大陆 controller 无法 `pull_tasks`”这一条链。 + +实际完成: + +1. 修复 `domaincheck-node-agent` 读取不到同步目标地址的问题 +2. 以 `www` 提交并推送: + - `74dc009` + - `fix: inherit sync env in node agent` +3. 在两台大陆节点刷新 `domaincheck-node-agent.service` +4. 重启并验证: + - `mainland-controller-01` + - `mainland-worker-01` +5. 从海外控制面再次发起: + - `runtime.pull_tasks` + - `job_id = 87` +6. 验证 `mainland-controller-01` 返回成功: + - `pull_state = success` + - `items_total = 50` +7. 验证大陆两台节点当前都处于: + - `participation_state = running` + +## 当前最高优先级 + +唯一最高优先级: + +- 收敛检测总览统计口径 + +原因: + +- 现在不是“没跑” +- 而是“已经跑了,但总览没完整体现大陆执行量” + +## 当前状态摘要 + +当前结论: + +- 检测执行链已跑通 +- 大陆 controller 已能拉任务 +- 大陆 controller / worker 已进入执行中 + +直接证据: + +- `ops/nodes` + - `mainland-controller-01` + - `items_total = 200` + - `items_running = 5` + - `mainland-worker-01` + - `items_total = 70` + - `items_running = 5` +- `job 87` + - `runtime.pull_tasks` + - `status = success` + +## 当前唯一剩余问题 + +唯一剩余问题: + +- `detect/queue-summary` +- `detect/job/active` + +仍主要沿用旧的海外活跃任务口径,没有把大陆“拉批后本地执行”的量体现在同一汇总里。 + +这会导致页面观感像“大陆没跑”,但实际上已经在跑。 + +## 下一轮如果继续,只做什么 + +下一轮只做这一件事: + +- 对齐以下三处的数据口径: + - `ops/nodes` + - `detect/queue-summary` + - `detect/job/active` + +目标: + +- 让后台页面既能显示大陆节点正在执行 +- 又能把这部分执行量合并进统一检测进度 + +## 现在不要做的事情 + +不要做: + +- 新页面 +- 新模块 +- 运维中枢增强 +- 正式发布 +- 与检测总览收口无关的任何工作 + +## 推荐模型与推理等级 + +继续推荐: + +- `GPT-5.4` +- `high` + +原因: + +- 当前是数据口径排查与小范围实现 +- 不需要切超高 + +## 给下一轮的任务卡 + +```text +任务名:G1-检测总览口径统一批 + +目标: +1. 查清 ops/nodes 与 detect/job/active 的数据来源差异 +2. 明确大陆 controller 拉批执行为什么没有进入统一检测总览 +3. 修好后再跑一轮小批量检测验证 + +范围限制: +1. 只做检测总览口径统一 +2. 不新增页面 +3. 不扩展控制面功能 +4. 不进入发布动作 + +完成判定: +1. 点击启动检测后 +2. 后台能看到大陆节点参与 +3. detect/queue-summary 不再只显示 overseas-control-01 与 unassigned +``` + +## 一句话结论 + +这轮已经把“大陆 controller 拉不到任务”这个硬阻塞清掉了;现在真正剩下的不是执行问题,而是后台统计口径还没把大陆执行量完整显示出来。 diff --git a/docs/ops_center_runtime/IMPLEMENTATION_STATUS.md b/docs/ops_center_runtime/IMPLEMENTATION_STATUS.md new file mode 100644 index 0000000..55f03ac --- /dev/null +++ b/docs/ops_center_runtime/IMPLEMENTATION_STATUS.md @@ -0,0 +1,166 @@ +# IMPLEMENTATION_STATUS + +更新时间:2026-04-19 02:25 CST + +## 当前真实状态 + +阶段判断: + +- 海外单脑接管能力:约 `95%` +- 分布式检测真实跑通程度:约 `89%` +- 距离“可稳定上线并放心用后台发起检测”:约 `90%` + +当前最重要的新事实: + +- 大陆节点接管闭环已完成 +- acceptance 主链已完成 +- `runtime.pull_tasks` 的环境阻塞已解决 +- 大陆检测链已开始真实执行 +- 当前剩余问题已收敛为“检测总览统计口径未统一” + +## 本轮代码与环境收口 + +本轮代码变更: + +- 提交:`74dc009` +- 主题:`fix: inherit sync env in node agent` + +本轮实际变更点: + +- `domain-api/deploy/systemd/domain-node-agent.service` + - 增加: + - `EnvironmentFile=-/etc/default/domaincheck-api` +- `domain-api/deploy/multi-region/templates/domaincheck-node-agent.env.example` + - 补齐: + - `runtime.start_detection` + - `runtime.stop_detection` + - `runtime.pull_tasks` + +本轮环境动作: + +- 已在 `mainland-controller-01` 刷新并重启 `domaincheck-node-agent` +- 已在 `mainland-worker-01` 刷新并重启 `domaincheck-node-agent` +- 已确认两台节点 unit 包含: + - `/etc/default/domaincheck-api` + - `/etc/default/domaincheck-node-agent` + - `SupplementaryGroups=systemd-journal` + +## 关键验证证据 + +关键成功证据: + +- `mainland-controller-01` 新运维任务: + - `job_id = 87` + - `action = runtime.pull_tasks` + - `status = success` +- 返回结果: + - `pull_state = success` + - `items_total = 50` + - `summary_text = 待检测任务批次拉取并入库成功` + +远端日志证据: + +- `mainland-controller-01` + - `job start: id=87 ... action=runtime.pull_tasks` + - `job complete: ... ok=True` + +节点运行证据: + +- `mainland-controller-01` + - `participation_state = running` + - `items_running = 5` + - `items_total = 200` +- `mainland-worker-01` + - `participation_state = running` + - `items_running = 5` + - `items_total = 70` + +## 代码闭环 + +当前已经被验证打通的链路: + +- detect.start 触发远端动作排队 +- mainland controller 的 sync-agent / pull_tasks 目标地址读取 +- Node Agent 远端执行 `runtime.pull_tasks` +- 大陆 controller 拉批入库 +- 大陆 controller / worker 进入执行态 + +这意味着: + +- “点开始检测后大陆节点完全不参与”这个问题已经被修掉 + +## 外部条件闭环 + +已经闭合: + +- SSH +- node handover +- bootstrap +- register / heartbeat +- acceptance +- journald 权限 +- node-agent 同步目标地址继承 + +当前不再依赖用户补充的外部条件: + +- 不需要再补 `SYNC_TARGET_API_BASE_URL` +- 不需要再手工改 controller 的 node-agent env +- 不需要重装系统 + +## 当前剩余问题 + +当前唯一剩余问题: + +- 控制面“检测总览”口径仍未和大陆拉批执行口径统一 + +具体表现: + +- `ops/nodes` + - 明确显示大陆 controller / worker 正在执行 +- 但: + - `/api/v1/detect/queue-summary` + - `/api/v1/detect/job/active` + - 仍主要显示旧的: + - `overseas-control-01` + - `unassigned` + +因此当前判断是: + +- 执行链路已经通 +- 展示与统计口径还没完全通 + +## 当前是否可以直接上线 + +当前结论: + +- 可以继续做检测功能测试 +- 但还不建议宣称“检测总览已经完全可信、可以直接签收上线” + +原因不是执行失败,而是: + +- 总览统计仍会让人误判“大陆没跑” + +## 当前优先级判断 + +最高优先级: + +- 收敛 `detect/queue-summary` / `detect/job/active` 的统计口径 + +当前不应继续推进: + +- 新运维页面 +- 控制面增强 +- 新模块 +- 发布动作 +- 与检测总览收口无关的任何功能 + +## 完成下一轮后的预期 + +如果检测总览口径收敛成功: + +- 后台将能同时看到: + - 海外触发 + - 大陆拉批 + - 大陆执行 + - 统一进度统计 +- 整体可上线程度预计提升到 `94%~96%` diff --git a/docs/ops_center_runtime/NIGHT_PACKAGE_20260419_0121.md b/docs/ops_center_runtime/NIGHT_PACKAGE_20260419_0121.md new file mode 100644 index 0000000..b8a0177 --- /dev/null +++ b/docs/ops_center_runtime/NIGHT_PACKAGE_20260419_0121.md @@ -0,0 +1,41 @@ +# NIGHT PACKAGE 2026-04-19 01:21 + +执行窗口: + +- 开始时间:`2026-04-19 01:21 CST` +- 截止时间:`2026-04-20 12:00 CST` + +范围边界: + +- 只围绕当前收口目标执行 +- 不进入新实现 +- 不扩展控制面功能 +- 不新增页面、模块或专题文档 + +夜间任务包内容: + +1. 周期性复查: + - `go-live-summary` + - `stack-diagnosis` + - `release-launchpad` + - `playbook-runs` + - `overseas-control-01 scene-log` +2. 若日志样本再次出现缺口: + - 自动执行 `log-sync-recover` + - 自动执行一次 `run_inspection_participating` +3. 若只剩历史 `playbook_runs_need_attention`: + - 最多补 3 轮安全的 `inspection.standard` + - 目的仅为刷新近期成功 run,推动历史告警窗口收敛 +4. 到达以下任一条件即停止: + - 到 `2026-04-20 12:00 CST` + - 或者: + - `log_sync_state = full_capture` + - `issue_total = 0` + - `problem_runs_total = 0` + +输出位置: + +- 运行日志目录: + - [docs/ops_center_runtime/night_runs](/www/wwwroot/getDomain/docs/ops_center_runtime/night_runs) +- 停止后自动生成报告: + - `night_run_*_report.md` diff --git a/docs/ops_center_runtime/TASK_BOARD.md b/docs/ops_center_runtime/TASK_BOARD.md new file mode 100644 index 0000000..e71ec3d --- /dev/null +++ b/docs/ops_center_runtime/TASK_BOARD.md @@ -0,0 +1,139 @@ +# TASK_BOARD + +更新时间:2026-04-19 02:25 CST + +## 当前主批次 + +唯一主批次:`F1-分布式检测实跑收口批` + +目标: + +- 不进入新功能实现 +- 不扩展控制面页面 +- 只确认“海外发起检测 -> 大陆 controller 拉任务 -> 大陆节点执行”已经真实跑通 +- 收敛剩余唯一问题:控制面检测总览口径仍停留在旧活跃任务视图 + +## 当前结果快照 + +本轮新增完成: + +- 已修复 `domaincheck-node-agent` 读取不到同步目标地址的问题 +- 修复方式: + - `domaincheck-node-agent.service` 追加读取 `/etc/default/domaincheck-api` + - 新提交:`74dc009` +- 已在以下节点完成刷入与重启: + - `mainland-controller-01` + - `mainland-worker-01` + +本轮关键验证: + +- `mainland-controller-01` + - `runtime.pull_tasks` + - `job_id = 87` + - `status = success` + - `items_total = 50` + - `pull_state = success` +- controller 远端日志已确认: + - `job start: runtime.pull_tasks` + - `job complete ... ok=True` + +当前节点执行状态: + +- `mainland-controller-01` + - `agent_state = online_busy` + - `participation_state = running` + - `items_total = 200` + - `items_running = 5` +- `mainland-worker-01` + - `agent_state = online_busy` + - `participation_state = running` + - `items_total = 70` + - `items_running = 5` +- `overseas-control-01` + - `agent_state = online_busy` + +## 当前结论 + +检测不是“没跑”。 + +当前真实状态是: + +- 分布式检测执行链已经跑通 +- 大陆 controller 已经能从海外控制面主动拉回待检测批次 +- 大陆 controller 与 mainland worker 都已经进入执行中 + +当前未完全收敛的不是执行,而是统计口径: + +- `/api/v1/detect/queue-summary` +- `/api/v1/detect/job/active` + +仍主要显示旧的海外活跃任务统计: + +- `overseas-control-01` +- `unassigned` + +尚未把大陆侧“拉批后本地执行”的结果完整并回收到同一检测总览口径中。 + +## 当前唯一主问题 + +唯一主问题:`检测执行已跑通,但控制面检测总览仍是旧口径` + +表现: + +- `ops/nodes` 能看到大陆节点正在跑 +- `runtime.pull_tasks` 已成功 +- 但 `detect/queue-summary` 与 `detect/job/active` 仍没有把大陆执行量体现在同一总览里 + +问题分类: + +- 不是接管问题 +- 不是环境问题 +- 不是同步目标地址问题 +- 是“检测总览聚合口径 / 运行态映射”问题 + +## 候选批次 + +### Candidate G1 + +名称:检测总览口径统一批 + +进入条件: + +- 用户确认下一轮继续只收敛检测总览,不扩其他功能 + +范围: + +- 复核 `detect job` 与 `ops/nodes` 的数据来源差异 +- 明确大陆拉批执行应如何回写到统一检测总览 +- 修复后再做一次检测实跑验证 + +### Candidate G2 + +名称:节点能力口径清理批 + +进入条件: + +- 需要把节点展示中的 `capabilities` 与现网真实支持动作同步 + +范围: + +- 只处理 `OPS_AGENT_CAPABILITIES` 的现网漂移 +- 不碰检测链路主体 + +## 暂停项 + +继续暂停: + +- 新页面 +- 新模块 +- 运维中枢增强 +- 发布动作 +- 与“检测实跑收口”无关的任何工作 + +## 下一轮唯一动作 + +下一轮唯一应该做的事情: + +- 对齐 `detect/queue-summary`、`detect/job/active` 与 `ops/nodes` 三者的数据口径 +- 明确为什么大陆执行已发生,但总览仍显示旧任务统计 +- 修好后再跑一次小批量检测验证 diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012740.pid b/docs/ops_center_runtime/night_runs/night_run_20260419_012740.pid new file mode 100644 index 0000000..a37e558 --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012740.pid @@ -0,0 +1 @@ +680339 diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012740/cycle_1/go_live.json.tmp b/docs/ops_center_runtime/night_runs/night_run_20260419_012740/cycle_1/go_live.json.tmp new file mode 100644 index 0000000..e69de29 diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929.pid b/docs/ops_center_runtime/night_runs/night_run_20260419_012929.pid new file mode 100644 index 0000000..303961c --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929.pid @@ -0,0 +1 @@ +683962 diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/go_live.json b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/go_live.json new file mode 100644 index 0000000..8595482 --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/go_live.json @@ -0,0 +1 @@ +{"code":0,"message":"ok","data":{"base_url":"http://127.0.0.1:8100","generated_at":"2026-04-19 01:29:38","go_live_status":"attention","publish_ready":false,"publish_status":"attention","publish_status_label":"可发布但建议先复核","publish_summary":"当前没有硬阻断,但仍有上线前关注项,建议先完成复核再正式发版。","stack_status":"attention","contracts_ready":true,"contracts_total":11,"launchpad_status":"attention","launchpad_status_label":"待补执行面","launchpad_recommended_action_code":"fix_managed_nodes","launchpad_recommended_target_node_code":"","launchpad_recommended_recovery_label":"","launchpad_recommended_recovery_summary":"来自 overview.recommendation.primary_action_code","launchpad_onboarding_bootstrap_pending_nodes":0,"launchpad_onboarding_acceptance_ready_nodes":0,"route_surface_complete":true,"route_surface_missing_keys":[],"route_surface_declares_bootstrap_plan":true,"runtime_schema_stale":false,"repository_capabilities":{"supports_install_command_block":true,"supports_multi_layout_bootstrap":true},"managed_enabled":3,"remote_access_ready":3,"queue_dead_letter_nodes":0,"activity_start_delivery_issue_total":0,"participating_nodes_total":3,"log_sync_enabled":true,"log_sync_state":"full_capture","log_sync_mode":"full","log_sync_covered_nodes":3,"log_sync_missing_node_codes":[],"next_step_action_code":"focus_playbook_run","next_step_reason":"来自 overview.recommendation.primary_action_code","operator_lane":"ops_jobs","operator_title":"按总检默认下一步继续处理","operator_primary_command_key":"focus_playbook_run","publish_blocking_reasons":[],"publish_warnings":["stack_diagnosis=attention","release_launchpad=attention"],"blocking_reasons":[],"warnings":["stack_diagnosis=attention","release_launchpad=attention"],"recommended_commands":{"stack_summary":"bash domain-api/deploy/multi-region/check_ops_center_stack.sh http://127.0.0.1:8100 summary","contracts":"bash domain-api/deploy/multi-region/check_ops_contracts.sh http://127.0.0.1:8100","ops_plane":"bash domain-api/deploy/multi-region/check_ops_plane.sh http://127.0.0.1:8100","release_hub":"bash domain-api/deploy/multi-region/check_release_hub.sh http://127.0.0.1:8100","inspection":"bash domain-api/deploy/multi-region/check_ops_inspection.sh http://127.0.0.1:8100","overview":"bash domain-api/deploy/multi-region/drive_ops_center.sh overview http://127.0.0.1:8100","go_live_recover":"bash domain-api/deploy/multi-region/drive_ops_center.sh go-live-recover http://127.0.0.1:8100","doctor_export":"bash domain-api/deploy/multi-region/drive_ops_center.sh doctor-export /tmp/domaincheck-go-live http://127.0.0.1:8100","next_step":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","log_sync_logs":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_worker_logs_participating","log_sync_inspection":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 run_inspection_participating"},"source_refs":{"stack_diagnosis_contract_key":"ops_stack_diagnosis_contract","contracts_registry_version":"2026-04-18","runtime_build_commit_sha":"246838ae4c07","release_focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_launchpad"}}},"detail_code":null} \ No newline at end of file diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/launchpad.json b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/launchpad.json new file mode 100644 index 0000000..58069b7 --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/launchpad.json @@ -0,0 +1 @@ +{"code":0,"message":"ok","data":{"channel":"stable","control_plane_base_url":"http://127.0.0.1:8100","latest_package":{"available":true,"package_name":"domaincheck_release_20260418_215930","package_type":"tar.gz","generated_at":"2026-04-18T21:59:32","commit_sha":"246838ae4c07","commit_ref":"main","smoke_test_ok":true,"smoke_test_message":"all checks passed (10/10)","final_release_ok":true,"final_release_report_available":true,"final_release_report_ok":true,"final_release_report_matches_latest":true,"final_release_report_package_name":"domaincheck_release_20260418_215930","final_release_report_stale_reason":"","final_release_gate_decision":"ready","final_release_gate_blocked_reasons":[],"final_release_gate_verify_ok":true,"final_release_gate_smoke_test_ok":true,"download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","sha256_download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/sha256","reason":"","manifest":{"artifact_kind":"tar.gz","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"2026-04-18T21:59:32","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""}},"latest_release":{"id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","commit_sha":"246838ae4c07","status":"ready","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_013833/download","artifact_url_present":true,"checksum":"9da7b89bf9796098f234f388944bab959114fbe667680e5582306b32e77b489d","created_by":"cli/ops-center/latest-package","created_at":"2026-04-18 01:38:35","updated_at":"2026-04-18 01:38:35","is_preview":false,"manifest":{"artifact_kind":"","package_name":"domaincheck_release_20260418_013833","package_size_bytes":0,"checksum":"9da7b89bf9796098f234f388944bab959114fbe667680e5582306b32e77b489d","build_id":"","commit_sha":"246838ae4c07","built_at":"2026-04-18 01:38:35","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""},"status_label":"已就绪","summary":"domaincheck_release_20260418_013833 已就绪,制品与校验信息齐备,可进入 Rollout。","summary_text":"domaincheck_release_20260418_013833 已就绪,制品与校验信息齐备,可进入 Rollout。","focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_detail"}},"release_summary":{"status_counts":{"ready":2},"total":2,"active_by_channel":{},"rollout_status_counts":{},"rollouts_total":0},"runtime_build_info":{"source":"latest_release","package_name":"domaincheck_release_20260418_215930","commit_sha":"246838ae4c07","commit_ref":"main","generated_at":"2026-04-18T21:59:32","route_surface_complete":true,"route_surface_missing_keys":[],"route_surface_declares_bootstrap_plan":true,"repository_supports_install_command_block":true,"runtime_schema_stale":false},"latest_package_preview":{"message":"最新发布包预览已生成","preview_source":"latest_package_draft","deduplicated":false,"release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","commit_sha":"246838ae4c07","status":"ready","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","artifact_url_present":true,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","created_by":"api/release-launchpad","created_at":"","updated_at":"","is_preview":true,"manifest":{"artifact_kind":"","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""},"status_label":"已就绪","summary":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","summary_text":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_detail"}},"package":{"available":true,"package_name":"domaincheck_release_20260418_215930","package_type":"tar.gz","generated_at":"2026-04-18T21:59:32","commit_sha":"246838ae4c07","commit_ref":"main","smoke_test_ok":true,"smoke_test_message":"all checks passed (10/10)","final_release_ok":true,"final_release_report_available":true,"final_release_report_ok":true,"final_release_report_matches_latest":true,"final_release_report_package_name":"domaincheck_release_20260418_215930","final_release_report_stale_reason":"","final_release_gate_decision":"ready","final_release_gate_blocked_reasons":[],"final_release_gate_verify_ok":true,"final_release_gate_smoke_test_ok":true,"download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","sha256_download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/sha256","reason":"","manifest":{"artifact_kind":"tar.gz","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"2026-04-18T21:59:32","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""}},"resolved_artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","resolved_sha256_download_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/sha256"},"worker_rollout_preview":{"message":"智能 Rollout 预检未通过","preview_source":"latest_package_draft","release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","commit_sha":"246838ae4c07","status":"ready","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","artifact_url_present":true,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","created_by":"api/release-launchpad","created_at":"","updated_at":"","is_preview":true,"manifest":{"artifact_kind":"","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""},"status_label":"已就绪","summary":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","summary_text":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_detail"}},"package":{"available":true,"package_name":"domaincheck_release_20260418_215930","package_type":"tar.gz","generated_at":"2026-04-18T21:59:32","commit_sha":"246838ae4c07","commit_ref":"main","smoke_test_ok":true,"smoke_test_message":"all checks passed (10/10)","final_release_ok":true,"final_release_report_available":true,"final_release_report_ok":true,"final_release_report_matches_latest":true,"final_release_report_package_name":"domaincheck_release_20260418_215930","final_release_report_stale_reason":"","final_release_gate_decision":"ready","final_release_gate_blocked_reasons":[],"final_release_gate_verify_ok":true,"final_release_gate_smoke_test_ok":true,"download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","sha256_download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/sha256","reason":"","manifest":{"artifact_kind":"tar.gz","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"2026-04-18T21:59:32","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""}},"smart_rollout":{"available":true,"mode":"worker","mode_label":"Worker 灰度","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","reason":"","summary":"已按 Worker 灰度 预选 1 台节点,执行路径采用 远端 Agent;优先锁定 mainland;首批 1 台,后续每批 1 台。","dominant_region":"mainland","target_node_codes":["mainland-worker-01"],"target_nodes":[{"node_code":"mainland-worker-01","region":"mainland","role":"worker","status":"busy","current_load":9,"remote_agent_ready":true,"ssh_ready":true,"execution_ready":true,"detect_participating":true,"delivery_queue_state":"healthy"}],"policy":{"auto_start":true,"auto_dispatch":false,"auto_advance":true,"auto_approve":false,"continue_on_failure":false,"continue_on_partial":false,"execution_mode":"remote-agent","first_batch_size":1,"batch_size":1,"deploy_payload":{"restart_services":["domaincheck-worker"],"health_check_urls":[],"health_check_services":["domaincheck-worker"],"health_check_timeout_seconds":10,"health_check_retries":2,"health_check_interval_seconds":2,"rollback_on_failure":true,"switch_current":true}}},"policy_preview":{"risk_level":"high","approval_required":true,"blocked":true,"blocking_reasons":["当前筛选条件未命中任何可 rollout 的目标节点。"],"approval_reasons":["该动作属于 high 风险动作,建议审批后执行。"],"warnings":[],"recommendations":[],"target_summary":{"nodes_total":0,"worker_nodes":0,"control_nodes":0,"effective_worker_nodes":0,"busy_nodes":0,"participating_nodes":0,"offline_nodes":0},"batch_plan":{"first_batch_size":0,"batch_size":0,"batches_total":0,"first_batch_nodes":[],"remaining_after_first_batch":0},"cluster_guardrails":{"online_worker_nodes":3,"dedicated_online_worker_nodes":1,"online_control_nodes":2,"busy_nodes":["mainland-worker-01","overseas-control-01"]},"release_gate":{"status":"no_targets","status_label":"无默认目标","status_type":"warning","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","summary":"当前没有在线且可视为有效执行面的预览 Rollout 目标节点。","release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","status":"ready","status_label":"已就绪","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","is_preview":true},"is_preview_release":true,"target_nodes":[],"target_node_codes":[],"blocking_reasons":[],"warning_reasons":[],"recommendations":[],"operational_readiness":{"summary":{"nodes_total":0,"managed_nodes":0,"managed_enabled_nodes":0,"agent_online_nodes":0,"ssh_ready_nodes":0,"remote_agent_ready_nodes":0,"execution_ready_nodes":0,"inspection_healthy_nodes":0,"inspection_running_nodes":0,"inspection_attention_nodes":0,"inspection_missing_nodes":0},"rows":[]},"summary_text":"当前没有在线且可视为有效执行面的预览 Rollout 目标节点。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate"}}},"rollout_preview":{"release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","commit_sha":"246838ae4c07","status":"ready","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","artifact_url_present":true,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","created_by":"api/release-launchpad","created_at":"","updated_at":"","is_preview":true,"manifest":{"artifact_kind":"","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""},"status_label":"已就绪","summary":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","summary_text":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_detail"}},"execution_mode":"remote-agent","execution_mode_label":"远端 Agent","target_nodes_total":0,"target_node_codes":["mainland-worker-01"],"expected_batches_total":0,"expected_jobs_total":0,"gate":{"status":"no_targets","status_label":"无默认目标","status_type":"warning","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","summary":"当前没有在线且可视为有效执行面的预览 Rollout 目标节点。","release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","status":"ready","status_label":"已就绪","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","is_preview":true},"is_preview_release":true,"target_nodes":[],"target_node_codes":[],"blocking_reasons":[],"warning_reasons":[],"recommendations":[],"operational_readiness":{"summary":{"nodes_total":0,"managed_nodes":0,"managed_enabled_nodes":0,"agent_online_nodes":0,"ssh_ready_nodes":0,"remote_agent_ready_nodes":0,"execution_ready_nodes":0,"inspection_healthy_nodes":0,"inspection_running_nodes":0,"inspection_attention_nodes":0,"inspection_missing_nodes":0},"rows":[]},"summary_text":"当前没有在线且可视为有效执行面的预览 Rollout 目标节点。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate"}},"batch_plan":{"first_batch_size":0,"batch_size":0,"batches_total":0,"first_batch_nodes":[],"remaining_after_first_batch":0},"summary":"当前没有在线且可视为有效执行面的预览 Rollout 目标节点。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"rollout_preview"},"summary_text":"当前没有在线且可视为有效执行面的预览 Rollout 目标节点。"},"requires_confirmation":false,"blocked":true,"rollout_created":false,"release_deduplicated":false},"control_rollout_preview":{"message":"智能 Rollout 需要确认","preview_source":"latest_package_draft","release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","commit_sha":"246838ae4c07","status":"ready","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","artifact_url_present":true,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","created_by":"api/release-launchpad","created_at":"","updated_at":"","is_preview":true,"manifest":{"artifact_kind":"","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""},"status_label":"已就绪","summary":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","summary_text":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_detail"}},"package":{"available":true,"package_name":"domaincheck_release_20260418_215930","package_type":"tar.gz","generated_at":"2026-04-18T21:59:32","commit_sha":"246838ae4c07","commit_ref":"main","smoke_test_ok":true,"smoke_test_message":"all checks passed (10/10)","final_release_ok":true,"final_release_report_available":true,"final_release_report_ok":true,"final_release_report_matches_latest":true,"final_release_report_package_name":"domaincheck_release_20260418_215930","final_release_report_stale_reason":"","final_release_gate_decision":"ready","final_release_gate_blocked_reasons":[],"final_release_gate_verify_ok":true,"final_release_gate_smoke_test_ok":true,"download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","sha256_download_path":"/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/sha256","reason":"","manifest":{"artifact_kind":"tar.gz","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"2026-04-18T21:59:32","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""}},"smart_rollout":{"available":true,"mode":"control","mode_label":"Control 发布","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","reason":"","summary":"已按 Control 发布 预选 1 台节点,执行路径采用 远端 Agent;优先锁定 mainland;首批 1 台,后续每批 1 台。","dominant_region":"mainland","target_node_codes":["mainland-controller-01"],"target_nodes":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"remote_agent_ready":true,"ssh_ready":true,"execution_ready":true,"detect_participating":true,"delivery_queue_state":"healthy"}],"policy":{"auto_start":true,"auto_dispatch":false,"auto_advance":true,"auto_approve":false,"continue_on_failure":false,"continue_on_partial":false,"execution_mode":"remote-agent","first_batch_size":1,"batch_size":1,"deploy_payload":{"restart_services":["domaincheck-api","domaincheck-worker","domaincheck-sync-agent"],"health_check_urls":["http://127.0.0.1:8100/health"],"health_check_services":["domaincheck-api","domaincheck-worker","domaincheck-sync-agent"],"health_check_timeout_seconds":10,"health_check_retries":2,"health_check_interval_seconds":2,"rollback_on_failure":true,"switch_current":true}}},"policy_preview":{"risk_level":"high","approval_required":true,"blocked":false,"blocking_reasons":[],"approval_reasons":["本次 rollout 覆盖 control 节点,建议强制审批,并把 control 放在最后一批。","该动作属于 high 风险动作,建议审批后执行。"],"warnings":["命中了 1 个当前有负载的节点,建议优先确认任务是否可中断。","命中了 1 个正在参与检测的节点,建议优先滚动到空闲节点。"],"recommendations":["有 1 台目标节点已进入验收窗口,建议先执行 onboarding.acceptance。","建议把 control 节点放在 rollout 末尾,并确保 worker 已先通过健康检查。","建议先等参与检测的节点降为空闲,再推进该批次。"],"target_summary":{"nodes_total":1,"worker_nodes":0,"control_nodes":1,"effective_worker_nodes":1,"busy_nodes":1,"participating_nodes":1,"offline_nodes":0},"batch_plan":{"first_batch_size":1,"batch_size":1,"batches_total":1,"first_batch_nodes":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"is_effective_worker":true,"detect_participating":true,"is_enabled":true,"deploy_channel":"stable"}],"remaining_after_first_batch":0},"cluster_guardrails":{"online_worker_nodes":3,"dedicated_online_worker_nodes":1,"online_control_nodes":2,"busy_nodes":["mainland-worker-01","overseas-control-01"]},"release_gate":{"status":"preview_ready","status_label":"预览可发 Rollout","status_type":"success","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","summary":"当前基于最新发布包推导出的 Release 预览与目标节点均已通过门禁,已具备进入定向或灰度 Rollout 的条件。","release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","status":"ready","status_label":"已就绪","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","is_preview":true},"is_preview_release":true,"target_nodes":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"is_effective_worker":true,"detect_participating":true,"is_enabled":true,"ssh_host":"121.204.244.188","ssh_user":"root","ssh_port":22,"auth_mode":"key","deploy_channel":"stable","title":"mainland-controller-01"}],"target_node_codes":["mainland-controller-01"],"blocking_reasons":[],"warning_reasons":[],"recommendations":["有 1 台目标节点已进入验收窗口,建议先执行 onboarding.acceptance。"],"operational_readiness":{"summary":{"nodes_total":1,"managed_nodes":1,"managed_enabled_nodes":1,"agent_online_nodes":1,"ssh_ready_nodes":1,"remote_agent_ready_nodes":1,"execution_ready_nodes":1,"inspection_healthy_nodes":1,"inspection_running_nodes":0,"inspection_attention_nodes":0,"inspection_missing_nodes":0,"queue_retrying_nodes":0,"queue_dead_letter_nodes":0,"onboarding_bootstrap_pending_nodes":0,"onboarding_acceptance_ready_nodes":1,"onboarding_noop_nodes":0},"rows":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"agent_managed":true,"is_managed":true,"is_enabled":true,"agent_state":"online_busy","agent_state_label":"已接管","agent_reason":"Agent 最近心跳 2 秒前,节点当前负载 5。","is_agent_online":true,"has_ssh_access":true,"ssh_ready":true,"remote_agent_ready":true,"execution_ready":true,"execution_mode":"remote-agent","execution_mode_label":"远端 Agent","inspection_status":"healthy","inspection_status_label":"巡检通过","inspection_reason":"最近一轮健康快照、Worker 日志和诊断包均已成功。","inspection_success_count":3,"inspection_ready":true,"inspection_status_map":{"health.snapshot":"success","logs.collect":"success","diagnostics.collect":"success"},"delivery_queue_state":"healthy","delivery_queue_label":"正常","delivery_queue_reason":"当前没有待重试回执,也没有死信记录。 最近一次队列冲刷:2026-04-19 14:29:22。","delivery_queue_pending_count":0,"delivery_queue_dead_letter_count":0,"current_release_version":"","desired_release_version":"domaincheck_release_20260418_215930","onboarding_stage_code":"acceptance_ready","onboarding_stage_label":"待接管验收","onboarding_summary":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","recovery_action":"run_acceptance","recovery_label":"执行接管验收","recovery_summary":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","recovery_command_hint":"bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli","recovery_window":"acceptance","risk_level":"low","blocking_reasons":[],"warning_reasons":[],"recommended_action_code":"","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate","node_code":"mainland-controller-01"}}]},"summary_text":"当前基于最新发布包推导出的 Release 预览与目标节点均已通过门禁,已具备进入定向或灰度 Rollout 的条件。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate"}}},"rollout_preview":{"release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","commit_sha":"246838ae4c07","status":"ready","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","artifact_url_present":true,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","created_by":"api/release-launchpad","created_at":"","updated_at":"","is_preview":true,"manifest":{"artifact_kind":"","package_name":"domaincheck_release_20260418_215930","package_size_bytes":0,"checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","build_id":"","commit_sha":"246838ae4c07","built_at":"","included_services":[],"required_env_keys":[],"health_checks":[],"rollback_hint":""},"status_label":"已就绪","summary":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","summary_text":"domaincheck_release_20260418_215930 已就绪,制品与校验信息齐备,可进入 Rollout。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_detail"}},"execution_mode":"remote-agent","execution_mode_label":"远端 Agent","target_nodes_total":1,"target_node_codes":["mainland-controller-01"],"expected_batches_total":1,"expected_jobs_total":1,"gate":{"status":"preview_ready","status_label":"预览可发 Rollout","status_type":"success","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","summary":"当前基于最新发布包推导出的 Release 预览与目标节点均已通过门禁,已具备进入定向或灰度 Rollout 的条件。","release":{"id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","status":"ready","status_label":"已就绪","artifact_url":"http://127.0.0.1:8100/api/v1/ops/releases/packages/domaincheck_release_20260418_215930/download","is_preview":true},"is_preview_release":true,"target_nodes":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"is_effective_worker":true,"detect_participating":true,"is_enabled":true,"ssh_host":"121.204.244.188","ssh_user":"root","ssh_port":22,"auth_mode":"key","deploy_channel":"stable","title":"mainland-controller-01"}],"target_node_codes":["mainland-controller-01"],"blocking_reasons":[],"warning_reasons":[],"recommendations":["有 1 台目标节点已进入验收窗口,建议先执行 onboarding.acceptance。"],"operational_readiness":{"summary":{"nodes_total":1,"managed_nodes":1,"managed_enabled_nodes":1,"agent_online_nodes":1,"ssh_ready_nodes":1,"remote_agent_ready_nodes":1,"execution_ready_nodes":1,"inspection_healthy_nodes":1,"inspection_running_nodes":0,"inspection_attention_nodes":0,"inspection_missing_nodes":0,"queue_retrying_nodes":0,"queue_dead_letter_nodes":0,"onboarding_bootstrap_pending_nodes":0,"onboarding_acceptance_ready_nodes":1,"onboarding_noop_nodes":0},"rows":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"agent_managed":true,"is_managed":true,"is_enabled":true,"agent_state":"online_busy","agent_state_label":"已接管","agent_reason":"Agent 最近心跳 2 秒前,节点当前负载 5。","is_agent_online":true,"has_ssh_access":true,"ssh_ready":true,"remote_agent_ready":true,"execution_ready":true,"execution_mode":"remote-agent","execution_mode_label":"远端 Agent","inspection_status":"healthy","inspection_status_label":"巡检通过","inspection_reason":"最近一轮健康快照、Worker 日志和诊断包均已成功。","inspection_success_count":3,"inspection_ready":true,"inspection_status_map":{"health.snapshot":"success","logs.collect":"success","diagnostics.collect":"success"},"delivery_queue_state":"healthy","delivery_queue_label":"正常","delivery_queue_reason":"当前没有待重试回执,也没有死信记录。 最近一次队列冲刷:2026-04-19 14:29:22。","delivery_queue_pending_count":0,"delivery_queue_dead_letter_count":0,"current_release_version":"","desired_release_version":"domaincheck_release_20260418_215930","onboarding_stage_code":"acceptance_ready","onboarding_stage_label":"待接管验收","onboarding_summary":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","recovery_action":"run_acceptance","recovery_label":"执行接管验收","recovery_summary":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","recovery_command_hint":"bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli","recovery_window":"acceptance","risk_level":"low","blocking_reasons":[],"warning_reasons":[],"recommended_action_code":"","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate","node_code":"mainland-controller-01"}}]},"summary_text":"当前基于最新发布包推导出的 Release 预览与目标节点均已通过门禁,已具备进入定向或灰度 Rollout 的条件。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate"}},"batch_plan":{"first_batch_size":1,"batch_size":1,"batches_total":1,"first_batch_nodes":[{"node_code":"mainland-controller-01","region":"mainland","role":"control","status":"online","current_load":5,"is_effective_worker":true,"detect_participating":true,"is_enabled":true,"deploy_channel":"stable"}],"remaining_after_first_batch":0},"summary":"当前基于最新发布包推导出的 Release 预览与目标节点均已通过门禁,已具备进入定向或灰度 Rollout 的条件。","focus_ref":{"kind":"release_hub","release_id":0,"release_version":"domaincheck_release_20260418_215930","channel":"stable","rollout_id":0,"rollout_code":"","section":"rollout_preview"},"summary_text":"当前基于最新发布包推导出的 Release 预览与目标节点均已通过门禁,已具备进入定向或灰度 Rollout 的条件。"},"requires_confirmation":true,"blocked":false,"rollout_created":false,"release_deduplicated":false},"launchpad_status":{"status":"blocked","status_label":"发布门禁阻断","summary":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","recommended_action_code":"run_acceptance","recommended_command":"bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli","recommended_execution_mode":"remote-agent","recommended_execution_mode_label":"远端 Agent","recommended_target_node_code":"mainland-controller-01","recommended_recovery_label":"执行接管验收","summary_text":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_launchpad"},"onboarding_bootstrap_pending_nodes":0,"onboarding_acceptance_ready_nodes":1,"onboarding_gap_nodes_total":1,"recommended_recovery_summary":"Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。","recommended_onboarding_stage_code":"acceptance_ready","recommended_onboarding_stage_label":"待接管验收"}},"detail_code":null} \ No newline at end of file diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/playbook_runs.json b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/playbook_runs.json new file mode 100644 index 0000000..c036d5c --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/playbook_runs.json @@ -0,0 +1 @@ +{"code":0,"message":"ok","data":{"runs":[{"run_code":"pbr-df533c6f4a","playbook_key":"inspection.standard","playbook_title":"标准巡检","group_key":"diagnostics","group_title":"标准巡检","requested_by":"cli/codex/run_inspection_participating","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":3,"expected_jobs_total":9,"target_nodes_total":3,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"created_at":"2026-04-19 01:20:34","updated_at":"2026-04-19 01:20:47","status_counts":{"success":9},"jobs_total":9,"success_jobs_total":9,"terminal_jobs_total":9,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":3,"nodes_total":3,"status_counts":{"success":3},"terminal_jobs_total":3,"success_jobs_total":3,"latest_job":{"id":68,"job_code":"ops-20260419012035-dfa01d","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:20:35","started_at":"2026-04-19 01:20:38","finished_at":"2026-04-19 01:20:40","updated_at":"2026-04-19 01:20:40","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-df533c6f4a","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":2,"jobs_total":3,"nodes_total":3,"status_counts":{"success":3},"terminal_jobs_total":3,"success_jobs_total":3,"latest_job":{"id":71,"job_code":"ops-20260419012035-634387","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:20:35","started_at":"2026-04-19 01:20:40","finished_at":"2026-04-19 01:20:42","updated_at":"2026-04-19 01:20:42","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-df533c6f4a","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}},{"step_key":"diagnostics","title":"收集诊断包","template_key":"diagnostics.collect","order":3,"jobs_total":3,"nodes_total":3,"status_counts":{"success":3},"terminal_jobs_total":3,"success_jobs_total":3,"latest_job":{"id":73,"job_code":"ops-20260419012036-4babe3","action":"diagnostics.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:20:36","started_at":"2026-04-19 01:20:43","finished_at":"2026-04-19 01:20:47","updated_at":"2026-04-19 01:20:47","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"收集诊断包 已完成收口。","summary_text":"收集诊断包 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-df533c6f4a","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"diagnostics","focus_step_title":"收集诊断包","event_key":"","node_code":""}}],"latest_job":{"id":73,"job_code":"ops-20260419012036-4babe3","action":"diagnostics.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:20:36","started_at":"2026-04-19 01:20:43","finished_at":"2026-04-19 01:20:47","updated_at":"2026-04-19 01:20:47","error_message":""},"status":"success","completion_percent":100.0,"problem_steps":[],"active_steps":[],"focus_level":"success","focus_step_key":"","focus_step_title":"","focus_summary":"当前整轮编排已全部收口。","status_label":"成功","steps_total":3,"steps_success":3,"steps_running":0,"steps_problem":0,"steps_terminal":3,"summary":"当前整轮编排已全部收口。","summary_text":"当前整轮编排已全部收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-df533c6f4a","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"","focus_step_title":"","event_key":"","node_code":""}},{"run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","playbook_title":"接管后验收","group_key":"onboarding","group_title":"接管验收","requested_by":"cli","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":5,"expected_jobs_total":5,"target_nodes_total":1,"target_node_codes":["mainland-controller-01"],"created_at":"2026-04-19 01:12:57","updated_at":"2026-04-19 01:13:27","status_counts":{"success":5},"jobs_total":5,"success_jobs_total":5,"terminal_jobs_total":5,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":56,"job_code":"ops-20260419011257-998c9e","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:12:57","started_at":"2026-04-19 01:13:03","finished_at":"2026-04-19 01:13:06","updated_at":"2026-04-19 01:13:06","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"node_agent_status","title":"采集 Node Agent 状态","template_key":"service.status","order":2,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":58,"job_code":"ops-20260419011258-daf92f","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:07","finished_at":"2026-04-19 01:13:09","updated_at":"2026-04-19 01:13:09","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集 Node Agent 状态 已完成收口。","summary_text":"采集 Node Agent 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_status","focus_step_title":"采集 Node Agent 状态","event_key":"","node_code":""}},{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","template_key":"logs.collect","order":3,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":60,"job_code":"ops-20260419011258-48e2cf","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:09","finished_at":"2026-04-19 01:13:12","updated_at":"2026-04-19 01:13:12","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"收集 Node Agent 日志 已完成收口。","summary_text":"收集 Node Agent 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"step_key":"worker_status","title":"采集 Worker 状态","template_key":"service.status","order":4,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":62,"job_code":"ops-20260419011258-591b52","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:14","finished_at":"2026-04-19 01:13:19","updated_at":"2026-04-19 01:13:19","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集 Worker 状态 已完成收口。","summary_text":"采集 Worker 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_status","focus_step_title":"采集 Worker 状态","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":5,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":64,"job_code":"ops-20260419011258-a55cec","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:22","finished_at":"2026-04-19 01:13:27","updated_at":"2026-04-19 01:13:27","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}}],"latest_job":{"id":64,"job_code":"ops-20260419011258-a55cec","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:22","finished_at":"2026-04-19 01:13:27","updated_at":"2026-04-19 01:13:27","error_message":""},"status":"success","completion_percent":100.0,"problem_steps":[],"active_steps":[],"focus_level":"success","focus_step_key":"","focus_step_title":"","focus_summary":"当前整轮编排已全部收口。","status_label":"成功","steps_total":5,"steps_success":5,"steps_running":0,"steps_problem":0,"steps_terminal":5,"summary":"当前整轮编排已全部收口。","summary_text":"当前整轮编排已全部收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"","focus_step_title":"","event_key":"","node_code":""}},{"run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","playbook_title":"接管后验收","group_key":"onboarding","group_title":"接管验收","requested_by":"cli","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":5,"expected_jobs_total":5,"target_nodes_total":1,"target_node_codes":["mainland-worker-01"],"created_at":"2026-04-19 01:12:57","updated_at":"2026-04-19 01:13:17","status_counts":{"success":5},"jobs_total":5,"success_jobs_total":5,"terminal_jobs_total":5,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":57,"job_code":"ops-20260419011257-0818f7","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:12:57","started_at":"2026-04-19 01:12:59","finished_at":"2026-04-19 01:13:02","updated_at":"2026-04-19 01:13:02","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"node_agent_status","title":"采集 Node Agent 状态","template_key":"service.status","order":2,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":59,"job_code":"ops-20260419011258-165f37","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:02","finished_at":"2026-04-19 01:13:04","updated_at":"2026-04-19 01:13:04","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集 Node Agent 状态 已完成收口。","summary_text":"采集 Node Agent 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_status","focus_step_title":"采集 Node Agent 状态","event_key":"","node_code":""}},{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","template_key":"logs.collect","order":3,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":61,"job_code":"ops-20260419011258-b326cf","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:04","finished_at":"2026-04-19 01:13:07","updated_at":"2026-04-19 01:13:07","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"收集 Node Agent 日志 已完成收口。","summary_text":"收集 Node Agent 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"step_key":"worker_status","title":"采集 Worker 状态","template_key":"service.status","order":4,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":63,"job_code":"ops-20260419011258-154d4e","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:11","finished_at":"2026-04-19 01:13:14","updated_at":"2026-04-19 01:13:14","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集 Worker 状态 已完成收口。","summary_text":"采集 Worker 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_status","focus_step_title":"采集 Worker 状态","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":5,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":65,"job_code":"ops-20260419011258-9eef54","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:14","finished_at":"2026-04-19 01:13:17","updated_at":"2026-04-19 01:13:17","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}}],"latest_job":{"id":65,"job_code":"ops-20260419011258-9eef54","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:12:58","started_at":"2026-04-19 01:13:14","finished_at":"2026-04-19 01:13:17","updated_at":"2026-04-19 01:13:17","error_message":""},"status":"success","completion_percent":100.0,"problem_steps":[],"active_steps":[],"focus_level":"success","focus_step_key":"","focus_step_title":"","focus_summary":"当前整轮编排已全部收口。","status_label":"成功","steps_total":5,"steps_success":5,"steps_running":0,"steps_problem":0,"steps_terminal":5,"summary":"当前整轮编排已全部收口。","summary_text":"当前整轮编排已全部收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"","focus_step_title":"","event_key":"","node_code":""}},{"run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","playbook_title":"接管后验收","group_key":"onboarding","group_title":"接管验收","requested_by":"cli","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":5,"expected_jobs_total":5,"target_nodes_total":1,"target_node_codes":["mainland-worker-01"],"created_at":"2026-04-19 01:10:09","updated_at":"2026-04-19 01:10:29","status_counts":{"success":4,"failed":1},"jobs_total":5,"success_jobs_total":4,"terminal_jobs_total":5,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":47,"job_code":"ops-20260419011009-cbd7d6","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:13","finished_at":"2026-04-19 01:10:15","updated_at":"2026-04-19 01:10:15","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"node_agent_status","title":"采集 Node Agent 状态","template_key":"service.status","order":2,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":49,"job_code":"ops-20260419011009-98c6d6","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:16","finished_at":"2026-04-19 01:10:18","updated_at":"2026-04-19 01:10:18","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集 Node Agent 状态 已完成收口。","summary_text":"采集 Node Agent 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_status","focus_step_title":"采集 Node Agent 状态","event_key":"","node_code":""}},{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","template_key":"logs.collect","order":3,"jobs_total":1,"nodes_total":1,"status_counts":{"failed":1},"terminal_jobs_total":1,"success_jobs_total":0,"latest_job":{"id":51,"job_code":"ops-20260419011009-f0a298","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:18","finished_at":"2026-04-19 01:10:21","updated_at":"2026-04-19 01:10:21","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."},"status":"attention","status_label":"待关注","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"step_key":"worker_status","title":"采集 Worker 状态","template_key":"service.status","order":4,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":53,"job_code":"ops-20260419011010-52f4b0","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:10","started_at":"2026-04-19 01:10:22","finished_at":"2026-04-19 01:10:26","updated_at":"2026-04-19 01:10:26","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集 Worker 状态 已完成收口。","summary_text":"采集 Worker 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_status","focus_step_title":"采集 Worker 状态","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":5,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":55,"job_code":"ops-20260419011010-9bea3c","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:10","started_at":"2026-04-19 01:10:27","finished_at":"2026-04-19 01:10:29","updated_at":"2026-04-19 01:10:29","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}}],"latest_job":{"id":55,"job_code":"ops-20260419011010-9bea3c","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:10","started_at":"2026-04-19 01:10:27","finished_at":"2026-04-19 01:10:29","updated_at":"2026-04-19 01:10:29","error_message":""},"status":"attention","completion_percent":100.0,"problem_steps":[{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","status":"attention","reason_label":"失败","latest_job":{"id":51,"job_code":"ops-20260419011009-f0a298","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-worker-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:18","finished_at":"2026-04-19 01:10:21","updated_at":"2026-04-19 01:10:21","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."}}],"active_steps":[],"focus_level":"danger","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","status_label":"待关注","steps_total":5,"steps_success":4,"steps_running":0,"steps_problem":1,"steps_terminal":5,"summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","playbook_title":"接管后验收","group_key":"onboarding","group_title":"接管验收","requested_by":"cli","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":5,"expected_jobs_total":5,"target_nodes_total":1,"target_node_codes":["mainland-controller-01"],"created_at":"2026-04-19 01:10:09","updated_at":"2026-04-19 01:10:27","status_counts":{"success":4,"failed":1},"jobs_total":5,"success_jobs_total":4,"terminal_jobs_total":5,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":46,"job_code":"ops-20260419011009-e075fe","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:09","finished_at":"2026-04-19 01:10:13","updated_at":"2026-04-19 01:10:13","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"node_agent_status","title":"采集 Node Agent 状态","template_key":"service.status","order":2,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":48,"job_code":"ops-20260419011009-0a0c70","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:14","finished_at":"2026-04-19 01:10:18","updated_at":"2026-04-19 01:10:18","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集 Node Agent 状态 已完成收口。","summary_text":"采集 Node Agent 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_status","focus_step_title":"采集 Node Agent 状态","event_key":"","node_code":""}},{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","template_key":"logs.collect","order":3,"jobs_total":1,"nodes_total":1,"status_counts":{"failed":1},"terminal_jobs_total":1,"success_jobs_total":0,"latest_job":{"id":50,"job_code":"ops-20260419011009-c091a0","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:19","finished_at":"2026-04-19 01:10:21","updated_at":"2026-04-19 01:10:21","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."},"status":"attention","status_label":"待关注","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"step_key":"worker_status","title":"采集 Worker 状态","template_key":"service.status","order":4,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":52,"job_code":"ops-20260419011010-1b63ee","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:10","started_at":"2026-04-19 01:10:21","finished_at":"2026-04-19 01:10:23","updated_at":"2026-04-19 01:10:23","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集 Worker 状态 已完成收口。","summary_text":"采集 Worker 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_status","focus_step_title":"采集 Worker 状态","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":5,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":54,"job_code":"ops-20260419011010-fb40be","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:10","started_at":"2026-04-19 01:10:24","finished_at":"2026-04-19 01:10:27","updated_at":"2026-04-19 01:10:27","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}}],"latest_job":{"id":54,"job_code":"ops-20260419011010-fb40be","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:10","started_at":"2026-04-19 01:10:24","finished_at":"2026-04-19 01:10:27","updated_at":"2026-04-19 01:10:27","error_message":""},"status":"attention","completion_percent":100.0,"problem_steps":[{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","status":"attention","reason_label":"失败","latest_job":{"id":50,"job_code":"ops-20260419011009-c091a0","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-controller-01","created_at":"2026-04-19 01:10:09","started_at":"2026-04-19 01:10:19","finished_at":"2026-04-19 01:10:21","updated_at":"2026-04-19 01:10:21","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."}}],"active_steps":[],"focus_level":"danger","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","status_label":"待关注","steps_total":5,"steps_success":4,"steps_running":0,"steps_problem":1,"steps_terminal":5,"summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","playbook_title":"接管后验收","group_key":"onboarding","group_title":"接管验收","requested_by":"nightly-b1","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":5,"expected_jobs_total":5,"target_nodes_total":1,"target_node_codes":["mainland-controller-01"],"created_at":"2026-04-19 00:33:57","updated_at":"2026-04-19 00:49:41","status_counts":{"success":4,"failed":1},"jobs_total":5,"success_jobs_total":4,"terminal_jobs_total":5,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":41,"job_code":"ops-20260419003357-2be293","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:20","finished_at":"2026-04-19 00:49:22","updated_at":"2026-04-19 00:49:22","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"node_agent_status","title":"采集 Node Agent 状态","template_key":"service.status","order":2,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":42,"job_code":"ops-20260419003357-1e65a2","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:23","finished_at":"2026-04-19 00:49:26","updated_at":"2026-04-19 00:49:26","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集 Node Agent 状态 已完成收口。","summary_text":"采集 Node Agent 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_status","focus_step_title":"采集 Node Agent 状态","event_key":"","node_code":""}},{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","template_key":"logs.collect","order":3,"jobs_total":1,"nodes_total":1,"status_counts":{"failed":1},"terminal_jobs_total":1,"success_jobs_total":0,"latest_job":{"id":43,"job_code":"ops-20260419003357-34e18d","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:28","finished_at":"2026-04-19 00:49:30","updated_at":"2026-04-19 00:49:30","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."},"status":"attention","status_label":"待关注","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"step_key":"worker_status","title":"采集 Worker 状态","template_key":"service.status","order":4,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":44,"job_code":"ops-20260419003357-6ad871","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:31","finished_at":"2026-04-19 00:49:34","updated_at":"2026-04-19 00:49:34","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"采集 Worker 状态 已完成收口。","summary_text":"采集 Worker 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_status","focus_step_title":"采集 Worker 状态","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":5,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":45,"job_code":"ops-20260419003357-e6ef6a","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:38","finished_at":"2026-04-19 00:49:41","updated_at":"2026-04-19 00:49:41","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}}],"latest_job":{"id":45,"job_code":"ops-20260419003357-e6ef6a","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:38","finished_at":"2026-04-19 00:49:41","updated_at":"2026-04-19 00:49:41","error_message":""},"status":"attention","completion_percent":100.0,"problem_steps":[{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","status":"attention","reason_label":"失败","latest_job":{"id":43,"job_code":"ops-20260419003357-34e18d","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-controller-01","created_at":"2026-04-19 00:33:57","started_at":"2026-04-19 00:49:28","finished_at":"2026-04-19 00:49:30","updated_at":"2026-04-19 00:49:30","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."}}],"active_steps":[],"focus_level":"danger","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","status_label":"待关注","steps_total":5,"steps_success":4,"steps_running":0,"steps_problem":1,"steps_terminal":5,"summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","playbook_title":"接管后验收","group_key":"onboarding","group_title":"接管验收","requested_by":"nightly-a1","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":5,"expected_jobs_total":5,"target_nodes_total":1,"target_node_codes":["mainland-worker-01"],"created_at":"2026-04-19 00:06:25","updated_at":"2026-04-19 00:49:34","status_counts":{"success":4,"failed":1},"jobs_total":5,"success_jobs_total":4,"terminal_jobs_total":5,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":36,"job_code":"ops-20260419000625-6f7db9","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:17","finished_at":"2026-04-19 00:49:20","updated_at":"2026-04-19 00:49:20","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"node_agent_status","title":"采集 Node Agent 状态","template_key":"service.status","order":2,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":37,"job_code":"ops-20260419000625-0ecb78","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:20","finished_at":"2026-04-19 00:49:22","updated_at":"2026-04-19 00:49:22","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集 Node Agent 状态 已完成收口。","summary_text":"采集 Node Agent 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_status","focus_step_title":"采集 Node Agent 状态","event_key":"","node_code":""}},{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","template_key":"logs.collect","order":3,"jobs_total":1,"nodes_total":1,"status_counts":{"failed":1},"terminal_jobs_total":1,"success_jobs_total":0,"latest_job":{"id":38,"job_code":"ops-20260419000625-4de21f","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:25","finished_at":"2026-04-19 00:49:27","updated_at":"2026-04-19 00:49:27","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."},"status":"attention","status_label":"待关注","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先查看该步骤事件流。","focus_ref":{"kind":"playbook_run","run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"step_key":"worker_status","title":"采集 Worker 状态","template_key":"service.status","order":4,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":39,"job_code":"ops-20260419000625-093ea8","action":"service.status","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:28","finished_at":"2026-04-19 00:49:31","updated_at":"2026-04-19 00:49:31","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"采集 Worker 状态 已完成收口。","summary_text":"采集 Worker 状态 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_status","focus_step_title":"采集 Worker 状态","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":5,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":40,"job_code":"ops-20260419000625-28c085","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:33","finished_at":"2026-04-19 00:49:34","updated_at":"2026-04-19 00:49:34","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-worker-01"],"summary":"收集 Worker 日志 已完成收口。","summary_text":"收集 Worker 日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}}],"latest_job":{"id":40,"job_code":"ops-20260419000625-28c085","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:33","finished_at":"2026-04-19 00:49:34","updated_at":"2026-04-19 00:49:34","error_message":""},"status":"attention","completion_percent":100.0,"problem_steps":[{"step_key":"node_agent_logs","title":"收集 Node Agent 日志","status":"attention","reason_label":"失败","latest_job":{"id":38,"job_code":"ops-20260419000625-4de21f","action":"logs.collect","status":"failed","status_label":"failed","target_node_code":"mainland-worker-01","created_at":"2026-04-19 00:06:25","started_at":"2026-04-19 00:49:25","finished_at":"2026-04-19 00:49:27","updated_at":"2026-04-19 00:49:27","error_message":"Hint: You are currently not seeing messages from other users and the system.\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\n Pass -q to turn off this notice.\nNo journal files were opened due to insufficient permissions."}}],"active_steps":[],"focus_level":"danger","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","status_label":"待关注","steps_total":5,"steps_success":4,"steps_running":0,"steps_problem":1,"steps_terminal":5,"summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","summary_text":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-137e8b258b","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"run_code":"pbr-4df9934383","playbook_key":"scene.logs.key","playbook_title":"关键现场日志","group_key":"scene","group_title":"执行现场","requested_by":"cli/ops-center/open_worker_logs_participating","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":1,"expected_jobs_total":3,"target_nodes_total":3,"target_node_codes":["overseas-control-01","mainland-controller-01","mainland-worker-01"],"created_at":"2026-04-18 22:50:32","updated_at":"2026-04-19 00:49:18","status_counts":{"success":3},"jobs_total":3,"success_jobs_total":3,"terminal_jobs_total":3,"steps":[{"step_key":"worker_logs","title":"收集 Worker 关键日志","template_key":"logs.collect","order":1,"jobs_total":3,"nodes_total":3,"status_counts":{"success":3},"terminal_jobs_total":3,"success_jobs_total":3,"latest_job":{"id":34,"job_code":"ops-20260418225032-8688b9","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 22:50:32","started_at":"2026-04-19 00:49:15","finished_at":"2026-04-19 00:49:18","updated_at":"2026-04-19 00:49:18","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"收集 Worker 关键日志 已完成收口。","summary_text":"收集 Worker 关键日志 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-4df9934383","playbook_key":"scene.logs.key","group_key":"scene","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 关键日志","event_key":"","node_code":""}}],"latest_job":{"id":34,"job_code":"ops-20260418225032-8688b9","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 22:50:32","started_at":"2026-04-19 00:49:15","finished_at":"2026-04-19 00:49:18","updated_at":"2026-04-19 00:49:18","error_message":""},"status":"success","completion_percent":100.0,"problem_steps":[],"active_steps":[],"focus_level":"success","focus_step_key":"","focus_step_title":"","focus_summary":"当前整轮编排已全部收口。","status_label":"成功","steps_total":1,"steps_success":1,"steps_running":0,"steps_problem":0,"steps_terminal":1,"summary":"当前整轮编排已全部收口。","summary_text":"当前整轮编排已全部收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-4df9934383","playbook_key":"scene.logs.key","group_key":"scene","focus_step_key":"","focus_step_title":"","event_key":"","node_code":""}},{"run_code":"pbr-de71dbbda4","playbook_key":"inspection.standard","playbook_title":"标准巡检","group_key":"diagnostics","group_title":"标准巡检","requested_by":"cli/ops-center/run_inspection_participating","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":3,"expected_jobs_total":9,"target_nodes_total":3,"target_node_codes":["overseas-control-01","mainland-controller-01","mainland-worker-01"],"created_at":"2026-04-18 22:45:53","updated_at":"2026-04-19 00:49:14","status_counts":{"success":8,"failed":1},"jobs_total":9,"success_jobs_total":8,"terminal_jobs_total":9,"steps":[{"step_key":"health","title":"采集健康快照","template_key":"health.snapshot","order":1,"jobs_total":3,"nodes_total":3,"status_counts":{"success":3},"terminal_jobs_total":3,"success_jobs_total":3,"latest_job":{"id":22,"job_code":"ops-20260418224553-beaa1b","action":"health.snapshot","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:49:00","finished_at":"2026-04-19 00:49:05","updated_at":"2026-04-19 00:49:05","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"采集健康快照 已完成收口。","summary_text":"采集健康快照 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-de71dbbda4","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"health","focus_step_title":"采集健康快照","event_key":"","node_code":""}},{"step_key":"worker_logs","title":"收集 Worker 日志","template_key":"logs.collect","order":2,"jobs_total":3,"nodes_total":3,"status_counts":{"failed":1,"success":2},"terminal_jobs_total":3,"success_jobs_total":2,"latest_job":{"id":27,"job_code":"ops-20260418224553-600be5","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:49:06","finished_at":"2026-04-19 00:49:07","updated_at":"2026-04-19 00:49:07","error_message":""},"status":"attention","status_label":"待关注","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"收集 Worker 日志 当前出现失败,建议先查看该步骤事件流。","summary_text":"收集 Worker 日志 当前出现失败,建议先查看该步骤事件流。","focus_ref":{"kind":"playbook_run","run_code":"pbr-de71dbbda4","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}},{"step_key":"diagnostics","title":"收集诊断包","template_key":"diagnostics.collect","order":3,"jobs_total":3,"nodes_total":3,"status_counts":{"success":3},"terminal_jobs_total":3,"success_jobs_total":3,"latest_job":{"id":31,"job_code":"ops-20260418224553-897976","action":"diagnostics.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:49:10","finished_at":"2026-04-19 00:49:14","updated_at":"2026-04-19 00:49:14","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"收集诊断包 已完成收口。","summary_text":"收集诊断包 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-de71dbbda4","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"diagnostics","focus_step_title":"收集诊断包","event_key":"","node_code":""}}],"latest_job":{"id":31,"job_code":"ops-20260418224553-897976","action":"diagnostics.collect","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:49:10","finished_at":"2026-04-19 00:49:14","updated_at":"2026-04-19 00:49:14","error_message":""},"status":"attention","completion_percent":100.0,"problem_steps":[{"step_key":"worker_logs","title":"收集 Worker 日志","status":"attention","reason_label":"失败","latest_job":{"id":27,"job_code":"ops-20260418224553-600be5","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:49:06","finished_at":"2026-04-19 00:49:07","updated_at":"2026-04-19 00:49:07","error_message":""}}],"active_steps":[],"focus_level":"danger","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","focus_summary":"收集 Worker 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","status_label":"待关注","steps_total":3,"steps_success":2,"steps_running":0,"steps_problem":1,"steps_terminal":3,"summary":"收集 Worker 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","summary_text":"收集 Worker 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-de71dbbda4","playbook_key":"inspection.standard","group_key":"diagnostics","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 日志","event_key":"","node_code":""}},{"run_code":"pbr-8c7cf4e19e","playbook_key":"scene.logs.key","playbook_title":"关键现场日志","group_key":"scene","group_title":"执行现场","requested_by":"cli/ops-center/open_worker_logs_participating","execution_mode":"remote-agent","execution_mode_label":"远端 Agent","stop_on_failure":true,"step_count":1,"expected_jobs_total":3,"target_nodes_total":3,"target_node_codes":["overseas-control-01","mainland-controller-01","mainland-worker-01"],"created_at":"2026-04-18 22:45:53","updated_at":"2026-04-19 00:49:00","status_counts":{"failed":1,"success":2},"jobs_total":3,"success_jobs_total":2,"terminal_jobs_total":3,"steps":[{"step_key":"worker_logs","title":"收集 Worker 关键日志","template_key":"logs.collect","order":1,"jobs_total":3,"nodes_total":3,"status_counts":{"failed":1,"success":2},"terminal_jobs_total":3,"success_jobs_total":2,"latest_job":{"id":21,"job_code":"ops-20260418224553-32c368","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:48:56","finished_at":"2026-04-19 00:49:00","updated_at":"2026-04-19 00:49:00","error_message":""},"status":"attention","status_label":"待关注","completion_percent":100.0,"target_node_codes":["mainland-controller-01","mainland-worker-01","overseas-control-01"],"summary":"收集 Worker 关键日志 当前出现失败,建议先查看该步骤事件流。","summary_text":"收集 Worker 关键日志 当前出现失败,建议先查看该步骤事件流。","focus_ref":{"kind":"playbook_run","run_code":"pbr-8c7cf4e19e","playbook_key":"scene.logs.key","group_key":"scene","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 关键日志","event_key":"","node_code":""}}],"latest_job":{"id":21,"job_code":"ops-20260418224553-32c368","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:48:56","finished_at":"2026-04-19 00:49:00","updated_at":"2026-04-19 00:49:00","error_message":""},"status":"attention","completion_percent":100.0,"problem_steps":[{"step_key":"worker_logs","title":"收集 Worker 关键日志","status":"attention","reason_label":"失败","latest_job":{"id":21,"job_code":"ops-20260418224553-32c368","action":"logs.collect","status":"success","status_label":"成功","target_node_code":"mainland-worker-01","created_at":"2026-04-18 22:45:53","started_at":"2026-04-19 00:48:56","finished_at":"2026-04-19 00:49:00","updated_at":"2026-04-19 00:49:00","error_message":""}}],"active_steps":[],"focus_level":"danger","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 关键日志","focus_summary":"收集 Worker 关键日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","status_label":"待关注","steps_total":1,"steps_success":0,"steps_running":0,"steps_problem":1,"steps_terminal":1,"summary":"收集 Worker 关键日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","summary_text":"收集 Worker 关键日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-8c7cf4e19e","playbook_key":"scene.logs.key","group_key":"scene","focus_step_key":"worker_logs","focus_step_title":"收集 Worker 关键日志","event_key":"","node_code":""}},{"run_code":"pbr-772e6e9e7b","playbook_key":"onboarding.bootstrap","playbook_title":"生成节点接入工单","group_key":"onboarding","group_title":"接管验收","requested_by":"cli","execution_mode":"control-plane","execution_mode_label":"控制面","stop_on_failure":true,"step_count":1,"expected_jobs_total":1,"target_nodes_total":1,"target_node_codes":["mainland-controller-01"],"created_at":"2026-04-18 19:44:21","updated_at":"2026-04-18 19:44:21","status_counts":{"success":1},"jobs_total":1,"success_jobs_total":1,"terminal_jobs_total":1,"steps":[{"step_key":"bootstrap","title":"生成 Node Agent 接入工单","template_key":"node.bootstrap","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":20,"job_code":"ops-20260418194421-bc277a","action":"node.bootstrap","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 19:44:21","started_at":"2026-04-18 19:44:21","finished_at":"2026-04-18 19:44:21","updated_at":"2026-04-18 19:44:21","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"生成 Node Agent 接入工单 已完成收口。","summary_text":"生成 Node Agent 接入工单 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-772e6e9e7b","playbook_key":"onboarding.bootstrap","group_key":"onboarding","focus_step_key":"bootstrap","focus_step_title":"生成 Node Agent 接入工单","event_key":"","node_code":""}}],"latest_job":{"id":20,"job_code":"ops-20260418194421-bc277a","action":"node.bootstrap","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 19:44:21","started_at":"2026-04-18 19:44:21","finished_at":"2026-04-18 19:44:21","updated_at":"2026-04-18 19:44:21","error_message":""},"status":"success","completion_percent":100.0,"problem_steps":[],"active_steps":[],"focus_level":"success","focus_step_key":"","focus_step_title":"","focus_summary":"接入工单已生成,目标节点已完成 Agent 接入,可继续执行接管后验收。","bootstrap_state_counts":{"total":1,"online":1,"pending_bootstrap":0,"runtime_only":0,"stale":0,"unmanaged":0,"unknown":0},"status_label":"成功","steps_total":1,"steps_success":1,"steps_running":0,"steps_problem":0,"steps_terminal":1,"summary":"接入工单已生成,目标节点已完成 Agent 接入,可继续执行接管后验收。","summary_text":"接入工单已生成,目标节点已完成 Agent 接入,可继续执行接管后验收。","focus_ref":{"kind":"playbook_run","run_code":"pbr-772e6e9e7b","playbook_key":"onboarding.bootstrap","group_key":"onboarding","focus_step_key":"","focus_step_title":"","event_key":"","node_code":""}},{"run_code":"pbr-d7a22054b2","playbook_key":"onboarding.bootstrap","playbook_title":"生成节点接入工单","group_key":"onboarding","group_title":"接管验收","requested_by":"cli","execution_mode":"control-plane","execution_mode_label":"控制面","stop_on_failure":true,"step_count":1,"expected_jobs_total":1,"target_nodes_total":1,"target_node_codes":["mainland-controller-01"],"created_at":"2026-04-18 19:44:00","updated_at":"2026-04-18 19:44:00","status_counts":{"success":1},"jobs_total":1,"success_jobs_total":1,"terminal_jobs_total":1,"steps":[{"step_key":"bootstrap","title":"生成 Node Agent 接入工单","template_key":"node.bootstrap","order":1,"jobs_total":1,"nodes_total":1,"status_counts":{"success":1},"terminal_jobs_total":1,"success_jobs_total":1,"latest_job":{"id":19,"job_code":"ops-20260418194400-41b4dd","action":"node.bootstrap","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 19:44:00","started_at":"2026-04-18 19:44:00","finished_at":"2026-04-18 19:44:00","updated_at":"2026-04-18 19:44:00","error_message":""},"status":"success","status_label":"成功","completion_percent":100.0,"target_node_codes":["mainland-controller-01"],"summary":"生成 Node Agent 接入工单 已完成收口。","summary_text":"生成 Node Agent 接入工单 已完成收口。","focus_ref":{"kind":"playbook_run","run_code":"pbr-d7a22054b2","playbook_key":"onboarding.bootstrap","group_key":"onboarding","focus_step_key":"bootstrap","focus_step_title":"生成 Node Agent 接入工单","event_key":"","node_code":""}}],"latest_job":{"id":19,"job_code":"ops-20260418194400-41b4dd","action":"node.bootstrap","status":"success","status_label":"成功","target_node_code":"mainland-controller-01","created_at":"2026-04-18 19:44:00","started_at":"2026-04-18 19:44:00","finished_at":"2026-04-18 19:44:00","updated_at":"2026-04-18 19:44:00","error_message":""},"status":"success","completion_percent":100.0,"problem_steps":[],"active_steps":[],"focus_level":"success","focus_step_key":"","focus_step_title":"","focus_summary":"接入工单已生成,目标节点已完成 Agent 接入,可继续执行接管后验收。","bootstrap_state_counts":{"total":1,"online":1,"pending_bootstrap":0,"runtime_only":0,"stale":0,"unmanaged":0,"unknown":0},"status_label":"成功","steps_total":1,"steps_success":1,"steps_running":0,"steps_problem":0,"steps_terminal":1,"summary":"接入工单已生成,目标节点已完成 Agent 接入,可继续执行接管后验收。","summary_text":"接入工单已生成,目标节点已完成 Agent 接入,可继续执行接管后验收。","focus_ref":{"kind":"playbook_run","run_code":"pbr-d7a22054b2","playbook_key":"onboarding.bootstrap","group_key":"onboarding","focus_step_key":"","focus_step_title":"","event_key":"","node_code":""}}],"summary":{"total":12,"filtered_total":17,"unfiltered_total":17,"success":6,"running":0,"attention":6,"queued":0,"scanned_jobs":64,"available_status_counts":{"success":11,"attention":6},"available_group_counts":{"diagnostics":2,"onboarding":13,"scene":2},"filters":{"status":"","group_key":"","query":""}}},"detail_code":null} \ No newline at end of file diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/scene_overseas_control_01.json b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/scene_overseas_control_01.json new file mode 100644 index 0000000..dc41d5d --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/scene_overseas_control_01.json @@ -0,0 +1 @@ +{"code":0,"message":"ok","data":{"node_code":"overseas-control-01","available":true,"status":"full_capture","status_label":"全量观察","status_type":"success","summary":"节点当前正在参与检测,已保留 1 条现场日志样本。","log_sync_enabled":true,"mode":"full","mode_label":"全量回传","records_total":0,"records_visible":0,"records_truncated":false,"records":[],"latest_record":{},"source_summary":{"node_code":"overseas-control-01","line_count":1,"key_line_count":0,"full_line_count":1,"last_at":"2026-04-19 01:20:35","last_line":"[2026-04-19 01:20:35] [overseas-control-01] 2026-04-19 01:00:18.866 | INFO | __main__:start_detection:2575 - 开始执行域名检测任务"},"missing_reason_code":"","missing_reason":"","node":{"node_code":"overseas-control-01","region":"overseas","role":"control","status":"busy","current_load":25,"last_heartbeat_at":"2026-04-19 01:29:45"},"participation":{"detect_participating":true,"participation_state":"running","participation_label":"执行中","participation_reason":"当前正在执行 4 项检测任务。","participation_bucket":"dispatch_active","participation_bucket_label":"执行/已领","is_dispatch_active":true},"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_observability_contract","contract_keys":["ops_observability_contract","ops_stack_diagnosis_contract"],"contracts":[{"key":"ops_observability_contract","title":"Ops Observability Contract","status":"active","version":"v1","summary":"冻结 execution scene / inspection overview / activity stream / delivery queue 的正式观察面 contract。","primary_endpoint":"/api/v1/ops/overview","schema_doc_path":"docs/schemas/ops_observability_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_observability_contract","discovery_endpoints":["/api/v1/ops/overview","/api/v1/ops/inspection-overview","/api/v1/ops/activity-stream","/api/v1/ops/nodes/{node_code}/scene-log","/api/v1/ops/nodes/{node_code}/delivery-queue","/api/v1/ops/nodes/{node_code}/delivery-queue/records","/api/v1/ops/nodes/{node_code}/delivery-queue/flush","/api/v1/ops/nodes/{node_code}/delivery-queue/replay","/api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/replay","/api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/discard"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","ops_driver_contract","ops_playbook_contract","ops_stack_diagnosis_contract"]},{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]}]}},"detail_code":null} \ No newline at end of file diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/stack.json b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/stack.json new file mode 100644 index 0000000..9732fc1 --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/stack.json @@ -0,0 +1 @@ +{"code":0,"message":"ok","data":{"generated_at":"2026-04-19 01:29:44","diagnosis":{"contract_key":"ops_stack_diagnosis_contract","contract_version":"v1","registry_version":"2026-04-18","base_url":"http://127.0.0.1:8100","contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_stack_diagnosis_contract","contract_keys":["ops_stack_diagnosis_contract","ops_driver_contract","ops_playbook_contract"],"contracts":[{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]},{"key":"ops_driver_contract","title":"Ops Driver Contract","status":"active","version":"v1","summary":"冻结 overview / driver-feed / codex-brief / driver-actions / runbook sequence 的统一驾驶 contract。","primary_endpoint":"/api/v1/ops/driver-feed","schema_doc_path":"docs/schemas/ops_driver_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_driver_contract","discovery_endpoints":["/api/v1/ops/overview","/api/v1/ops/driver-feed","/api/v1/ops/codex-brief","/api/v1/ops/driver-actions/resolve","/api/v1/ops/driver-actions/execute-resolved","/api/v1/ops/codex-actions/resolve","/api/v1/ops/codex-actions/execute","/api/v1/ops/activity-stream","/api/v1/ops/runbook","/api/v1/ops/runbook/sequences/{sequence_key}/resolve","/api/v1/ops/runbook/sequences/{sequence_key}/execute","/api/v1/ops/driver-actions/preview","/api/v1/ops/driver-actions/execute"],"related_contract_keys":["release_hub_contract","ops_playbook_contract","ops_observability_contract","ops_stack_diagnosis_contract"]},{"key":"ops_playbook_contract","title":"Ops Playbook Contract","status":"active","version":"v1","summary":"冻结 playbook catalog / preview / run / events 的正式编排 contract。","primary_endpoint":"/api/v1/ops/playbooks","schema_doc_path":"docs/schemas/ops_playbook_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_playbook_contract","discovery_endpoints":["/api/v1/ops/playbooks","/api/v1/ops/playbooks/preview","/api/v1/ops/playbooks/execute","/api/v1/ops/playbook-runs","/api/v1/ops/playbook-runs/{run_code}","/api/v1/ops/playbook-runs/{run_code}/events","/api/v1/ops/playbook-runs/{run_code}/rerun","/api/v1/ops/playbook-runs/{run_code}/cancel"],"related_contract_keys":["ops_job_contract","release_hub_contract","ops_driver_contract","ops_observability_contract","ops_stack_diagnosis_contract"]}]},"surface_status":"healthy","automation_status":"attention","stack_status":"attention","issue_total":1,"blocking_issue_total":0,"warning_issue_total":1,"info_issue_total":0,"missing_surfaces":[],"launchpad_recommended_target_node_code":"","launchpad_recommended_recovery_label":"","launchpad_recommended_recovery_summary":"","launchpad_onboarding_bootstrap_pending_nodes":0,"launchpad_onboarding_acceptance_ready_nodes":0,"next_step":{"action_code":"focus_playbook_run","source":"overview.primary","reason":"来自 overview.recommendation.primary_action_code","command":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","focus_ref":{},"contract_keys":["ops_stack_diagnosis_contract"],"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_stack_diagnosis_contract","contract_keys":["ops_stack_diagnosis_contract"],"contracts":[{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]}]}},"recommended_actions":[{"action_code":"focus_playbook_run","source":"overview.primary","title":"先处理异常编排","reason":"来自 overview.recommendation.primary_action_code","focus_ref":{},"command":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","contract_keys":["ops_stack_diagnosis_contract"],"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_stack_diagnosis_contract","contract_keys":["ops_stack_diagnosis_contract"],"contracts":[{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]}]}},{"action_code":"fix_managed_nodes","source":"release_hub.launchpad","title":"待补执行面","reason":"来自 release_launchpad.launchpad_status.recommended_action_code","focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_launchpad"},"command":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 fix_managed_nodes","contract_keys":["ops_stack_diagnosis_contract","ops_agent_protocol","release_hub_contract"],"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_stack_diagnosis_contract","contract_keys":["ops_stack_diagnosis_contract","ops_agent_protocol","release_hub_contract"],"contracts":[{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]},{"key":"ops_agent_protocol","title":"Node Agent Protocol","status":"active","version":"v1","summary":"冻结海外控制面与大陆 Node Agent 之间的认证、注册、心跳、拉任务、完成回执与事件回放 contract。","primary_endpoint":"/api/v1/ops/agent/bootstrap-plan","schema_doc_path":"docs/schemas/ops_agent_protocol.md","detail_endpoint":"/api/v1/ops/contracts/ops_agent_protocol","discovery_endpoints":["/api/v1/ops/agent/tokens","/api/v1/ops/agent/bootstrap-plan","/api/v1/ops/agent/register","/api/v1/ops/agent/heartbeat","/api/v1/ops/agent/pull","/api/v1/ops/agent/jobs/{job_id}/start","/api/v1/ops/agent/jobs/{job_id}/complete","/api/v1/ops/agent/jobs/{job_id}/events","/api/v1/ops/nodes/{node_code}/delivery-queue","/api/v1/ops/nodes/{node_code}/delivery-queue/records","/api/v1/ops/nodes/{node_code}/delivery-queue/flush","/api/v1/ops/nodes/{node_code}/delivery-queue/replay","/api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/replay","/api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/discard"],"related_contract_keys":["ops_job_contract","ops_observability_contract","ops_stack_diagnosis_contract"]},{"key":"release_hub_contract","title":"Release Hub Contract","status":"active","version":"v1","summary":"冻结 Release / Rollout / Launchpad / Default Gate 的对象模型与门禁口径。","primary_endpoint":"/api/v1/ops/releases/launchpad","schema_doc_path":"docs/schemas/release_hub_contract.md","detail_endpoint":"/api/v1/ops/contracts/release_hub_contract","discovery_endpoints":["/api/v1/ops/overview","/api/v1/ops/releases","/api/v1/ops/releases/latest","/api/v1/ops/releases/launchpad","/api/v1/ops/releases/{release_id}","/api/v1/ops/releases/{release_id}/rollouts","/api/v1/ops/rollouts/{rollout_id}","/api/v1/ops/rollouts/{rollout_id}/jobs","/api/v1/ops/rollouts/{rollout_id}/advance"],"related_contract_keys":["ops_job_contract","ops_driver_contract","ops_playbook_contract","ops_stack_diagnosis_contract"]}]}},{"action_code":"open_playbook_run_latest_events","source":"overview.secondary","title":"先处理异常编排","reason":"来自 overview.recommendation.secondary_action_code","focus_ref":{},"command":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_playbook_run_latest_events","contract_keys":["ops_stack_diagnosis_contract"],"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_stack_diagnosis_contract","contract_keys":["ops_stack_diagnosis_contract"],"contracts":[{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]}]}}],"operator_decision":{"lane":"ops_jobs","priority":"medium","reason_code":"playbook_runs_need_attention","title":"按总检默认下一步继续处理","summary":"来自 overview.recommendation.primary_action_code","next_focus":{},"primary_command_key":"focus_playbook_run","secondary_command_key":""},"next_actions":{"primary_command_key":"focus_playbook_run","primary_command":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","secondary_command_key":"","secondary_command":""},"recommended_commands":{"stack-diagnosis":"bash domain-api/deploy/multi-region/drive_ops_center.sh stack-diagnosis http://127.0.0.1:8100 summary","stack-check":"bash domain-api/deploy/multi-region/drive_ops_center.sh stack-diagnosis http://127.0.0.1:8100 summary","doctor":"bash domain-api/deploy/multi-region/drive_ops_center.sh doctor http://127.0.0.1:8100","contracts":"bash domain-api/deploy/multi-region/drive_ops_center.sh contracts http://127.0.0.1:8100","runtime-refresh-recover":"bash domain-api/deploy/multi-region/drive_ops_center.sh runtime-refresh-recover http://127.0.0.1:8100","api-restart":"bash domain-api/deploy/multi-region/drive_ops_center.sh runtime-refresh-recover http://127.0.0.1:8100","focus_playbook_run":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","fix_managed_nodes":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 fix_managed_nodes","open_playbook_run_latest_events":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_playbook_run_latest_events","focus-release-hub":"bash domain-api/deploy/multi-region/drive_ops_center.sh release-launchpad http://127.0.0.1:8100","release-launchpad-worker":"bash domain-api/deploy/multi-region/drive_ops_center.sh release-launchpad http://127.0.0.1:8100 worker"},"issues":[{"code":"playbook_runs_need_attention","severity":"warning","layer":"playbook_runs","summary":"最近存在未完全收口的 playbook run,建议先确认执行序列是否停在中间步骤。","detail":"problem_runs_total=3","action_code":"","focus_ref":{},"commands":["bash domain-api/deploy/multi-region/drive_ops_center.sh doctor http://127.0.0.1:8100"],"contract_keys":["ops_stack_diagnosis_contract","ops_playbook_contract"],"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_stack_diagnosis_contract","contract_keys":["ops_stack_diagnosis_contract","ops_playbook_contract"],"contracts":[{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]},{"key":"ops_playbook_contract","title":"Ops Playbook Contract","status":"active","version":"v1","summary":"冻结 playbook catalog / preview / run / events 的正式编排 contract。","primary_endpoint":"/api/v1/ops/playbooks","schema_doc_path":"docs/schemas/ops_playbook_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_playbook_contract","discovery_endpoints":["/api/v1/ops/playbooks","/api/v1/ops/playbooks/preview","/api/v1/ops/playbooks/execute","/api/v1/ops/playbook-runs","/api/v1/ops/playbook-runs/{run_code}","/api/v1/ops/playbook-runs/{run_code}/events","/api/v1/ops/playbook-runs/{run_code}/rerun","/api/v1/ops/playbook-runs/{run_code}/cancel"],"related_contract_keys":["ops_job_contract","release_hub_contract","ops_driver_contract","ops_observability_contract","ops_stack_diagnosis_contract"]}]}}],"operator_hints":["当前总检可用,但仍有缺口;建议先按 next_step 收口,再进入更细粒度脚本。"],"quick_commands":["bash domain-api/deploy/multi-region/drive_ops_center.sh stack-diagnosis http://127.0.0.1:8100 summary","bash domain-api/deploy/multi-region/drive_ops_center.sh doctor http://127.0.0.1:8100","bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 fix_managed_nodes","bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_playbook_run_latest_events"]},"api":{"http_status":"200","available":true,"service":"domain-api","version":"0.1.0","api_prefix":"/api/v1","worker_mode":"linux-systemd"},"surface_matrix":{"status_counts":{"total":9,"available":9,"missing":0},"items":[{"name":"api","available":true,"elapsed_ms":0},{"name":"contracts","available":true,"elapsed_ms":0},{"name":"link_snapshot","available":true,"elapsed_ms":0},{"name":"overview","available":true,"elapsed_ms":4655},{"name":"managed_nodes","available":true,"elapsed_ms":297},{"name":"release_hub","available":true,"elapsed_ms":0},{"name":"runtime_build_info","available":true,"elapsed_ms":0},{"name":"playbook_runs","available":true,"elapsed_ms":304},{"name":"activity_stream","available":true,"elapsed_ms":0}]},"contracts":{"http_status":"200","available":true,"elapsed_ms":0,"registry_version":"2026-04-18","schema_version":"v1","contracts_total":11,"contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract","ops_stack_diagnosis_contract","ops_doctor_decision_contract","ops_go_live_signoff_contract","ops_go_live_bundle_contract","ops_go_live_review_contract"],"error":""},"link_snapshot":{"http_status":"200","available":true,"elapsed_ms":0,"status":"attention","headline":"存在 1 个结果批次仍待推送。","next_action_codes":["focus_playbook_run","open_playbook_run_latest_events"],"error":""},"overview":{"http_status":"200","available":true,"elapsed_ms":4655,"recommendation_key":"playbook-run-attention-pbr-9cf64aa152","recommendation_title":"先处理异常编排","primary_action_code":"focus_playbook_run","secondary_action_code":"open_playbook_run_latest_events","focus_ref":{},"execution_scene_summary":"有效执行节点 3 台;正在执行/领任务 3 台;近窗刚有吞吐 0 台;在线但未参与 0 台","execution_scene_counts":{"dispatch_active":3,"recent_only":0,"standby":0,"load_syncing":0},"log_sync":{"status":"full_capture","status_label":"全量观察","mode":"full","mode_label":"全量回传","enabled":true,"covered_participating_nodes":3,"participating_nodes_total":3,"missing_sample_node_codes":[]},"error":""},"managed_nodes":{"http_status":"200","available":true,"elapsed_ms":297,"total":3,"managed_total":3,"managed_enabled":3,"online":3,"agent_ready":3,"ssh_ready":2,"remote_access_ready":3,"participating":3,"dispatch_active":3,"standby":0,"load_syncing":0,"queue_retrying_nodes":0,"queue_dead_letter_nodes":0,"queue_pending_records":0,"queue_dead_letter_records":0,"agent_state_counts":{"online_busy":3},"remote_access_state_counts":{"hybrid_ready":2,"agent_ready":1},"delivery_queue_state_counts":{"healthy":3},"problem_nodes":[],"first_handover_gap_node_code":"","first_handover_gap":{},"error":""},"release_hub":{"http_status":"200","available":true,"elapsed_ms":0,"latest_release_version":"domaincheck_release_20260418_013833","latest_release_status":"ready","default_rollout_gate_status":"ready","default_rollout_gate_label":"可发 Rollout","default_rollout_gate_summary":"默认 Release 与默认 Rollout 目标均已通过门禁,当前可直接进入定向或灰度 Rollout。","default_rollout_gate_focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"default_rollout_gate"},"launchpad_status":"attention","launchpad_status_label":"待补执行面","launchpad_summary":"最新发布包已经可用,但当前没有可用于智能 Rollout 的在线启用执行节点,需要先补齐托管或接入状态。","recommended_action_code":"fix_managed_nodes","recommended_execution_mode":"","focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_launchpad"},"error":""},"runtime_build_info":{"http_status":"200","available":true,"elapsed_ms":0,"source":"latest_release","package_name":"domaincheck_release_20260418_215930","generated_at":"2026-04-18T21:59:32","commit_sha":"246838ae4c07","commit_ref":"main","checksum":"69fe1074c472c4a7601f1d7f0a78866fc1a2011949e46d5d26ef603fc1aa5f7a","manifest_path":"/www/wwwroot/getDomain/release/latest_release.json","repository_capabilities":{"supports_install_command_block":true,"supports_multi_layout_bootstrap":true},"route_surface":{"mode":"registered","registered_paths_total":150,"surface_flags":{"ops_contracts":true,"ops_contract_detail":true,"ops_stack_diagnosis":true,"ops_node_handover":true,"ops_node_onboarding":true,"ops_node_onboarding_bootstrap_preview":true,"ops_node_onboarding_bootstrap_execute":true,"ops_node_onboarding_acceptance_preview":true,"ops_node_onboarding_acceptance_execute":true,"ops_node_onboarding_recovery_preview":true,"ops_node_onboarding_recovery_execute":true,"ops_node_scene_log":true,"ops_node_handover_bootstrap_plan":true,"ops_driver_feed":true,"ops_codex_brief":true,"ops_activity_stream":true,"ops_runbook_resolve":true,"ops_runbook_execute":true,"runtime_build_info":true},"expected_paths":{"ops_contracts":"/api/v1/ops/contracts","ops_contract_detail":"/api/v1/ops/contracts/{contract_key}","ops_stack_diagnosis":"/api/v1/ops/stack-diagnosis","ops_node_handover":"/api/v1/ops/nodes/{node_code}/handover","ops_node_onboarding":"/api/v1/ops/nodes/{node_code}/onboarding","ops_node_onboarding_bootstrap_preview":"/api/v1/ops/nodes/{node_code}/onboarding/bootstrap/preview","ops_node_onboarding_bootstrap_execute":"/api/v1/ops/nodes/{node_code}/onboarding/bootstrap/execute","ops_node_onboarding_acceptance_preview":"/api/v1/ops/nodes/{node_code}/onboarding/acceptance/preview","ops_node_onboarding_acceptance_execute":"/api/v1/ops/nodes/{node_code}/onboarding/acceptance/execute","ops_node_onboarding_recovery_preview":"/api/v1/ops/nodes/{node_code}/onboarding/recovery/preview","ops_node_onboarding_recovery_execute":"/api/v1/ops/nodes/{node_code}/onboarding/recovery/execute","ops_node_scene_log":"/api/v1/ops/nodes/{node_code}/scene-log","ops_node_handover_bootstrap_plan":"/api/v1/ops/nodes/{node_code}/handover/bootstrap-plan","ops_driver_feed":"/api/v1/ops/driver-feed","ops_codex_brief":"/api/v1/ops/codex-brief","ops_activity_stream":"/api/v1/ops/activity-stream","ops_runbook_resolve":"/api/v1/ops/runbook/sequences/{sequence_key}/resolve","ops_runbook_execute":"/api/v1/ops/runbook/sequences/{sequence_key}/execute","runtime_build_info":"/api/v1/runtime/build-info"},"missing_keys":[],"missing_paths":[],"surface_complete":true},"route_surface_missing_keys":[],"route_surface_declares_bootstrap_plan":true,"runtime_schema_stale":false,"error":""},"playbook_runs":{"http_status":"200","available":true,"elapsed_ms":304,"recent_total":6,"problem_runs_total":3,"problem_runs":[{"run_code":"pbr-9cf64aa152","status":"attention","status_label":"待关注","focus_step_key":"node_agent_logs","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-9cf64aa152","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"run_code":"pbr-2be333d3ce","status":"attention","status_label":"待关注","focus_step_key":"node_agent_logs","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-2be333d3ce","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}},{"run_code":"pbr-bfed43ea46","status":"attention","status_label":"待关注","focus_step_key":"node_agent_logs","focus_summary":"收集 Node Agent 日志 当前出现失败,建议先看事件,再决定是否整轮重跑。","focus_ref":{"kind":"playbook_run","run_code":"pbr-bfed43ea46","playbook_key":"onboarding.acceptance","group_key":"onboarding","focus_step_key":"node_agent_logs","focus_step_title":"收集 Node Agent 日志","event_key":"","node_code":""}}],"error":""},"activity_stream":{"http_status":"200","available":true,"elapsed_ms":0,"recent_total":8,"status_counts":{"running":2,"success":3,"attention":3},"top_items":[{"kind":"execution_scene","status":"running","summary":"有效执行节点 3 台;正在执行/领任务 3 台;近窗刚有吞吐 0 台;在线但未参与 0 台","occurred_at":"2026-04-19 01:29:39","focus_ref":{"kind":"execution_scene","scene_key":"execution-scene:live"},"source_focus_ref":{},"ui_intent_kind":"focus_execution_scene"},{"kind":"playbook_run","status":"success","summary":"当前整轮编排已全部收口。","occurred_at":"2026-04-19 01:20:47","focus_ref":{"kind":"playbook_run","run_code":"pbr-df533c6f4a","focus_step_key":"","focus_step_title":""},"source_focus_ref":{},"ui_intent_kind":"playbook_run_detail"},{"kind":"log_sync","status":"running","summary":"参与节点已经全部被覆盖,当前处于全量日志观察模式,适合短时深度排障。","occurred_at":"2026-04-19 01:20:42","focus_ref":{"kind":"execution_scene","scene_key":"execution-scene:log-sync"},"source_focus_ref":{},"ui_intent_kind":"focus_execution_scene"},{"kind":"playbook_run","status":"success","summary":"当前整轮编排已全部收口。","occurred_at":"2026-04-19 01:13:27","focus_ref":{"kind":"playbook_run","run_code":"pbr-9ce5c85f17","focus_step_key":"","focus_step_title":""},"source_focus_ref":{},"ui_intent_kind":"playbook_run_detail"},{"kind":"playbook_run","status":"success","summary":"当前整轮编排已全部收口。","occurred_at":"2026-04-19 01:13:17","focus_ref":{"kind":"playbook_run","run_code":"pbr-389abd618c","focus_step_key":"","focus_step_title":""},"source_focus_ref":{},"ui_intent_kind":"playbook_run_detail"}],"error":""}},"detail_code":null} \ No newline at end of file diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/summary.env b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/summary.env new file mode 100644 index 0000000..6d50ea4 --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929/cycle_1/summary.env @@ -0,0 +1,16 @@ +CYCLES="1" +GO_LIVE_STATUS="attention" +PUBLISH_READY="false" +LOG_SYNC_STATE="full_capture" +LOG_SYNC_MISSING_NODE_CODES="" +STACK_STATUS="attention" +ISSUE_TOTAL="1" +BLOCKING_ISSUE_TOTAL="0" +ISSUE_CODES="playbook_runs_need_attention" +LAUNCHPAD_STATUS="blocked" +LAUNCHPAD_RECOMMENDED_ACTION="run_acceptance" +PROBLEM_RUNS_TOTAL="0" +PROBLEM_RUN_CODE="" +SCENE_STATUS="full_capture" +SCENE_LINE_COUNT="1" +GENERATED_AT="2026-04-19 01:29:38" diff --git a/docs/ops_center_runtime/night_runs/night_run_20260419_012929_summary.json b/docs/ops_center_runtime/night_runs/night_run_20260419_012929_summary.json new file mode 100644 index 0000000..b590cd9 --- /dev/null +++ b/docs/ops_center_runtime/night_runs/night_run_20260419_012929_summary.json @@ -0,0 +1,20 @@ +{ + "cycles": 1, + "go_live_status": "attention", + "publish_ready": false, + "log_sync_state": "full_capture", + "log_sync_missing_node_codes": [], + "stack_status": "attention", + "issue_total": 1, + "blocking_issue_total": 0, + "issue_codes": [ + "playbook_runs_need_attention" + ], + "launchpad_status": "blocked", + "launchpad_recommended_action": "run_acceptance", + "problem_runs_total": 0, + "problem_run_code": "", + "scene_status": "full_capture", + "scene_line_count": 1, + "generated_at": "2026-04-19 01:29:38" +} \ No newline at end of file