docs: move ops runtime tracking under docs
This commit is contained in:
@@ -1,135 +0,0 @@
|
||||
# IMPLEMENTATION_STATUS
|
||||
|
||||
更新时间:2026-04-18 23:44 CST
|
||||
|
||||
## 当前真实状态
|
||||
|
||||
阶段判断:
|
||||
|
||||
- 方案与控制面能力建设:约 `78%`
|
||||
- 真正达到“可稳定上线、海外单脑统一接管大陆节点”:约 `58%`
|
||||
|
||||
当前已成立的事实:
|
||||
|
||||
- Ops Center 主控骨架已基本形成
|
||||
- Release Hub 已具备主流程判断能力
|
||||
- Node Agent 协议、日志回传、runbook / playbook、节点接管视图已具备基础可用性
|
||||
- `drive_ops_center.sh` 已补齐:
|
||||
- `nodes`
|
||||
- `node-bind-ssh`
|
||||
|
||||
当前最新节点状态:
|
||||
|
||||
- `managed_enabled = 3`
|
||||
- `agent_ready = 1`
|
||||
- `ssh_ready = 2`
|
||||
- `remote_access_ready = 3`
|
||||
- `pending_bootstrap = 2`
|
||||
|
||||
节点明细:
|
||||
|
||||
- `overseas-control-01`
|
||||
- `agent_ready`
|
||||
- `remote_access_ready = true`
|
||||
- `mainland-controller-01`
|
||||
- `pending_bootstrap`
|
||||
- `remote_access_state = ssh_ready`
|
||||
- `SSH 已录入`
|
||||
- `node-handover` 已完成
|
||||
- `node-bootstrap-plan` 已生成
|
||||
- 手工 SSH 登录已验证可达
|
||||
- `mainland-worker-01`
|
||||
- `pending_bootstrap`
|
||||
- `remote_access_state = ssh_ready`
|
||||
- `SSH 已录入`
|
||||
- `node-handover` 已完成
|
||||
- `node-bootstrap-plan` 已生成
|
||||
|
||||
## 代码闭环
|
||||
|
||||
已完成或基本具备:
|
||||
|
||||
- 统一查看节点接管状态
|
||||
- 统一生成与查看 handover / bootstrap 入口
|
||||
- 统一查看 go-live summary
|
||||
- 统一查看日志回传覆盖情况
|
||||
- 统一为托管节点录入 SSH 入口的 CLI 能力
|
||||
- controller 与 worker 的 bootstrap plan 均可生成
|
||||
|
||||
当前代码侧已验证的事实:
|
||||
|
||||
- 当前主控链路可以把大陆节点从 `agent_pending` 推到 `ssh_ready`
|
||||
- `node-handover` 与 `node-bootstrap-plan` 输出一致
|
||||
- `go-live-summary` 已看到 `remote_access_ready = 3`
|
||||
|
||||
结论:
|
||||
|
||||
- 当前代码侧已经可以支撑“接管闭环”的执行顺序
|
||||
- 当前主阻塞已不再是“还缺哪块控制面功能”
|
||||
- 但当前 SSH 自动执行链仍默认走密钥模式,和今晚实际提供的密码登录方式不一致
|
||||
|
||||
## 外部条件闭环
|
||||
|
||||
当前仍未闭合的条件:
|
||||
|
||||
- 大陆两台虽然已录入 SSH,但远端仓库未具备 node-agent 所需文件
|
||||
- 尚未实际触发大陆两台的 bootstrap 执行
|
||||
- 尚未建立两台大陆节点的 register / heartbeat 闭环
|
||||
- 尚未完成两台大陆节点的 acceptance 闭环
|
||||
|
||||
结论:
|
||||
|
||||
- 当前最关键的剩余问题不是新代码,而是大陆节点环境与当前控制面能力不匹配
|
||||
- 今晚已经不是“SSH 信息缺失”阶段,而是“大陆节点代码/部署内容缺失”阶段
|
||||
|
||||
## 明确依赖用户提供的信息
|
||||
|
||||
今晚已经提供并验证:
|
||||
|
||||
- `mainland-controller-01`
|
||||
- `ssh_host=121.204.244.188`
|
||||
- `ssh_user=root`
|
||||
- 手工 SSH 登录成功
|
||||
- `mainland-worker-01`
|
||||
- `ssh_host=121.204.244.248`
|
||||
- `ssh_user=root`
|
||||
|
||||
当前仍需要你明天确认或操作的,不再是 SSH 信息,而是以下环境修正路径二选一:
|
||||
|
||||
1. 让大陆两台节点仓库更新到包含 node-agent 文件的最新版本
|
||||
2. 或把 node-agent 相关文件以正式发布包 / 正式部署方式补齐到大陆两台
|
||||
|
||||
在上述环境未修正前,以下链路不能真正闭合:
|
||||
|
||||
- `bootstrap execute`
|
||||
- `register / heartbeat`
|
||||
- `acceptance`
|
||||
- `pending_bootstrap -> 已接管`
|
||||
|
||||
## 当前优先级判断
|
||||
|
||||
最高优先级:
|
||||
|
||||
- 大陆节点接管闭环最小路径
|
||||
|
||||
当前不应继续优先推进:
|
||||
|
||||
- 控制面横向增强
|
||||
- 额外专题设计
|
||||
- 与接管闭环无直接关系的发布能力细化
|
||||
- 新的页面或模块
|
||||
|
||||
本轮新增阻塞分类结论:
|
||||
|
||||
- controller 阻塞:环境问题
|
||||
- worker 阻塞:环境问题
|
||||
- 自动 SSH 执行器限制:权限 / 外部条件问题
|
||||
- 现链路仅按 `auth_mode=key` 工作
|
||||
- 今晚实际验证可用的是密码 SSH
|
||||
|
||||
## 完成主批次后的预期
|
||||
|
||||
如果大陆两台环境补齐,并按最小闭环顺序完成:
|
||||
|
||||
- 项目整体可稳定上线进度预计提升到 `60%~65%`
|
||||
- 一旦 `remote_access_ready=3/3` 成立,后续就从“架构推进期”切换到“发布收口期”
|
||||
148
TASK_BOARD.md
148
TASK_BOARD.md
@@ -1,148 +0,0 @@
|
||||
# TASK_BOARD
|
||||
|
||||
更新时间:2026-04-18 23:44 CST
|
||||
|
||||
## 当前主批次
|
||||
|
||||
唯一主批次:大陆节点接管闭环最小路径
|
||||
|
||||
目标:
|
||||
|
||||
- 把当前项目的最高优先级从“继续扩控制面能力”切到“完成大陆节点接管闭环”
|
||||
- 只围绕这条最短路径推进:
|
||||
- 录入大陆节点 SSH 入口
|
||||
- 生成并复核 bootstrap plan
|
||||
- 执行 bootstrap
|
||||
- 建立 register / heartbeat
|
||||
- 执行 acceptance
|
||||
- 将 `remote_access_ready` 从 `1/3` 提升到 `3/3`
|
||||
|
||||
主批次成功标准:
|
||||
|
||||
- `mainland-controller-01` 不再是 `pending_bootstrap`
|
||||
- `mainland-worker-01` 不再是 `pending_bootstrap`
|
||||
- `ssh_ready >= 2`
|
||||
- `remote_access_ready = 3/3`
|
||||
- 大陆两台都具备可由海外控制面统一发起标准动作的条件
|
||||
|
||||
当前完成度:
|
||||
|
||||
- 已完成 `SSH` 入口录入
|
||||
- 已完成 `node-handover`
|
||||
- 已完成 `node-bootstrap-plan`
|
||||
- 已确认 `ssh_ready = 2`
|
||||
- 已确认 `remote_access_ready = 3/3`
|
||||
- 尚未完成:
|
||||
- `bootstrap`
|
||||
- `register / heartbeat`
|
||||
- `acceptance`
|
||||
- `pending_bootstrap -> 已接管`
|
||||
|
||||
## 主批次拆解
|
||||
|
||||
### Batch A1
|
||||
|
||||
名称:大陆节点接管闭环最小路径
|
||||
|
||||
边界:
|
||||
|
||||
- 只处理大陆两台:
|
||||
- `mainland-controller-01`
|
||||
- `mainland-worker-01`
|
||||
- 只处理接管闭环,不扩展控制面功能
|
||||
|
||||
任务:
|
||||
|
||||
- 已完成:
|
||||
- 录入 `mainland-controller-01` SSH 信息
|
||||
- 录入 `mainland-worker-01` SSH 信息
|
||||
- 复查 `ops/nodes`
|
||||
- 逐台执行 `node-handover`
|
||||
- 逐台生成并复核 `node-bootstrap-plan`
|
||||
- 复核:
|
||||
- `go-live-summary`
|
||||
- `ops/nodes`
|
||||
- `release-launchpad`
|
||||
- 待完成:
|
||||
- 执行 controller 的 `bootstrap -> register -> heartbeat`
|
||||
- 执行 worker 的 `bootstrap -> register -> heartbeat`
|
||||
- 对两台执行 `acceptance`
|
||||
- 让两台都退出 `pending_bootstrap`
|
||||
|
||||
阻塞:
|
||||
|
||||
- controller 与 worker 两台大陆节点都缺少 node-agent 相关运行文件
|
||||
- 远端实际缺失:
|
||||
- `domain-api/app/node_agent.py`
|
||||
- `domain-api/app/api/routes/ops_agent.py`
|
||||
- `domain-api/app/services/ops_agent_service.py`
|
||||
- `domain-api/deploy/systemd/domain-node-agent.service`
|
||||
- `domain-api/deploy/multi-region/templates/domaincheck-node-agent.env.example`
|
||||
- 因此 bootstrap 片段虽然已生成,但远端执行时无法完成安装与启动
|
||||
|
||||
阻塞分类:
|
||||
|
||||
- 主阻塞:环境问题
|
||||
- 大陆节点上的仓库内容落后于当前控制面所需的 node-agent 文件集
|
||||
- 次阻塞:权限 / 外部条件问题
|
||||
- 当前 SSH 自动执行链默认走密钥模式
|
||||
- 今晚实际可用的是密码 SSH
|
||||
|
||||
## 候选批次
|
||||
|
||||
### Candidate B1
|
||||
|
||||
名称:接管闭环完成后的上线收口
|
||||
|
||||
前提:
|
||||
|
||||
- 仅在 `remote_access_ready = 3/3` 后进入
|
||||
|
||||
范围:
|
||||
|
||||
- 重新跑 `go-live-check / stack-diagnosis / release-launchpad`
|
||||
- 收口发布前证据
|
||||
- 判断是否达到正式上线窗口
|
||||
|
||||
### Candidate B2
|
||||
|
||||
名称:接管闭环完成后的值班自动化细化
|
||||
|
||||
前提:
|
||||
|
||||
- 仅在大陆两台已进入稳定接管状态后进入
|
||||
|
||||
范围:
|
||||
|
||||
- 优化运维体验
|
||||
- 优化日常值班命令
|
||||
- 优化交接与证据导出
|
||||
|
||||
当前不进入:
|
||||
|
||||
- `Candidate B1`
|
||||
- `Candidate B2`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前虽然 `remote_access_ready = 3/3`
|
||||
- 但大陆两台仍停在 `pending_bootstrap`
|
||||
- A1 还没有真正闭环
|
||||
|
||||
## 暂停项
|
||||
|
||||
以下事项当前不继续推进:
|
||||
|
||||
- 新的控制面专题设计
|
||||
- 新页面、新模块、新专题文档扩写
|
||||
- 与大陆节点接管闭环无直接关系的 Release Hub 增强
|
||||
- 与大陆节点接管闭环无直接关系的 runbook / playbook 扩展
|
||||
- 更细粒度日志分析功能
|
||||
- 额外的驾驶视图、美化项、信息聚合项
|
||||
- 任何不能直接推动 `bootstrap -> acceptance -> remote_access_ready=3/3` 的功能开发
|
||||
|
||||
## 当前判断
|
||||
|
||||
当前项目不缺“继续横向扩能力”的理由,缺的是“把最后两台大陆节点真正纳入闭环”的收口动作。
|
||||
|
||||
因此下一轮只围绕主批次 A1 推进。
|
||||
138
docs/ops_center_runtime/HANDOFF_20260419_0007.md
Normal file
138
docs/ops_center_runtime/HANDOFF_20260419_0007.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# HANDOFF 2026-04-19 00:07
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 完成 `mainland-controller-01` 的公网控制面重接入
|
||||
|
||||
原因:
|
||||
|
||||
- `mainland-worker-01` 已经接管成功
|
||||
- `remote_access_ready = 3/3` 已达成
|
||||
- 当前只剩 controller 一台仍处于 `pending_bootstrap`
|
||||
|
||||
## 本轮已完成的步骤
|
||||
|
||||
已完成:
|
||||
|
||||
1. 复核当前 `nodes / go-live-summary / release-launchpad`
|
||||
2. 确认两台大陆节点都已拉到最新代码
|
||||
3. 重新生成 controller / worker 的 bootstrap plan
|
||||
4. 在 `mainland-controller-01` 成功执行一次 bootstrap 落地
|
||||
5. 在 `mainland-worker-01` 使用公网控制面地址完成:
|
||||
- bootstrap
|
||||
- register
|
||||
- heartbeat
|
||||
6. 确认 `mainland-worker-01` 已变为:
|
||||
- `agent_state = online_busy`
|
||||
- `remote_access_state = hybrid_ready`
|
||||
7. 为 `mainland-worker-01` 发起接管后验收:
|
||||
- `playbook_run_code = pbr-137e8b258b`
|
||||
|
||||
## 当前结果
|
||||
|
||||
当前摘要:
|
||||
|
||||
- `ssh_ready = 2`
|
||||
- `agent_ready = 2`
|
||||
- `remote_access_ready = 3/3`
|
||||
- `pending_bootstrap = 1`
|
||||
|
||||
节点结果:
|
||||
|
||||
- `mainland-worker-01`
|
||||
- 已接管成功
|
||||
- 已开始 acceptance
|
||||
- `mainland-controller-01`
|
||||
- 仍未完成接管
|
||||
- 当前是唯一剩余阻塞点
|
||||
|
||||
## 两台节点推进到哪一步
|
||||
|
||||
### mainland-worker-01
|
||||
|
||||
已推进到:
|
||||
|
||||
- SSH 已录入
|
||||
- `node-handover` 已完成
|
||||
- `node-bootstrap-plan` 已生成
|
||||
- 使用公网控制面地址完成 bootstrap
|
||||
- register / heartbeat 已建立
|
||||
- 节点已进入:
|
||||
- `online_busy`
|
||||
- `hybrid_ready`
|
||||
- acceptance 已发起
|
||||
|
||||
当前状态判断:
|
||||
|
||||
- 该节点已经不再属于接管主阻塞
|
||||
|
||||
### mainland-controller-01
|
||||
|
||||
已推进到:
|
||||
|
||||
- SSH 已录入
|
||||
- `node-handover` 已完成
|
||||
- `node-bootstrap-plan` 已生成
|
||||
- 首次 bootstrap 已真正落地安装并启动 node-agent
|
||||
|
||||
准确卡点:
|
||||
|
||||
- 第一次写入的 `OPS_CONTROL_PLANE_BASE_URL` 是:
|
||||
- `http://127.0.0.1:8100`
|
||||
- 对远端大陆节点来说,这会指向它自己,而不是海外控制面
|
||||
- 随后尝试把它改为公网控制面地址时,SSH 握手被远端重置
|
||||
|
||||
阻塞分类:
|
||||
|
||||
- 主阻塞:外部条件 / 网络或权限问题
|
||||
- 不是:
|
||||
- 新功能缺失
|
||||
- 代码目录缺文件
|
||||
- Node Agent 主链路不可用
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做这一件事:
|
||||
|
||||
- 让 `mainland-controller-01` 成功执行“公网控制面地址版本”的 bootstrap 命令块
|
||||
|
||||
最小动作顺序:
|
||||
|
||||
1. 恢复 controller SSH 可执行窗口
|
||||
2. 重新写入:
|
||||
- `/etc/default/domaincheck-node-agent`
|
||||
3. 确认其中:
|
||||
- `OPS_CONTROL_PLANE_BASE_URL=http://152.53.37.118:8100`
|
||||
4. 重启:
|
||||
- `domaincheck-node-agent`
|
||||
5. 回控制面确认 controller 建立 register / heartbeat
|
||||
6. 对 controller 执行 acceptance
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
当前不建议切低模型的原因:
|
||||
|
||||
- 剩下的是收口型、多条件联动的现场问题
|
||||
- 需要稳定地分辨代码、执行参数、SSH 状态和节点真实状态
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 控制面增强项
|
||||
- 新架构文档扩写
|
||||
- Candidate B2
|
||||
- 任何与 controller 收口无直接关系的工作
|
||||
|
||||
## 一句话结论
|
||||
|
||||
今晚不是“要不要重装系统”的阶段,而是已经把大陆接管闭环收束到只剩一台 controller:`worker-01` 已接管成功,`controller-01` 只差把 node-agent 正确指向海外公网控制面并完成一次成功回连。
|
||||
151
docs/ops_center_runtime/HANDOFF_20260419_0030.md
Normal file
151
docs/ops_center_runtime/HANDOFF_20260419_0030.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# HANDOFF 2026-04-19 00:30
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 完成 acceptance 与发布前证据收口
|
||||
|
||||
原因:
|
||||
|
||||
- A1 大陆节点接管闭环已经完成
|
||||
- 当前 3 台节点都已在线
|
||||
- 当前真正剩余的不是接管问题,而是:
|
||||
- acceptance 还没全部收口
|
||||
- playbook run 仍有排队 / attention
|
||||
- log sync 仍是 partial
|
||||
|
||||
## 本轮已完成的步骤
|
||||
|
||||
已完成:
|
||||
|
||||
1. 复跑 `go-live-check`
|
||||
2. 复跑 `stack-diagnosis`
|
||||
3. 复跑 `release-launchpad`
|
||||
4. 复核 `mainland-worker-01` acceptance run:
|
||||
- `pbr-137e8b258b`
|
||||
5. 确认 `mainland-controller-01` 已成功进入:
|
||||
- `online_busy`
|
||||
- `hybrid_ready`
|
||||
6. 确认全局摘要:
|
||||
- `pending_bootstrap = 0`
|
||||
- `agent_ready = 3`
|
||||
- `remote_access_ready = 3`
|
||||
|
||||
## 当前结果
|
||||
|
||||
当前已经成立的事实:
|
||||
|
||||
- 大陆节点接管闭环已完成
|
||||
- controller / worker 都已接管成功
|
||||
- delivery queue 当前 3/3 健康
|
||||
- 发布包与 release gate 已就绪
|
||||
|
||||
当前仍未收口的事实:
|
||||
|
||||
- `mainland-worker-01` acceptance run `pbr-137e8b258b`
|
||||
- 仍是 `queued / running`
|
||||
- `release-launchpad` 当前推荐动作:
|
||||
- `run_acceptance`
|
||||
- 推荐目标:
|
||||
- `mainland-controller-01`
|
||||
- `log_sync_state = partial_coverage`
|
||||
- 未覆盖节点:
|
||||
- `mainland-worker-01`
|
||||
- `mainland-controller-01`
|
||||
|
||||
## 最新总检摘要
|
||||
|
||||
### go-live-summary
|
||||
|
||||
- `go_live_status = attention`
|
||||
- `publish_status = attention`
|
||||
- `publish_blocking_reasons = []`
|
||||
- `next_step_action_code = focus_playbook_run`
|
||||
|
||||
说明:
|
||||
|
||||
- 当前不是硬阻断
|
||||
- 但仍建议先把 playbook / acceptance 收口完再正式发版
|
||||
|
||||
### stack-diagnosis
|
||||
|
||||
- `surface_status = healthy`
|
||||
- `automation_status = attention`
|
||||
- `stack_status = attention`
|
||||
- 当前主推荐动作:
|
||||
- `focus_playbook_run`
|
||||
|
||||
说明:
|
||||
|
||||
- 接口面与 contract 面没有缺口
|
||||
- 当前问题集中在运行编排层
|
||||
|
||||
### release-launchpad
|
||||
|
||||
- latest package 可用
|
||||
- latest release `ready`
|
||||
- default rollout gate `ready`
|
||||
- 当前 launchpad 仍显示阻断,原因不是 bootstrap,而是:
|
||||
- `Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。`
|
||||
- 推荐命令:
|
||||
- `bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli`
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做这一件事:
|
||||
|
||||
- 对 `mainland-controller-01` 发起 acceptance
|
||||
|
||||
然后依次做:
|
||||
|
||||
1. 观察 `mainland-controller-01` acceptance run
|
||||
2. 跟完 `mainland-worker-01` acceptance run `pbr-137e8b258b`
|
||||
3. 复查 playbook runs 是否仍有 `focus_playbook_run`
|
||||
4. 再复查 `go-live-summary / stack-diagnosis / release-launchpad`
|
||||
5. 输出上线签收证据摘要
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前阶段是高耦合收口,不适合切低模型
|
||||
- 需要稳定判断 acceptance、playbook、launchpad 和 go-live 之间的关联关系
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 控制面增强项
|
||||
- 新架构文档扩写
|
||||
- 接管之外的自动化扩展
|
||||
- 在 acceptance 未收口前提前进入正式 Rollout
|
||||
|
||||
## 上线前证据摘要
|
||||
|
||||
已经具备的证据:
|
||||
|
||||
- 大陆节点接管完成
|
||||
- 3 台节点均在线
|
||||
- 发布包可用
|
||||
- smoke test 通过
|
||||
- final release gate 为 `ready`
|
||||
- delivery queue 当前 3/3 健康
|
||||
|
||||
尚缺的证据:
|
||||
|
||||
- controller acceptance 完成结果
|
||||
- worker acceptance 完成结果
|
||||
- playbook run 队列收口结果
|
||||
- log sync partial 覆盖问题的最终结论
|
||||
|
||||
## 一句话结论
|
||||
|
||||
项目已经从“接管闭环阶段”进入“上线前证据收口阶段”;当前真正拦在前面的,只剩 acceptance 与 playbook / log sync 这类运行证据,还不是代码能力问题。
|
||||
119
docs/ops_center_runtime/HANDOFF_20260419_0035.md
Normal file
119
docs/ops_center_runtime/HANDOFF_20260419_0035.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# HANDOFF 2026-04-19 00:35
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 跟完 acceptance run,拿到可签收终态
|
||||
|
||||
原因:
|
||||
|
||||
- acceptance 已全部发起
|
||||
- 当前接管不是问题
|
||||
- 当前真正卡住的是 acceptance 与 playbook run 收口
|
||||
|
||||
## 本轮已完成的步骤
|
||||
|
||||
已完成:
|
||||
|
||||
1. 对 `mainland-controller-01` 发起 acceptance
|
||||
2. 复核 `mainland-worker-01` acceptance run `pbr-137e8b258b`
|
||||
3. 复查:
|
||||
- `go-live-summary`
|
||||
- `stack-diagnosis`
|
||||
- `release-launchpad`
|
||||
4. 输出 acceptance 收口链判断
|
||||
|
||||
## 本轮执行摘要
|
||||
|
||||
### controller acceptance 结果
|
||||
|
||||
- `run_code = pbr-bfed43ea46`
|
||||
- 当前状态:`running`
|
||||
- 当前焦点:`health`
|
||||
- 当前说明:
|
||||
- `采集健康快照 当前处于排队中,建议先看事件和节点现场输出。`
|
||||
|
||||
### worker acceptance 结果
|
||||
|
||||
- `run_code = pbr-137e8b258b`
|
||||
- 当前状态:`running`
|
||||
- 当前焦点:`health`
|
||||
- 当前说明:
|
||||
- `采集健康快照 当前处于排队中,建议先看事件和节点现场输出。`
|
||||
|
||||
### go-live-summary 当前状态
|
||||
|
||||
- `go_live_status = attention`
|
||||
- `publish_status = attention`
|
||||
- `stack_status = attention`
|
||||
- `launchpad_status = attention`
|
||||
- `next_step_action_code = focus_playbook_run`
|
||||
|
||||
### stack-diagnosis 当前状态
|
||||
|
||||
- `surface_status = healthy`
|
||||
- `automation_status = attention`
|
||||
- `stack_status = attention`
|
||||
- `issue_total = 2`
|
||||
- `blocking_issue_total = 0`
|
||||
- `next_step_action_code = focus_playbook_run`
|
||||
|
||||
### release-launchpad 当前推荐动作
|
||||
|
||||
- `status = blocked`
|
||||
- `recommended_action_code = run_acceptance`
|
||||
- `recommended_target_node_code = mainland-controller-01`
|
||||
- 当前推荐摘要:
|
||||
- `Node Agent 已在线,可以直接跑 onboarding.acceptance 做标准接管验收。`
|
||||
|
||||
### log_sync_state`
|
||||
|
||||
- 当前仍为:`partial_coverage`
|
||||
- 未覆盖节点:
|
||||
- `mainland-worker-01`
|
||||
- `mainland-controller-01`
|
||||
|
||||
## 当前是否可上线签收
|
||||
|
||||
当前结论:
|
||||
|
||||
- 还不可上线签收
|
||||
|
||||
## 若仍不可签收,唯一剩余阻塞是什么
|
||||
|
||||
唯一剩余阻塞:
|
||||
|
||||
- acceptance playbook run 尚未完成,没有形成可签收终态
|
||||
|
||||
更具体地说:
|
||||
|
||||
- `pbr-bfed43ea46` 仍在运行
|
||||
- `pbr-137e8b258b` 仍在运行
|
||||
- 两条 run 都停在排队中的 `health` 步骤
|
||||
|
||||
当前没有出现明确失败,但也没有成功收口。
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 正式发布动作
|
||||
- 新功能开发
|
||||
- 新模块 / 新页面
|
||||
- 任何与 acceptance 收口无关的实现
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做:
|
||||
|
||||
- 再次检查 `pbr-bfed43ea46` 与 `pbr-137e8b258b` 是否进入终态
|
||||
|
||||
如果 acceptance 仍卡在排队:
|
||||
|
||||
- 只记录阻塞点、影响范围和建议下一步
|
||||
- 不扩展实现来绕过阻塞
|
||||
|
||||
## 一句话结论
|
||||
|
||||
acceptance 收口链已经全部发起,但两条验收编排都还停在队列中;当前离上线签收只差“acceptance 形成终态证据”,不是差功能,不是差接管,也不是差发布包。
|
||||
135
docs/ops_center_runtime/HANDOFF_20260419_0050.md
Normal file
135
docs/ops_center_runtime/HANDOFF_20260419_0050.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# HANDOFF 2026-04-19 00:50 CST
|
||||
|
||||
## 本轮目标
|
||||
|
||||
只处理 acceptance 队列不消费问题,不扩功能,不新增页面或专题文档。
|
||||
|
||||
## 本轮完成
|
||||
|
||||
### 1. 定位并修复 acceptance 队列卡死
|
||||
|
||||
定位结果:
|
||||
|
||||
- 两台大陆节点都在持续请求 `/api/v1/ops/agent/pull?limit=1`
|
||||
- 控制面收到请求,但 acceptance job 长时间停留在 `queued`
|
||||
- 数据库中存在大量 `idle in transaction`
|
||||
- 事务卡在:
|
||||
- `UPDATE ops_job_steps ...`
|
||||
- 随后又在新连接里 `INSERT INTO ops_job_events ...`
|
||||
|
||||
代码修复:
|
||||
|
||||
- 文件:
|
||||
- [domain-api/app/services/ops_agent_service.py](/www/wwwroot/getDomain/domain-api/app/services/ops_agent_service.py)
|
||||
- 修改点:
|
||||
- `agent_pull_jobs`
|
||||
- `agent_mark_job_started`
|
||||
- `agent_complete_job`
|
||||
- 修复方式:
|
||||
- 把 `append_ops_job_event(...)` 从事务内部移到 `conn.commit()` 之后执行
|
||||
|
||||
### 2. 补充最小回归测试
|
||||
|
||||
- 文件:
|
||||
- [domain-api/tests/test_ops_agent_service.py](/www/wwwroot/getDomain/domain-api/tests/test_ops_agent_service.py)
|
||||
- 新增测试:
|
||||
- `test_agent_pull_jobs_commits_before_appending_event`
|
||||
- `test_agent_mark_job_started_commits_before_appending_event`
|
||||
- `test_agent_complete_job_commits_before_appending_event`
|
||||
- 验证结果:
|
||||
- `cd /www/wwwroot/getDomain/domain-api && /opt/domaincheck/domainCheck/.venv/bin/python -m unittest tests.test_ops_agent_service -q`
|
||||
- `Ran 27 tests ... OK`
|
||||
|
||||
### 3. 运行态验证
|
||||
|
||||
- `domaincheck-api` 已完成强制切换到新版本进程
|
||||
- 新进程:
|
||||
- `MainPID = 614403`
|
||||
- 运行态日志已确认:
|
||||
- `job 36 / 41` 开始执行
|
||||
- 已出现 `start / events / complete`
|
||||
|
||||
## 当前 acceptance 结果
|
||||
|
||||
### `mainland-worker-01`
|
||||
|
||||
- run:
|
||||
- `pbr-137e8b258b`
|
||||
- 当前状态:
|
||||
- `attention`
|
||||
- 结果:
|
||||
- `4 success + 1 failed`
|
||||
- 唯一失败步骤:
|
||||
- `node_agent_logs`
|
||||
- 失败原因:
|
||||
- `No journal files were opened due to insufficient permissions`
|
||||
|
||||
### `mainland-controller-01`
|
||||
|
||||
- run:
|
||||
- `pbr-bfed43ea46`
|
||||
- 当前状态:
|
||||
- `attention`
|
||||
- 结果:
|
||||
- `4 success + 1 failed`
|
||||
- 唯一失败步骤:
|
||||
- `node_agent_logs`
|
||||
- 失败原因:
|
||||
- `No journal files were opened due to insufficient permissions`
|
||||
|
||||
## 当前总检状态
|
||||
|
||||
`go-live-summary`:
|
||||
|
||||
- `go_live_status = attention`
|
||||
- `publish_status = attention`
|
||||
- `stack_status = attention`
|
||||
- `launchpad_status = attention`
|
||||
- `log_sync_state = full_capture`
|
||||
|
||||
`stack-diagnosis`:
|
||||
|
||||
- `surface_status = healthy`
|
||||
- `automation_status = attention`
|
||||
- `stack_status = attention`
|
||||
- `issue_total = 1`
|
||||
- `blocking_issue_total = 0`
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 修复远端 Node Agent 读取 `journalctl` 的权限
|
||||
|
||||
## 当前是否可上线签收
|
||||
|
||||
结论:
|
||||
|
||||
- 还不可上线签收
|
||||
|
||||
唯一剩余阻塞:
|
||||
|
||||
- 两条 acceptance run 都因为 `node_agent_logs` 权限不足而未全绿
|
||||
|
||||
## 下一轮若继续,必须先做什么
|
||||
|
||||
只做这一件事:
|
||||
|
||||
- 处理 `domaincheck-node-agent` 所在服务账号的 journal 读取权限
|
||||
|
||||
推荐最小方向:
|
||||
|
||||
- 让运行 Node Agent 的账号具备读取 systemd journal 的权限
|
||||
- 然后重跑 acceptance,或只补采失败日志步骤
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
- 不要回退这次热修复
|
||||
- 不要重新 bootstrap
|
||||
- 不要重装大陆机器
|
||||
- 不要扩控制面功能
|
||||
- 不要进入正式发布
|
||||
|
||||
## 本轮一句话结论
|
||||
|
||||
接管主链已经打通,acceptance 队列事务 bug 已修复;当前只剩远端 `journalctl` 权限问题,解决后即可继续冲刺上线签收。
|
||||
156
docs/ops_center_runtime/HANDOFF_20260419_0101.md
Normal file
156
docs/ops_center_runtime/HANDOFF_20260419_0101.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# HANDOFF 2026-04-19 01:01
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 完成 `B2-Journal Permission Closure`
|
||||
|
||||
原因:
|
||||
|
||||
- 接管链路已经闭合
|
||||
- acceptance 队列不消费问题已经修复
|
||||
- 当前两条 acceptance run 只剩同一个失败步骤:
|
||||
- `node_agent_logs`
|
||||
|
||||
## 本轮处理结论
|
||||
|
||||
本轮没有继续扩展控制面,也没有新增页面或模块。
|
||||
|
||||
本轮只做了两件事:
|
||||
|
||||
1. 把剩余阻塞进一步收敛为明确的环境落点:
|
||||
- `domaincheck-node-agent.service` 以 `www:www` 运行
|
||||
- 现网 unit 没有给 `www` 补 `journalctl` 所需的读取权限
|
||||
2. 在仓库中的 Node Agent systemd 模板加入:
|
||||
- `SupplementaryGroups=systemd-journal`
|
||||
|
||||
文件变更:
|
||||
|
||||
- `domain-api/deploy/systemd/domain-node-agent.service`
|
||||
|
||||
## 当前状态
|
||||
|
||||
当前摘要:
|
||||
|
||||
- `pending_bootstrap = 0`
|
||||
- `agent_ready = 3`
|
||||
- `remote_access_ready = 3`
|
||||
- `managed_enabled = 3`
|
||||
- `log_sync_state = full_capture`
|
||||
|
||||
acceptance 摘要:
|
||||
|
||||
- `mainland-worker-01`
|
||||
- `run_code = pbr-137e8b258b`
|
||||
- `status = attention`
|
||||
- `4 success + 1 failed`
|
||||
- 唯一失败步骤:`node_agent_logs`
|
||||
- `mainland-controller-01`
|
||||
- `run_code = pbr-bfed43ea46`
|
||||
- `status = attention`
|
||||
- `4 success + 1 failed`
|
||||
- 唯一失败步骤:`node_agent_logs`
|
||||
|
||||
## 当前唯一剩余阻塞
|
||||
|
||||
唯一剩余阻塞:
|
||||
|
||||
- 两台大陆节点尚未刷新新的 Node Agent systemd unit
|
||||
|
||||
准确卡点:
|
||||
|
||||
- 代码仓库里已经补了 `SupplementaryGroups=systemd-journal`
|
||||
- 但远端节点上的 `/etc/systemd/system/domaincheck-node-agent.service` 还需要重新安装并重启服务
|
||||
|
||||
阻塞分类:
|
||||
|
||||
- 环境 / 权限问题
|
||||
|
||||
不是以下问题:
|
||||
|
||||
- 不是 SSH 问题
|
||||
- 不是 agent offline
|
||||
- 不是 acceptance 队列不消费
|
||||
- 不是控制面接口 404
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做这一件事:
|
||||
|
||||
- 在 `mainland-controller-01` 与 `mainland-worker-01` 上重新安装最新 Node Agent unit,并重启 `domaincheck-node-agent`
|
||||
|
||||
然后依次做:
|
||||
|
||||
1. 确认 unit 已带 `SupplementaryGroups=systemd-journal`
|
||||
2. 重跑两台节点 acceptance
|
||||
3. 复核:
|
||||
- `go-live-summary`
|
||||
- `stack-diagnosis`
|
||||
- `release-launchpad`
|
||||
4. 输出上线签收证据摘要
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前阶段是最后一段 acceptance 收口
|
||||
- 需要稳定处理运行证据与状态对账
|
||||
- 不需要切到超高推理
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新功能
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 控制面增强项
|
||||
- 新专题文档
|
||||
- 正式发布动作
|
||||
|
||||
## 给用户的下一任务卡片
|
||||
|
||||
按下面这张任务卡,在两台大陆节点执行:
|
||||
|
||||
```text
|
||||
任务名:B2-Journal Permission Closure / 节点 Unit 刷新
|
||||
|
||||
目标:
|
||||
1. 让 domaincheck-node-agent 具备 journal 读取权限
|
||||
2. 为 acceptance 清掉 node_agent_logs 的最后失败点
|
||||
|
||||
执行节点:
|
||||
- mainland-controller-01
|
||||
- mainland-worker-01
|
||||
|
||||
执行步骤:
|
||||
1. 进入最新代码目录
|
||||
2. 重新安装最新 unit 文件到 /etc/systemd/system/domaincheck-node-agent.service
|
||||
3. daemon-reload
|
||||
4. restart domaincheck-node-agent
|
||||
5. 输出 unit 关键行与服务状态
|
||||
|
||||
执行命令:
|
||||
cd /www/wwwroot/getDomain
|
||||
install -m 0644 domain-api/deploy/systemd/domain-node-agent.service /etc/systemd/system/domaincheck-node-agent.service
|
||||
systemctl daemon-reload
|
||||
systemctl restart domaincheck-node-agent
|
||||
systemctl cat domaincheck-node-agent | grep -E '^(User|Group|SupplementaryGroups)='
|
||||
systemctl status domaincheck-node-agent --no-pager -l
|
||||
journalctl -u domaincheck-node-agent -n 60 --no-pager
|
||||
|
||||
回传给 Codex 的结果:
|
||||
1. systemctl cat 中是否出现 SupplementaryGroups=systemd-journal
|
||||
2. systemctl status 是否 active (running)
|
||||
3. journalctl 是否还出现 insufficient permissions
|
||||
```
|
||||
|
||||
## 一句话结论
|
||||
|
||||
项目现在不缺接管能力,也不缺控制面能力;真正剩下的只是把两台大陆节点上的 Node Agent unit 刷新到最新,然后重跑 acceptance 把最后一个 `node_agent_logs` 失败点清掉。
|
||||
126
docs/ops_center_runtime/HANDOFF_20260419_0107.md
Normal file
126
docs/ops_center_runtime/HANDOFF_20260419_0107.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# HANDOFF 2026-04-19 01:07
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 执行 `C1-Acceptance Re-run`
|
||||
|
||||
原因:
|
||||
|
||||
- `B2-Journal Permission Closure` 在运行态上已经具备完成证据
|
||||
- 两台节点都已出现:
|
||||
- `logs.collect ok=True`
|
||||
- `domaincheck-node-agent` 重启成功
|
||||
- `registered: Agent 注册成功`
|
||||
- 当前只剩旧 acceptance 结果尚未被新运行态覆盖
|
||||
|
||||
## 本轮确认到的运行态证据
|
||||
|
||||
`mainland-controller-01`:
|
||||
|
||||
- 旧执行记录中已经出现:
|
||||
- `action=logs.collect ok=True`
|
||||
- 之后又完成:
|
||||
- `domaincheck-node-agent.service` 停止 / 启动
|
||||
- `starting node agent`
|
||||
- `registered: Agent 注册成功`
|
||||
|
||||
`mainland-worker-01`:
|
||||
|
||||
- 旧执行记录中已经出现:
|
||||
- `action=logs.collect ok=True`
|
||||
- 之后也完成:
|
||||
- `domaincheck-node-agent.service` 停止 / 启动
|
||||
- `starting node agent`
|
||||
- `registered: Agent 注册成功`
|
||||
|
||||
## 当前判断
|
||||
|
||||
当前可以做出的最小结论:
|
||||
|
||||
- 日志权限收口在运行面已经打通
|
||||
- Node Agent 当前在线且能重新注册
|
||||
- 不需要再回头排查 SSH、bootstrap、heartbeat、队列消费
|
||||
|
||||
当前还不能直接签收上线的唯一原因:
|
||||
|
||||
- 控制面上仍保留旧 acceptance run 的 `attention` 结果
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做这一件事:
|
||||
|
||||
- 分别对 `mainland-controller-01` 和 `mainland-worker-01` 重跑 acceptance
|
||||
|
||||
然后依次做:
|
||||
|
||||
1. 记录新的 acceptance `run_code`
|
||||
2. 确认两条 run 是否全绿
|
||||
3. 复核:
|
||||
- `go-live-summary`
|
||||
- `stack-diagnosis`
|
||||
- `release-launchpad`
|
||||
4. 输出上线签收证据摘要
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前还是高耦合收口阶段
|
||||
- 但已经不需要切超高推理
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新功能
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 控制面增强项
|
||||
- 新专题文档
|
||||
- 正式发布动作
|
||||
|
||||
## 给用户的下一任务卡片
|
||||
|
||||
按下面这张任务卡继续:
|
||||
|
||||
```text
|
||||
任务名:C1-Acceptance Re-run / 两台大陆节点验收重跑
|
||||
|
||||
目标:
|
||||
1. 用新的运行态覆盖旧 acceptance 失败结果
|
||||
2. 确认 mainland-controller-01 与 mainland-worker-01 是否都已通过标准接管验收
|
||||
3. 为上线签收准备最终证据
|
||||
|
||||
执行位置:
|
||||
- 海外控制面主机
|
||||
|
||||
执行命令:
|
||||
cd /www/wwwroot/getDomain
|
||||
bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-controller-01 cli
|
||||
bash domain-api/deploy/multi-region/drive_ops_center.sh node-acceptance-run http://127.0.0.1:8100 mainland-worker-01 cli
|
||||
|
||||
然后继续执行:
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/go-live-summary
|
||||
echo
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/stack-diagnosis
|
||||
echo
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/release-launchpad
|
||||
|
||||
回传给 Codex 的结果:
|
||||
1. controller 新的 acceptance run_code 和最终状态
|
||||
2. worker 新的 acceptance run_code 和最终状态
|
||||
3. go-live-summary 当前状态
|
||||
4. stack-diagnosis 当前状态
|
||||
5. release-launchpad 当前推荐动作
|
||||
```
|
||||
|
||||
## 一句话结论
|
||||
|
||||
当前项目已经越过“日志权限修复”这一步,下一步不需要再修环境,只需要把两台大陆节点的 acceptance 重跑一遍,用新结果完成上线前签收判断。
|
||||
133
docs/ops_center_runtime/HANDOFF_20260419_0115.md
Normal file
133
docs/ops_center_runtime/HANDOFF_20260419_0115.md
Normal file
@@ -0,0 +1,133 @@
|
||||
# HANDOFF 2026-04-19 01:15
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 执行 `D1-Evidence Convergence`
|
||||
|
||||
原因:
|
||||
|
||||
- 大陆两台 acceptance 已经全绿
|
||||
- 当前不再是接管链路问题
|
||||
- 当前剩余的是“上线签收证据口径”没有完全收敛
|
||||
|
||||
## 本轮已完成
|
||||
|
||||
`mainland-controller-01`:
|
||||
|
||||
- acceptance 新 run:
|
||||
- `pbr-9ce5c85f17`
|
||||
- 最终结果:
|
||||
- `status = success`
|
||||
- `5 success / 0 failed`
|
||||
|
||||
`mainland-worker-01`:
|
||||
|
||||
- acceptance 新 run:
|
||||
- `pbr-389abd618c`
|
||||
- 最终结果:
|
||||
- `status = success`
|
||||
- `5 success / 0 failed`
|
||||
|
||||
总检现状:
|
||||
|
||||
- `go_live_status = attention`
|
||||
- `stack_status = attention`
|
||||
- `publish_status = attention`
|
||||
- `launchpad_status = attention`
|
||||
- `log_sync_state = partial_coverage`
|
||||
- `log_sync_missing_node_codes = [overseas-control-01]`
|
||||
- `stack_diagnosis.issue_total = 2`
|
||||
- `stack_diagnosis.blocking_issue_total = 0`
|
||||
|
||||
## 当前判断
|
||||
|
||||
当前已经可以确认:
|
||||
|
||||
- 大陆节点接管闭环已完成
|
||||
- acceptance 已通过
|
||||
- Node Agent 日志权限问题已真实解除
|
||||
|
||||
当前还不能直接签收上线的原因只有一类:
|
||||
|
||||
- 证据口径还没完全收敛
|
||||
|
||||
具体表现为两处:
|
||||
|
||||
- `overseas-control-01` 还没有补齐日志样本,导致 `log_sync_state = partial_coverage`
|
||||
- `stack-diagnosis / release-launchpad` 仍保留历史 attention / stale 推荐
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做这一件事:
|
||||
|
||||
- 给 `overseas-control-01` 补一轮场景日志样本
|
||||
|
||||
然后依次做:
|
||||
|
||||
1. 复查 `go-live-summary`
|
||||
2. 复查 `stack-diagnosis`
|
||||
3. 复查 `release-launchpad`
|
||||
4. 判断 attention 是否已收敛到可上线签收
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前还是收口判断阶段
|
||||
- 需要稳定地辨别“真实阻塞”和“口径残留”
|
||||
- 但还不需要切到超高推理
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新功能
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 控制面增强项
|
||||
- 新专题文档
|
||||
- 正式发布动作
|
||||
|
||||
## 下一个任务卡
|
||||
|
||||
```text
|
||||
任务名:D1-Evidence Convergence / overseas-control-01 日志补样与总检复核
|
||||
|
||||
目标:
|
||||
1. 补齐 overseas-control-01 的场景日志样本
|
||||
2. 观察 go-live-summary 是否从 partial_coverage 收敛
|
||||
3. 观察 stack-diagnosis 与 release-launchpad 是否仍保留历史残留口径
|
||||
4. 判断是否达到可上线签收状态
|
||||
|
||||
执行位置:
|
||||
- 海外控制面主机
|
||||
|
||||
执行命令:
|
||||
cd /www/wwwroot/getDomain
|
||||
bash domain-api/deploy/multi-region/drive_ops_center.sh scene-node-log http://127.0.0.1:8100 overseas-control-01 120 full
|
||||
|
||||
然后继续执行:
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/go-live-summary
|
||||
echo
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/stack-diagnosis
|
||||
echo
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/releases/launchpad
|
||||
|
||||
回传给 Codex 的结果:
|
||||
1. overseas-control-01 场景日志是否成功拉到样本
|
||||
2. go-live-summary 当前状态
|
||||
3. stack-diagnosis 当前 issue_total / blocking_issue_total
|
||||
4. release-launchpad 当前推荐动作
|
||||
5. 当前是否已经达到可上线签收
|
||||
```
|
||||
|
||||
## 一句话结论
|
||||
|
||||
acceptance 这一关已经过了;下一轮不该再回头修大陆节点,而是只把 `overseas-control-01` 的日志样本和总检残留口径收敛掉。
|
||||
128
docs/ops_center_runtime/HANDOFF_20260419_0121.md
Normal file
128
docs/ops_center_runtime/HANDOFF_20260419_0121.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# HANDOFF 2026-04-19 01:21
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 执行 `E1-Historical Signal Convergence`
|
||||
|
||||
原因:
|
||||
|
||||
- `overseas-control-01` 的日志样本已经补齐
|
||||
- `go-live-summary.log_sync_state` 已恢复为 `full_capture`
|
||||
- 当前不再是样本缺口问题,而是历史 attention 口径未完全收敛
|
||||
|
||||
## 本轮已完成
|
||||
|
||||
日志样本补齐:
|
||||
|
||||
- 已执行 `log-sync-recover`
|
||||
- 已执行 `run_inspection_participating`
|
||||
- 新巡检 run:
|
||||
- `pbr-df533c6f4a`
|
||||
- `overseas-control-01`:
|
||||
- `scene-log status = full_capture`
|
||||
- `line_count = 1`
|
||||
|
||||
总检变化:
|
||||
|
||||
- `go-live-summary`
|
||||
- `log_sync_state: partial_coverage -> full_capture`
|
||||
- `log_sync_missing_node_codes: [overseas-control-01] -> []`
|
||||
- `stack-diagnosis`
|
||||
- `issue_total: 2 -> 1`
|
||||
- `blocking_issue_total = 0`
|
||||
- `playbook_runs.problem_runs_total`
|
||||
- `4 -> 3`
|
||||
|
||||
## 当前判断
|
||||
|
||||
当前已经可以确认:
|
||||
|
||||
- 大陆节点接管闭环已完成
|
||||
- acceptance 已通过
|
||||
- participating 节点日志样本覆盖已完成
|
||||
|
||||
当前还不能直接签收上线的原因只有一类:
|
||||
|
||||
- 历史 playbook / launchpad attention 口径还没完全消化掉
|
||||
|
||||
具体表现为两处:
|
||||
|
||||
- `stack-diagnosis` 还剩:
|
||||
- `playbook_runs_need_attention`
|
||||
- `release-launchpad` 还保留旧推荐:
|
||||
- `run_acceptance`
|
||||
- `mainland-controller-01`
|
||||
|
||||
## 下一轮若继续,第一步先做什么
|
||||
|
||||
下一轮第一步只做这一件事:
|
||||
|
||||
- 聚焦 `focus_playbook_run`
|
||||
|
||||
然后依次做:
|
||||
|
||||
1. 查清 `problem_runs_total = 3` 是不是纯历史旧 run
|
||||
2. 复查 `stack-diagnosis`
|
||||
3. 复查 `release-launchpad`
|
||||
4. 判断当前是否已经达到可上线签收
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前已经进入最后的收口判断阶段
|
||||
- 需要高质量区分“真阻塞”和“历史口径残留”
|
||||
- 还不需要切超高推理
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新功能
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 控制面增强项
|
||||
- 新专题文档
|
||||
- 正式发布动作
|
||||
|
||||
## 下一个任务卡
|
||||
|
||||
```text
|
||||
任务名:E1-Historical Signal Convergence / 历史 playbook attention 收口
|
||||
|
||||
目标:
|
||||
1. 确认 stack-diagnosis 剩余的唯一告警是不是纯历史旧 run
|
||||
2. 判断 release-launchpad 的旧 acceptance 推荐是否只是残留口径
|
||||
3. 为上线签收做最后判断
|
||||
|
||||
执行位置:
|
||||
- 海外控制面主机
|
||||
|
||||
执行命令:
|
||||
cd /www/wwwroot/getDomain
|
||||
bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run
|
||||
echo
|
||||
bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_playbook_run_latest_events
|
||||
echo
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/stack-diagnosis
|
||||
echo
|
||||
curl -s http://127.0.0.1:8100/api/v1/ops/releases/launchpad
|
||||
|
||||
回传给 Codex 的结果:
|
||||
1. focus_playbook_run 指向的具体 run_code
|
||||
2. 该 run 是否是历史旧 run
|
||||
3. stack-diagnosis 当前 issue_total
|
||||
4. release-launchpad 当前推荐动作
|
||||
5. 当前是否已经达到可上线签收
|
||||
```
|
||||
|
||||
## 一句话结论
|
||||
|
||||
日志样本缺口已经补上了;下一轮不该再碰接管和日志链路,而是只把最后那条历史 attention 口径清掉。
|
||||
129
docs/ops_center_runtime/HANDOFF_20260419_0225.md
Normal file
129
docs/ops_center_runtime/HANDOFF_20260419_0225.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# HANDOFF 2026-04-19 02:25
|
||||
|
||||
## 本轮完成了什么
|
||||
|
||||
本轮只收口了“大陆 controller 无法 `pull_tasks`”这一条链。
|
||||
|
||||
实际完成:
|
||||
|
||||
1. 修复 `domaincheck-node-agent` 读取不到同步目标地址的问题
|
||||
2. 以 `www` 提交并推送:
|
||||
- `74dc009`
|
||||
- `fix: inherit sync env in node agent`
|
||||
3. 在两台大陆节点刷新 `domaincheck-node-agent.service`
|
||||
4. 重启并验证:
|
||||
- `mainland-controller-01`
|
||||
- `mainland-worker-01`
|
||||
5. 从海外控制面再次发起:
|
||||
- `runtime.pull_tasks`
|
||||
- `job_id = 87`
|
||||
6. 验证 `mainland-controller-01` 返回成功:
|
||||
- `pull_state = success`
|
||||
- `items_total = 50`
|
||||
7. 验证大陆两台节点当前都处于:
|
||||
- `participation_state = running`
|
||||
|
||||
## 当前最高优先级
|
||||
|
||||
唯一最高优先级:
|
||||
|
||||
- 收敛检测总览统计口径
|
||||
|
||||
原因:
|
||||
|
||||
- 现在不是“没跑”
|
||||
- 而是“已经跑了,但总览没完整体现大陆执行量”
|
||||
|
||||
## 当前状态摘要
|
||||
|
||||
当前结论:
|
||||
|
||||
- 检测执行链已跑通
|
||||
- 大陆 controller 已能拉任务
|
||||
- 大陆 controller / worker 已进入执行中
|
||||
|
||||
直接证据:
|
||||
|
||||
- `ops/nodes`
|
||||
- `mainland-controller-01`
|
||||
- `items_total = 200`
|
||||
- `items_running = 5`
|
||||
- `mainland-worker-01`
|
||||
- `items_total = 70`
|
||||
- `items_running = 5`
|
||||
- `job 87`
|
||||
- `runtime.pull_tasks`
|
||||
- `status = success`
|
||||
|
||||
## 当前唯一剩余问题
|
||||
|
||||
唯一剩余问题:
|
||||
|
||||
- `detect/queue-summary`
|
||||
- `detect/job/active`
|
||||
|
||||
仍主要沿用旧的海外活跃任务口径,没有把大陆“拉批后本地执行”的量体现在同一汇总里。
|
||||
|
||||
这会导致页面观感像“大陆没跑”,但实际上已经在跑。
|
||||
|
||||
## 下一轮如果继续,只做什么
|
||||
|
||||
下一轮只做这一件事:
|
||||
|
||||
- 对齐以下三处的数据口径:
|
||||
- `ops/nodes`
|
||||
- `detect/queue-summary`
|
||||
- `detect/job/active`
|
||||
|
||||
目标:
|
||||
|
||||
- 让后台页面既能显示大陆节点正在执行
|
||||
- 又能把这部分执行量合并进统一检测进度
|
||||
|
||||
## 现在不要做的事情
|
||||
|
||||
不要做:
|
||||
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 运维中枢增强
|
||||
- 正式发布
|
||||
- 与检测总览收口无关的任何工作
|
||||
|
||||
## 推荐模型与推理等级
|
||||
|
||||
继续推荐:
|
||||
|
||||
- `GPT-5.4`
|
||||
- `high`
|
||||
|
||||
原因:
|
||||
|
||||
- 当前是数据口径排查与小范围实现
|
||||
- 不需要切超高
|
||||
|
||||
## 给下一轮的任务卡
|
||||
|
||||
```text
|
||||
任务名:G1-检测总览口径统一批
|
||||
|
||||
目标:
|
||||
1. 查清 ops/nodes 与 detect/job/active 的数据来源差异
|
||||
2. 明确大陆 controller 拉批执行为什么没有进入统一检测总览
|
||||
3. 修好后再跑一轮小批量检测验证
|
||||
|
||||
范围限制:
|
||||
1. 只做检测总览口径统一
|
||||
2. 不新增页面
|
||||
3. 不扩展控制面功能
|
||||
4. 不进入发布动作
|
||||
|
||||
完成判定:
|
||||
1. 点击启动检测后
|
||||
2. 后台能看到大陆节点参与
|
||||
3. detect/queue-summary 不再只显示 overseas-control-01 与 unassigned
|
||||
```
|
||||
|
||||
## 一句话结论
|
||||
|
||||
这轮已经把“大陆 controller 拉不到任务”这个硬阻塞清掉了;现在真正剩下的不是执行问题,而是后台统计口径还没把大陆执行量完整显示出来。
|
||||
166
docs/ops_center_runtime/IMPLEMENTATION_STATUS.md
Normal file
166
docs/ops_center_runtime/IMPLEMENTATION_STATUS.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# IMPLEMENTATION_STATUS
|
||||
|
||||
更新时间:2026-04-19 02:25 CST
|
||||
|
||||
## 当前真实状态
|
||||
|
||||
阶段判断:
|
||||
|
||||
- 海外单脑接管能力:约 `95%`
|
||||
- 分布式检测真实跑通程度:约 `89%`
|
||||
- 距离“可稳定上线并放心用后台发起检测”:约 `90%`
|
||||
|
||||
当前最重要的新事实:
|
||||
|
||||
- 大陆节点接管闭环已完成
|
||||
- acceptance 主链已完成
|
||||
- `runtime.pull_tasks` 的环境阻塞已解决
|
||||
- 大陆检测链已开始真实执行
|
||||
- 当前剩余问题已收敛为“检测总览统计口径未统一”
|
||||
|
||||
## 本轮代码与环境收口
|
||||
|
||||
本轮代码变更:
|
||||
|
||||
- 提交:`74dc009`
|
||||
- 主题:`fix: inherit sync env in node agent`
|
||||
|
||||
本轮实际变更点:
|
||||
|
||||
- `domain-api/deploy/systemd/domain-node-agent.service`
|
||||
- 增加:
|
||||
- `EnvironmentFile=-/etc/default/domaincheck-api`
|
||||
- `domain-api/deploy/multi-region/templates/domaincheck-node-agent.env.example`
|
||||
- 补齐:
|
||||
- `runtime.start_detection`
|
||||
- `runtime.stop_detection`
|
||||
- `runtime.pull_tasks`
|
||||
|
||||
本轮环境动作:
|
||||
|
||||
- 已在 `mainland-controller-01` 刷新并重启 `domaincheck-node-agent`
|
||||
- 已在 `mainland-worker-01` 刷新并重启 `domaincheck-node-agent`
|
||||
- 已确认两台节点 unit 包含:
|
||||
- `/etc/default/domaincheck-api`
|
||||
- `/etc/default/domaincheck-node-agent`
|
||||
- `SupplementaryGroups=systemd-journal`
|
||||
|
||||
## 关键验证证据
|
||||
|
||||
关键成功证据:
|
||||
|
||||
- `mainland-controller-01` 新运维任务:
|
||||
- `job_id = 87`
|
||||
- `action = runtime.pull_tasks`
|
||||
- `status = success`
|
||||
- 返回结果:
|
||||
- `pull_state = success`
|
||||
- `items_total = 50`
|
||||
- `summary_text = 待检测任务批次拉取并入库成功`
|
||||
|
||||
远端日志证据:
|
||||
|
||||
- `mainland-controller-01`
|
||||
- `job start: id=87 ... action=runtime.pull_tasks`
|
||||
- `job complete: ... ok=True`
|
||||
|
||||
节点运行证据:
|
||||
|
||||
- `mainland-controller-01`
|
||||
- `participation_state = running`
|
||||
- `items_running = 5`
|
||||
- `items_total = 200`
|
||||
- `mainland-worker-01`
|
||||
- `participation_state = running`
|
||||
- `items_running = 5`
|
||||
- `items_total = 70`
|
||||
|
||||
## 代码闭环
|
||||
|
||||
当前已经被验证打通的链路:
|
||||
|
||||
- detect.start 触发远端动作排队
|
||||
- mainland controller 的 sync-agent / pull_tasks 目标地址读取
|
||||
- Node Agent 远端执行 `runtime.pull_tasks`
|
||||
- 大陆 controller 拉批入库
|
||||
- 大陆 controller / worker 进入执行态
|
||||
|
||||
这意味着:
|
||||
|
||||
- “点开始检测后大陆节点完全不参与”这个问题已经被修掉
|
||||
|
||||
## 外部条件闭环
|
||||
|
||||
已经闭合:
|
||||
|
||||
- SSH
|
||||
- node handover
|
||||
- bootstrap
|
||||
- register / heartbeat
|
||||
- acceptance
|
||||
- journald 权限
|
||||
- node-agent 同步目标地址继承
|
||||
|
||||
当前不再依赖用户补充的外部条件:
|
||||
|
||||
- 不需要再补 `SYNC_TARGET_API_BASE_URL`
|
||||
- 不需要再手工改 controller 的 node-agent env
|
||||
- 不需要重装系统
|
||||
|
||||
## 当前剩余问题
|
||||
|
||||
当前唯一剩余问题:
|
||||
|
||||
- 控制面“检测总览”口径仍未和大陆拉批执行口径统一
|
||||
|
||||
具体表现:
|
||||
|
||||
- `ops/nodes`
|
||||
- 明确显示大陆 controller / worker 正在执行
|
||||
- 但:
|
||||
- `/api/v1/detect/queue-summary`
|
||||
- `/api/v1/detect/job/active`
|
||||
- 仍主要显示旧的:
|
||||
- `overseas-control-01`
|
||||
- `unassigned`
|
||||
|
||||
因此当前判断是:
|
||||
|
||||
- 执行链路已经通
|
||||
- 展示与统计口径还没完全通
|
||||
|
||||
## 当前是否可以直接上线
|
||||
|
||||
当前结论:
|
||||
|
||||
- 可以继续做检测功能测试
|
||||
- 但还不建议宣称“检测总览已经完全可信、可以直接签收上线”
|
||||
|
||||
原因不是执行失败,而是:
|
||||
|
||||
- 总览统计仍会让人误判“大陆没跑”
|
||||
|
||||
## 当前优先级判断
|
||||
|
||||
最高优先级:
|
||||
|
||||
- 收敛 `detect/queue-summary` / `detect/job/active` 的统计口径
|
||||
|
||||
当前不应继续推进:
|
||||
|
||||
- 新运维页面
|
||||
- 控制面增强
|
||||
- 新模块
|
||||
- 发布动作
|
||||
- 与检测总览收口无关的任何功能
|
||||
|
||||
## 完成下一轮后的预期
|
||||
|
||||
如果检测总览口径收敛成功:
|
||||
|
||||
- 后台将能同时看到:
|
||||
- 海外触发
|
||||
- 大陆拉批
|
||||
- 大陆执行
|
||||
- 统一进度统计
|
||||
- 整体可上线程度预计提升到 `94%~96%`
|
||||
41
docs/ops_center_runtime/NIGHT_PACKAGE_20260419_0121.md
Normal file
41
docs/ops_center_runtime/NIGHT_PACKAGE_20260419_0121.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# NIGHT PACKAGE 2026-04-19 01:21
|
||||
|
||||
执行窗口:
|
||||
|
||||
- 开始时间:`2026-04-19 01:21 CST`
|
||||
- 截止时间:`2026-04-20 12:00 CST`
|
||||
|
||||
范围边界:
|
||||
|
||||
- 只围绕当前收口目标执行
|
||||
- 不进入新实现
|
||||
- 不扩展控制面功能
|
||||
- 不新增页面、模块或专题文档
|
||||
|
||||
夜间任务包内容:
|
||||
|
||||
1. 周期性复查:
|
||||
- `go-live-summary`
|
||||
- `stack-diagnosis`
|
||||
- `release-launchpad`
|
||||
- `playbook-runs`
|
||||
- `overseas-control-01 scene-log`
|
||||
2. 若日志样本再次出现缺口:
|
||||
- 自动执行 `log-sync-recover`
|
||||
- 自动执行一次 `run_inspection_participating`
|
||||
3. 若只剩历史 `playbook_runs_need_attention`:
|
||||
- 最多补 3 轮安全的 `inspection.standard`
|
||||
- 目的仅为刷新近期成功 run,推动历史告警窗口收敛
|
||||
4. 到达以下任一条件即停止:
|
||||
- 到 `2026-04-20 12:00 CST`
|
||||
- 或者:
|
||||
- `log_sync_state = full_capture`
|
||||
- `issue_total = 0`
|
||||
- `problem_runs_total = 0`
|
||||
|
||||
输出位置:
|
||||
|
||||
- 运行日志目录:
|
||||
- [docs/ops_center_runtime/night_runs](/www/wwwroot/getDomain/docs/ops_center_runtime/night_runs)
|
||||
- 停止后自动生成报告:
|
||||
- `night_run_*_report.md`
|
||||
139
docs/ops_center_runtime/TASK_BOARD.md
Normal file
139
docs/ops_center_runtime/TASK_BOARD.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# TASK_BOARD
|
||||
|
||||
更新时间:2026-04-19 02:25 CST
|
||||
|
||||
## 当前主批次
|
||||
|
||||
唯一主批次:`F1-分布式检测实跑收口批`
|
||||
|
||||
目标:
|
||||
|
||||
- 不进入新功能实现
|
||||
- 不扩展控制面页面
|
||||
- 只确认“海外发起检测 -> 大陆 controller 拉任务 -> 大陆节点执行”已经真实跑通
|
||||
- 收敛剩余唯一问题:控制面检测总览口径仍停留在旧活跃任务视图
|
||||
|
||||
## 当前结果快照
|
||||
|
||||
本轮新增完成:
|
||||
|
||||
- 已修复 `domaincheck-node-agent` 读取不到同步目标地址的问题
|
||||
- 修复方式:
|
||||
- `domaincheck-node-agent.service` 追加读取 `/etc/default/domaincheck-api`
|
||||
- 新提交:`74dc009`
|
||||
- 已在以下节点完成刷入与重启:
|
||||
- `mainland-controller-01`
|
||||
- `mainland-worker-01`
|
||||
|
||||
本轮关键验证:
|
||||
|
||||
- `mainland-controller-01`
|
||||
- `runtime.pull_tasks`
|
||||
- `job_id = 87`
|
||||
- `status = success`
|
||||
- `items_total = 50`
|
||||
- `pull_state = success`
|
||||
- controller 远端日志已确认:
|
||||
- `job start: runtime.pull_tasks`
|
||||
- `job complete ... ok=True`
|
||||
|
||||
当前节点执行状态:
|
||||
|
||||
- `mainland-controller-01`
|
||||
- `agent_state = online_busy`
|
||||
- `participation_state = running`
|
||||
- `items_total = 200`
|
||||
- `items_running = 5`
|
||||
- `mainland-worker-01`
|
||||
- `agent_state = online_busy`
|
||||
- `participation_state = running`
|
||||
- `items_total = 70`
|
||||
- `items_running = 5`
|
||||
- `overseas-control-01`
|
||||
- `agent_state = online_busy`
|
||||
|
||||
## 当前结论
|
||||
|
||||
检测不是“没跑”。
|
||||
|
||||
当前真实状态是:
|
||||
|
||||
- 分布式检测执行链已经跑通
|
||||
- 大陆 controller 已经能从海外控制面主动拉回待检测批次
|
||||
- 大陆 controller 与 mainland worker 都已经进入执行中
|
||||
|
||||
当前未完全收敛的不是执行,而是统计口径:
|
||||
|
||||
- `/api/v1/detect/queue-summary`
|
||||
- `/api/v1/detect/job/active`
|
||||
|
||||
仍主要显示旧的海外活跃任务统计:
|
||||
|
||||
- `overseas-control-01`
|
||||
- `unassigned`
|
||||
|
||||
尚未把大陆侧“拉批后本地执行”的结果完整并回收到同一检测总览口径中。
|
||||
|
||||
## 当前唯一主问题
|
||||
|
||||
唯一主问题:`检测执行已跑通,但控制面检测总览仍是旧口径`
|
||||
|
||||
表现:
|
||||
|
||||
- `ops/nodes` 能看到大陆节点正在跑
|
||||
- `runtime.pull_tasks` 已成功
|
||||
- 但 `detect/queue-summary` 与 `detect/job/active` 仍没有把大陆执行量体现在同一总览里
|
||||
|
||||
问题分类:
|
||||
|
||||
- 不是接管问题
|
||||
- 不是环境问题
|
||||
- 不是同步目标地址问题
|
||||
- 是“检测总览聚合口径 / 运行态映射”问题
|
||||
|
||||
## 候选批次
|
||||
|
||||
### Candidate G1
|
||||
|
||||
名称:检测总览口径统一批
|
||||
|
||||
进入条件:
|
||||
|
||||
- 用户确认下一轮继续只收敛检测总览,不扩其他功能
|
||||
|
||||
范围:
|
||||
|
||||
- 复核 `detect job` 与 `ops/nodes` 的数据来源差异
|
||||
- 明确大陆拉批执行应如何回写到统一检测总览
|
||||
- 修复后再做一次检测实跑验证
|
||||
|
||||
### Candidate G2
|
||||
|
||||
名称:节点能力口径清理批
|
||||
|
||||
进入条件:
|
||||
|
||||
- 需要把节点展示中的 `capabilities` 与现网真实支持动作同步
|
||||
|
||||
范围:
|
||||
|
||||
- 只处理 `OPS_AGENT_CAPABILITIES` 的现网漂移
|
||||
- 不碰检测链路主体
|
||||
|
||||
## 暂停项
|
||||
|
||||
继续暂停:
|
||||
|
||||
- 新页面
|
||||
- 新模块
|
||||
- 运维中枢增强
|
||||
- 发布动作
|
||||
- 与“检测实跑收口”无关的任何工作
|
||||
|
||||
## 下一轮唯一动作
|
||||
|
||||
下一轮唯一应该做的事情:
|
||||
|
||||
- 对齐 `detect/queue-summary`、`detect/job/active` 与 `ops/nodes` 三者的数据口径
|
||||
- 明确为什么大陆执行已发生,但总览仍显示旧任务统计
|
||||
- 修好后再跑一次小批量检测验证
|
||||
@@ -0,0 +1 @@
|
||||
680339
|
||||
@@ -0,0 +1 @@
|
||||
683962
|
||||
@@ -0,0 +1 @@
|
||||
{"code":0,"message":"ok","data":{"base_url":"http://127.0.0.1:8100","generated_at":"2026-04-19 01:29:38","go_live_status":"attention","publish_ready":false,"publish_status":"attention","publish_status_label":"可发布但建议先复核","publish_summary":"当前没有硬阻断,但仍有上线前关注项,建议先完成复核再正式发版。","stack_status":"attention","contracts_ready":true,"contracts_total":11,"launchpad_status":"attention","launchpad_status_label":"待补执行面","launchpad_recommended_action_code":"fix_managed_nodes","launchpad_recommended_target_node_code":"","launchpad_recommended_recovery_label":"","launchpad_recommended_recovery_summary":"来自 overview.recommendation.primary_action_code","launchpad_onboarding_bootstrap_pending_nodes":0,"launchpad_onboarding_acceptance_ready_nodes":0,"route_surface_complete":true,"route_surface_missing_keys":[],"route_surface_declares_bootstrap_plan":true,"runtime_schema_stale":false,"repository_capabilities":{"supports_install_command_block":true,"supports_multi_layout_bootstrap":true},"managed_enabled":3,"remote_access_ready":3,"queue_dead_letter_nodes":0,"activity_start_delivery_issue_total":0,"participating_nodes_total":3,"log_sync_enabled":true,"log_sync_state":"full_capture","log_sync_mode":"full","log_sync_covered_nodes":3,"log_sync_missing_node_codes":[],"next_step_action_code":"focus_playbook_run","next_step_reason":"来自 overview.recommendation.primary_action_code","operator_lane":"ops_jobs","operator_title":"按总检默认下一步继续处理","operator_primary_command_key":"focus_playbook_run","publish_blocking_reasons":[],"publish_warnings":["stack_diagnosis=attention","release_launchpad=attention"],"blocking_reasons":[],"warnings":["stack_diagnosis=attention","release_launchpad=attention"],"recommended_commands":{"stack_summary":"bash domain-api/deploy/multi-region/check_ops_center_stack.sh http://127.0.0.1:8100 summary","contracts":"bash domain-api/deploy/multi-region/check_ops_contracts.sh http://127.0.0.1:8100","ops_plane":"bash domain-api/deploy/multi-region/check_ops_plane.sh http://127.0.0.1:8100","release_hub":"bash domain-api/deploy/multi-region/check_release_hub.sh http://127.0.0.1:8100","inspection":"bash domain-api/deploy/multi-region/check_ops_inspection.sh http://127.0.0.1:8100","overview":"bash domain-api/deploy/multi-region/drive_ops_center.sh overview http://127.0.0.1:8100","go_live_recover":"bash domain-api/deploy/multi-region/drive_ops_center.sh go-live-recover http://127.0.0.1:8100","doctor_export":"bash domain-api/deploy/multi-region/drive_ops_center.sh doctor-export /tmp/domaincheck-go-live http://127.0.0.1:8100","next_step":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 focus_playbook_run","log_sync_logs":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 open_worker_logs_participating","log_sync_inspection":"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 run_inspection_participating"},"source_refs":{"stack_diagnosis_contract_key":"ops_stack_diagnosis_contract","contracts_registry_version":"2026-04-18","runtime_build_commit_sha":"246838ae4c07","release_focus_ref":{"kind":"release_hub","release_id":2,"release_version":"domaincheck_release_20260418_013833","channel":"stable","rollout_id":0,"rollout_code":"","section":"release_launchpad"}}},"detail_code":null}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
{"code":0,"message":"ok","data":{"node_code":"overseas-control-01","available":true,"status":"full_capture","status_label":"全量观察","status_type":"success","summary":"节点当前正在参与检测,已保留 1 条现场日志样本。","log_sync_enabled":true,"mode":"full","mode_label":"全量回传","records_total":0,"records_visible":0,"records_truncated":false,"records":[],"latest_record":{},"source_summary":{"node_code":"overseas-control-01","line_count":1,"key_line_count":0,"full_line_count":1,"last_at":"2026-04-19 01:20:35","last_line":"[2026-04-19 01:20:35] [overseas-control-01] 2026-04-19 01:00:18.866 | INFO | __main__:start_detection:2575 - 开始执行域名检测任务"},"missing_reason_code":"","missing_reason":"","node":{"node_code":"overseas-control-01","region":"overseas","role":"control","status":"busy","current_load":25,"last_heartbeat_at":"2026-04-19 01:29:45"},"participation":{"detect_participating":true,"participation_state":"running","participation_label":"执行中","participation_reason":"当前正在执行 4 项检测任务。","participation_bucket":"dispatch_active","participation_bucket_label":"执行/已领","is_dispatch_active":true},"contract_navigation":{"detail_endpoint_pattern":"/api/v1/ops/contracts/{contract_key}","primary_contract_key":"ops_observability_contract","contract_keys":["ops_observability_contract","ops_stack_diagnosis_contract"],"contracts":[{"key":"ops_observability_contract","title":"Ops Observability Contract","status":"active","version":"v1","summary":"冻结 execution scene / inspection overview / activity stream / delivery queue 的正式观察面 contract。","primary_endpoint":"/api/v1/ops/overview","schema_doc_path":"docs/schemas/ops_observability_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_observability_contract","discovery_endpoints":["/api/v1/ops/overview","/api/v1/ops/inspection-overview","/api/v1/ops/activity-stream","/api/v1/ops/nodes/{node_code}/scene-log","/api/v1/ops/nodes/{node_code}/delivery-queue","/api/v1/ops/nodes/{node_code}/delivery-queue/records","/api/v1/ops/nodes/{node_code}/delivery-queue/flush","/api/v1/ops/nodes/{node_code}/delivery-queue/replay","/api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/replay","/api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/discard"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","ops_driver_contract","ops_playbook_contract","ops_stack_diagnosis_contract"]},{"key":"ops_stack_diagnosis_contract","title":"Ops Stack Diagnosis Contract","status":"active","version":"v1","summary":"冻结海外单脑总检入口的统一诊断 contract,供页面、CLI、Codex、按钮共享同一份第一现场判断。","primary_endpoint":"/api/v1/ops/stack-diagnosis","schema_doc_path":"docs/schemas/ops_stack_diagnosis_contract.md","detail_endpoint":"/api/v1/ops/contracts/ops_stack_diagnosis_contract","discovery_endpoints":["/api/v1/ops/go-live-summary","/api/v1/ops/stack-diagnosis","/api/v1/ops/contracts","/api/v1/ops/link-snapshot","/api/v1/ops/overview","/api/v1/ops/nodes","/api/v1/ops/releases/launchpad","/api/v1/ops/playbook-runs","/api/v1/ops/activity-stream"],"related_contract_keys":["ops_job_contract","ops_agent_protocol","release_hub_contract","ops_driver_contract","ops_playbook_contract","ops_observability_contract"]}]}},"detail_code":null}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,16 @@
|
||||
CYCLES="1"
|
||||
GO_LIVE_STATUS="attention"
|
||||
PUBLISH_READY="false"
|
||||
LOG_SYNC_STATE="full_capture"
|
||||
LOG_SYNC_MISSING_NODE_CODES=""
|
||||
STACK_STATUS="attention"
|
||||
ISSUE_TOTAL="1"
|
||||
BLOCKING_ISSUE_TOTAL="0"
|
||||
ISSUE_CODES="playbook_runs_need_attention"
|
||||
LAUNCHPAD_STATUS="blocked"
|
||||
LAUNCHPAD_RECOMMENDED_ACTION="run_acceptance"
|
||||
PROBLEM_RUNS_TOTAL="0"
|
||||
PROBLEM_RUN_CODE=""
|
||||
SCENE_STATUS="full_capture"
|
||||
SCENE_LINE_COUNT="1"
|
||||
GENERATED_AT="2026-04-19 01:29:38"
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"cycles": 1,
|
||||
"go_live_status": "attention",
|
||||
"publish_ready": false,
|
||||
"log_sync_state": "full_capture",
|
||||
"log_sync_missing_node_codes": [],
|
||||
"stack_status": "attention",
|
||||
"issue_total": 1,
|
||||
"blocking_issue_total": 0,
|
||||
"issue_codes": [
|
||||
"playbook_runs_need_attention"
|
||||
],
|
||||
"launchpad_status": "blocked",
|
||||
"launchpad_recommended_action": "run_acceptance",
|
||||
"problem_runs_total": 0,
|
||||
"problem_run_code": "",
|
||||
"scene_status": "full_capture",
|
||||
"scene_line_count": 1,
|
||||
"generated_at": "2026-04-19 01:29:38"
|
||||
}
|
||||
Reference in New Issue
Block a user