feat: add ops center and node onboarding flow

This commit is contained in:
Your Name
2026-04-18 23:52:51 +08:00
parent 246838ae4c
commit b9c29481b5
142 changed files with 89727 additions and 186 deletions

View File

@@ -0,0 +1,816 @@
# domainCheck Ops Agent Protocol
## 1. 目标
这份文档用于冻结 `Node Agent <-> Overseas Control Plane` 的正式 contract。
适用范围:
- `POST /api/v1/ops/agent/tokens`
- `POST /api/v1/ops/agent/bootstrap-plan`
- `GET /api/v1/ops/nodes/{node_code}/handover`
- `POST /api/v1/ops/nodes/{node_code}/handover/bootstrap-plan`
- `POST /api/v1/ops/agent/register`
- `POST /api/v1/ops/agent/heartbeat`
- `POST /api/v1/ops/agent/pull`
- `POST /api/v1/ops/agent/jobs/{job_id}/start`
- `POST /api/v1/ops/agent/jobs/{job_id}/complete`
- `POST /api/v1/ops/agent/jobs/{job_id}/events`
设计原则:
- Agent 只执行声明过的结构化动作
- 控制面负责调度、门禁、编排和审计
- 所有回执都必须可重放、可去重、可死信化
---
## 2. 公共响应包裹
所有接口统一返回:
```json
{
"code": 0,
"message": "ok",
"detail_code": "",
"data": {}
}
```
约束:
- `code=0` 表示成功
- `code=1` 表示业务失败
- `detail_code` 用于结构化错误语义
- `message` 只承担可读说明,不承担机器判断主逻辑
常见 `detail_code`
- `agent_token_invalid`
- `agent_token_expired`
- `agent_token_node_mismatch`
- `ops_job_not_owned_by_agent`
- `ops_job_invalid_status_transition`
- `release_checksum_mismatch`
- `release_health_check_failed`
---
## 3. 认证与请求头
Agent 相关接口统一使用:
- `Content-Type: application/json`
- `X-Domaincheck-Agent-Token: <token>`
其中:
- `tokens`
- `bootstrap-plan`
不要求 Agent Token。
其余接口都要求 Token。
---
## 4. 公共节点载荷
`register``heartbeat` 当前共享 `_base_payload()`
最小字段:
```json
{
"node_code": "mainland-worker-02",
"region": "mainland",
"role": "worker",
"title": "mainland-worker-02",
"hostname": "host-a",
"ip": "10.0.0.12",
"agent_version": "0.1.0",
"capabilities": [
"service.start",
"service.stop",
"service.restart",
"service.status",
"health.snapshot",
"logs.collect",
"diagnostics.collect",
"deploy.release"
],
"labels": {},
"metadata": {
"service_names": {
"api": "domaincheck-api",
"worker": "domaincheck-worker",
"sync_agent": "domaincheck-sync-agent",
"node_agent": "domaincheck-node-agent"
},
"delivery_queue": {
"state": "healthy",
"label": "正常",
"reason": "当前没有待重试回执,也没有死信记录。",
"pending_count": 0,
"dead_letter_count": 0,
"last_flush_at": "",
"oldest_pending_at": "",
"oldest_pending_request_id": "",
"oldest_pending_kind": "",
"oldest_dead_letter_at": "",
"oldest_dead_letter_request_id": "",
"oldest_dead_letter_kind": ""
}
}
}
```
说明:
- `service_names` 是控制面识别节点服务拓扑的最小入口
- `delivery_queue` 是控制面识别 Agent 回执现场的最小入口
---
## 5. Bootstrap Contract
### 5.1 Issue Token
`POST /api/v1/ops/agent/tokens`
请求体最小字段:
```json
{
"node_code": "mainland-worker-02",
"issued_by": "web-ui",
"expires_in_hours": 72,
"metadata": {
"source": "ops-center"
}
}
```
返回体关键字段:
- `token`
- `token_preview`
- `record_id`
- `node_code`
- `expires_at`
- `created_at`
### 5.2 Bootstrap Plan
`POST /api/v1/ops/agent/bootstrap-plan`
请求体最小字段:
```json
{
"node_code": "mainland-worker-02",
"node_region": "mainland",
"node_role": "worker",
"issued_by": "web-ui",
"expires_in_hours": 72,
"control_plane_base_url": "https://ops.example.com",
"root_dir": "/opt/domaincheck",
"metadata": {
"source": "ops-center"
}
}
```
返回体关键字段:
- Token 相关字段
- `control_plane_base_url`
- `bootstrap_plan.node_code`
- `bootstrap_plan.node_region`
- `bootstrap_plan.node_role`
- `bootstrap_plan.root_dir`
- `bootstrap_plan.env_file`
- `bootstrap_plan.service_name`
- `bootstrap_plan.service_file`
- `bootstrap_plan.install_script_path`
- `bootstrap_plan.env_content`
- `bootstrap_plan.command_lines`
- `bootstrap_plan.command_block`
- `bootstrap_plan.health_checks`
- `bootstrap_plan.health_check_block`
- `bootstrap_plan.bootstrap_script_name`
- `bootstrap_plan.bootstrap_script_path`
- `bootstrap_plan.bootstrap_script_content`
- `bootstrap_plan.bootstrap_run_script_block`
约束:
- `bootstrap_plan` 是标准接入方案对象
- 页面、Codex、CLI 必须复用这一个对象
- 不允许各端再次手写 env 或 shell 逻辑
---
## 6. Runtime Contract
### 6.1 Register
`POST /api/v1/ops/agent/register`
语义:
- 校验 Token 与 `node_code`
- Upsert Agent 运行态
- 将节点推入托管目录候选态
最小成功返回字段:
- `node_code`
- `expires_at`
- `capabilities`
### 6.2 Heartbeat
`POST /api/v1/ops/agent/heartbeat`
语义:
- 刷新 `last_seen_at`
- 刷新节点身份、服务名和队列快照
- 维持控制面中的 Agent 在线态
最小成功返回字段:
- `node_code`
- `server_time`
- `expires_at`
### 6.3 Pull
`POST /api/v1/ops/agent/pull?limit=1`
请求体最小字段:
```json
{
"node_code": "mainland-worker-02"
}
```
返回体关键字段:
```json
{
"jobs": [
{
"id": 123,
"job_code": "ops-xxx",
"action": "health.snapshot",
"target_node_code": "mainland-worker-02",
"status": "dispatching",
"execution_mode": "remote-agent",
"payload": {},
"metadata": {},
"policy": {},
"steps": []
}
],
"count": 1
}
```
约束:
- Agent 只能拿到属于自己的任务
- Agent 不负责选择任务
- 调度权始终在控制面
### 6.3.1 Agent Job Envelope
为了避免 Agent 再从自然语言动作名里“猜执行上下文”,`pull.data.jobs[]` 当前已经按统一任务包裹返回。
当前 `pull.data` 顶层也会额外带:
- `protocol_version`
- `envelope_type`
- `node_code`
- `limit`
- `count`
推荐最小结构:
```json
{
"protocol_version": "ops-agent/v1",
"envelope_type": "agent_job",
"id": 123,
"job_id": 123,
"job_code": "ops-20260418-001",
"job_type": "ops_action",
"action": "logs.collect",
"target_node_code": "mainland-worker-02",
"status": "dispatching",
"execution_mode": "remote-agent",
"step_key": "worker_logs",
"step_title": "收集 Worker 日志",
"job_ref": {
"job_id": 123,
"job_code": "ops-20260418-001",
"action": "logs.collect",
"target_node_code": "mainland-worker-02"
},
"step_ref": {
"step_id": 456,
"step_key": "worker_logs",
"step_title": "收集 Worker 日志"
},
"payload": {
"service_name": "domaincheck-worker",
"lines": 120,
"include_agent_logs": false
},
"policy": {
"timeout_seconds": 120,
"stop_on_failure": true,
"auto_approve": true
},
"metadata": {
"requested_by": "playbook:inspection.standard",
"source": "ops-center"
},
"release_context": {
"release_id": 0,
"release_version": "",
"rollout_id": 0,
"rollout_code": ""
},
"focus_ref": {
"kind": "ops_job",
"job_id": 123,
"job_code": "ops-20260418-001",
"action": "logs.collect",
"target_node_code": "mainland-worker-02"
}
}
```
正式要求:
- Agent 必须优先消费:
- `action`
- `payload`
- `policy`
- Agent 不应通过:
- `step_title`
- `summary`
- `notes`
去反推真实执行参数
- 与发布相关的任务应通过:
- `release_context`
传递 release / rollout 关联,而不是让 Agent 自己查询“当前版本”
这层的意义是:
- `ops job`
- 仍是正式执行颗粒度
- `Agent Job Envelope`
- 是 Node Agent 拿到的稳定执行视图
这样同一套 Agent 才能同时承接:
- 标准巡检
- 日志收集
- 诊断采样
- Release 部署
- Rollout 回滚
### 6.4 Start
`POST /api/v1/ops/agent/jobs/{job_id}/start`
请求体最小字段:
```json
{
"node_code": "mainland-worker-02"
}
```
语义:
- `dispatching` 表示已派发
- `running` 表示节点已真实开工
### 6.5 Complete
`POST /api/v1/ops/agent/jobs/{job_id}/complete`
请求体最小字段:
```json
{
"node_code": "mainland-worker-02",
"client_request_id": "complete-ops-123-1",
"status": "success",
"stdout": "",
"stderr": "",
"result": {
"summary": "ok"
},
"error_message": ""
}
```
允许状态:
- `success`
- `failed`
- `partially_succeeded`
幂等约束:
- 使用 `client_request_id`
- 控制面写入 `ops_jobs.last_agent_complete_request_id`
- 同一完成回执可安全重放,不应再次制造副作用
推荐补充字段:
- `duration_ms`
- `result.summary_text`
- `result.focus_ref`
当前成功返回里也会额外带:
- `completion_summary.job_id`
- `completion_summary.job_code`
- `completion_summary.status`
- `completion_summary.status_label`
- `completion_summary.result_summary_text`
- `completion_summary.focus_ref`
- `completion_summary.deduplicated`
这样控制面在不展开 stdout/stderr 的情况下,也能直接把:
- 任务收口摘要
- 后续跳转落点
并入 activity / inspection / rollout 观察面。
### 6.6 Events
`POST /api/v1/ops/agent/jobs/{job_id}/events`
请求体最小字段:
```json
{
"node_code": "mainland-worker-02",
"client_event_id": "event-ops-123-checksum-1",
"step_id": 0,
"event_type": "deploy_checksum_verified",
"level": "info",
"message": "checksum ok",
"payload": {
"checksum": "sha256:..."
}
}
```
去重约束:
- 使用 `client_event_id`
- 控制面通过唯一索引去重
- 同一事件可安全重放,不应重复落库
推荐补充字段:
- `occurred_at`
- `summary_text`
- `focus_ref`
当前成功返回里也会额外带:
- `event`
- 即标准化后的事件对象
- 内含 `event_key / summary_text / occurred_at / focus_ref`
这样事件流就不再只是“原始日志点”,而能直接成为:
- playbook run events
- rollout events
- activity-stream
共同消费的现场片段。
---
## 7. Agent 本地补发队列
当前主链路已经正式具备:
- `pending` 队列
- `dead-letter` 队列
- 定时 `_flush_delivery_queue()`
- 永久性错误转死信
当前重点覆盖:
- `jobs/{id}/complete`
- `jobs/{id}/events`
队列状态:
- `healthy`
- `retrying`
- `dead_letter`
语义:
- `healthy`:无积压,无死信
- `retrying`存在待补发回执Agent 会继续自动重放
- `dead_letter`:存在永久失败记录,需人工介入
控制面消费方式:
- 节点维度:
- `delivery_queue_state`
- `delivery_queue_label`
- `delivery_queue_reason`
- `delivery_queue_pending_count`
- `delivery_queue_dead_letter_count`
- 汇总维度:
- `queue_retrying_nodes`
- `queue_dead_letter_nodes`
- `queue_pending_records`
- `queue_dead_letter_records`
### 7.1 当前控制面已可见的最小现场
当前至少已经可以通过:
- `GET /api/v1/ops/nodes`
看到每台托管节点的队列现场:
```json
{
"node_code": "mainland-worker-02",
"delivery_queue_state": "dead_letter",
"delivery_queue_label": "死信 2",
"delivery_queue_reason": "当前存在 2 条死信记录,建议优先查看节点日志或诊断编排。",
"delivery_queue_pending_count": 0,
"delivery_queue_dead_letter_count": 2,
"delivery_queue_last_flush_at": "2026-04-18 06:10:00",
"delivery_queue_oldest_pending_at": "",
"delivery_queue_oldest_pending_request_id": "",
"delivery_queue_oldest_pending_kind": "",
"delivery_queue_oldest_dead_letter_at": "2026-04-18 05:59:00",
"delivery_queue_oldest_dead_letter_request_id": "complete-ops-123-1",
"delivery_queue_oldest_dead_letter_kind": "job_complete"
}
```
这已经足够让:
- 页面显示“节点存在死信”
- `overview / driver-feed / codex-brief` 产出驾驶建议
- CLI 快速判断现场是不是要先走日志或诊断
但这还不够支撑正式运维,因为它只能“看见死信”,还不能“处理死信”。
### 7.2 Delivery Queue Record 正式对象
后续控制面不应再把死信理解成一个纯计数器,而要把每条待补发 / 死信记录升级成正式对象。
推荐最小结构:
```json
{
"record_id": "dq-mainland-worker-02-20260418-001",
"node_code": "mainland-worker-02",
"state": "dead_letter",
"request_kind": "job_complete",
"client_request_id": "complete-ops-123-1",
"job_id": 123,
"job_code": "ops-20260418-001",
"target_api": "/api/v1/ops/agent/jobs/123/complete",
"detail_code": "ops_job_invalid_status_transition",
"error_message": "当前任务状态不允许 complete",
"attempt_count": 6,
"first_queued_at": "2026-04-18 05:58:00",
"last_attempt_at": "2026-04-18 05:59:00",
"next_retry_at": "",
"payload_preview": {
"status": "success"
},
"response_preview": {
"code": 1,
"detail_code": "ops_job_invalid_status_transition"
}
}
```
正式约束:
- `record_id`
- 是控制面侧唯一主键
- `client_request_id`
- 是 Agent 幂等键
- `detail_code`
- 是死信聚类主键,不能只靠 `message`
- `payload_preview / response_preview`
- 用于页面和 Codex 快速判断,不必默认展开完整原始报文
推荐状态:
- `pending`
- `retrying`
- `dead_letter`
- `replayed`
- `discarded`
### 7.3 死信操作面正式入口
为了让“看见死信”之后不用回节点本地处理,推荐把操作面固定成下面这组接口。
当前已落地的第一阶段能力:
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue`
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue/records`
当前控制面返回的记录可见性为:
- `record_visibility=head_only`
也就是:
- 当前已经能稳定查看节点级队列总览
- 也能看到 `pending / dead_letter` 的头部记录摘要
- 但还没有把节点本地全量 `pending/*.json / dead-letter/*.json` 正式同步到控制面
所以这两条 GET 入口现在属于“正式只读观察面”,而不是完整操作面。
#### a. 单节点队列总览
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue`
最小返回体:
```json
{
"node_code": "mainland-worker-02",
"summary": {
"state": "dead_letter",
"pending_count": 0,
"dead_letter_count": 2,
"last_flush_at": "2026-04-18 06:10:00"
},
"oldest_pending_record": {},
"oldest_dead_letter_record": {}
}
```
#### b. 队列记录列表
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue/records`
推荐筛选参数:
- `state=pending|retrying|dead_letter|replayed|discarded`
- `request_kind=job_complete|job_event`
- `detail_code=...`
- `group_by=detail_code|request_kind|target_api`
- `limit=50`
当带 `group_by` 时,返回值应优先给出聚类摘要,而不是只返回平铺明细。
#### c. 单条死信重放
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/replay`
当前已落地,但要明确当前阶段约束:
- 控制面仍然是 `record_visibility=head_only`
- 所以单条动作目前只允许针对“当前可见头部记录”
- 真正的执行不是控制面直接改远端文件,而是创建正式 `ops job`
- 由目标节点 `Node Agent` 执行 `delivery.queue.replay`
最小请求体:
```json
{
"requested_by": "web-ui",
"reason": "确认任务状态已修复,重放这条 complete 回执"
}
```
#### d. 批量重放
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/replay`
当前已落地为正式操作面。
推荐请求体:
```json
{
"requested_by": "codex",
"selector": {
"state": "dead_letter",
"detail_code": "ops_job_invalid_status_transition"
},
"limit": 20
}
```
#### e. 丢弃死信
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/discard`
当前已落地,但与单条重放一样,仍然只允许针对当前可见头部死信记录发起单条动作。
最小请求体:
```json
{
"discarded_by": "web-ui",
"reason": "确认这条回执不再需要补发"
}
```
#### f. 主动冲刷队列
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/flush`
当前已落地为正式操作面,执行方式同样是:
- 控制面创建 `ops job`
- 目标节点 `Node Agent` 执行 `delivery.queue.flush`
语义:
- 不是“强行成功”
- 而是立即触发一次 Agent 补发冲刷
- 结果仍然回到 `pending / retrying / dead_letter / replayed`
### 7.4 为什么必须做成正式操作面
这层不是为了让页面多一个按钮,而是为了保证:
- 页面可以处理死信
- CLI 可以处理死信
- 海外 Codex 驾驶员可以处理死信
三者都不需要绕回:
- 手工 SSH 登录节点
- 手工删本地文件
- 手工重放某条 HTTP 请求
正式规则应该固定成:
- “死信查看” 走控制面对象
- “死信重放 / 丢弃 / 冲刷” 走控制面对象
- Agent 只负责执行,不负责决定如何处理死信
---
## 8. Job Status Enum
当前 Agent 主链路约束:
- `queued`
- `dispatching`
- `running`
- `success`
- `failed`
- `partially_succeeded`
语义:
- `queued`:任务已创建,尚未派发
- `dispatching`:控制面已派发,但节点尚未确认开工
- `running`:节点已真实执行
- `success`:成功完成
- `failed`:失败完成
- `partially_succeeded`:部分成功,需关注
---
## 9. 实现边界
Agent 只执行已声明动作:
- `service.start`
- `service.stop`
- `service.restart`
- `service.status`
- `health.snapshot`
- `logs.collect`
- `diagnostics.collect`
- `deploy.release`
不允许控制面长期依赖任意 shell 下发。
`shell executor` 只能作为受限兜底能力存在,并必须挂审批与审计。

View File

@@ -0,0 +1,56 @@
# ops_doctor_decision_contract
## Purpose
冻结 `doctor-decision` 主决策 contract让页面、CLI、Codex 都消费同一份“当前先看哪一面、先执行什么动作、为什么”的正式结论。
## Producer
- backend service `app.services.ops_service.get_ops_doctor_decision`
- API `GET /api/v1/ops/doctor-decision`
## Primary Consumers
- `domain-web` Ops Center 顶部总检入口与主决策详情面板
- `domain-api/deploy/multi-region/drive_ops_center.sh doctor-decision`
- 海外 Codex 驾驶员 / 自动驾驶策略层
## Required Payload
- `status`
- `status_label`
- `headline`
- `detail`
- `decision`
- `summary`
- `recommended_commands`
- `contract_navigation`
## Decision Shape
- `decision.status`
- `decision.reason_code`
- `decision.headline`
- `decision.detail`
- `decision.preferred_surface`
- `decision.next_action_code`
- `decision.recommended_commands`
## Summary Shape
- `summary.required_failures`
- `summary.optional_unavailable`
- `summary.scene_log_reports_total`
- `summary.scene_log_reports_ok`
- `summary.scene_log_status_counts`
- `summary.scene_log_problem_node_codes`
- `summary.contract_surface_gaps`
- `summary.launchpad_recommended_target_node_code`
- `summary.launchpad_recommended_recovery_label`
- `summary.launchpad_recommended_recovery_summary`
## Notes
- 优先读取最新 `go-live bundle` 中的 `doctor_decision` 产物。
- 如果 bundle 尚未导出,后端必须回落到 live `stack-diagnosis + go-live-summary` 兜底,而不是直接返回空。
- `preferred_surface` 必须稳定可消费,不能把面板跳转逻辑继续留给前端猜测。

View File

@@ -0,0 +1,797 @@
# domainCheck Ops Driver Contract
## 1. 目标
这份文档用于冻结 Overseas Ops Center / Codex Driver / CLI 共用的驾驶 contract。
核心原则:
- 后端负责推荐和解释
- 前端负责展示和触发
- Codex 负责选择入口和上下文
- 是否允许自动执行由后端统一门禁
- 页面与 Codex 都不能各自再推导一套优先级
---
## 2. 入口接口
当前驾驶 contract 主要来自:
- `GET /api/v1/ops/overview`
- `GET /api/v1/ops/driver-feed`
- `GET /api/v1/ops/codex-brief`
- `POST /api/v1/ops/driver-actions/preview`
- `POST /api/v1/ops/driver-actions/resolve`
- `POST /api/v1/ops/driver-actions/execute`
- `POST /api/v1/ops/driver-actions/execute-resolved`
- `POST /api/v1/ops/codex-actions/resolve`
- `POST /api/v1/ops/codex-actions/execute`
补充配套:
- `GET /api/v1/ops/playbooks`
- `GET /api/v1/ops/playbook-runs`
- `GET /api/v1/ops/playbook-runs/{run_code}`
- `GET /api/v1/ops/playbook-runs/{run_code}/events`
- `GET /api/v1/ops/activity-stream`
---
## 3. Driver Recommendation
来源:
- `overview.driver_recommendations`
最小字段:
```json
{
"key": "node-agent-dead-letter",
"priority": "先处理 Agent 死信",
"title": "先处理 Agent 死信",
"summary": "当前已有节点出现死信记录,继续堆动作会放大控制面与节点现场的不一致。",
"reason": "Node Agent 回执队列已经出现死信。",
"level_label": "最高优先",
"tag_type": "danger",
"primary_label": "查看 Worker 日志",
"secondary_label": "打开标准巡检",
"primary_action_code": "open_worker_logs",
"secondary_action_code": "open_diagnostics",
"node_codes": [
"mainland-worker-01"
],
"primary_node_codes": [
"mainland-worker-01"
],
"secondary_node_codes": [
"mainland-worker-01"
],
"primary_action_payload": {},
"secondary_action_payload": {},
"focus_ref": {
"kind": "ops_job",
"job_id": 101,
"job_code": "ops-20260418064500-a1b2c3"
}
}
```
要求:
- 推荐不只给文案
- 必须给动作码、节点和预填 payload
- 主次动作允许节点和 payload 不同
- 推荐对象允许直接携带 `focus_ref / primary_focus_ref / secondary_focus_ref`
- 新客户端优先用 `focus_ref` 做统一定位,而不是再从动作码反推页面落点
---
## 4. Priority Recommendation
来源:
- `overview.recommendation`
作用:
- 给首页顶部、CLI 摘要、Codex 首屏判断提供“当前第一优先动作”
最小字段:
```json
{
"source": "driver_recommendations",
"key": "node-agent-dead-letter",
"priority": "先处理 Agent 死信",
"title": "先处理 Agent 死信",
"summary": "当前已有节点出现死信记录。",
"reason": "Node Agent 回执队列已经出现死信。",
"primary_action_code": "open_worker_logs",
"secondary_action_code": "open_diagnostics"
}
```
---
## 5. Driver Feed
来源:
- `GET /api/v1/ops/driver-feed`
作用:
- 提供比 `overview` 更直接给驾驶员消费的聚合结果
建议最小结构:
```json
{
"top_recommendation": {},
"driver_recommendations": [],
"runbook_sequences": [],
"activity_focus": [],
"automation_coverage": {}
}
```
推荐补充约束:
- `top_recommendation`
- 应该总是来自同一份 `entries` 列表,而不是额外拼一条“影子建议”
- 这样页面、CLI、海外 Codex 在按 `key` 选中时才不会出现口径漂移
- `driver_recommendations`
- 普通驾驶建议,优先 `executor_kind=driver_action`
- `runbook_sequences`
- 标准作业路径,优先 `executor_kind=runbook_sequence`
- `activity_focus`
- 更偏观察与跟踪,不应伪装成“可立即执行”的驾驶动作
- `activity_focus[*]`
- 至少应保留:
- `activity_key`
- `activity_kind`
- `summary`
- `status`
- `status_label`
- `occurred_at`
- `focus_ref`
- `source_focus_ref`
- `ui_intent`
- 如果存在可落点观察入口
- 允许补 `focus_action_code=focus_activity_item`
- 但它的定位仍然是观察入口,而不是标准执行动作
- `entries[*].focus_ref`
- 是 driver-feed 的统一定位对象
- 页面 / CLI / Codex 均应优先消费
- `entries[*].primary_focus_ref` / `entries[*].secondary_focus_ref`
- 是主次动作各自的稳定落点
- 不允许调用方再根据 `primary_action_code` 自己猜“该跳 Release Hub、Launchpad 还是 Rollout Jobs”
- `automation_coverage`
- 是 Driver Feed 的自动化收口摘要
- 用于统一回答:
- 当前是否已经达到上线闸门
- 当前还有多少动作停留在 preview-only
- 当前后端是否已经足够接管首屏默认动作
- 页面、CLI、Codex 不允许自行再统计一份“自动化覆盖率”
推荐最小结构:
```json
{
"launch_ready": false,
"launch_status": "attention",
"summary": "仍有 3 个驾驶动作停留在 preview-only暂未达到上线收口标准。",
"total_actions": 12,
"backend_handled_total": 9,
"preview_only_total": 3,
"ui_only_total": 0,
"blocked_total": 0
}
```
其中 `runbook_sequences``activity_focus` 应继续区分:
- 这是建议动作
- 还是固定标准作业路径
- 还是最近值得优先处理的活动
推荐 CLI 消费顺序:
1. 显式 `entry_key`
2. `top_recommendation`
3. `entries[0]`
这样海外单入口就能稳定实现:
- `driver-focus-preview`
- `driver-focus-run`
---
## 6. Codex Brief
来源:
- `GET /api/v1/ops/codex-brief`
作用:
- 不让 Codex 自己从页面文案里猜“能不能自动执行”
最小字段:
```json
{
"mode": "guarded_auto",
"recommended_behavior": "confirm_then_execute",
"summary": "当前建议先处理 Agent 死信,再进入标准巡检。",
"automation_coverage": {
"launch_ready": false,
"launch_status": "attention",
"summary": "仍有 3 个驾驶动作停留在 preview-only。"
},
"target": {
"kind": "driver_action",
"action_code": "open_worker_logs",
"node_codes": [
"mainland-worker-01"
],
"action_payload": {}
}
}
```
推荐 `mode`
- `safe_auto`
- `guarded_auto`
- `mixed`
- `ui_only`
- `blocked`
推荐 `recommended_behavior`
- `auto_execute`
- `confirm_then_execute`
- `resolve_first`
- `open_ui`
- `blocked`
补充约束:
- `codex-brief.automation_coverage`
- 应与 `driver-feed.automation_coverage` 保持同源
- 供海外 Codex 驾驶员直接判断:
- 是否允许默认走自动执行
- 是否必须先预览或人工确认
- 是否已经达到“可以上线”的自动化收口标准
- `codex-brief.activity_focus`
- 应直接透传 `driver-feed.activity_focus`
- 供 Codex 在“当前没有可执行主线,但有观察焦点”时继续判断
- `codex-brief.focus`
- 优先使用 `entries[0]`
-`entries` 为空且 `activity_focus` 不为空时
- 允许回退到首条带 `focus_action_code``activity_focus`
- 这时推荐行为应偏向 `open_ui`
- 不应伪装成自动执行动作
---
## 7. Driver Action Execute
来源:
- `POST /api/v1/ops/driver-actions/execute`
作用:
- 把能标准化的驾驶动作收进统一后端执行入口
最小请求体:
```json
{
"action_code": "run_standard_inspection",
"node_codes": [
"mainland-worker-01"
],
"action_payload": {}
}
```
最小返回体:
```json
{
"handled": true,
"mode": "playbook",
"ui_intent": {
"kind": "playbook_run_detail",
"run_code": "run-20260418-001"
}
}
```
要求:
- 不只返回 handled / unhandled
- 还要返回 `mode`
- 还要返回 `ui_intent`
---
## 8. Driver Action Preview
来源:
- `POST /api/v1/ops/driver-actions/preview`
作用:
- 给 OpsCenter / CLI / Codex Driver 统一生成驾驶动作契约预览
- 冻结 `execution_chain / target_api / request_payload / 主次动作 section`
- 不再让前端自己拼请求体和执行建议
最小请求体:
```json
{
"source_label": "driver-feed",
"title": "版本发布与放量",
"summary": "当前可以进入正式发布路径。",
"reason": "统一回到 Release / Rollout 闭环。",
"executor_kind": "runbook_sequence",
"sequence_key": "release_progression",
"primary_label": "Worker 灰度",
"secondary_label": "查看版本区",
"secondary_action_code": "focus_release_hub",
"requested_by": "web-ui/ops-center"
}
```
最小返回体:
```json
{
"contract_key": "ops_driver_contract",
"contract_version": "v1",
"contract_schema_doc_path": "docs/schemas/ops_driver_contract.md",
"execution_chain": "ops-driver -> runbook-sequence -> resolved driver action / ui-intent",
"focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad"
},
"primary_focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad",
"mode": "worker"
},
"secondary_focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_detail"
},
"primary": {
"label": "Worker 灰度",
"action_code": "create_release_rollout_worker",
"target_api": "/api/v1/ops/runbook/sequences/release_progression/execute",
"target_method": "POST",
"recommendation_label": "建议确认后执行",
"automation_level_label": "guarded_auto",
"risk_level_label": "high",
"focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad",
"mode": "worker"
},
"request_payload": {
"secondary": false,
"requested_by": "web-ui/ops-center",
"node_codes": [
"mainland-controller-01"
],
"action_payload": {
"release_id": 7,
"focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad",
"mode": "worker"
}
}
}
}
}
```
要求:
- preview 必须和 execute 走同一套 action 解析与 runbook resolve 逻辑
- `primary` / `secondary` section 必须可直接给页面抽屉和 CLI 文本渲染复用
- `request_payload` 必须是真正可发送到执行接口的 body而不是页面内部临时对象
- runbook sequence preview 必须经过后端 resolve不能由页面自己猜 action_code
- preview 返回体必须保留:
- `focus_ref`
- `primary_focus_ref`
- `secondary_focus_ref`
- `primary.focus_ref`
- `secondary.focus_ref`
---
## 8.3. Driver Action Resolve / Execute Resolved
来源:
- `POST /api/v1/ops/driver-actions/resolve`
- `POST /api/v1/ops/driver-actions/execute-resolved`
作用:
-`overview.driver_recommendations` / `driver-feed` / 页面按钮 / CLI 对普通驾驶动作的门禁判断统一下沉到后端
- 页面不再根据 `preview` 自己推导 `直接执行 / 等确认 / 进入 UI / 阻断`
- 后端统一输出:
- `preview_request`
- `preview`
- `gate`
- `selected_section`
- `execution_result`
`resolve` 最小请求体:
```json
{
"source_label": "driver-feed",
"title": "开启关键日志回传",
"executor_kind": "driver_action",
"primary_action_code": "enable_log_sync_key",
"primary_action_payload": {},
"requested_by": "web-ui/driver-feed",
"secondary": false,
"confirm": false
}
```
`resolve` 最小返回体:
```json
{
"preview_request": {},
"preview": {},
"selected_section": "primary",
"focus_ref": {
"kind": "execution_scene",
"scene_key": "scene.logs"
},
"gate": {
"decision": "execute_now",
"decision_label": "直接执行",
"decision_reason": "当前动作属于 safe_auto可直接执行。",
"will_execute": true,
"recommendation": "auto_execute",
"focus_ref": {
"kind": "execution_scene",
"scene_key": "scene.logs"
},
"action_code": "enable_log_sync_key",
"target_api": "/api/v1/ops/driver-actions/execute"
}
}
```
`execute-resolved` 约束:
- 只有 `gate.will_execute=true` 才允许真正执行
- `confirmation_required` 只有显式传入 `confirm=true` 才会放行
- `open_ui / resolve_first / blocked` 必须返回非执行态,由页面或 CLI 继续展示契约
- `driver-focus-preview / driver-focus-run`
- 允许先从 `driver-feed.top_recommendation` 选出目标项
- 也允许显式传入 `activity_focus.key`
- 当目标来自 `activity_focus`
- 应自动收口成 `focus_activity_item`
- 只允许进入对应工作区,不应被当成标准执行动作
- 再把它收口成同一份 `resolve / execute-resolved` 请求体
- 不允许 CLI 再自己二次推导 gate
- resolve 返回体必须保留:
- `preview_request.focus_ref`
- `preview_request.primary_focus_ref`
- `preview_request.secondary_focus_ref`
- `gate.focus_ref`
- 顶层 `focus_ref`
---
## 8.5. Codex Action Resolve / Execute
来源:
- `POST /api/v1/ops/codex-actions/resolve`
- `POST /api/v1/ops/codex-actions/execute`
作用:
- 不再让 CLI / Codex 自己从 `codex-brief + preview` 拼一套门禁
- 由后端统一输出:
- 当前选中的 `selected_entry`
- 对应的 `preview`
- 统一门禁结论 `gate`
- 最终是否允许执行
`resolve` 最小请求体:
```json
{
"entry_key": "runbook:release_progression",
"requested_by": "cli/ops-center/codex-focus-preview",
"source_label": "cli/codex-focus-preview",
"include_secondary": true,
"confirm": false
}
```
`resolve` 最小返回体:
```json
{
"entry_key": "runbook:release_progression",
"selected_entry": {},
"preview_request": {},
"preview": {},
"focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad",
"mode": "worker"
},
"gate": {
"decision": "confirmation_required",
"decision_label": "等待确认",
"recommendation": "confirm_then_execute",
"recommendation_label": "建议确认后执行",
"will_execute": false,
"confirm_required": true,
"focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad",
"mode": "worker"
},
"target_api": "/api/v1/ops/runbook/sequences/release_progression/execute",
"target_method": "POST",
"request_payload": {
"secondary": false,
"requested_by": "cli/ops-center/codex-focus-preview",
"action_payload": {
"release_id": 7
}
}
}
}
```
`execute` 要求:
- 必须复用 `resolve` 的同一套选中逻辑和门禁逻辑
- `safe_auto`
- 允许直接执行
- `guarded_auto`
- 只有 `confirm=true` 时允许执行
- `resolve_first / open_ui / blocked`
- 一律不得执行
- 返回体必须保留:
- `selected_entry`
- `preview_request`
- `preview`
- `gate`
- `focus_ref`
- `execution_result`
- `executed`
---
## 9. UI Intent
`ui_intent` 的职责不是让前端猜去哪,而是由后端直接声明:
- 应打开哪个详情
- 应聚焦哪轮 run
- 应落到哪个发布入口
当前建议固定支持:
- `playbook_run_detail`
- `playbook_run_latest_events`
- `job_events`
- `rollout_jobs`
- `managed_node_handover`
- `managed_node_edit`
- `open_release_dialog`
- `open_rollout_dialog`
- `focus_release_hub`
- `focus_ref`
页面只做路由和抽屉承载,不做推荐解释。
---
## 10. Runbook Sequence
作用:
- 表达固定标准作业路径在“此刻”的推荐下一步
最小字段:
```json
{
"key": "standard_inspection",
"title": "标准巡检",
"status": "warning",
"status_label": "待补接管",
"summary": "当前还没有满足标准巡检条件的节点。",
"target_node_codes": [],
"focus_ref": {
"kind": "release_hub",
"release_id": 7,
"section": "release_launchpad"
},
"primary_label": "直接执行标准巡检",
"primary_action_code": "run_standard_inspection",
"primary_action_payload": {},
"primary_focus_ref": {},
"secondary_label": "打开巡检编排",
"secondary_action_code": "open_playbook_dialog",
"secondary_action_payload": {
"playbook_key": "inspection.standard"
},
"secondary_focus_ref": {}
}
```
要求:
- Runbook Sequence 是标准路径对象
- 不是普通活动记录
- 不是普通推荐卡
- `focus_ref`
- 表示这条标准路径整体对应的统一观察落点
- `primary_focus_ref` / `secondary_focus_ref`
- 表示主次动作各自应回看的稳定区域
---
## 11. Activity Stream
来源:
- `GET /api/v1/ops/activity-stream`
作用:
- 把最近异常、编排、任务、Rollout 统一压成可钻取活动流
每条 activity 最小字段:
```json
{
"kind": "playbook_run",
"activity_key": "run-20260418-001",
"status": "failed",
"occurred_at": "2026-04-18 06:00:00",
"summary": "标准巡检在 diagnostics.collect 步骤失败。",
"ui_intent": {
"kind": "playbook_run_detail",
"run_code": "run-20260418-001"
}
}
```
推荐 `kind`
- `playbook_run`
- `ops_job`
- `rollout`
- `runbook_sequence`
---
## 12. 单脑驾驶舱首屏约束
Ops Center 首屏不是自由拼装页面,而是 Driver Contract 的正式消费者。
首屏必须优先消费同一组 contract
- `overview.recommendation`
- `driver-feed.top_recommendation`
- `driver-feed.automation_coverage`
- `codex-brief`
首屏建议固定为三层:
1. 驾驶结论
2. 驾驶动作条
3. 驾驶回执条
正式要求:
- 驾驶结论
- 直接展示当前第一优先动作、自动化收口状态、是否达到上线闸门
- 驾驶动作条
- 只放“默认下一步 / 定位下一步 / 看契约 / 复制命令 / 进入主处理面板”这类高频动作
- 这些按钮不能自己发明新规则,必须消费已有 `action_code / focus_ref / preview`
- 驾驶回执条
- 是首屏动作后的统一反馈区
- 用于展示:
- 已打开哪个契约
- 已定位哪个焦点
- 已复制哪个命令
- 后端动作执行成功 / 警告 / 失败
- 页面不应再把动作结果散落在多个局部 toast 里
也就是说:
- 首屏负责决策与触发
- 下方详情区负责展开与下钻
- 不允许详情区反向重写首屏推荐
---
## 13. 建议优先级顺序
当前 contract 应优先遵守:
1. 先看异常编排
2. 再看异常活动
3. 再处理 Agent 死信 / 回执积压
4. 再优先执行首个缺口节点的接入收口或接管验收
5. 再处理现场日志回传和标准巡检
6. 最后进入 Release / Rollout
这条顺序必须由后端维护,不应让前端或 Codex 再自行拼装。
补充约束:
- 当后端已经把首个缺口节点收敛成 `bootstrap_run / run_acceptance` 时:
- `driver_feed.top_recommendation`
- `codex_brief.focus`
- `stack_diagnosis.next_step`
都必须优先指向这一个节点和这一个动作
- `fix_managed_nodes` 只能保留为兜底入口,不能再覆盖已经明确收敛的单节点恢复动作
---
## 14. 页面 / Codex / CLI 共用约束
同一条驾驶建议必须由后端一次性给出:
- 推荐标题
- 推荐理由
- 主动作
- 次动作
- 节点范围
- 预填参数
- UI 落点
- 是否允许自动执行
这样三类入口才会真正共享一套驾驶 contract
- OpsCenter 页面
- 海外 Codex 驾驶员
- CLI / 运维检查脚本
并且三类入口对 `recommended_behavior` 的执行语义也必须一致:
- `auto_execute`
- 可直接执行
- `confirm_then_execute`
- 必须显式确认后才能执行
- `resolve_first`
- 只能先审阅和补上下文,不能直接执行
- `open_ui`
- 只能进入对应工作区,不能直接执行
- `blocked`
- 明确阻断,不能执行

View File

@@ -0,0 +1,59 @@
# ops_go_live_bundle_contract
## 目标
把发布前证据包与 `manifest.json` 复核结论压成一份统一 contract
- Ops Center 交付面板
- CLI `go-live-export / go-live-review`
- 海外 Codex 驾驶员
共同消费。
它解决的问题不是“现在能不能上线”,而是:
> 现在有没有一份足够完整、足够可信、可交付可复盘的上线证据包
## 主接口
- `GET /api/v1/ops/go-live-bundle`
## 核心字段
- `status`
- `missing`
- `ready`
- `attention`
- `blocked`
- `status_label`
- `headline`
- `bundle_available`
- `report_dir`
- `manifest_path`
- `artifact_total`
- `failure_total`
- `critical_failures`
- `noncritical_failures`
- `env_audit`
- `summary`
- `recommended_commands`
## 判定原则
- 如果当前还没有 bundle / manifest
- `status = missing`
- 如果 manifest 明确标记 `review_status_hint=blocked`
- `status = blocked`
- 如果 manifest 已存在,但:
- 仍有非关键失败
- 环境审计提示 attention
- launchpad 摘要存在漂移
-`status = attention`
- 只有当 manifest 可读、review 为 ready、关键摘要一致
- 才允许 `status = ready`
## 设计约束
- 页面不直接触发 shell 导出 bundle
- 后端默认只读最新 bundle / manifest不负责替代 CLI 执行导出
- 当 bundle 缺失时,必须明确给出固定命令,而不是让操作者自己猜路径

View File

@@ -0,0 +1,37 @@
# ops_go_live_review_contract
## Purpose
冻结 `go-live-review` 复核结论 contract让页面、CLI、Codex 都能消费同一份 bundle manifest 复核结果,而不是各自再解读一遍 artifact。
## Producer
- backend service `app.services.ops_service.get_ops_go_live_review`
- API `GET /api/v1/ops/go-live-review`
- CLI `bash domain-api/deploy/multi-region/drive_ops_center.sh go-live-review`
## Primary Consumers
- `domain-web` Ops Center 证据复核面板
- 海外 Codex 驾驶员
- 发布前人工值班复核
## Required Payload
- `status`
- `status_label`
- `headline`
- `artifact_total`
- `failure_total`
- `critical_failures`
- `noncritical_failures`
- `env_audit`
- `launchpad_alignment`
- `recommended_next_steps`
- `recommended_commands`
## Notes
- 这层是 manifest 的正式“复核解释层”,不是单纯重复 bundle 基础信息。
- 如果 `launchpad_alignment.consistent=false`,即使关键文件都存在,也必须至少回到 `attention`
- 页面和 Codex 应优先消费这层 `headline / recommended_next_steps`,而不是自行拼 review 口径。

View File

@@ -0,0 +1,65 @@
# ops_go_live_signoff_contract
## 目标
把发布前最终签收判断压成一份统一 contract
- Ops Center 首屏
- CLI `go-live-signoff`
- 海外 Codex 驾驶员
共同消费。
它不是替代:
- `go-live-summary`
- `go-live-review`
- `doctor-decision`
- `stack-diagnosis`
- `driver-feed`
- `codex-brief`
而是在这些 contract 之上给出最终一句:
> 现在能不能签字上线
## 主接口
- `GET /api/v1/ops/go-live-signoff`
## 核心字段
- `signoff_status`
- `ready`
- `attention`
- `blocked`
- `status_label`
- `headline`
- `release_gate`
- `blocked_reasons`
- `attention_reasons`
- `decision`
- `launchpad_alignment`
- `recommended_commands`
- `report_dir`
- `manifest_path`
## 判定原则
- 只要 `go_live_summary / go_live_review / doctor_decision / publish_status / stack_status / automation / release_launchpad` 中任一明确阻断
- `signoff_status = blocked`
- 如果没有阻断,但存在:
- `go-live-review=attention|missing`
- `doctor-decision=attention|missing`
- `attention`
- launchpad 摘要漂移
- 自动化仍未完全进入 ready
-`signoff_status = attention`
- 只有当收口、正式复核、主决策、发布、自动化、launchpad 摘要一致
- 才允许 `signoff_status = ready`
## 设计约束
- 页面不允许再自己拼一套“看起来能上”
- CLI 不允许绕过这个 contract 单独下结论
- Codex 驾驶员优先看这一份,再决定是否继续下钻

View File

@@ -0,0 +1,740 @@
# domainCheck Ops Job Contract
## 1. 目标
这份文档用于冻结 `ops job` 的正式 contract。
它是整套海外单脑控制面里最核心的执行对象。
正式定义:
> `ops job` 是一条可审计、可编排、可派发、可回执、可取消的标准运维任务记录。
它不是:
- 一条 shell 命令
- 一个页面按钮点击结果
- 一个仅供 agent 消费的内部对象
它应该同时服务于:
- 页面按钮
- CLI
- 海外 Codex 驾驶员
- Node Agent
- Release / Rollout
- Playbook Run
也就是说:
- `playbook run`
- 是编排聚合对象
- `rollout`
- 是发布推进对象
- `ops job`
- 是真正的执行颗粒度对象
---
## 2. 入口接口
当前已落地的主入口:
- `GET /api/v1/ops/jobs`
- `GET /api/v1/ops/jobs/{job_id}`
- `GET /api/v1/ops/jobs/{job_id}/events`
- `POST /api/v1/ops/jobs`
- `POST /api/v1/ops/jobs/batch`
- `POST /api/v1/ops/policy/preview`
- `POST /api/v1/ops/jobs/{job_id}/approve`
- `POST /api/v1/ops/jobs/{job_id}/cancel`
- `POST /api/v1/ops/jobs/{job_id}/dispatch`
配套对象来源:
- `ops_jobs`
- `ops_job_steps`
- `ops_job_events`
正式约束:
- 页面、CLI、Codex 不能各自定义第 2 套任务模型
- `playbook run` 只能聚合 `ops job`
- `rollout` 只能批量生成 `ops job`
- `Node Agent` 只能拉取和回执 `ops job`
---
## 3. 当前实现里的 Job Object
来源:
- `GET /api/v1/ops/jobs`
- `GET /api/v1/ops/jobs/{job_id}`
- 创建 / 审批 / 取消 / 派发后的返回体中的 `job`
最小字段:
```json
{
"id": 101,
"job_code": "ops-20260418064500-a1b2c3",
"action": "logs.collect",
"target_type": "node",
"target_node_code": "mainland-worker-01",
"target_node_codes": [
"mainland-worker-01"
],
"status": "queued",
"status_label": "排队中",
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"requested_by": "web-ui/ops-center",
"payload": {},
"metadata": {},
"result": {},
"summary": "目标节点 mainland-worker-01 / 发起 web-ui/ops-center / 方式 Node Agent",
"summary_text": "目标节点 mainland-worker-01 / 发起 web-ui/ops-center / 方式 Node Agent",
"error_message": "",
"created_at": "2026-04-18 06:45:00",
"started_at": "",
"finished_at": "",
"updated_at": "2026-04-18 06:45:00",
"risk_level": "medium",
"approval_required": false,
"approval_status": "approved",
"approval_status_label": "已审批",
"approved_by": "",
"approved_at": "",
"blocked_reason": "",
"cancellation_reason": "",
"dispatched_at": "",
"target_selector": {},
"policy": {},
"rollout_id": 0,
"steps_total": 0,
"steps_running": 0,
"steps_success": 0,
"steps_failed": 0,
"steps_terminal": 0,
"steps_loaded": false,
"focus_ref": {
"kind": "ops_job",
"job_id": 101,
"job_code": "ops-20260418064500-a1b2c3",
"action": "logs.collect",
"target_node_code": "mainland-worker-01"
},
"steps": []
}
```
正式要求:
- `job_code`
- 是跨页面、CLI、事件流的稳定任务标识
- `action`
- 是执行语义主键,不应用 `title` 或自然语言代替
- `target_type`
- 定义任务作用范围,不应只靠是否传了 `target_node_code` 来猜
- `execution_mode`
- 是正式执行方式,不是附属备注
- `policy`
- 是创建时冻结的策略快照,不应由页面二次推导
- `status_label / approval_status_label`
- 是给页面、CLI、Codex 直接消费的人类可读标签
- `summary_text`
- 是任务摘要 contract不要求调用方再用状态和节点自己拼一句中文
- `steps_total / steps_running / steps_success / steps_failed / steps_terminal`
- 是任务步骤聚合统计,避免前端和 CLI 再自行遍历 steps 统计
- `focus_ref`
- 是统一定位对象后续页面、CLI、Codex 应优先消费它,而不是自己重组跳转参数
---
## 4. Job 状态机
### 4.1 当前实现中已使用的状态
当前后端已实际使用:
- `queued`
- `awaiting_approval`
- `blocked`
- `running`
- `success`
- `failed`
- `cancelled`
这些状态已经是正式 contract 的最小闭环。
### 4.2 推荐终局扩展状态
终局可继续扩展为:
- `draft`
- `approved`
- `queued`
- `dispatching`
- `running`
- `verifying`
- `partially_succeeded`
- `rollback_running`
- `rolled_back`
- `success`
- `failed`
- `cancelled`
- `timed_out`
但要明确区分:
- 当前实际 API 已稳定承诺的,是最小状态集
- 终局扩展状态不能破坏当前状态语义
---
## 5. Execution Mode Contract
当前实现里正式出现的执行方式:
- `remote-agent`
- `local-runtime`
- `control-plane`
- `ssh`
语义必须固定:
### 5.1 `remote-agent`
- 默认正式执行方式
- 由目标节点 Node Agent 拉取并执行
### 5.2 `local-runtime`
- 仅用于当前节点本机即时执行
- 主要用于本机 runtime 动作
### 5.3 `control-plane`
- 由海外控制面直接执行
- 适用于控制面内生动作
### 5.4 `ssh`
- 只作为救援和过渡执行器
- 不能成为大规模正式发布的默认方式
正式约束:
- 发布默认优先 `remote-agent`
- `ssh` 只应是兜底链路
- 页面、CLI、Codex 不应再自行创造第 5 种 execution mode
---
## 6. Target Type Contract
当前 contract 至少应允许:
- `node`
- `batch`
- `nodes`
- `rollout`
语义建议固定为:
### 6.1 `node`
- 单节点任务
### 6.2 `batch` / `nodes`
- 多节点批量任务
- 当前通常通过 `/ops/jobs/batch` 创建
### 6.3 `rollout`
- 该任务属于某轮 release rollout 推进的一部分
正式要求:
- 如果任务来自 rollout必须能通过 `rollout_id` 回溯
- 如果任务来自 playbook应通过 `metadata` 标出 run 归属
---
## 7. Job Step Contract
来源:
- `GET /api/v1/ops/jobs/{job_id}`
最小字段:
```json
{
"id": 201,
"step_key": "dispatch",
"title": "调度动作",
"node_code": "mainland-worker-01",
"status": "queued",
"stdout": "",
"stderr": "",
"result": {},
"created_at": "2026-04-18 06:45:00",
"started_at": "",
"finished_at": "",
"updated_at": "2026-04-18 06:45:00"
}
```
当前实现说明:
- 现阶段每条 `ops job` 至少会先生成一个 `dispatch` step
- 以后如果引入真正 DAG / 多步骤执行,也应继续沿用 `ops_job_steps`
正式约束:
- `step_key`
- 是步骤主键,不应用标题代替
- `status`
- 必须与 job 状态联动,但不要求完全相同
- `stdout / stderr`
- 是步骤级输出,不应与整条 job 的 `result` 混成一层
---
## 8. Job Event Contract
来源:
- `GET /api/v1/ops/jobs/{job_id}/events`
最小字段建议:
```json
{
"id": 301,
"event_key": "job-event:301",
"job_id": 101,
"step_id": 201,
"node_code": "mainland-worker-01",
"client_event_id": "evt-001",
"event_type": "job_dispatch_requested",
"level": "info",
"level_label": "信息",
"message": "任务等待 node-agent 拉取: 101",
"summary": "任务等待 node-agent 拉取: 101",
"payload": {},
"has_payload": false,
"occurred_at": "2026-04-18 06:45:03",
"created_at": "2026-04-18 06:45:03"
}
```
当前代码里事件来源包括:
- job created
- job blocked
- job awaiting approval
- job approved
- job cancelled
- job dispatch requested
- job executed locally / over ssh / on control plane
- agent start / complete / custom events
正式要求:
- `events` 是任务时间线,不是调试附属品
- 页面、CLI、Codex 都应从这里读“任务做到哪一步了”
- 不应重新从 stdout/stderr 推断关键状态
补充约束:
- `event_key`
- 是事件行的稳定前端 / CLI 标识
- `summary`
- 是给页面、CLI、Codex 直接消费的短摘要,不要求调用方再拼接 `message`
- `level_label`
- 是面向人看的等级标签
- `occurred_at`
- 是统一观察面时间字段,允许与 `created_at` 同值
### 8.1 Event List Summary Contract
当前 `GET /api/v1/ops/jobs/{job_id}/events` 建议同时返回:
```json
{
"summary": {
"job_id": 101,
"returned_total": 12,
"level_counts": {
"info": 8,
"warning": 3,
"error": 1
},
"event_type_counts": {
"job_created": 1,
"job_dispatch_requested": 1,
"executor_received": 1
},
"node_counts": {
"mainland-worker-01": 12
},
"latest_at": "2026-04-18 06:45:08",
"filters": {
"limit": 50
}
}
}
```
正式要求:
- 事件接口不只给一串数组
- 至少要给:
- 当前返回了多少条
- 等级分布
- 事件类型分布
- 最近时间
- 这样页面、CLI、Codex 才能先判断“有没有异常事件”再决定是否展开明细
---
## 9. List Contract 与 Compact 模式
来源:
- `GET /api/v1/ops/jobs`
当前实现特点:
- 默认 `list_ops_jobs(..., compact=True)`
- 返回 compact job
- `steps` 被置空
- `payload / metadata / result` 会被压缩
因此建议把列表 contract 明确成:
```json
{
"jobs": [
{
"id": 101,
"job_code": "ops-20260418064500-a1b2c3",
"action": "logs.collect",
"status": "queued",
"execution_mode": "remote-agent",
"requested_by": "web-ui/ops-center",
"payload": {
"tail_lines": 120
},
"metadata": {},
"result": {},
"result_summary": {},
"is_compact": true,
"steps": []
}
]
}
```
正式要求:
- 列表页默认只承诺 compact 视图
- 详情页再承诺 full job + steps
- 页面不应假设列表结果天然带完整 steps
---
## 10. Create Contract
来源:
- `POST /api/v1/ops/jobs`
最小请求体:
```json
{
"action": "logs.collect",
"target_type": "node",
"target_node_code": "mainland-worker-01",
"execution_mode": "remote-agent",
"requested_by": "web-ui/ops-center",
"payload": {
"tail_lines": 120
},
"metadata": {
"source": "ops-center"
},
"auto_approve": false,
"run_now": true
}
```
当前实现还支持:
- `target_selector`
- `rollout_id`
最小返回体:
```json
{
"job": {},
"executed_immediately": false
}
```
关键语义:
- `executed_immediately=true`
- 说明任务已被控制面即时执行,而不是停留在队列
- `executed_immediately=false`
- 说明任务进入了:
- `queued`
- `awaiting_approval`
- `blocked`
正式要求:
- 创建接口不能只回答“建成功了”
- 必须明确回答:
- 当前状态是什么
- 是否立即执行
- 如果没执行,是在等审批、等 agent、还是被阻断
---
## 11. Batch Create Contract
来源:
- `POST /api/v1/ops/jobs/batch`
最小请求体建议:
```json
{
"template_key": "diagnostics.collect",
"target_node_codes": [
"mainland-worker-01",
"mainland-worker-02"
],
"execution_mode": "remote-agent",
"requested_by": "web-ui/ops-center",
"payload": {},
"metadata": {},
"auto_approve": true
}
```
最小返回体建议:
```json
{
"jobs": [],
"results": []
}
```
正式要求:
- batch 只是创建方式,不是新的执行对象
- batch 最终仍要展开成多条 `ops job`
- 单条失败不能让结果完全丢失,必须保留逐项结果
---
## 12. Policy Preview Contract
来源:
- `POST /api/v1/ops/policy/preview`
- `ops_jobs.policy`
这层是创建前的正式门禁对象。
当前创建任务时,后端已先做:
- 风险等级判断
- 阻断判断
- 审批要求判断
建议最小返回字段:
```json
{
"blocked": false,
"blocking_reasons": [],
"approval_required": true,
"approval_reasons": [
"当前目标包含 control 节点"
],
"warnings": [],
"recommendations": [],
"risk_level": "high",
"target_summary": {},
"batch_plan": {},
"cluster_guardrails": {},
"operational_readiness": {}
}
```
正式要求:
- 页面、CLI、Codex 都应复用同一份 policy preview
- 不能各自再写一套“能不能执行”的判断逻辑
---
## 13. Approval / Cancel / Dispatch Contract
### 13.1 Approve
来源:
- `POST /api/v1/ops/jobs/{job_id}/approve`
最小请求体:
```json
{
"approved_by": "admin"
}
```
语义:
-`awaiting_approval` 可审批
- 审批后回到 `queued`
### 13.2 Cancel
来源:
- `POST /api/v1/ops/jobs/{job_id}/cancel`
最小请求体:
```json
{
"cancelled_by": "admin",
"reason": "本轮灰度暂停"
}
```
语义:
- `success / failed / cancelled` 不可取消
- 取消后 job 进入 `cancelled`
- steps 同步进入 `cancelled` 或保持终态
### 13.3 Dispatch
来源:
- `POST /api/v1/ops/jobs/{job_id}/dispatch`
语义:
- `queued` 才可派发
- `control-plane`
- 立即由控制面执行
- `local-runtime`
- 当前节点本机即时执行
- `ssh`
- 由 SSH 执行器执行
- `remote-agent`
- 保持 `queued`,等待对应 agent 拉取
这点必须明确:
> dispatch 不等于一定会立刻变成 running。
对于 `remote-agent`,它可能只是:
- 写入事件
- 保持排队
- 等节点稍后 pull
---
## 14. 与 Playbook / Rollout / Agent 的边界
### 14.1 与 Playbook
- `playbook run`
- 负责展开、聚合、重跑、取消整轮编排
- `ops job`
- 负责单条执行记录
### 14.2 与 Rollout
- `rollout`
- 负责发布批次推进
- `ops job`
- 负责每个节点每步的实际任务
### 14.3 与 Node Agent
- 控制面创建 `ops job`
- agent 拉取 `ops job`
- agent 回写 `ops_job_events`
- agent 推进 job / step 状态
### 14.4 与 Observability
- `ops_observability_contract`
- 看现场事实
- `ops_job_contract`
- 看执行动作本身
这两层不能混:
- 观察面告诉你“谁在跑、谁异常”
- 任务面告诉你“这一条任务如何被创建、审批、派发、执行、取消”
---
## 15. 终局约束
后续不应再回退到下面几种模式:
### 1. 页面点按钮直接跑 shell
错误。
正确方式:
- 页面创建 `ops job`
### 2. Codex 直接 SSH 改现场
错误。
正确方式:
- Codex 创建 `ops job` / `playbook run`
### 3. 发布系统不经过任务层
错误。
正确方式:
- rollout 批量生成 `ops job`
### 4. 死信治理不留痕
错误。
正确方式:
- `flush / replay / discard` 都生成正式 `ops job`
所以可以把这份文档看成整套系统的“执行订单母本”。
后续只要这份 contract 稳住页面、CLI、Codex、Agent、Release、Playbook 就都能围绕同一个中心对象协作。

View File

@@ -0,0 +1,702 @@
# domainCheck Ops Observability Contract
## 1. 目标
这份文档用于冻结海外单脑控制面里“看现场”这一层的正式 contract。
它不负责:
- 发布决策
- Driver 推荐
- Playbook 编排
它负责回答 3 类问题:
1. 哪些节点现在真正处于执行现场
2. 这些节点最近巡检收口结果是什么
3. Node Agent 回执队列现在是否健康
也就是说,这份 contract 的职责是:
> 让页面、CLI、Codex 看到同一份现场事实,而不是各自从日志和状态字串里猜
---
## 2. 入口接口
建议把下面这些接口视为同一观察面的一组正式入口:
- `GET /api/v1/ops/overview`
- `GET /api/v1/ops/inspection-overview`
- `GET /api/v1/ops/activity-stream`
- `GET /api/v1/ops/nodes/{node_code}/handover`
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue`
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue/records`
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/flush`
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/replay`
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/replay`
- `POST /api/v1/ops/nodes/{node_code}/delivery-queue/records/{record_id}/discard`
配套关系:
- `overview`
- 首页收口
- `inspection-overview`
- 节点巡检收口
- `activity-stream`
- 最近异常活动
- `handover`
- 单节点接管阻断与下一步建议
- `delivery-queue`
- Node Agent 回执健康
正式要求:
- 页面不能自己再定义第 5 套“观察口径”
- CLI 不应重新从数据库拼现场
- Codex 不应通过自然语言日志推断核心状态
---
## 3. Execution Scene Contract
来源:
- `GET /api/v1/ops/overview`
- `overview.execution_scene`
最小字段:
```json
{
"summary": "当前存在 2 台有效执行节点,其中 1 台正在领任务1 台在线待命。",
"dispatch_active_nodes": [
{
"node_code": "mainland-worker-01",
"region": "mainland",
"role": "worker",
"status": "busy",
"participation_state": "dispatch_active",
"participation_label": "正在执行",
"participation_reason": "当前存在已领任务和执行中工作负载",
"current_load": 9,
"items_claimed": 9,
"items_running": 5,
"items_completed_recent": 14
}
],
"recent_only_nodes": [],
"standby_nodes": [
{
"node_code": "mainland-controller-01",
"region": "mainland",
"role": "control",
"status": "online",
"participation_state": "standby",
"participation_label": "在线待命",
"participation_reason": "节点在线、有效,但当前没有领任务、执行中任务或近窗吞吐",
"current_load": 0
}
],
"load_syncing_nodes": [],
"counts": {
"dispatch_active": 1,
"recent_only": 0,
"standby": 1,
"load_syncing": 0
}
}
```
分组语义必须固定:
- `dispatch_active_nodes`
- 当前正在领任务、执行任务、或仍有实时工作负载
- `recent_only_nodes`
- 当前不再运行,但近窗刚有吞吐
- `standby_nodes`
- 在线、有效,但当前不参与
- `load_syncing_nodes`
- 节点与控制面状态尚未完全同步
正式约束:
- “在线”不等于“参与检测”
- “有效执行节点”不等于“正在跑”
- 页面展示、CLI 摘要、Codex 判断必须沿用这 4 类分组
节点参与态附加字段也应固定:
- `participation_state`
- 机器判断主键,推荐值:
- `dispatch_active`
- `recent_only`
- `standby`
- `load_syncing`
- `participation_label`
- 直接给页面、CLI、Codex 展示的人类可读标签
- `participation_reason`
- 用一句稳定摘要解释为什么节点落在当前分组
正式要求:
- 页面必须直接区分:
- 在线但未参与
- 正在执行或领任务
- 近窗刚参与但当前已收口
- 新消费方不允许再从:
- `status`
- `current_load`
- `items_running`
手工推断“这个节点到底算不算正在参与”
---
## 4. Log Sync Runtime Contract
来源:
- `overview.execution_scene.log_sync`
-`overview.log_sync`
最小字段:
```json
{
"status": "partial_coverage",
"status_label": "部分覆盖",
"mode": "key",
"mode_label": "关键回传",
"enabled": true,
"source_nodes": [
"mainland-worker-01"
],
"source_node_summaries": [
{
"node_code": "mainland-worker-01",
"line_count": 42,
"key_line_count": 42,
"full_line_count": 0,
"last_at": "2026-04-18 06:40:00",
"last_line": "域名检测完成: demo.com"
}
],
"covered_participating_node_summaries": [
{
"node_code": "mainland-worker-01",
"region": "mainland",
"role": "worker",
"participation_state": "dispatch_active",
"participation_label": "正在执行",
"participation_reason": "当前存在已领任务和执行中工作负载",
"line_count": 42,
"key_line_count": 42,
"full_line_count": 0,
"last_at": "2026-04-18 06:40:00",
"last_line": "域名检测完成: demo.com"
}
],
"missing_participating_node_summaries": [
{
"node_code": "mainland-worker-02",
"region": "mainland",
"role": "worker",
"participation_state": "dispatch_active",
"participation_label": "正在执行",
"participation_reason": "当前存在已领任务和执行中工作负载",
"missing_reason": "no_sample"
}
],
"covered_participating_nodes": 1,
"participating_nodes_total": 2,
"missing_sample_node_codes": [
"mainland-worker-02"
],
"last_line_at": "2026-04-18 06:40:00",
"summary": "当前已覆盖 1/2 台参与节点,建议继续观察未回传样本节点。"
}
```
推荐状态:
- `disabled`
- `waiting_sample`
- `partial_coverage`
- `healthy`
- `full_capture`
这层是“现场可见性”最关键的辅助状态。
正式要求:
- 如果存在参与节点但日志回传还没开,优先推荐开关键回传
- 如果日志回传已开但没有样本,优先推荐抓 Worker 日志
- 页面和 Codex 都不能再自己定义另一套日志覆盖结论
节点级日志覆盖字段也应固定:
- `source_nodes`
- 为兼容旧页面保留的节点编码列表
- `source_node_summaries`
- 当前已有日志样本的节点级统计对象
- `covered_participating_node_summaries`
- 已参与且已拿到样本的节点对象
- `missing_participating_node_summaries`
- 已参与但尚未拿到样本的节点对象
`source_node_summaries[]` 至少应返回:
- `node_code`
- `line_count`
- `key_line_count`
- `full_line_count`
- `last_at`
- `last_line`
`missing_participating_node_summaries[]` 至少应返回:
- `node_code`
- `region`
- `role`
- `participation_state`
- `participation_label`
- `participation_reason`
- `missing_reason`
推荐 `missing_reason`
- `no_sample`
- `delayed_flush`
- `filtered_out`
兼容要求:
- 旧页面仍可继续消费:
- `source_nodes`
- `missing_sample_node_codes`
- 新页面、CLI、Codex 应优先消费:
- `source_node_summaries`
- `covered_participating_node_summaries`
- `missing_participating_node_summaries`
补充节点级现场入口:
- `GET /api/v1/ops/nodes/{node_code}/scene-log`
最小字段:
```json
{
"node_code": "mainland-worker-01",
"status": "healthy",
"status_label": "关键覆盖",
"mode": "key",
"mode_label": "关键回传",
"log_sync_enabled": true,
"summary": "节点当前正在参与检测,已保留 42 条现场日志样本。",
"node": {
"region": "mainland",
"role": "worker",
"status": "busy",
"current_load": 6,
"last_heartbeat_at": "2026-04-18 06:40:00"
},
"participation": {
"detect_participating": true,
"participation_state": "claimed",
"participation_label": "已领待跑",
"participation_bucket": "dispatch_active",
"participation_reason": "已领取任务,等待线程继续执行。"
},
"source_summary": {
"line_count": 42,
"key_line_count": 42,
"full_line_count": 0,
"last_at": "2026-04-18 06:40:00",
"last_line": "[2026-04-18 06:40:00] [mainland-worker-01] 域名检测完成: demo.com"
},
"records_total": 42,
"records_visible": 80,
"records": [
{
"created_at": "2026-04-18 06:38:00",
"node_code": "mainland-worker-01",
"message": "开始检测域名: demo.com",
"mode": "key",
"line": "[2026-04-18 06:38:00] [mainland-worker-01] 开始检测域名: demo.com"
}
]
}
```
节点级入口的正式语义:
- 这是“看某一台节点当前现场”的正式入口,不允许页面再自己从聚合样本中二次猜测
- `status` 至少覆盖:
- `healthy`
- `full_capture`
- `historical_sample`
- `waiting_sample`
- `missing_sample`
- `disabled`
- `standby`
- `unknown`
- 当节点当前在线但未参与时,必须明确返回 `standby`
- 当节点正在参与但还没有样本时,必须明确返回 `waiting_sample`
- 当节点已有历史样本但当前不在参与面中时,必须明确返回 `historical_sample`
---
## 5. Inspection Overview Contract
来源:
- `GET /api/v1/ops/inspection-overview`
- `overview.inspection`
最小字段:
```json
{
"status": "attention",
"status_label": "待处理",
"summary": "存在 1 台节点最近缺少 Worker 日志收口。",
"counts": {
"healthy_nodes": 1,
"attention_nodes": 1,
"problem_nodes": 0
},
"rows": [
{
"node_code": "mainland-worker-01",
"problem_kind": "missing_worker_logs",
"problem_label": "缺少 Worker 日志收口",
"problem_level": "warning",
"recommended_action_code": "open_worker_logs",
"ui_intent": {
"kind": "node_inspection_detail",
"node_code": "mainland-worker-01"
},
"latest_health_snapshot": {},
"latest_worker_logs": {},
"latest_diagnostics": {}
}
]
}
```
设计要求:
- 巡检不是几条离散 job 记录
- 必须按节点重新聚合
- 必须直接给出:
- `problem_kind`
- `problem_label`
- `problem_level`
- `recommended_action_code`
- `ui_intent`
这样页面、CLI、Codex 才能真正消费“节点巡检结论”,而不是再从几十条 job 记录里回推问题。
---
## 6. Inspection Row Contract
`rows[]` 中每个节点建议最少包含:
- `node_code`
- `region`
- `role`
- `effective_worker`
- `problem_kind`
- `problem_label`
- `problem_level`
- `recommended_action_code`
- `ui_intent`
- `latest_health_snapshot`
- `latest_worker_logs`
- `latest_diagnostics`
其中 3 类最近结果建议统一结构:
```json
{
"status": "success",
"status_label": "成功",
"occurred_at": "2026-04-18 06:35:00",
"job_code": "ops-20260418-001",
"summary": "最近一次 Worker 日志收集成功,共采样 120 行。"
}
```
正式边界:
- `logs.collect` 默认指 Worker 日志收口
- Node Agent 辅助排障日志不应混成同一种巡检结果
---
## 7. Delivery Queue Summary Contract
来源:
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue`
最小字段:
```json
{
"node": {
"node_code": "mainland-worker-01",
"title": "mainland-worker-01"
},
"summary": {
"state": "dead_letter",
"label": "死信",
"reason": "当前有 2 条记录进入死信。",
"pending_count": 3,
"dead_letter_count": 2,
"oldest_pending_at": "2026-04-18 06:20:00",
"oldest_dead_letter_at": "2026-04-18 06:18:00"
},
"record_visibility": "head_only",
"capabilities": {
"can_flush": true,
"can_replay": true,
"can_discard": true
}
}
```
推荐状态:
- `healthy`
- `retrying`
- `dead_letter`
- `stalled`
正式约束:
- 控制面要明确告诉操作者当前是 `head_only` 还是 `full_projection`
- 不允许页面假装能列出远端完整目录
- 不允许因为能看见死信就绕过 `ops job` 直接改文件
---
## 8. Delivery Queue Record Contract
来源:
- `GET /api/v1/ops/nodes/{node_code}/delivery-queue/records`
当前阶段建议最小字段:
```json
{
"records": [
{
"record_id": 101,
"state": "dead_letter",
"state_label": "死信",
"request_id": "req-20260418-001",
"kind": "job_complete",
"created_at": "2026-04-18 06:18:00",
"updated_at": "2026-04-18 06:19:00",
"summary": "某次任务完成回执多次补发失败。",
"visibility": "head_only"
}
],
"summary": {
"returned_records": 1,
"record_visibility": "head_only"
}
}
```
当前阶段要求:
- 单条 `replay / discard` 只针对控制面可见记录
- 当前 `head_only` 不等于未来不能扩展
- 以后若进入 `full_projection`,扩的是可见范围,不是重写操作模型
---
## 9. Delivery Queue Action Contract
来源:
- `flush`
- `replay`
- `record replay`
- `record discard`
正式要求:
- 所有动作都必须落成正式 `ops job`
- 返回体要能告诉调用方:
- 是否已受理
- 生成了什么 job
- 建议跳转到哪里观察
最小返回体建议:
```json
{
"accepted": true,
"job_code": "ops-20260418-queue-001",
"ui_intent": {
"kind": "job_events",
"job_code": "ops-20260418-queue-001"
}
}
```
这里的边界必须比“临时脚本”更严格:
- `discard` 必须带原因
- `replay` 应支持限流 / limit
- `flush` 应支持显式 `requested_by`
---
## 10. Activity Stream Contract
来源:
- `GET /api/v1/ops/activity-stream`
最小字段建议:
```json
{
"items": [
{
"kind": "ops_job",
"activity_key": "ops-job:101",
"title": "logs.collect",
"subtitle": "ops-20260418064500-a1b2c3",
"summary": "目标节点 mainland-worker-01 / 发起 web-ui / 方式 Node Agent",
"summary_text": "目标节点 mainland-worker-01 / 发起 web-ui / 方式 Node Agent",
"status": "running",
"status_label": "执行中",
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"target_node_codes": [
"mainland-worker-01"
],
"occurred_at": "2026-04-18 06:45:08",
"ui_intent_kind": "job_events",
"focus_ref": {
"kind": "ops_job",
"job_id": 101,
"job_code": "ops-20260418064500-a1b2c3",
"action": "logs.collect",
"target_node_code": "mainland-worker-01"
}
}
],
"summary": {
"status": "running",
"status_label": "执行中",
"summary_text": "最近活动里存在正在推进的现场、任务、编排或回传。",
"filtered_total": 6,
"unfiltered_total": 18,
"latest_at": "2026-04-18 06:45:08"
}
}
```
正式要求:
- `activity-stream` 不是页面内部 table 数据
- 它必须是:
- 页面
- CLI
- Codex 驾驶员
- 自动驾驶编排器
共同消费的“最近活动摘要”
字段语义需要固定:
- `summary_text`
- 是直接消费的活动短摘要
- `status_label`
- 是稳定的人类可读状态标签
- `target_node_codes`
- 是活动关联的节点口径,不允许调用方再从自然语言里猜
- `ui_intent_kind`
- 是当前活动默认落点类型
- `focus_ref`
- 是跨页面 / CLI / Codex 的统一定位对象
正式边界:
- 活动流应保留老字段以兼容现有页面
- 新消费方优先用:
- `summary_text`
- `status_label`
- `focus_ref`
- 不允许新客户端再从 `title + subtitle + summary` 手工解析定位参数
## 11. 与 Driver / Playbook / Release 的边界关系
这份 contract 只负责观察面和现场治理,不负责替代其他 contract。
边界应明确为:
- `ops_driver_contract`
- 决定“现在优先做什么”
- `ops_playbook_contract`
- 决定“标准多步动作怎么编排”
- `release_hub_contract`
- 决定“版本怎么发、怎么停、怎么回滚”
- `ops_observability_contract`
- 决定“现场事实是什么、当前缺口在哪里”
这样后续系统再扩展时,不会又回到“所有东西都写在 overview 一坨大 json 里”的状态。
---
## 12. 终局验收标准
如果这份 contract 真正落地,海外单脑控制面应该做到:
### 1. 不登录大陆机器也能回答现场问题
例如:
- 谁在跑
- 谁在线但没跑
- 谁近窗刚跑过
- 谁日志没回来
- 谁队列死信了
### 2. 不复制大日志也能先做判断
先看:
- execution scene
- inspection overview
- activity stream
- delivery queue
再决定是否需要全量日志或 diagnostics bundle。
### 3. 页面、CLI、Codex 看到的是同一现场
不能出现:
- 页面说节点正在执行
- CLI 说节点待命
- Codex 说节点离线
### 4. 所有高风险动作都有正式观察落点
也就是:
- replay 后能看 job
- discard 后能看审计
- inspection 后能看节点收口
- rollout 后能看 launchpad / gate / rollout jobs
这样这套系统才能真正从“调试台”升级成“驾驶舱”。

View File

@@ -0,0 +1,563 @@
# domainCheck Ops Playbook Contract
## 1. 目标
这份文档用于冻结 `playbook catalog / preview / run / events` 的正式 contract。
这层的定位不是“页面里的一个弹窗”,而是:
- 控制面的正式编排对象
- 海外 Codex 驾驶员可消费的标准作业对象
- Driver / Runbook / Release / Inspection 之间的统一中间层
核心原则:
- 一次标准编排 = 一次 `playbook run`
- 页面、CLI、Codex 必须共用同一份 `playbook` / `playbook run` 结构
- `playbook run` 只负责创建与聚合正式 `ops job`
- 不允许前端再把“标准巡检 / 接管验收 / 现场日志”退回成零散按钮逻辑
建议与 [ops_job_contract.md](/www/wwwroot/getDomain/docs/schemas/ops_job_contract.md:1) 配套阅读。
---
## 2. 当前接口面
当前已落地的主入口:
- `GET /api/v1/ops/playbooks`
- `POST /api/v1/ops/playbooks/preview`
- `POST /api/v1/ops/playbooks/execute`
- `GET /api/v1/ops/playbook-runs`
- `GET /api/v1/ops/playbook-runs/{run_code}`
- `GET /api/v1/ops/playbook-runs/{run_code}/events`
- `POST /api/v1/ops/playbook-runs/{run_code}/rerun`
- `POST /api/v1/ops/playbook-runs/{run_code}/cancel`
并且这层已经被以下入口复用:
- `driver action -> open_playbook_dialog`
- `runbook sequence -> secondary_resolution`
- `inspection`
- `activity-stream`
所以后续不应再新造第四套“标准作业对象”。
---
## 3. Playbook Catalog Object
来源:
- `GET /api/v1/ops/playbooks`
- `preview / execute` 返回体中的 `playbook`
最小字段:
```json
{
"key": "inspection.standard",
"group_key": "diagnostics",
"group_title": "标准巡检",
"title": "标准巡检",
"description": "按 健康快照 -> Worker 日志 -> 诊断包 的顺序,对目标节点做一次标准联调巡检。",
"target_scope": "node-set",
"default_execution_mode": "remote-agent",
"default_execution_mode_label": "Node Agent",
"execution_modes": [
"remote-agent",
"ssh"
],
"execution_mode_options": [
{
"value": "remote-agent",
"label": "Node Agent",
"is_default": true
},
{
"value": "ssh",
"label": "SSH",
"is_default": false
}
],
"default_auto_approve": true,
"stop_on_failure": true,
"steps": [
{
"step_key": "health",
"title": "采集健康快照",
"template_key": "health.snapshot",
"payload": {}
}
]
}
```
正式约束:
- `key`
- 是 Playbook 的稳定主键
- `group_key`
- 用于页面、CLI、Codex 做标准分组,不应用 `title`
- `execution_modes`
- 是能力白名单,不是建议文案
- `stop_on_failure`
- 是编排级行为,不是某一步骤的临时说明
---
## 4. Playbook Preview Contract
来源:
- `POST /api/v1/ops/playbooks/preview`
最小请求体:
```json
{
"playbook_key": "inspection.standard",
"node_codes": [
"mainland-worker-01"
],
"execution_mode": "remote-agent",
"auto_approve": true,
"requested_by": "web-ui"
}
```
最小返回体:
```json
{
"playbook": {},
"requested_by": "web-ui",
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"auto_approve": true,
"target_nodes_total": 1,
"target_node_codes": [
"mainland-worker-01"
],
"step_count": 3,
"expected_jobs_total": 3,
"template_keys": [
"health.snapshot",
"logs.collect",
"diagnostics.collect"
],
"steps": [
{
"order": 1,
"step_key": "health",
"title": "采集健康快照",
"template_key": "health.snapshot",
"action": "health.snapshot",
"payload": {},
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"auto_approve": true,
"target_nodes_total": 1,
"expected_jobs": 1
}
]
}
```
预览阶段必须回答:
- 最终执行方式是什么
- 总共会展开多少步
- 每步会展开成什么模板
- 总共会生成多少个 `ops job`
这样页面、CLI、Codex 才能在执行前看见“这次会做什么”。
---
## 5. Playbook Run Object
来源:
- `GET /api/v1/ops/playbook-runs`
- `GET /api/v1/ops/playbook-runs/{run_code}`
- `POST /api/v1/ops/playbooks/execute`
最小字段:
```json
{
"run_code": "playbook-20260418-001",
"playbook_key": "inspection.standard",
"playbook_title": "标准巡检",
"group_key": "diagnostics",
"group_title": "标准巡检",
"requested_by": "web-ui",
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"stop_on_failure": true,
"step_count": 3,
"expected_jobs_total": 3,
"target_nodes_total": 1,
"target_node_codes": [
"mainland-worker-01"
],
"created_at": "2026-04-18 06:30:00",
"updated_at": "2026-04-18 06:31:10",
"status": "running",
"status_label": "收口中",
"status_counts": {
"queued": 1,
"running": 1,
"success": 1
},
"jobs_total": 3,
"success_jobs_total": 1,
"terminal_jobs_total": 1,
"completion_percent": 33.3,
"steps_total": 3,
"steps_success": 1,
"steps_running": 1,
"steps_problem": 0,
"steps_terminal": 1,
"focus_level": "warning",
"focus_step_key": "worker_logs",
"focus_step_title": "收集 Worker 日志",
"focus_summary": "收集 Worker 日志 当前仍在执行,先等待回执或检查对应节点事件流。",
"summary": "收集 Worker 日志 当前仍在执行,先等待回执或检查对应节点事件流。",
"summary_text": "收集 Worker 日志 当前仍在执行,先等待回执或检查对应节点事件流。",
"focus_ref": {
"kind": "playbook_run",
"run_code": "playbook-20260418-001",
"playbook_key": "inspection.standard",
"group_key": "diagnostics",
"focus_step_key": "worker_logs",
"focus_step_title": "收集 Worker 日志",
"event_key": "",
"node_code": ""
},
"problem_steps": [],
"active_steps": [],
"latest_job": {},
"steps": []
}
```
正式定义:
> `playbook run` 是一轮标准编排在控制面的聚合对象,不是某一条单独任务。
也就是说:
- `ops job` 是执行颗粒度
- `playbook run` 是观察、重跑、取消、钻取的编排颗粒度
---
## 6. Playbook Step Object
`playbook run.steps[]` 的最小结构:
```json
{
"step_key": "worker_logs",
"title": "收集 Worker 日志",
"template_key": "logs.collect",
"order": 2,
"jobs_total": 1,
"nodes_total": 1,
"target_node_codes": [
"mainland-worker-01"
],
"status_counts": {
"running": 1
},
"terminal_jobs_total": 0,
"success_jobs_total": 0,
"status": "running",
"status_label": "收口中",
"completion_percent": 0.0,
"summary": "收集 Worker 日志 当前处于执行中,可继续观察该步骤事件。",
"summary_text": "收集 Worker 日志 当前处于执行中,可继续观察该步骤事件。",
"focus_ref": {
"kind": "playbook_run",
"run_code": "playbook-20260418-001",
"playbook_key": "inspection.standard",
"group_key": "diagnostics",
"focus_step_key": "worker_logs",
"focus_step_title": "收集 Worker 日志",
"event_key": "",
"node_code": ""
},
"latest_job": {}
}
```
推荐 step 状态:
- `queued`
- `running`
- `success`
- `attention`
规则:
- 只要该步骤出现失败、阻断、取消等终态问题,应收口为 `attention`
- 页面与 Codex 都应优先盯 `problem_steps / active_steps`,而不是自己遍历所有子任务推断
---
## 7. Playbook Event Stream
来源:
- `GET /api/v1/ops/playbook-runs/{run_code}/events`
最小返回体:
```json
{
"run_code": "playbook-20260418-001",
"playbook_run": {},
"events": [
{
"job_id": 123,
"job_code": "ops-20260418-123",
"run_code": "playbook-20260418-001",
"job_status": "running",
"job_status_label": "收口中",
"action": "logs.collect",
"target_node_code": "mainland-worker-01",
"step_key": "worker_logs",
"step_title": "收集 Worker 日志",
"event_type": "job_running",
"level": "info",
"level_label": "信息",
"message": "开始收集 Worker 日志",
"summary": "开始收集 Worker 日志",
"summary_text": "开始收集 Worker 日志",
"payload": {},
"event_key": "job-event:1001",
"occurred_at": "2026-04-18 06:31:00",
"focus_ref": {
"kind": "playbook_run",
"run_code": "playbook-20260418-001",
"playbook_key": "inspection.standard",
"group_key": "diagnostics",
"focus_step_key": "worker_logs",
"focus_step_title": "收集 Worker 日志",
"event_key": "job-event:1001",
"node_code": "mainland-worker-01"
},
"created_at": "2026-04-18 06:31:00"
}
],
"summary": {
"total": 1,
"returned_total": 1,
"available_step_counts": {
"worker_logs": 1
},
"available_node_counts": {
"mainland-worker-01": 1
},
"level_counts": {
"info": 1
},
"event_type_counts": {
"job_running": 1
},
"job_status_counts": {
"running": 1
},
"latest_at": "2026-04-18 06:31:00",
"filters": {
"step_key": "",
"node_code": "",
"limit": 80
}
}
}
```
这层必须支持:
-`step_key` 过滤
-`node_code` 过滤
- 快速看到最近活动时间
否则 `playbook run` 只能看总览,不能真正钻取。
---
## 8. 正式 Playbook Key 语义
当前最值得冻结成正式对象的 key
### 8.1 `onboarding.bootstrap`
定位:
- 生成节点接入工单
执行方式:
-`control-plane`
正式语义:
- 不直接接管节点
- 只创建标准接入方案对象
### 8.2 `onboarding.acceptance`
定位:
- 新节点接管后标准验收
当前标准步骤:
1. `health.snapshot`
2. `service.status(node-agent)`
3. `logs.collect(node-agent)`
4. `service.status(worker)`
5. `logs.collect(worker)`
### 8.3 `inspection.standard`
定位:
- 标准巡检
当前标准步骤:
1. `health.snapshot`
2. `logs.collect(worker)`
3. `diagnostics.collect`
正式要求:
- 这是 release / rollout 之前最核心的健康观察面
### 8.4 `scene.logs.key`
定位:
- 关键现场日志样本
正式要求:
- 默认比 `scene.logs.full` 更轻
- 适合先看现场,再决定是否升级取证
### 8.5 `scene.logs.full`
定位:
- 全量现场取证
正式要求:
- 应明确比 `scene.logs.key` 更重
- 默认应同时补诊断包
### 8.6 `scene.diagnostics`
定位:
- 轻量诊断包
正式要求:
- 适合作为参与节点的常规取证动作
---
## 9. 执行模式 Contract
推荐模式:
- `remote-agent`
- `ssh`
- `control-plane`
正式约束:
- `remote-agent`
- 正式日常运维默认模式
- `ssh`
- 仅用于首发接管与应急救援
- `control-plane`
- 只适合控制面本地对象生成或极少量控制面动作
页面、CLI、Codex 不能自己发明第四种临时模式。
---
## 10. Playbook Run 状态机
推荐状态:
- `queued`
- `running`
- `success`
- `attention`
含义:
- `queued`
- 子任务刚创建,尚未出现有效执行推进
- `running`
- 已有子任务进入 `dispatching / running / awaiting_approval`
- `success`
- 全部子任务收口成功
- `attention`
- 任一步骤出现失败、阻断、取消或部分成功,需要人工关注
正式规则:
- 页面上的“标准巡检状态”
- `activity-stream` 中的 playbook run 摘要
- Driver / Codex 的推荐动作
都必须共用这一套状态语义。
---
## 11. 与 Driver / Runbook / Release 的关系
正式收口规则:
- Driver 不直接“模拟巡检”
- 应优先跳转或创建 `playbook run`
- Runbook Sequence 不直接展开 shell
- 应优先收口到 `driver action``playbook`
- Release / Rollout 不自己维护另一套巡检摘要
- 应优先消费 `inspection.standard` 的最近结果
也就是说:
- `scene` 是现场观察对象
- `inspection` 是标准健康对象
- `release / rollout` 是发布对象
三者不能再各自维护不同的“日志 / 诊断 / 巡检”定义。
---
## 12. 正式要求
后续继续演进时,必须保持:
1. `playbook catalog`
- 是稳定 contract而不是页面枚举
2. `playbook preview`
- 是执行前的唯一结构化预览面
3. `playbook run`
- 是页面、CLI、Codex 共同观察的一轮编排对象
4. `playbook events`
- 是整轮编排的钻取视角
5. 接管验收、标准巡检、现场观察
- 优先都收进 `playbook`
这样这套编排才能成为真正的正式平台能力,而不是一组“看起来像流程”的按钮。

View File

@@ -0,0 +1,372 @@
# domainCheck Ops Stack Diagnosis Contract
## 1. 目标
这份文档用于冻结海外单脑控制面的“总检入口” contract。
它的职责不是替代:
- `overview`
- `link-snapshot`
- `inspection-overview`
- `release launchpad`
- `activity-stream`
而是把这些正式 contract 再收口成一份:
> 第一现场诊断结果
让下面这些消费者都先看同一份判断:
- 海外 Ops Center 页面
- 海外 Codex 驾驶员
- CLI 总检脚本
- 后台按钮自动诊断
正式原则:
- 总检不能只回一段说明文字
- 总检必须给出结构化问题清单
- 总检必须给出默认下一步动作
- 总检必须给出可以直接执行的推荐命令
---
## 2. 入口接口
主入口:
- `GET /api/v1/ops/go-live-summary`
- `GET /api/v1/ops/stack-diagnosis`
配套来源:
- `GET /api/v1/ops/contracts`
- `GET /api/v1/ops/link-snapshot`
- `GET /api/v1/ops/overview`
- `GET /api/v1/ops/nodes`
- `GET /api/v1/ops/releases/launchpad`
- `GET /api/v1/ops/playbook-runs`
- `GET /api/v1/ops/activity-stream`
推荐 query
- `base_url`
用途:
- `go-live-summary`
- 给 Ops Center 顶部收口卡、CLI 总检、Codex 驾驶员直接消费
- 返回更偏“上线判断”的稳定摘要
- 用于生成 `next_step.command`
- 用于生成 `quick_commands`
- 让 CLI / Codex / 页面复制出来的命令口径一致
---
## 3. 顶层结构
建议最小结构:
```json
{
"generated_at": "2026-04-18 09:00:00",
"diagnosis": {},
"api": {},
"surface_matrix": {},
"contracts": {},
"link_snapshot": {},
"overview": {},
"managed_nodes": {},
"release_hub": {},
"playbook_runs": {},
"activity_stream": {}
}
```
正式要求:
- 顶层每一段都必须是稳定 key
- 页面、CLI、Codex 不允许自己再拼第 2 套总检结构
- `go-live-summary`
-`stack-diagnosis` 的稳定摘要层
- 必须至少给出:
- `go_live_status`
- `blocking_reasons`
- `warnings`
- `launchpad_recommended_target_node_code`
- `launchpad_recommended_recovery_label`
- `launchpad_recommended_recovery_summary`
- `launchpad_onboarding_bootstrap_pending_nodes`
- `launchpad_onboarding_acceptance_ready_nodes`
- `next_step_action_code`
- `operator_title`
- `recommended_commands`
- `playbook_runs.problem_runs[*]`
- 需要保留 `focus_ref`
- 这样 CLI / Codex / 页面才能直接定位到问题编排的正式焦点
- `activity_stream.top_items[*]`
- 需要保留:
- `occurred_at`
- `focus_ref`
- `source_focus_ref`
- `ui_intent_kind`
- 这样总检不会把“观察摘要”再次压扁成一段不可定位的纯文本
- `go-live-summary` + `stack-diagnosis` + `driver-feed.automation_coverage`
- 必须能共同驱动 Ops Center 首屏
- 不允许页面自己从局部接口重新推导一份“收口状态”
---
## 4. Diagnosis Contract
来源:
- `stack_diagnosis.diagnosis`
最小字段:
```json
{
"contract_key": "ops_stack_diagnosis_contract",
"contract_version": "v1",
"registry_version": "2026-04-18",
"base_url": "http://127.0.0.1:8100",
"surface_status": "partial",
"automation_status": "blocked",
"stack_status": "blocked",
"issue_total": 4,
"blocking_issue_total": 1,
"warning_issue_total": 3,
"missing_surfaces": [
"contracts"
],
"launchpad_recommended_target_node_code": "mainland-worker-01",
"launchpad_recommended_recovery_label": "待执行接入",
"launchpad_recommended_recovery_summary": "mainland-worker-01 还缺接入收口,先执行 bootstrap。",
"launchpad_onboarding_bootstrap_pending_nodes": 1,
"launchpad_onboarding_acceptance_ready_nodes": 0,
"next_step": {
"action_code": "bootstrap_run",
"source": "issue:managed_nodes_agent_pending",
"reason": "mainland-worker-01 还缺接入收口,先执行 bootstrap。",
"command": "bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 bootstrap_run",
"focus_ref": {
"kind": "managed_node",
"node_code": "mainland-worker-01"
}
},
"recommended_actions": [],
"issues": [],
"operator_hints": [],
"quick_commands": []
}
```
状态语义必须固定:
- `surface_status`
- `healthy`
- `partial`
- `broken`
- `automation_status`
- `ready`
- `attention`
- `blocked`
- `stack_status`
- 总结论,面向人和程序
- 推荐值为 `ready / attention / blocked`
正式要求:
- `next_step` 是默认下一步,而不是“可能动作之一”
- 当首个缺口节点已经明确收敛为 `bootstrap_run / run_acceptance` 时,`next_step.action_code` 不应再退回成泛化的 `fix_managed_nodes`
- 当存在 `runtime_build_schema_stale` 时,`next_step.action_code` 必须优先收敛为 `api-restart`
- `issues` 必须按可处理性组织,而不是只堆原始异常文本
- `quick_commands` 必须是可以直接复制执行的命令
- `next_step.focus_ref`
- 是首屏“定位下一步”的正式落点
- 页面、CLI、Codex 不允许再根据 `action_code` 反推要跳去哪个区域
- `diagnosis.launchpad_recommended_target_node_code`
- 代表 stack-diagnosis 已经把 Release Hub 收敛出的首个接管缺口直接下沉到总检摘要
- 页面、CLI、doctor-export、Codex 不应再分别回源二次推导
- `diagnosis.launchpad_onboarding_bootstrap_pending_nodes / launchpad_onboarding_acceptance_ready_nodes`
- 代表当前接管缺口的阶段性计数
- 用来帮助总检直接区分“待 bootstrap”与“待 acceptance”
- `go-live-summary.launchpad_recommended_target_node_code`
- 是 launchpad 已经收敛出的首个接管缺口节点
- 页面、CLI、Codex 应优先直接显示它,而不是重新扫描 gap rows
- `go-live-summary.launchpad_onboarding_bootstrap_pending_nodes / launchpad_onboarding_acceptance_ready_nodes`
- 用于区分当前是“待接入”还是“待验收”
- 这两个计数应作为上线收口阶段的正式 warning 来源,而不是页面本地再统计
---
## 5. Issue Contract
`diagnosis.issues[]` 最小字段:
```json
{
"code": "managed_nodes_agent_pending",
"severity": "blocked",
"layer": "managed_nodes",
"summary": "托管节点虽然已经纳入 Ops Center但 0 台 remote-agent 就绪,当前第一动作已收敛为首台节点接入收口。",
"detail": "managed_enabled=3remote_access_ready=0首个缺口节点 mainland-worker-01待执行接入。",
"action_code": "bootstrap_run",
"focus_ref": {
"kind": "managed_node",
"node_code": "mainland-worker-01"
},
"commands": [
"bash domain-api/deploy/multi-region/drive_ops_center.sh driver-resolve http://127.0.0.1:8100 bootstrap_run",
"bash domain-api/deploy/multi-region/drive_ops_center.sh node-bootstrap-plan http://127.0.0.1:8100 mainland-worker-01"
]
}
```
严重级别建议固定为:
- `blocked`
- `warning`
- `info`
正式要求:
- `summary` 用于列表展示
- `detail` 用于展开解释
- `commands` 用于终端直接执行
- 不允许只有中文文案,没有结构化定位信息
对于运行时版本漂移,还应允许出现这类 issue
```json
{
"code": "runtime_build_schema_stale",
"severity": "warning",
"layer": "runtime_build_info.schema",
"summary": "仓库已经具备新的节点接管能力,但运行中的 build-info 仍未声明对应路由键,当前更像 API 还没重启到最新代码。",
"detail": "supports_install_command_block=trueroute_surface_declares_bootstrap_plan=false",
"action_code": "api-restart",
"focus_ref": {
"kind": "runtime_build_info",
"section": "schema",
"expected_route_key": "ops_node_handover_bootstrap_plan"
},
"commands": [
"bash domain-api/deploy/multi-region/drive_ops_center.sh runtime-refresh-recover http://127.0.0.1:8100",
"bash domain-api/deploy/multi-region/drive_ops_center.sh stack-diagnosis http://127.0.0.1:8100 summary",
"bash domain-api/deploy/multi-region/drive_ops_center.sh node-bootstrap-plan http://127.0.0.1:8100 mainland-worker-01"
]
}
```
---
## 6. Surface Matrix Contract
来源:
- `stack_diagnosis.surface_matrix`
最小字段:
```json
{
"status_counts": {
"total": 8,
"available": 7,
"missing": 1
},
"items": [
{
"name": "contracts",
"available": false
}
]
}
```
---
## 7. Runtime Build Info Extension
`stack_diagnosis.runtime_build_info` 以及 `/api/v1/runtime/build-info` 建议至少包含:
```json
{
"repository_capabilities": {
"supports_install_command_block": true,
"supports_multi_layout_bootstrap": true
},
"route_surface_declares_bootstrap_plan": false,
"runtime_schema_stale": true
}
```
语义要求:
- `repository_capabilities`
- 表示当前仓库代码本身具备的能力
- `route_surface_declares_bootstrap_plan`
- 表示运行中的 API build-info 是否已经声明节点级 bootstrap-plan 路由键
- `runtime_schema_stale`
- 表示“仓库能力已到位,但运行中的 API 结构还停留在旧版”
- 这是典型的“服务没重启到最新代码”信号,不应误判成“功能没开发完”
作用:
- 让页面和 CLI 直观看出“缺的是哪一层”
- 避免只看到最终 blocked却不知道缺的是 contract、launchpad 还是 activity-stream
---
## 7. 与其他 Contract 的关系
`ops_stack_diagnosis_contract` 的定位是:
- 不替代 `ops_driver_contract`
- 不替代 `ops_observability_contract`
- 不替代 `release_hub_contract`
它负责:
- 把它们统一收口成第一现场诊断
对于 Ops Center 页面还必须补充一条:
- 首屏负责“结论 + 默认下一步 + 复制命令 + 动作回执”
- 下方详情区负责“issue / surface / launchpad / activity / runbook”的展开与下钻
- 不允许详情区覆盖首屏的默认下一步结论
所以正确消费顺序应该是:
1. 先看 `stack-diagnosis`
2. 再根据 `issues / next_step / focus_ref` 下钻到:
- `overview`
- `link-snapshot`
- `inspection-overview`
- `release launchpad`
- `activity-stream`
---
## 8. CLI / Codex / 页面统一约束
CLI
- `check_ops_center_stack.sh` 应优先调用 `GET /api/v1/ops/stack-diagnosis`
Codex
- 默认先读取 `stack-diagnosis`
- 再决定是继续 `driver-resolve``doctor` 还是细分检查
页面:
- 顶部总检卡片必须优先消费 `diagnosis`
- 不能重新从多个 endpoint 拼一份影子状态
只有这样,海外单脑控制面才能真正进入:
> 单入口观察,分层下钻,统一执行

View File

@@ -0,0 +1,609 @@
# domainCheck Release Hub Contract
## 1. 目标
这份文档用于冻结 Release / Rollout / Launchpad / Gate 的正式 contract。
核心原则:
- 先有 Release再有 Rollout
- Release 是不可变版本对象
- Rollout 是分批推进对象
- Gate 是是否可发的统一门禁对象
- Launchpad 是给页面与 Codex 消费的发布驾驶舱对象
建议与 [ops_job_contract.md](/www/wwwroot/getDomain/docs/schemas/ops_job_contract.md:1) 配套阅读,因为 Rollout 的真正执行颗粒度最终仍然是 `ops job`
---
## 2. Release Object
来源:
- `POST /api/v1/ops/releases`
- `GET /api/v1/ops/releases`
- `GET /api/v1/ops/releases/latest`
- `GET /api/v1/ops/releases/{id}`
- `POST /api/v1/ops/releases/{id}/activate`
最小字段:
```json
{
"id": 1,
"release_version": "2026.04.18+bd6bcb2",
"channel": "stable",
"commit_sha": "bd6bcb2",
"status": "ready",
"status_label": "已就绪",
"artifact_url": "https://release.example.com/domaincheck-2026.04.18.tgz",
"checksum": "sha256:...",
"notes": "",
"metadata": {},
"created_by": "www",
"created_at": "2026-04-18 05:30:00",
"activated_at": "2026-04-18 05:35:00",
"summary": "2026.04.18+bd6bcb2 已就绪,制品与校验信息齐备,可进入 Rollout。",
"summary_text": "2026.04.18+bd6bcb2 已就绪,制品与校验信息齐备,可进入 Rollout。",
"focus_ref": {
"kind": "release_hub",
"release_id": 1,
"release_version": "2026.04.18+bd6bcb2",
"channel": "stable",
"rollout_id": 0,
"rollout_code": "",
"section": "release_detail"
}
}
```
约束:
- 同一 `release_version` 只对应一份发布物
- Release 创建后不应重新指向另一份 artifact
- `artifact_url + checksum` 是发布可验证性的最小基线
推荐状态:
- `draft`
- `ready`
- `active`
- `archived`
---
## 3. Rollout Object
来源:
- `POST /api/v1/ops/releases/{id}/rollouts`
- `GET /api/v1/ops/rollouts/{id}`
- `GET /api/v1/ops/rollouts/{id}/jobs`
- `POST /api/v1/ops/rollouts/{id}/advance`
最小字段:
```json
{
"id": 11,
"release_id": 1,
"rollout_code": "rollout-20260418-001",
"target_selector": {
"region": "mainland",
"role": "worker"
},
"target_nodes": [
"mainland-worker-01",
"mainland-controller-01"
],
"policy": {
"batch_size": 1,
"require_approval_between_batches": true,
"rollback_on_failure": true
},
"status": "running",
"status_label": "执行中",
"batch_cursor": 1,
"batches_total": 2,
"jobs_total": 2,
"jobs_created": 1,
"result_summary": {},
"target_node_codes": [
"mainland-worker-01",
"mainland-controller-01"
],
"summary": "当前已覆盖 1/2 台节点,仍有 1 条任务执行中。",
"summary_text": "当前已覆盖 1/2 台节点,仍有 1 条任务执行中。",
"focus_ref": {
"kind": "release_rollout",
"rollout_id": 11,
"rollout_code": "rollout-20260418-001",
"release_id": 1,
"section": "rollout_jobs"
}
}
```
正式定义:
> 某个 Release 在某批目标节点上的一次分批推进计划
推荐状态:
- `planned`
- `running`
- `awaiting_approval`
- `ready_for_next_batch`
- `halted`
- `completed`
- `completed_with_issues`
---
## 4. Default Rollout Gate
来源:
- `GET /api/v1/ops/overview`
- `overview.release_hub.default_rollout_gate`
这层是 Release Hub 最关键的后端 gate contract。
最小字段:
```json
{
"status": "attention",
"status_label": "待处理",
"status_type": "warning",
"summary": "当前已有 Release但默认目标节点中仍有未接管或未通过巡检的节点。",
"summary_text": "当前已有 Release但默认目标节点中仍有未接管或未通过巡检的节点。",
"release": {
"id": 1,
"release_version": "2026.04.18+bd6bcb2",
"status": "ready",
"status_label": "已就绪"
},
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"target_node_codes": [
"mainland-worker-01"
],
"default_target_node_codes": [
"mainland-worker-01"
],
"blocking_reasons": [],
"warning_reasons": [
"存在未通过标准巡检的节点"
],
"recommendations": [
"先执行标准巡检,再进入 Rollout"
],
"operational_readiness": {
"summary": "remote-agent 就绪 1/2标准巡检通过 1/2",
"rows": []
},
"focus_ref": {
"kind": "release_hub",
"release_id": 1,
"release_version": "2026.04.18+bd6bcb2",
"channel": "stable",
"rollout_id": 0,
"rollout_code": "",
"section": "default_rollout_gate"
}
}
```
推荐状态:
- `missing_release`
- `release_not_ready`
- `artifact_missing`
- `no_targets`
- `blocked`
- `attention`
- `ready`
要求:
- 首页驾驶建议
- Release 页面门禁
- Rollout 预检
- Codex 自动驾驶
必须共用这同一份 Gate。
---
## 5. Default Rollout Execution
来源:
- `overview.release_hub.default_rollout_execution`
- `overview.release_hub.default_rollout_gate_options`
作用:
- 告诉页面 / Codex 这轮默认应使用什么执行方式
- 给出其他可选 gate / mode
最小字段:
```json
{
"recommended_mode": "remote-agent",
"recommended_mode_label": "Node Agent",
"recommended_gate": {},
"gates": {
"remote-agent": {},
"ssh": {}
}
}
```
推荐执行模式:
- `remote-agent`
- `ssh`
- `control-plane`
说明:
- 正式发布优先 `remote-agent`
- `ssh` 只作为首发接管与紧急救援兜底
- `control-plane` 不应用来承载大规模 Worker 发布
---
## 6. Release Launchpad
来源:
- `overview.release_hub.launchpad`
作用:
- 给页面与 Codex 提供更偏“驾驶舱”的发布聚合视角
最小字段:
```json
{
"latest_package": {},
"latest_release": {},
"worker_rollout_preview": {},
"control_rollout_preview": {},
"launchpad_status": {
"status": "attention",
"status_label": "待处理",
"summary": "建议先完成 Worker 点状灰度,再决定是否推进控制面。",
"summary_text": "建议先完成 Worker 点状灰度,再决定是否推进控制面。",
"recommended_action_code": "open_release_deploy_worker",
"recommended_target_node_code": "mainland-worker-01",
"recommended_recovery_label": "签发接入工单",
"recommended_recovery_summary": "当前节点仍处于接入阶段,建议先签发 onboarding.bootstrap。",
"onboarding_bootstrap_pending_nodes": 1,
"onboarding_acceptance_ready_nodes": 0,
"recommended_mode": "remote-agent",
"recommended_mode_label": "Node Agent",
"focus_ref": {
"kind": "release_hub",
"release_id": 1,
"release_version": "2026.04.18+bd6bcb2",
"channel": "stable",
"rollout_id": 0,
"rollout_code": "",
"section": "release_launchpad",
"mode": "worker"
}
}
}
```
`launchpad_status` 至少应回答:
- 当前先补什么
- 推荐动作码是什么
- 当前首个缺口节点是谁
- 当前恢复动作的人类标签是什么
- 当前缺口更偏“待接入”还是“待验收”
- 推荐执行方式是什么
- 先看 Worker 预案还是先看 Control 预案
正式约束:
- `recommended_target_node_code`
- 一旦 launchpad 已经收敛出首个缺口节点,就应直接返回
- 页面、CLI、Codex 不应再分别从 gap rows 中自行挑第一台
- `recommended_recovery_label`
- 是给人看的动作标题,例如:
- `签发接入工单`
- `补执行面接管`
- `执行接管验收`
- `recommended_recovery_summary`
- 是解释“为什么当前默认不是发版,而是先补接管”的正式摘要
- `onboarding_bootstrap_pending_nodes / onboarding_acceptance_ready_nodes`
- 是 launchpad 汇总层面的正式计数
- 允许被 `go-live-summary`、CLI 摘要、Codex brief 直接复用
---
## 7. 发布动作 contract
当前发布相关动作不应再退回“通用发任务”语义,而应正式区分:
- `deploy.release.control`
- `deploy.release.worker`
- `deploy.release.custom`
这三个动作必须共用:
- 同一份 Gate 判断
- 同一份 Release 版本对象
- 同一份 Rollout 编排结果
---
## 8. Artifact 不可变要求
上线前建议把 Release 供应链固定为:
1. 构建产物
2. 生成 `checksum`
3. 生成 `manifest`
4. 上传 artifact
5. 创建 Release
6. 激活 channel
7. 创建 Rollout
Node Agent 侧部署链路要求:
1. 下载 artifact
2. 校验 checksum
3. 解压到 `releases/<version>`
4. 切换 `current`
5. 重启服务
6. 健康检查
7. 失败回滚
---
## 9. Rollout 与 Ops Job 的关系
约束:
- Rollout 是分批推进对象
- Ops Job 是单次动作对象
- Rollout 通过一批或多批 `deploy.release.*` Job 落地
控制面责任:
- 决定下一批是否生成 Job
- 汇总 Job 结果
- 反推 Rollout 状态
- 决定是否暂停、继续或回滚
Node Agent 不负责:
- 推进下一批
- 修改 Rollout 策略
- 修改 Release 状态
---
## 10. 页面 / Codex 共同消费约束
以下对象必须作为稳定后端 contract 存在:
- `default_rollout_gate`
- `default_rollout_execution`
- `launchpad_status`
- `worker_rollout_preview`
- `control_rollout_preview`
页面只负责展示和少量交互兜底。
Codex 只负责消费后端给出的:
- 当前是否可发
- 建议先发哪一侧
- 应该使用什么 mode
- 下一步推荐动作是什么
---
## 11. Artifact Manifest Contract
Release Hub 不能只停在:
- `artifact_url`
- `checksum`
这只能证明“有个包”,还不能证明“这个包会把哪些服务改成什么样”。
建议把 `release.metadata.manifest` 固定为正式对象。
最小结构:
```json
{
"artifact_kind": "tarball",
"package_name": "domaincheck-2026.04.18+bd6bcb2.tgz",
"package_size_bytes": 186542331,
"checksum": "sha256:...",
"build_id": "build-20260418-001",
"commit_sha": "bd6bcb2",
"built_at": "2026-04-18 05:20:00",
"included_services": [
"domaincheck-api",
"domaincheck-worker",
"domaincheck-sync-agent",
"domaincheck-node-agent"
],
"required_env_keys": [
"NODE_CODE",
"NODE_REGION",
"NODE_ROLE"
],
"health_checks": [
{
"name": "api_health",
"type": "http",
"url": "http://127.0.0.1:8100/health"
}
],
"rollback_hint": "切回上一个 current 并重启对应服务"
}
```
正式要求:
- Release 创建后,`manifest` 不应再被覆盖成另一份语义不同的发布物
- Node Agent、页面、Codex 必须共用这同一份 manifest
- 是否可发、怎么回滚、要验哪些服务,不应再散落在:
- shell 脚本
- 页面表单
- 人工脑补
---
## 12. Operational Readiness Row Contract
`default_rollout_gate.operational_readiness.rows[]` 不应只是“几行提醒文案”,而应成为逐节点门禁对象。
最小结构:
```json
{
"node_code": "mainland-worker-01",
"region": "mainland",
"role": "worker",
"agent_managed": true,
"execution_mode_ready": true,
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"inspection_status": "healthy",
"inspection_status_label": "已通过",
"current_release_version": "2026.04.11+1921318",
"desired_release_version": "2026.04.18+bd6bcb2",
"risk_level": "low",
"blocking_reasons": [],
"warning_reasons": [],
"recommended_action_code": "",
"focus_ref": {
"kind": "release_hub",
"release_id": 1,
"release_version": "2026.04.18+bd6bcb2",
"channel": "stable",
"rollout_id": 0,
"rollout_code": "",
"section": "default_rollout_gate"
}
}
```
正式要求:
- `rows[]` 应能直接回答:
- 这台节点能不能进下一轮 rollout
- 不能的话卡在哪
- 只是 warning 还是 hard block
- 页面、CLI、Codex 不应再从:
- inspection
- runtime
- nodes
三处数据手工拼门禁结论
推荐风险分级:
- `low`
- `medium`
- `high`
- `blocked`
---
## 13. Rollout Preview Contract
真正创建 Rollout 之前,还应保留一层正式 preview而不是由页面自己预估批次和风险。
建议入口:
- `POST /api/v1/ops/releases/{id}/rollouts/preview`
最小请求体:
```json
{
"release_id": 1,
"target_node_codes": [
"mainland-worker-01",
"mainland-controller-01"
],
"policy": {
"batch_size": 1,
"require_approval_between_batches": true,
"rollback_on_failure": true
},
"execution_mode": "remote-agent"
}
```
最小返回体:
```json
{
"release": {},
"execution_mode": "remote-agent",
"execution_mode_label": "Node Agent",
"target_nodes_total": 2,
"target_node_codes": [
"mainland-worker-01",
"mainland-controller-01"
],
"expected_batches_total": 2,
"expected_jobs_total": 2,
"gate": {},
"batch_plan": [
{
"batch_no": 1,
"node_codes": [
"mainland-worker-01"
],
"risk_level": "low"
},
{
"batch_no": 2,
"node_codes": [
"mainland-controller-01"
],
"risk_level": "medium"
}
],
"summary": "建议先灰度 1 台 Worker再推进 Controller。",
"summary_text": "建议先灰度 1 台 Worker再推进 Controller。",
"focus_ref": {
"kind": "release_hub",
"release_id": 1,
"release_version": "2026.04.18+bd6bcb2",
"channel": "stable",
"rollout_id": 0,
"rollout_code": "",
"section": "rollout_preview"
}
}
```
正式要求:
- preview 是:
- 页面发 rollout 前的统一预检
- Codex 自动驾驶发 rollout 前的统一预检
- CLI 批量发布前的统一预检
- 不允许前端重新自己算:
- 该分几批
- 哪台应该先发
- 哪台属于高风险
只有这样Release Hub 才是真正的发布控制面,而不是“能看版本的页面”。