This commit is contained in:
Your Name
2026-04-17 16:17:19 +08:00
parent 0e096947fc
commit bd6bcb240f
17 changed files with 863 additions and 23 deletions

View File

@@ -36,6 +36,10 @@
- `docs/18_domainCheck_CentOS9一键复制部署与更新文档.md` - `docs/18_domainCheck_CentOS9一键复制部署与更新文档.md`
### 9. 临时海外控制面联调清单
- `docs/20_domainCheck_临时海外控制面联调清单.md`
## 二、文档阅读顺序 ## 二、文档阅读顺序
### 1. 先看总体方案 ### 1. 先看总体方案
@@ -53,6 +57,7 @@
- `docs/16_domainCheck_多机检测与跨地域部署设计.md` - `docs/16_domainCheck_多机检测与跨地域部署设计.md`
- `docs/17_domainCheck_全流程部署实操手册.md` - `docs/17_domainCheck_全流程部署实操手册.md`
- `docs/18_domainCheck_CentOS9一键复制部署与更新文档.md` - `docs/18_domainCheck_CentOS9一键复制部署与更新文档.md`
- `docs/20_domainCheck_临时海外控制面联调清单.md`
### 3. 如果要回溯历史需求与问题 ### 3. 如果要回溯历史需求与问题

View File

@@ -0,0 +1,231 @@
# domainCheck 临时海外控制面联调清单
适用场景:
- 暂时不使用正式海外机器
- 先把大陆 `controller + worker` 对接到当前测试海外机
- 先跑通“控制面 + 同步 + 观测”全链路,再切回正式海外环境
当前临时海外控制面:
- 节点编码:`overseas-control-01`
- 节点角色:`overseas / control`
- 机器 IP`152.53.37.118`
- 后台地址:`http://152.53.37.118/`
- API 基址:`http://152.53.37.118:8100`
## 1. 当前目标拓扑
本次联调按下面的角色分工:
- 大陆 controller
- 承担 API 控制面
- 承担 Sync Agent
- 可以兼跑本机 Worker
- 大陆 worker
- 只承担检测执行
- 临时海外控制面
- 只承担 API 控制与同步接收
- 不承载本机 Worker
- 不承载 Sync Agent
## 2. 临时海外控制面应有状态
在临时海外控制面机器上执行:
```bash
bash domain-api/deploy/multi-region/check_temp_overseas_control.sh
```
预期:
- `domaincheck-api``active (running)`
- `domaincheck-worker``inactive (dead)` 或禁用
- `runtime.status.data.node`
- `code=overseas-control-01`
- `region=overseas`
- `role=control`
- `runtime.status.data.worker.expected_on_this_node=false`
- `runtime.status.data.sync_agent.expected_on_this_node=false`
- `runtime.status.data.detect.phase_label=当前节点不承载`
## 3. 大陆 controller 切到临时海外控制面
在大陆 controller 上执行:
```bash
sed -i 's#^SYNC_TARGET_API_BASE_URL=.*#SYNC_TARGET_API_BASE_URL=http://152.53.37.118:8100#' /etc/default/domaincheck-api
sed -i 's/^SYNC_PUSH_ENABLED=.*/SYNC_PUSH_ENABLED=true/' /etc/default/domaincheck-api
systemctl restart domaincheck-api
systemctl restart domaincheck-sync-agent
systemctl restart domaincheck-worker
sleep 3
```
然后检查:
```bash
curl -s http://127.0.0.1:8100/api/v1/runtime/readiness
echo
curl -s http://127.0.0.1:8100/api/v1/runtime/sync-summary
```
预期:
- `runtime/readiness` 至少不是大陆本机配置错误导致的 `blocking`
- `runtime/sync-summary.data.enabled=true`
- `runtime/sync-summary.data.target_api_base_url=http://152.53.37.118:8100`
## 4. 大陆 worker 检查项
在大陆 worker 上执行:
```bash
bash domain-api/deploy/multi-region/check_mainland_worker.sh
```
预期:
- `NODE_CODE=mainland-worker-01`
- `NODE_REGION=mainland`
- `NODE_ROLE=worker`
- `domaincheck-worker``active (running)`
如果大陆 controller 也兼跑检测,再执行:
```bash
systemctl status domaincheck-worker --no-pager -l
echo
curl -s http://127.0.0.1:8100/api/v1/runtime/status
```
预期:
- `worker.running=true`
- `worker.expected_on_this_node=true`
## 5. 最短联调命令
### 5.1 在大陆 controller 上
```bash
bash domain-api/deploy/multi-region/check_temp_link.sh http://127.0.0.1:8100 http://152.53.37.118:8100
```
关注点:
- `push_sync` 返回 `code=0` 或幂等成功
- 不再出现目标地址为空
- 不再出现 `SYNC_PUSH_ENABLED=false`
关注点:
- `cluster` 中至少能看到:
- `overseas-control-01`
- 大陆 `mainland-controller-01`
- 大陆 `mainland-worker-01`
- `sync-summary` 中能看到来自大陆的接收记录
- `readiness` 不再是“完全单节点海外态”
## 6. 后台页面应如何理解
打开:
- `http://152.53.37.118/`
在运行中心里看到下面这些内容时,属于正常:
- `本机 Worker当前节点不承载`
- `Sync Agent当前节点不承载`
- `当前检测阶段:当前节点不承载`
- `代理运行态:不适用`
这不是故障,而是因为当前节点就是临时海外控制面。
真正应该关注的是:
- `集群节点`
- `有效执行节点`
- `同步状态`
- `多机就绪度`
## 7. 判断“到底有几台 Worker 在工作”
如果页面上“在线 Worker 数”和“实际正在跑检测的节点数”看起来不一致,以数据库为准。
在大陆 controller 上执行:
```bash
bash domain-api/deploy/multi-region/check_worker_participation.sh
```
判断规则:
- `detect_worker_nodes`
- 看谁在线
- 看谁是 `worker_online=true`
- 看谁 `detect_participating=true`
- `detect_job_items`
- 看任务到底被哪台节点 `claimed_by`
- 这才是“当前真正干活的节点”
## 8. 常见误判
### 8.1 海外控制面显示“本机 Worker 不承载”
这是正常,不是故障。
### 8.2 页面显示“在线 Worker 1”但你部署了 2 台大陆机器
先区分:
- “有效执行节点”是可承担任务的在线节点数
- “当前参与检测节点”是当前真的在领任务、跑任务的节点
如果 controller 没兼跑检测,通常只会看到独立 worker 在真正执行。
### 8.3 controller 机器也开了 worker 服务,但没有领任务
这不一定是 bug可能只是当前调度没有分到它。应以 `detect_job_items.claimed_by` 为准,不要只看服务在线。
## 9. 联调完成后的回滚
如果你后续要切回正式海外机器,在大陆 controller 上把目标地址改回正式值即可:
```bash
sed -i 's#^SYNC_TARGET_API_BASE_URL=.*#SYNC_TARGET_API_BASE_URL=http://正式海外控制面IP:8100#' /etc/default/domaincheck-api
systemctl restart domaincheck-api
systemctl restart domaincheck-sync-agent
sleep 3
```
如果临时海外控制面不再使用,可在该机器上保留 `domaincheck-api` 供后续调试,也可以停掉:
```bash
systemctl stop domaincheck-api
```
## 10. 本次临时海外控制面结论
当前这台测试海外机已经整理为:
-`overseas-control`
- API 正常
- Worker 不承载
- Sync Agent 不承载
- 数据已清空并按首装态初始化
- 可直接作为大陆 controller 的临时同步接收端
## 11. 配套脚本
- 临时海外控制面自检:
- `domain-api/deploy/multi-region/check_temp_overseas_control.sh`
- 大陆 controller -> 临时海外控制面联调检查:
- `domain-api/deploy/multi-region/check_temp_link.sh`
- 三机拓扑总览检查:
- `domain-api/deploy/multi-region/check_temp_topology.sh`
- 当前活跃任务由谁真正执行:
- `domain-api/deploy/multi-region/check_worker_participation.sh`
- 大陆 worker 本机自检:
- `domain-api/deploy/multi-region/check_mainland_worker.sh`

View File

@@ -1,3 +1,5 @@
from __future__ import annotations
from uuid import uuid4 from uuid import uuid4
from fastapi import APIRouter from fastapi import APIRouter

View File

@@ -28,11 +28,13 @@ def fetch_overview() -> dict:
online_worker_nodes = int(cluster_summary.get("online_worker_nodes", 0) or 0) online_worker_nodes = int(cluster_summary.get("online_worker_nodes", 0) or 0)
dedicated_online_worker_nodes = int(cluster_summary.get("dedicated_online_worker_nodes", 0) or 0) dedicated_online_worker_nodes = int(cluster_summary.get("dedicated_online_worker_nodes", 0) or 0)
result["worker_status"] = "online" if runtime["worker"]["running"] else "offline" result["worker_status"] = "online" if runtime["worker"]["running"] else "offline"
result["local_worker_status"] = result["worker_status"] result["local_worker_status"] = "not-applicable" if not runtime["worker"].get("expected_on_this_node", True) else result["worker_status"]
result["cluster_worker_status"] = "online" if online_worker_nodes > 0 else "offline" result["cluster_worker_status"] = "online" if online_worker_nodes > 0 else "offline"
result["cluster_online_worker_nodes"] = online_worker_nodes result["cluster_online_worker_nodes"] = online_worker_nodes
result["cluster_dedicated_online_worker_nodes"] = dedicated_online_worker_nodes result["cluster_dedicated_online_worker_nodes"] = dedicated_online_worker_nodes
result["cluster_online_control_nodes"] = int(cluster_summary.get("online_control_nodes", 0) or 0) result["cluster_online_control_nodes"] = int(cluster_summary.get("online_control_nodes", 0) or 0)
result["api_status"] = "online" result["api_status"] = "online"
result["worker_mode"] = runtime["worker"]["mode"] result["worker_mode"] = runtime["worker"]["mode"]
result["node_region"] = runtime["node"]["region"]
result["node_role"] = runtime["node"]["role"]
return result return result

View File

@@ -270,6 +270,7 @@ def get_runtime_status() -> dict:
runtime_settings = get_runtime_settings() runtime_settings = get_runtime_settings()
worker_runtime = detect_worker_runtime() worker_runtime = detect_worker_runtime()
sync_agent_runtime = detect_sync_agent_runtime() sync_agent_runtime = detect_sync_agent_runtime()
worker_expected_on_this_node = not (settings.node_region == "overseas" and settings.node_role == "control")
api_pid = os.getpid() api_pid = os.getpid()
detect_snapshot = get_detect_status() detect_snapshot = get_detect_status()
latest_run = (detect_snapshot.get("runs") or [None])[0] or {} latest_run = (detect_snapshot.get("runs") or [None])[0] or {}
@@ -314,6 +315,35 @@ def get_runtime_status() -> dict:
"queue_health": queue_health, "queue_health": queue_health,
"capacity_plan": capacity_plan, "capacity_plan": capacity_plan,
} }
if not worker_expected_on_this_node:
detect_payload.update(
{
"phase_label": "当前节点不承载",
"phase_detail": "当前节点为海外控制面,仅承载 API 控制与同步接收,不执行本机检测任务。",
"recent_event": "",
"recent_warning": "",
"progress_percent": 0,
"active_thread_count": 0,
"max_thread_count": 0,
"available_proxy_count": 0,
"proxy_pool_count": 0,
"proxy_runtime_label": "不适用",
"proxy_runtime_detail": "当前节点不承载本机 Worker代理、线程与检测阶段信息在此节点上不适用。",
"proxy_runtime_reason": "not_applicable",
"proxy_supplier_empty": False,
"proxy_last_refresh_status": "",
"proxy_last_refresh_time": "",
"proxy_last_refresh_source_count": 0,
"proxy_last_refresh_total_items": 0,
"proxy_last_validated_count": 0,
"proxy_last_available_count": 0,
"proxy_source_stats": [],
"dependency_alerts": [],
"active_job": None,
"runs_count": 0,
"worker_online": False,
}
)
detect_payload["participating_nodes"] = _build_participating_detect_nodes( detect_payload["participating_nodes"] = _build_participating_detect_nodes(
cluster_snapshot=cluster_snapshot, cluster_snapshot=cluster_snapshot,
detect_snapshot=detect_payload, detect_snapshot=detect_payload,
@@ -351,6 +381,7 @@ def get_runtime_status() -> dict:
"mode": worker_runtime.get("mode", runtime_settings.get("worker_mode", "windows-local")), "mode": worker_runtime.get("mode", runtime_settings.get("worker_mode", "windows-local")),
"service_name": runtime_settings.get("worker_service_name", settings.worker_service_name), "service_name": runtime_settings.get("worker_service_name", settings.worker_service_name),
"running": worker_runtime.get("running", False), "running": worker_runtime.get("running", False),
"expected_on_this_node": worker_expected_on_this_node,
"process_count": worker_runtime.get("process_count", 0), "process_count": worker_runtime.get("process_count", 0),
"latest_start_time": worker_runtime.get("latest_start_time", ""), "latest_start_time": worker_runtime.get("latest_start_time", ""),
"message": worker_runtime.get("message", ""), "message": worker_runtime.get("message", ""),

View File

@@ -25,6 +25,11 @@
- `deploy/multi-region/bootstrap_mainland.sh` - `deploy/multi-region/bootstrap_mainland.sh`
- `deploy/multi-region/check_cluster.sh` - `deploy/multi-region/check_cluster.sh`
- `deploy/multi-region/check_mainland_controller.sh` - `deploy/multi-region/check_mainland_controller.sh`
- `deploy/multi-region/check_mainland_worker.sh`
- `deploy/multi-region/check_worker_participation.sh`
- `deploy/multi-region/check_temp_overseas_control.sh`
- `deploy/multi-region/check_temp_link.sh`
- `deploy/multi-region/check_temp_topology.sh`
- `deploy/multi-region/simulate_cluster_node.py` - `deploy/multi-region/simulate_cluster_node.py`
- `deploy/multi-region/simulate_multi_region.sh` - `deploy/multi-region/simulate_multi_region.sh`
- `deploy/multi-region/prune_cluster_nodes.py` - `deploy/multi-region/prune_cluster_nodes.py`
@@ -199,13 +204,48 @@ bash deploy/multi-region/check_mainland_controller.sh
这条命令会直接检查: 这条命令会直接检查:
- `/etc/default/domaincheck-worker` 是否存在 - `/etc/default/domaincheck-api` 是否存在
- `NODE_ROLE=control` 是否正确 - `NODE_ROLE=control` 是否正确
- `SYNC_PUSH_ENABLED` / `SYNC_TARGET_API_BASE_URL` 是否已配置 - `SYNC_PUSH_ENABLED` / `SYNC_TARGET_API_BASE_URL` 是否已配置
- `domaincheck-api`
- `domaincheck-worker` - `domaincheck-worker`
- `domaincheck-sync-agent` - `domaincheck-sync-agent`
是否已启用并处于运行态 是否已启用并处于运行态
如果要判断“在线 Worker 有几台”和“当前真正领任务执行的是哪台”是否一致,可以执行:
```bash
cd /opt/domaincheck/domain-api
bash deploy/multi-region/check_worker_participation.sh
```
这条命令会直接输出:
- `detect_worker_nodes` 当前在线节点概览
- 最新活跃任务的 `claimed_by / status / count`
- 一段压缩摘要 JSON
如果要检查纯大陆 worker 节点自己的配置和服务状态,可以执行:
```bash
cd /opt/domaincheck/domain-api
bash deploy/multi-region/check_mainland_worker.sh
```
如果当前是“大陆 controller + 大陆 worker + 临时海外控制面”的联调拓扑,还可以执行:
```bash
cd /opt/domaincheck/domain-api
bash deploy/multi-region/check_temp_topology.sh http://127.0.0.1:8100 http://152.53.37.118:8100
```
这条命令会同时汇总:
- 大陆控制面的 `runtime/cluster`
- 大陆控制面的 `runtime/readiness`
- 最新活跃任务到底由谁 `claimed_by`
- 临时海外控制面的 `runtime/cluster`
如果当前还无法真正部署到大陆机器,也可以先在国外测试机上做“单机模拟多节点联调”: 如果当前还无法真正部署到大陆机器,也可以先在国外测试机上做“单机模拟多节点联调”:
```bash ```bash

View File

@@ -1,8 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
ENV_FILE="${1:-/etc/default/domaincheck-worker}" ENV_FILE="${1:-/etc/default/domaincheck-api}"
WORKER_SERVICE="${WORKER_SERVICE:-domaincheck-worker}" WORKER_SERVICE="${WORKER_SERVICE:-domaincheck-worker}"
API_SERVICE="${API_SERVICE:-domaincheck-api}"
SYNC_AGENT_SERVICE="${SYNC_AGENT_SERVICE:-domaincheck-sync-agent}" SYNC_AGENT_SERVICE="${SYNC_AGENT_SERVICE:-domaincheck-sync-agent}"
PYTHON_BIN="${PYTHON_BIN:-python3}" PYTHON_BIN="${PYTHON_BIN:-python3}"
@@ -61,16 +62,22 @@ if errors:
PY PY
echo echo
echo "[3/5] systemd status" echo "[3/6] systemd status"
systemctl is-enabled "$API_SERVICE" || true
systemctl is-active "$API_SERVICE"
systemctl is-enabled "$WORKER_SERVICE" || true systemctl is-enabled "$WORKER_SERVICE" || true
systemctl is-active "$WORKER_SERVICE" systemctl is-active "$WORKER_SERVICE"
systemctl is-enabled "$SYNC_AGENT_SERVICE" || true systemctl is-enabled "$SYNC_AGENT_SERVICE" || true
systemctl is-active "$SYNC_AGENT_SERVICE" systemctl is-active "$SYNC_AGENT_SERVICE"
echo echo
echo "[4/5] worker service detail" echo "[4/6] api service detail"
systemctl status "$API_SERVICE" --no-pager -l | sed -n '1,25p'
echo
echo "[5/6] worker service detail"
systemctl status "$WORKER_SERVICE" --no-pager -l | sed -n '1,25p' systemctl status "$WORKER_SERVICE" --no-pager -l | sed -n '1,25p'
echo echo
echo "[5/5] sync-agent detail" echo "[6/6] sync-agent detail"
systemctl status "$SYNC_AGENT_SERVICE" --no-pager -l | sed -n '1,25p' systemctl status "$SYNC_AGENT_SERVICE" --no-pager -l | sed -n '1,25p'

View File

@@ -0,0 +1,70 @@
#!/usr/bin/env bash
set -euo pipefail
ENV_FILE="${1:-/etc/default/domaincheck-worker}"
WORKER_SERVICE="${WORKER_SERVICE:-domaincheck-worker}"
PYTHON_BIN="${PYTHON_BIN:-python3}"
echo "[1/4] environment file"
if [ ! -f "$ENV_FILE" ]; then
echo "missing env file: $ENV_FILE"
exit 1
fi
echo "env file: $ENV_FILE"
echo
echo "[2/4] key env summary"
"${PYTHON_BIN}" - <<'PY' "$ENV_FILE"
import json
import sys
from pathlib import Path
env_path = Path(sys.argv[1])
values = {}
for line in env_path.read_text(encoding="utf-8").splitlines():
text = line.strip()
if not text or text.startswith("#") or "=" not in text:
continue
key, value = text.split("=", 1)
values[key.strip()] = value.strip()
required = {
"NODE_CODE": values.get("NODE_CODE", ""),
"NODE_REGION": values.get("NODE_REGION", ""),
"NODE_ROLE": values.get("NODE_ROLE", ""),
"DB_HOST": values.get("DB_HOST", ""),
"DB_PORT": values.get("DB_PORT", ""),
"REDIS_HOST": values.get("REDIS_HOST", ""),
"REDIS_PORT": values.get("REDIS_PORT", ""),
"SYNC_TARGET_API_BASE_URL": values.get("SYNC_TARGET_API_BASE_URL", ""),
}
print(json.dumps(required, ensure_ascii=False, indent=2))
errors = []
if required["NODE_REGION"] != "mainland":
errors.append("NODE_REGION 必须为 mainland")
if required["NODE_ROLE"] != "worker":
errors.append("NODE_ROLE 必须为 worker")
if not required["NODE_CODE"]:
errors.append("NODE_CODE 不能为空")
if not required["DB_HOST"]:
errors.append("DB_HOST 不能为空")
if not required["REDIS_HOST"]:
errors.append("REDIS_HOST 不能为空")
if errors:
print()
print("env validation failed:")
for item in errors:
print(f"- {item}")
sys.exit(1)
PY
echo
echo "[3/4] systemd status"
systemctl is-enabled "$WORKER_SERVICE" || true
systemctl is-active "$WORKER_SERVICE"
echo
echo "[4/4] worker service detail"
systemctl status "$WORKER_SERVICE" --no-pager -l | sed -n '1,30p'

View File

@@ -0,0 +1,80 @@
#!/usr/bin/env bash
set -euo pipefail
MAINLAND_BASE_URL="${1:-http://127.0.0.1:8100}"
OVERSEAS_BASE_URL="${2:-http://152.53.37.118:8100}"
PYTHON_BIN="${PYTHON_BIN:-python3}"
echo "[1/6] mainland readiness"
MAINLAND_READINESS="$(curl -fsS "${MAINLAND_BASE_URL}/api/v1/runtime/readiness")"
echo "${MAINLAND_READINESS}"
echo
echo
echo "[2/6] mainland sync summary"
MAINLAND_SYNC="$(curl -fsS "${MAINLAND_BASE_URL}/api/v1/runtime/sync-summary")"
echo "${MAINLAND_SYNC}"
echo
echo
echo "[3/6] mainland push sync"
MAINLAND_PUSH="$(curl -fsS -X POST "${MAINLAND_BASE_URL}/api/v1/runtime/actions/push_sync")"
echo "${MAINLAND_PUSH}"
echo
echo
echo "[4/6] overseas cluster"
OVERSEAS_CLUSTER="$(curl -fsS "${OVERSEAS_BASE_URL}/api/v1/runtime/cluster")"
echo "${OVERSEAS_CLUSTER}"
echo
echo
echo "[5/6] overseas sync summary"
OVERSEAS_SYNC="$(curl -fsS "${OVERSEAS_BASE_URL}/api/v1/runtime/sync-summary")"
echo "${OVERSEAS_SYNC}"
echo
echo
echo "[6/6] condensed summary"
"${PYTHON_BIN}" - <<'PY' "$MAINLAND_READINESS" "$MAINLAND_SYNC" "$MAINLAND_PUSH" "$OVERSEAS_CLUSTER" "$OVERSEAS_SYNC"
import json
import sys
mainland_readiness = json.loads(sys.argv[1]).get("data", {})
mainland_sync = json.loads(sys.argv[2]).get("data", {})
mainland_push = json.loads(sys.argv[3])
overseas_cluster = json.loads(sys.argv[4]).get("data", {})
overseas_sync = json.loads(sys.argv[5]).get("data", {})
cluster_nodes = overseas_cluster.get("nodes") or []
cluster_summary = overseas_cluster.get("summary") or {}
type_counts = overseas_sync.get("type_counts") or {}
status_counts = overseas_sync.get("status_counts") or {}
print(
json.dumps(
{
"mainland": {
"readiness_status": mainland_readiness.get("status", ""),
"readiness_summary": mainland_readiness.get("summary", ""),
"sync_enabled": bool(mainland_sync.get("enabled", False)),
"target_api_base_url": mainland_sync.get("target_api_base_url", ""),
"push_sync_code": mainland_push.get("code"),
"push_sync_message": mainland_push.get("message", ""),
},
"overseas": {
"nodes_total": overseas_cluster.get("nodes_total", 0),
"online_control_nodes": cluster_summary.get("online_control_nodes", 0),
"online_worker_nodes": cluster_summary.get("online_worker_nodes", 0),
"node_codes": [str(item.get("node_code") or "") for item in cluster_nodes],
"runtime_ingest": type_counts.get("runtime_ingest", 0),
"detect_result_ingest": type_counts.get("detect_result_ingest", 0),
"status_received": status_counts.get("received", 0),
"status_success": status_counts.get("success", 0),
},
},
ensure_ascii=False,
indent=2,
)
)
PY

View File

@@ -0,0 +1,75 @@
#!/usr/bin/env bash
set -euo pipefail
BASE_URL="${1:-http://127.0.0.1:8100}"
PYTHON_BIN="${PYTHON_BIN:-python3}"
echo "[1/5] health"
curl -fsS "${BASE_URL}/health"
echo
echo
echo "[2/5] runtime status"
STATUS_JSON="$(curl -fsS "${BASE_URL}/api/v1/runtime/status")"
echo "${STATUS_JSON}"
echo
echo
echo "[3/5] runtime cluster"
curl -fsS "${BASE_URL}/api/v1/runtime/cluster"
echo
echo
echo "[4/5] runtime readiness"
curl -fsS "${BASE_URL}/api/v1/runtime/readiness"
echo
echo
echo "[5/5] condensed summary"
"${PYTHON_BIN}" - <<'PY' "$STATUS_JSON"
import json
import sys
data = json.loads(sys.argv[1]).get("data", {})
node = data.get("node") or {}
worker = data.get("worker") or {}
sync_agent = data.get("sync_agent") or {}
detect = data.get("detect") or {}
cluster = data.get("cluster") or {}
summary = cluster.get("summary") or {}
print(
json.dumps(
{
"node": {
"code": node.get("code", ""),
"region": node.get("region", ""),
"role": node.get("role", ""),
},
"worker": {
"running": bool(worker.get("running", False)),
"expected_on_this_node": bool(worker.get("expected_on_this_node", True)),
"message": worker.get("message", ""),
},
"sync_agent": {
"running": bool(sync_agent.get("running", False)),
"expected_on_this_node": bool(sync_agent.get("expected_on_this_node", False)),
"message": sync_agent.get("message", ""),
},
"detect": {
"phase_label": detect.get("phase_label", ""),
"proxy_runtime_label": detect.get("proxy_runtime_label", ""),
"active_job": (detect.get("active_job") or {}).get("job_code", ""),
},
"cluster": {
"nodes_total": cluster.get("nodes_total", 0),
"online_control_nodes": summary.get("online_control_nodes", 0),
"online_worker_nodes": summary.get("online_worker_nodes", 0),
"dedicated_online_worker_nodes": summary.get("dedicated_online_worker_nodes", 0),
},
},
ensure_ascii=False,
indent=2,
)
)
PY

View File

@@ -0,0 +1,126 @@
#!/usr/bin/env bash
set -euo pipefail
MAINLAND_BASE_URL="${1:-http://127.0.0.1:8100}"
OVERSEAS_BASE_URL="${2:-http://152.53.37.118:8100}"
PYTHON_BIN="${PYTHON_BIN:-python3}"
echo "[1/5] mainland runtime cluster"
MAINLAND_CLUSTER="$(curl -fsS "${MAINLAND_BASE_URL}/api/v1/runtime/cluster")"
echo "${MAINLAND_CLUSTER}"
echo
echo
echo "[2/5] mainland runtime readiness"
MAINLAND_READINESS="$(curl -fsS "${MAINLAND_BASE_URL}/api/v1/runtime/readiness")"
echo "${MAINLAND_READINESS}"
echo
echo
echo "[3/5] mainland worker participation"
PARTICIPATION_JSON="$(/www/server/pgsql/bin/psql -U postgres -d domain -At <<'SQL'
with latest_job as (
select id, job_code, status
from detect_jobs
where status in ('pending','running')
order by id desc
limit 1
),
node_stats as (
select
node_code,
role,
status,
coalesce(metadata_json->>'worker_online', '') as worker_online,
coalesce(metadata_json->>'detect_participating', '') as detect_participating
from detect_worker_nodes
),
item_stats as (
select
coalesce(claimed_by, '') as claimed_by,
status,
count(*) as cnt
from detect_job_items
where job_id = (select id from latest_job)
group by claimed_by, status
)
select json_build_object(
'job', (
select json_build_object(
'job_code', coalesce(job_code, ''),
'status', coalesce(status, '')
)
from latest_job
),
'online_nodes', (
select coalesce(json_agg(json_build_object(
'node_code', node_code,
'role', role,
'status', status,
'worker_online', worker_online,
'detect_participating', detect_participating
) order by node_code), '[]'::json)
from node_stats
where status in ('online', 'busy')
),
'claimed_distribution', (
select coalesce(json_agg(json_build_object(
'claimed_by', claimed_by,
'status', status,
'count', cnt
) order by claimed_by, status), '[]'::json)
from item_stats
)
);
SQL
)"
echo "${PARTICIPATION_JSON}"
echo
echo
echo "[4/5] overseas runtime cluster"
OVERSEAS_CLUSTER="$(curl -fsS "${OVERSEAS_BASE_URL}/api/v1/runtime/cluster")"
echo "${OVERSEAS_CLUSTER}"
echo
echo
echo "[5/5] condensed summary"
"${PYTHON_BIN}" - <<'PY' "$MAINLAND_CLUSTER" "$MAINLAND_READINESS" "$PARTICIPATION_JSON" "$OVERSEAS_CLUSTER"
import json
import sys
mainland_cluster = json.loads(sys.argv[1]).get("data", {})
mainland_readiness = json.loads(sys.argv[2]).get("data", {})
participation = json.loads(sys.argv[3])
overseas_cluster = json.loads(sys.argv[4]).get("data", {})
mainland_summary = mainland_cluster.get("summary") or {}
overseas_summary = overseas_cluster.get("summary") or {}
def node_codes(cluster):
return [str(item.get("node_code") or "") for item in (cluster.get("nodes") or [])]
print(
json.dumps(
{
"mainland": {
"readiness_status": mainland_readiness.get("status", ""),
"readiness_summary": mainland_readiness.get("summary", ""),
"online_worker_nodes": mainland_summary.get("online_worker_nodes", 0),
"dedicated_online_worker_nodes": mainland_summary.get("dedicated_online_worker_nodes", 0),
"online_control_nodes": mainland_summary.get("online_control_nodes", 0),
"node_codes": node_codes(mainland_cluster),
},
"participation": participation,
"overseas": {
"online_worker_nodes": overseas_summary.get("online_worker_nodes", 0),
"dedicated_online_worker_nodes": overseas_summary.get("dedicated_online_worker_nodes", 0),
"online_control_nodes": overseas_summary.get("online_control_nodes", 0),
"node_codes": node_codes(overseas_cluster),
},
},
ensure_ascii=False,
indent=2,
)
)
PY

View File

@@ -0,0 +1,97 @@
#!/usr/bin/env bash
set -euo pipefail
DB_NAME="${DB_NAME:-domain}"
DB_USER="${DB_USER:-postgres}"
PSQL_BIN="${PSQL_BIN:-/www/server/pgsql/bin/psql}"
echo "[1/3] cluster nodes"
"${PSQL_BIN}" -U "${DB_USER}" -d "${DB_NAME}" -Atc "
select
node_code || '|' ||
role || '|' ||
status || '|' ||
coalesce(current_load, 0) || '|' ||
to_char(last_heartbeat_at, 'YYYY-MM-DD HH24:MI:SS') || '|' ||
coalesce(metadata_json->>'worker_online', '') || '|' ||
coalesce(metadata_json->>'detect_participating', '')
from detect_worker_nodes
order by node_code;
"
echo
echo "[2/3] active job distribution"
"${PSQL_BIN}" -U "${DB_USER}" -d "${DB_NAME}" -Atc "
select
coalesce(claimed_by, '') || '|' ||
status || '|' ||
count(*)
from detect_job_items
where job_id = (
select id
from detect_jobs
where status in ('pending','running')
order by id desc
limit 1
)
group by claimed_by, status
order by claimed_by, status;
"
echo
echo "[3/3] condensed summary"
"${PSQL_BIN}" -U "${DB_USER}" -d "${DB_NAME}" -At <<'SQL'
with latest_job as (
select id, job_code, status
from detect_jobs
where status in ('pending','running')
order by id desc
limit 1
),
node_stats as (
select
node_code,
role,
status,
coalesce(metadata_json->>'worker_online', '') as worker_online,
coalesce(metadata_json->>'detect_participating', '') as detect_participating
from detect_worker_nodes
),
item_stats as (
select
coalesce(claimed_by, '') as claimed_by,
status,
count(*) as cnt
from detect_job_items
where job_id = (select id from latest_job)
group by claimed_by, status
)
select json_build_object(
'job', (
select json_build_object(
'job_code', coalesce(job_code, ''),
'status', coalesce(status, '')
)
from latest_job
),
'online_nodes', (
select coalesce(json_agg(json_build_object(
'node_code', node_code,
'role', role,
'status', status,
'worker_online', worker_online,
'detect_participating', detect_participating
) order by node_code), '[]'::json)
from node_stats
where status in ('online', 'busy')
),
'claimed_distribution', (
select coalesce(json_agg(json_build_object(
'claimed_by', claimed_by,
'status', status,
'count', cnt
) order by claimed_by, status), '[]'::json)
from item_stats
)
);
SQL

View File

@@ -24,8 +24,8 @@
<span class="status-pill" :class="runtime.apiOnline ? 'online' : 'offline'"> <span class="status-pill" :class="runtime.apiOnline ? 'online' : 'offline'">
API {{ runtime.apiOnline ? "在线" : "离线" }} API {{ runtime.apiOnline ? "在线" : "离线" }}
</span> </span>
<span class="status-pill" :class="runtime.localWorkerOnline ? 'online' : 'offline'"> <span class="status-pill" :class="runtime.localWorkerExpected ? (runtime.localWorkerOnline ? 'online' : 'offline') : 'neutral'">
本机 Worker {{ runtime.localWorkerOnline ? "在线" : "离线" }} 本机 Worker {{ runtime.localWorkerExpected ? (runtime.localWorkerOnline ? "在线" : "离线") : "不承载" }}
</span> </span>
<span class="status-pill" :class="runtime.effectiveWorkerNodes > 0 ? 'online' : 'offline'"> <span class="status-pill" :class="runtime.effectiveWorkerNodes > 0 ? 'online' : 'offline'">
有效执行节点 {{ runtime.effectiveWorkerNodes }} 有效执行节点 {{ runtime.effectiveWorkerNodes }}
@@ -57,6 +57,7 @@ const authStore = useAuthStore();
const runtime = ref({ const runtime = ref({
apiOnline: false, apiOnline: false,
localWorkerOnline: false, localWorkerOnline: false,
localWorkerExpected: true,
effectiveWorkerNodes: 0, effectiveWorkerNodes: 0,
workerMode: "windows-local" workerMode: "windows-local"
}); });
@@ -85,6 +86,7 @@ const loadRuntimeSummary = async () => {
runtime.value = { runtime.value = {
apiOnline: Boolean(response.data?.api?.pid), apiOnline: Boolean(response.data?.api?.pid),
localWorkerOnline: Boolean(response.data?.worker?.running), localWorkerOnline: Boolean(response.data?.worker?.running),
localWorkerExpected: Boolean(response.data?.worker?.expected_on_this_node ?? true),
effectiveWorkerNodes: Number(summary.online_worker_nodes || 0), effectiveWorkerNodes: Number(summary.online_worker_nodes || 0),
workerMode: response.data?.worker?.mode || "windows-local" workerMode: response.data?.worker?.mode || "windows-local"
}; };
@@ -92,6 +94,7 @@ const loadRuntimeSummary = async () => {
runtime.value = { runtime.value = {
apiOnline: false, apiOnline: false,
localWorkerOnline: false, localWorkerOnline: false,
localWorkerExpected: true,
effectiveWorkerNodes: 0, effectiveWorkerNodes: 0,
workerMode: "unknown" workerMode: "unknown"
}; };

View File

@@ -80,12 +80,20 @@ const loadOverview = async (showError = true) => {
loading.value = true; loading.value = true;
try { try {
const { data } = await dashboardApi.overview(); const { data } = await dashboardApi.overview();
const localWorkerValue =
data.local_worker_status === "not-applicable"
? "不承载"
: data.local_worker_status || data.worker_status;
const localWorkerNote =
data.local_worker_status === "not-applicable"
? `当前节点 ${data.node_region || "-"} / ${data.node_role || "-"},默认不承载本机 Worker`
: "当前节点本机 Worker 运行状态";
stats.value = [ stats.value = [
{ label: "总域名", value: String(data.domains_total), note: "数据库域名总量" }, { label: "总域名", value: String(data.domains_total), note: "数据库域名总量" },
{ label: "待检测", value: String(data.pending_total), note: "当前待处理域名" }, { label: "待检测", value: String(data.pending_total), note: "当前待处理域名" },
{ label: "黑名单", value: String(data.blacklist_total), note: "命中风险域名" }, { label: "黑名单", value: String(data.blacklist_total), note: "命中风险域名" },
{ label: "API", value: data.api_status, note: "当前 API 运行状态" }, { label: "API", value: data.api_status, note: "当前 API 运行状态" },
{ label: "本机 Worker", value: data.local_worker_status || data.worker_status, note: "当前节点本机 Worker 运行状态" }, { label: "本机 Worker", value: localWorkerValue, note: localWorkerNote },
{ {
label: "有效执行节点", label: "有效执行节点",
value: `${data.cluster_worker_status || "offline"} / ${data.cluster_online_worker_nodes || 0}`, value: `${data.cluster_worker_status || "offline"} / ${data.cluster_online_worker_nodes || 0}`,

View File

@@ -24,7 +24,9 @@
<div class="summary-card"> <div class="summary-card">
<div class="summary-title">本机 Worker</div> <div class="summary-title">本机 Worker</div>
<div class="summary-value">{{ runtime.worker?.service_name || "-" }}</div> <div class="summary-value">{{ runtime.worker?.service_name || "-" }}</div>
<el-tag :type="runtime.worker?.running ? 'success' : 'danger'">{{ runtime.worker?.running ? "本机在线" : "本机离线" }}</el-tag> <el-tag :type="localWorkerTagType">
{{ localWorkerSummaryText }}
</el-tag>
</div> </div>
<div class="summary-card"> <div class="summary-card">
<div class="summary-title">运行模式</div> <div class="summary-title">运行模式</div>
@@ -118,8 +120,8 @@
<span class="status-chip" :class="runtime.api?.pid ? 'ok' : 'neutral'"> <span class="status-chip" :class="runtime.api?.pid ? 'ok' : 'neutral'">
API {{ runtime.api?.pid ? "运行中" : "未知" }} API {{ runtime.api?.pid ? "运行中" : "未知" }}
</span> </span>
<span class="status-chip" :class="runtime.worker?.running ? 'ok' : 'danger'"> <span class="status-chip" :class="localWorkerChipClass">
本机 Worker {{ runtime.worker?.running ? "在线" : "离线" }} 本机 Worker {{ localWorkerStripText }}
</span> </span>
<span class="status-chip" :class="(runtime.cluster?.summary?.online_worker_nodes || 0) > 0 ? 'ok' : 'danger'"> <span class="status-chip" :class="(runtime.cluster?.summary?.online_worker_nodes || 0) > 0 ? 'ok' : 'danger'">
有效执行节点 {{ runtime.cluster?.summary?.online_worker_nodes || 0 }} 有效执行节点 {{ runtime.cluster?.summary?.online_worker_nodes || 0 }}
@@ -151,10 +153,10 @@
<el-descriptions-item label="API PID">{{ runtime.api?.pid || "-" }}</el-descriptions-item> <el-descriptions-item label="API PID">{{ runtime.api?.pid || "-" }}</el-descriptions-item>
<el-descriptions-item label="API 地址">{{ runtime.api?.health_url || "-" }}</el-descriptions-item> <el-descriptions-item label="API 地址">{{ runtime.api?.health_url || "-" }}</el-descriptions-item>
<el-descriptions-item label="Worker 服务">{{ runtime.worker?.service_name || "-" }}</el-descriptions-item> <el-descriptions-item label="Worker 服务">{{ runtime.worker?.service_name || "-" }}</el-descriptions-item>
<el-descriptions-item label="Worker 在线">{{ runtime.worker?.running ? "是" : "否" }}</el-descriptions-item> <el-descriptions-item label="Worker 在线">{{ localWorkerBooleanText }}</el-descriptions-item>
<el-descriptions-item label="运行模式">{{ runtime.worker?.mode || "-" }}</el-descriptions-item> <el-descriptions-item label="运行模式">{{ runtime.worker?.mode || "-" }}</el-descriptions-item>
<el-descriptions-item label="进程数">{{ runtime.worker?.process_count ?? "-" }}</el-descriptions-item> <el-descriptions-item label="进程数">{{ runtime.worker?.process_count ?? "-" }}</el-descriptions-item>
<el-descriptions-item label="运行消息">{{ runtime.worker?.message || "-" }}</el-descriptions-item> <el-descriptions-item label="运行消息">{{ localWorkerMessage }}</el-descriptions-item>
<el-descriptions-item label="Sync Agent 服务">{{ runtime.sync_agent?.service_name || "-" }}</el-descriptions-item> <el-descriptions-item label="Sync Agent 服务">{{ runtime.sync_agent?.service_name || "-" }}</el-descriptions-item>
<el-descriptions-item label="Sync Agent 在线">{{ runtime.sync_agent?.running ? "是" : "否" }}</el-descriptions-item> <el-descriptions-item label="Sync Agent 在线">{{ runtime.sync_agent?.running ? "是" : "否" }}</el-descriptions-item>
<el-descriptions-item label="Sync Agent 启动时间">{{ runtime.sync_agent?.latest_start_time || "-" }}</el-descriptions-item> <el-descriptions-item label="Sync Agent 启动时间">{{ runtime.sync_agent?.latest_start_time || "-" }}</el-descriptions-item>
@@ -166,8 +168,21 @@
<div class="action-panel"> <div class="action-panel">
<h3>运行时动作</h3> <h3>运行时动作</h3>
<div class="action-buttons"> <div class="action-buttons">
<el-button type="primary" :loading="actionLoading === 'start_worker'" @click="invokeAction('start_worker')">启动 Worker</el-button> <el-button
<el-button :loading="actionLoading === 'stop_worker'" @click="invokeAction('stop_worker')">停止 Worker</el-button> v-if="runtime.worker?.expected_on_this_node"
type="primary"
:loading="actionLoading === 'start_worker'"
@click="invokeAction('start_worker')"
>
启动 Worker
</el-button>
<el-button
v-if="runtime.worker?.expected_on_this_node"
:loading="actionLoading === 'stop_worker'"
@click="invokeAction('stop_worker')"
>
停止 Worker
</el-button>
<el-button type="warning" plain :loading="actionLoading === 'restart_api'" @click="invokeAction('restart_api')">重启 API</el-button> <el-button type="warning" plain :loading="actionLoading === 'restart_api'" @click="invokeAction('restart_api')">重启 API</el-button>
<el-button <el-button
v-if="runtime.sync_agent?.expected_on_this_node" v-if="runtime.sync_agent?.expected_on_this_node"
@@ -201,7 +216,15 @@
show-icon show-icon
type="info" type="info"
style="margin-top: 12px" style="margin-top: 12px"
:title="`当前本机 Worker${runtime.worker?.running ? '运行中' : '未运行'};有效执行节点:${runtime.cluster?.summary?.online_worker_nodes || 0};独立 Worker${runtime.cluster?.summary?.dedicated_online_worker_nodes || 0};当前 API 状态:${runtime.api?.pid ? '运行中' : '未知'}Sync Agent${runtime.sync_agent?.running ? '运行中' : '未运行'}。`" :title="`当前本机 Worker${localWorkerActionText};有效执行节点:${runtime.cluster?.summary?.online_worker_nodes || 0};独立 Worker${runtime.cluster?.summary?.dedicated_online_worker_nodes || 0};当前 API 状态:${runtime.api?.pid ? '运行中' : '未知'}Sync Agent${runtime.sync_agent?.running ? '运行中' : runtime.sync_agent?.expected_on_this_node ? '未运行' : '当前节点不承载'}。`"
/>
<el-alert
v-if="!runtime.worker?.expected_on_this_node"
:closable="false"
show-icon
type="info"
style="margin-top: 12px"
:title="`当前节点身份为 ${runtime.node?.region || '-'} / ${runtime.node?.role || '-'},按当前部署角色默认不承载本机 Worker检测执行应由大陆 controller / worker 节点承担。`"
/> />
<el-alert <el-alert
v-if="!runtime.sync_agent?.expected_on_this_node" v-if="!runtime.sync_agent?.expected_on_this_node"
@@ -800,6 +823,45 @@ const proxyDiagnosisText = () => {
return runtime.detect?.proxy_runtime_detail || "当前暂无代理运行说明"; return runtime.detect?.proxy_runtime_detail || "当前暂无代理运行说明";
}; };
const localWorkerExpected = computed(() => Boolean(runtime.worker?.expected_on_this_node ?? true));
const localWorkerSummaryText = computed(() => {
if (!localWorkerExpected.value) return "当前节点不承载";
return runtime.worker?.running ? "本机在线" : "本机离线";
});
const localWorkerStripText = computed(() => {
if (!localWorkerExpected.value) return "不承载";
return runtime.worker?.running ? "在线" : "离线";
});
const localWorkerBooleanText = computed(() => {
if (!localWorkerExpected.value) return "当前节点不承载";
return runtime.worker?.running ? "是" : "否";
});
const localWorkerMessage = computed(() => {
if (!localWorkerExpected.value) {
return `当前节点 ${runtime.node?.region || "-"} / ${runtime.node?.role || "-"},默认不承载本机 Worker`;
}
return runtime.worker?.message || "-";
});
const localWorkerActionText = computed(() => {
if (!localWorkerExpected.value) return "当前节点不承载";
return runtime.worker?.running ? "运行中" : "未运行";
});
const localWorkerTagType = computed(() => {
if (!localWorkerExpected.value) return "info";
return runtime.worker?.running ? "success" : "danger";
});
const localWorkerChipClass = computed(() => {
if (!localWorkerExpected.value) return "neutral";
return runtime.worker?.running ? "ok" : "danger";
});
const clusterIssueCount = computed(() => { const clusterIssueCount = computed(() => {
const summary = runtime.cluster?.summary || {}; const summary = runtime.cluster?.summary || {};
return Number(summary.status_counts?.stale || 0) + Number(summary.status_counts?.offline || 0); return Number(summary.status_counts?.stale || 0) + Number(summary.status_counts?.offline || 0);

View File

@@ -0,0 +1 @@
{}

View File

@@ -1,13 +1,13 @@
{ {
"proxy_enable": true, "proxy_enable": true,
"allow_direct": true, "allow_direct": true,
"proxy_url": "http://211.101.244.154:18008/getProxy_batch.php?group=B&count=50", "proxy_url": "http://211.101.244.154:18003/getProxy_batch.php?group=B&count=50",
"proxy_urls": [ "proxy_urls": [
"http://211.101.244.154:18008/getProxy_batch.php?group=B&count=50", "http://211.101.244.154:18003/getProxy_batch.php?group=B&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=C&count=50", "http://211.101.244.154:18003/getProxy_batch.php?group=C&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=D&count=50", "http://211.101.244.154:18003/getProxy_batch.php?group=D&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=E&count=50", "http://211.101.244.154:18003/getProxy_batch.php?group=E&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=F&count=50", "http://211.101.244.154:18008/getProxy_batch.php?group=F&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=G&count=50" "http://211.101.244.154:18003/getProxy_batch.php?group=G&count=50"
] ]
} }