docs(spider): record domestic bot recovery signals
This commit is contained in:
@@ -3847,6 +3847,92 @@ php code/scripts/seo_copy_front_verify.php chuanjiafeng.net category_list dian-y
|
||||
3. 当前最优模板文件是:
|
||||
- `前端站群服务器nginx配置/站点伪静态-受管蜘蛛放行版-动态页安全缓存.txt`
|
||||
|
||||
### 5.5 2026-04-17 晚间补充:国内蜘蛛有效窗口结论
|
||||
|
||||
这轮继续看蜘蛛日志时,发现:
|
||||
|
||||
1. `latest/spider-crawl.summary.json` 当前更像最新一轮 `ingest` 基础摘要
|
||||
2. 它没有展开成适合直接看趋势的聚合视角
|
||||
3. 所以 latest 看起来像“空壳”,不等于国内蜘蛛完全停抓
|
||||
|
||||
#### 手工筛选后的国内蜘蛛窗口
|
||||
|
||||
从最近约 180 个 `crawl-logs.summary.json` 中,只筛:
|
||||
|
||||
- `baiduspider`
|
||||
- `sogou`
|
||||
- `bytespider`
|
||||
|
||||
重新轻量聚合后,当前累计大致为:
|
||||
|
||||
1. 页面类型:
|
||||
- `home`:`144`
|
||||
- `other`:`96`
|
||||
- `robots`:`87`
|
||||
- `detail`:`37`
|
||||
- `category`:`34`
|
||||
- `play`:`27`
|
||||
2. 状态码:
|
||||
- `403`:`149`
|
||||
- `301`:`117`
|
||||
- `200`:`113`
|
||||
- `444`:`30`
|
||||
- `404`:`16`
|
||||
|
||||
#### 当前最重要的更新判断
|
||||
|
||||
1. 国内蜘蛛最近窗口里,`detail/play` 并没有消失
|
||||
2. `baiduspider` 深页修复方向仍成立,而且继续出现新的 `301 -> 200 MISS -> 200 HIT` 样本
|
||||
3. `sogou` 深页当前仍大量是 `403`
|
||||
4. 所以下一阶段不要把“百度恢复”和“搜狗恢复”混为一谈
|
||||
|
||||
#### 最近确认到的百度正向样本
|
||||
|
||||
1. `www.codohealth.com`
|
||||
- `/voddetail/gu-chuan-jin-zhen-qian-jin-jing-shi-feng-jian-lao-zu-zong-73272`
|
||||
- `200 HIT`
|
||||
2. `www.codohealth.com`
|
||||
- `/voddetail/xing-qiu-da-zhan-hei-shi-chuan-shuo-78631`
|
||||
- `301 -> 200 MISS -> 200 HIT`
|
||||
3. `sjzyunyang.com`
|
||||
- `/video-detail/jia-zheng-fu-san-tian-yuan-60513`
|
||||
- `301 -> 200 MISS -> 200 HIT`
|
||||
4. `gxhongzhuang.com`
|
||||
- `/voddetail/xi-xue-gui-ji-nv-184448`
|
||||
- `301 -> 200 MISS -> 200 HIT`
|
||||
5. `jingxifa.com`
|
||||
- `/video-detail/xing-jian-fu-guo-ji-di-wu-ji-107484`
|
||||
- `301 -> 200 MISS -> 200 HIT`
|
||||
6. `jingxifa.com`
|
||||
- `/video-bofang/hei-yi-nv-ren-de-xiang-shui-183894-default-1`
|
||||
- `301 -> 200 MISS -> 200 HIT`
|
||||
7. `sjzyunyang.com`
|
||||
- `/video-bofang/wu-lu-ke-tao-185896-default-1`
|
||||
- `301 -> 200 MISS -> 200 HIT`
|
||||
|
||||
#### 当前新增阻塞点
|
||||
|
||||
最近窗口里持续能看到:
|
||||
|
||||
1. `www.lgyz.net /vodplay/...`
|
||||
2. `www.vikau.com /vodplay/...`
|
||||
3. `www.vikau.com /voddetail/...`
|
||||
4. `www.jingxifa.com /video-detail/...`
|
||||
5. `www.jingxifa.com /video-bofang/...`
|
||||
|
||||
这些 `sogou` 深页当前大多表现为:
|
||||
|
||||
- `403`
|
||||
- `cache_status = NONE`
|
||||
|
||||
因此当前阶段更准确的表述是:
|
||||
|
||||
1. 页面恢复层面:成立
|
||||
2. 百度深页恢复层面:成立
|
||||
3. `latest` 空壳:只是展示层问题,不代表停抓
|
||||
4. 当前真正值得继续单独分析的新问题:
|
||||
- `sogou` 深页 `403`
|
||||
|
||||
### 6. 对后续 Codex 的明确提醒
|
||||
|
||||
后面如果再看到:
|
||||
|
||||
Reference in New Issue
Block a user