convert domainCheck to regular directory
This commit is contained in:
Submodule domainCheck deleted from 83ab7a79e8
12
domainCheck/.env.example
Normal file
12
domainCheck/.env.example
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
# 数据库配置
|
||||||
|
DB_HOST=localhost
|
||||||
|
DB_PORT=5432
|
||||||
|
DB_DATABASE=domain_scan_db
|
||||||
|
DB_USER=postgres
|
||||||
|
DB_PASSWORD=postgres
|
||||||
|
|
||||||
|
# Redis配置
|
||||||
|
REDIS_HOST=localhost
|
||||||
|
REDIS_PORT=6379
|
||||||
|
REDIS_PASSWORD=
|
||||||
|
REDIS_DB=0
|
||||||
10
domainCheck/.gitignore
vendored
Normal file
10
domainCheck/.gitignore
vendored
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
# 忽略整个目录
|
||||||
|
/node_modules/
|
||||||
|
/build/
|
||||||
|
/dist/
|
||||||
|
|
||||||
|
# 忽略所有压缩文件
|
||||||
|
*.spec
|
||||||
|
.env
|
||||||
|
.venv
|
||||||
|
.idea
|
||||||
221
domainCheck/DEPLOYMENT_GUIDE.md
Normal file
221
domainCheck/DEPLOYMENT_GUIDE.md
Normal file
@@ -0,0 +1,221 @@
|
|||||||
|
# 域名工具项目部署与使用文档
|
||||||
|
|
||||||
|
## 1. 项目概述
|
||||||
|
|
||||||
|
域名工具是一个用于域名筛选、检测和管理的综合性工具,主要功能包括:
|
||||||
|
- 域名批量导入
|
||||||
|
- 域名筛选与查询
|
||||||
|
- 域名状态检测(注册状态、使用状态、检测状态等)
|
||||||
|
- 域名详细信息检测(备案历史、快照年份、百度收录等)
|
||||||
|
- 域名数据导出(Excel、CSV)
|
||||||
|
- 批量更新域名状态
|
||||||
|
|
||||||
|
## 2. 环境要求
|
||||||
|
|
||||||
|
### 2.1 系统要求
|
||||||
|
- Windows 10/11 64位操作系统
|
||||||
|
- 至少4GB内存
|
||||||
|
- 至少500MB磁盘空间
|
||||||
|
|
||||||
|
### 2.2 软件依赖
|
||||||
|
- Python 3.9+
|
||||||
|
- PostgreSQL 12+
|
||||||
|
- Redis(用于缓存)+布隆过滤器(用于域名去重)
|
||||||
|
|
||||||
|
## 3. 安装步骤
|
||||||
|
|
||||||
|
### 3.1 克隆项目
|
||||||
|
```bash
|
||||||
|
git clone <项目仓库地址>
|
||||||
|
cd domainScanDemo
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 创建虚拟环境
|
||||||
|
```bash
|
||||||
|
python -m venv .venv
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.3 激活虚拟环境
|
||||||
|
```bash
|
||||||
|
# Windows
|
||||||
|
.venv\Scripts\activate
|
||||||
|
|
||||||
|
# Linux/Mac
|
||||||
|
source .venv/bin/activate
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.4 安装依赖
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.5 安装额外依赖(用于Excel导出)
|
||||||
|
```bash
|
||||||
|
pip install openpyxl
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. 配置说明
|
||||||
|
|
||||||
|
### 4.1 环境变量配置
|
||||||
|
|
||||||
|
复制 `.env.example` 文件为 `.env`,并根据实际情况修改配置:
|
||||||
|
|
||||||
|
```env
|
||||||
|
# 数据库连接信息
|
||||||
|
DATABASE_URL=postgresql://username:password@localhost:5432/domain_db
|
||||||
|
|
||||||
|
# Redis连接信息(可选)
|
||||||
|
REDIS_URL=redis://localhost:6379/0
|
||||||
|
|
||||||
|
# 日志配置
|
||||||
|
LOG_LEVEL=INFO
|
||||||
|
LOG_FILE=logs/app.log
|
||||||
|
|
||||||
|
# 线程配置
|
||||||
|
MAX_WORKERS=10
|
||||||
|
|
||||||
|
# 检测配置
|
||||||
|
DETECT_TIMEOUT=30
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 其他配置文件
|
||||||
|
|
||||||
|
- `detect_options.json` - 检测选项配置
|
||||||
|
- `proxy_config.json` - 代理配置
|
||||||
|
- `domain_suffixes.json` - 域名后缀配置
|
||||||
|
- `thread_count.json` - 线程数量配置
|
||||||
|
|
||||||
|
## 5. 数据库初始化
|
||||||
|
|
||||||
|
### 5.1 创建数据库
|
||||||
|
|
||||||
|
在PostgreSQL中创建数据库:
|
||||||
|
```sql
|
||||||
|
CREATE DATABASE domain_db;
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 初始化表结构
|
||||||
|
|
||||||
|
运行数据库初始化脚本:
|
||||||
|
```bash
|
||||||
|
python init_database.py
|
||||||
|
```
|
||||||
|
|
||||||
|
此脚本会创建以下表:
|
||||||
|
- `domains` - 域名基本信息表
|
||||||
|
- `domain_detections` - 域名检测详情表
|
||||||
|
- `sensitive_words` - 敏感词表
|
||||||
|
|
||||||
|
### 5.3 添加敏感词(可选)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python add_sensitive_words_table.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## 6. 运行项目
|
||||||
|
|
||||||
|
### 6.1 启动主应用
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python app/main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.2 启动检测端(可选,用于分布式检测)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python detect_worker.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## 7. 打包指南
|
||||||
|
|
||||||
|
### 7.1 打包主应用
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pyinstaller --onefile --windowed --add-data="app/sdk_leg.js;detect" --add-data="app/sdk_leg_env.js;detect" --add-data="app/chinaz.js;detect" --add-data="detect/module/crack_geetest2x.js;detect/module" --add-data="favicon.ico;." --add-data="new_logo.svg;." --add-data=".env;." --icon="favicon.ico" --name="域名工具" app/main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.2 打包检测端
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pyinstaller --onefile --windowed --add-data="detect/sdk_leg.js;detect" --add-data="detect/sdk_leg_env.js;detect" --add-data="detect/chinaz.js;detect" --add-data="detect/module/crack_geetest2x.js;detect/module" --add-data="favicon2.ico;." --add-data="new_logo.svg;." --add-data=".env;." --icon="favicon2.ico" --name="域名检测端" detect_worker.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.3 打包结果
|
||||||
|
|
||||||
|
打包完成后,可执行文件会生成在 `dist` 目录中:
|
||||||
|
- `域名工具.exe` - 主应用
|
||||||
|
- `域名检测端.exe` - 检测端
|
||||||
|
|
||||||
|
|
||||||
|
## 8. 使用说明
|
||||||
|
|
||||||
|
### 8.1 域名导入
|
||||||
|
|
||||||
|
1. 点击"域名导入"标签页
|
||||||
|
2. 选择导入方式(文件导入或聚名网爬取)
|
||||||
|
3. 选择文件或输入聚名网URL
|
||||||
|
4. 点击"开始导入"按钮
|
||||||
|
5. 等待导入完成
|
||||||
|
|
||||||
|
### 8.2 域名筛选
|
||||||
|
|
||||||
|
1. 点击"域名筛选"标签页
|
||||||
|
2. 设置筛选条件(注册状态、使用状态、检测状态等)
|
||||||
|
3. 点击"查询"按钮
|
||||||
|
4. 查看查询结果
|
||||||
|
5. 可选择域名进行批量更新或导出
|
||||||
|
|
||||||
|
### 8.3 域名检测
|
||||||
|
|
||||||
|
1. 确保检测端已启动
|
||||||
|
2. 在主应用中选择需要检测的域名
|
||||||
|
3. 点击"开始检测"按钮
|
||||||
|
4. 等待检测完成
|
||||||
|
5. 查看检测结果
|
||||||
|
|
||||||
|
### 8.4 数据导出
|
||||||
|
|
||||||
|
1. 在域名筛选界面选择需要导出的域名
|
||||||
|
2. 点击"导出"按钮
|
||||||
|
3. 选择导出格式(Excel或CSV)
|
||||||
|
4. 选择保存路径
|
||||||
|
5. 等待导出完成
|
||||||
|
|
||||||
|
## 9. 常见问题
|
||||||
|
|
||||||
|
### 9.1 数据库连接失败
|
||||||
|
|
||||||
|
- 检查 `.env` 文件中的数据库连接信息是否正确
|
||||||
|
- 确保 PostgreSQL 服务已启动
|
||||||
|
- 确保数据库用户有足够的权限
|
||||||
|
|
||||||
|
### 9.2 Redis 连接失败
|
||||||
|
|
||||||
|
- 检查 `.env` 文件中的 Redis 连接信息是否正确
|
||||||
|
- 确保 Redis 服务已启动
|
||||||
|
- 如果不需要 Redis 缓存,可以忽略此警告
|
||||||
|
|
||||||
|
### 9.3 检测失败
|
||||||
|
|
||||||
|
- 检查网络连接是否正常
|
||||||
|
- 检查代理配置是否正确
|
||||||
|
- 检查检测端是否已启动
|
||||||
|
|
||||||
|
### 9.4 导出失败
|
||||||
|
|
||||||
|
- 确保已安装 `openpyxl` 库(用于Excel导出)
|
||||||
|
- 确保导出路径有写入权限
|
||||||
|
- 检查导出文件格式是否正确
|
||||||
|
|
||||||
|
## 10. 技术支持
|
||||||
|
|
||||||
|
如果遇到问题,请查看日志文件 `logs/app.log` 获取详细错误信息,或联系技术支持。
|
||||||
|
|
||||||
|
## 11. 版本信息
|
||||||
|
|
||||||
|
- 项目版本:1.0.0
|
||||||
|
- 最后更新:2026-04-14
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**注意**:本文档仅供参考,具体配置和使用方法可能因环境不同而有所差异。
|
||||||
192
domainCheck/README_acw_sc__v2.md
Normal file
192
domainCheck/README_acw_sc__v2.md
Normal file
@@ -0,0 +1,192 @@
|
|||||||
|
# acw_sc__v2 生成器
|
||||||
|
|
||||||
|
从混淆的HTML代码中提取的 `acw_sc__v2` cookie 生成函数。
|
||||||
|
|
||||||
|
## 文件说明
|
||||||
|
|
||||||
|
### JavaScript版本
|
||||||
|
- **acw_sc__v2_generator.js** - JavaScript版本的生成函数,可直接在浏览器控制台中使用
|
||||||
|
|
||||||
|
### Python版本
|
||||||
|
- **acw_sc__v2_fixed.py** - 简化版Python生成器,适合快速使用
|
||||||
|
- **acw_sc__v2_complete.py** - 完整版Python生成器,包含更多功能和错误处理
|
||||||
|
- **acw_sc__v2_simple.py** - 最简化版本,直接使用硬编码密钥
|
||||||
|
|
||||||
|
## 使用方法
|
||||||
|
|
||||||
|
### JavaScript版本
|
||||||
|
|
||||||
|
在浏览器控制台中执行:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// 加载acw_sc__v2_generator.js文件后
|
||||||
|
var arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0';
|
||||||
|
var acwScV2 = generateAcwScV2(arg1);
|
||||||
|
console.log('acw_sc__v2的值:', acwScV2);
|
||||||
|
|
||||||
|
// 设置cookie(可选)
|
||||||
|
setAcwScV2Cookie(acwScV2);
|
||||||
|
```
|
||||||
|
|
||||||
|
### Python版本
|
||||||
|
|
||||||
|
#### 简化版(acw_sc__v2_fixed.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
from acw_sc__v2_fixed import generate_acw_sc_v2
|
||||||
|
|
||||||
|
arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0'
|
||||||
|
acw_sc_v2 = generate_acw_sc_v2(arg1)
|
||||||
|
print(f'acw_sc__v2的值: {acw_sc_v2}')
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 完整版(acw_sc__v2_complete.py)
|
||||||
|
|
||||||
|
```python
|
||||||
|
from acw_sc__v2_complete import AcwScV2Generator, generate_acw_sc_v2
|
||||||
|
|
||||||
|
# 方法1:使用便捷函数
|
||||||
|
arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0'
|
||||||
|
result = generate_acw_sc_v2(arg1)
|
||||||
|
print(f'acw_sc__v2的值: {result}')
|
||||||
|
|
||||||
|
# 方法2:使用生成器类
|
||||||
|
generator = AcwScV2Generator()
|
||||||
|
result = generator.generate(arg1)
|
||||||
|
print(f'acw_sc__v2的值: {result}')
|
||||||
|
|
||||||
|
# 方法3:批量生成
|
||||||
|
test_args = [
|
||||||
|
'1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0',
|
||||||
|
'A13D8B4174A10C57667C1CD92E54D01F449CA76E',
|
||||||
|
'F0286855D50CD9E96C1CDF3D4A913DB01EEB3321'
|
||||||
|
]
|
||||||
|
batch_results = generator.generate_batch(test_args)
|
||||||
|
for arg, res in batch_results.items():
|
||||||
|
print(f'{arg} -> {res}')
|
||||||
|
|
||||||
|
# 方法4:使用自定义密钥
|
||||||
|
custom_key = 'YXBwbHk=' # Base64编码的密钥
|
||||||
|
generator.set_key(custom_key)
|
||||||
|
result = generator.generate(arg1)
|
||||||
|
print(f'使用自定义密钥的acw_sc__v2值: {result}')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 核心算法
|
||||||
|
|
||||||
|
### 1. 字符重排
|
||||||
|
|
||||||
|
根据索引映射数组 `m` 重新排列输入字符串 `arg1`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
m = [0xf, 0x23, 0x1d, 0x18, 0x21, 0x10, 0x1, 0x26, 0xa, 0x9, 0x13, 0x1f, 0x28, 0x1b, 0x16, 0x17, 0x19, 0xd, 0x6, 0xb, 0x27, 0x12, 0x14, 0x8, 0xe, 0x15, 0x20, 0x1a, 0x2, 0x1e, 0x7, 0x4, 0x11, 0x5, 0x3, 0x1c, 0x22, 0x25, 0xc, 0x24]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. 密钥解码
|
||||||
|
|
||||||
|
将Base64编码的密钥解码为十六进制字符串:
|
||||||
|
|
||||||
|
```python
|
||||||
|
key_encoded = 'Aw5PDg9JAw4='
|
||||||
|
key_hex = base64.b64decode(key_encoded).hex()
|
||||||
|
# 结果: '030e4f0e0f49030e'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. 异或运算
|
||||||
|
|
||||||
|
将重排后的字符串与解码后的密钥进行按位异或:
|
||||||
|
|
||||||
|
```python
|
||||||
|
for x in range(0, min(len(u), len(p)), 2):
|
||||||
|
u_sub = u[x:x+2]
|
||||||
|
p_sub = p[x:x+2]
|
||||||
|
a = format(int(u_sub, 16) ^ int(p_sub, 16), '02x')
|
||||||
|
v += a
|
||||||
|
```
|
||||||
|
|
||||||
|
## 输入输出示例
|
||||||
|
|
||||||
|
### 输入
|
||||||
|
```python
|
||||||
|
arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 输出
|
||||||
|
```python
|
||||||
|
acw_sc__v2 = '5ad70a13004a0290'
|
||||||
|
```
|
||||||
|
|
||||||
|
## 注意事项
|
||||||
|
|
||||||
|
1. **输入长度**:`arg1` 必须是40位的十六进制字符串
|
||||||
|
2. **输入格式**:`arg1` 必须是有效的十六进制字符串
|
||||||
|
3. **密钥**:默认使用内置密钥,也可以自定义密钥
|
||||||
|
4. **输出格式**:输出为十六进制字符串
|
||||||
|
|
||||||
|
## 错误处理
|
||||||
|
|
||||||
|
完整版生成器包含以下错误处理:
|
||||||
|
|
||||||
|
- 输入长度验证
|
||||||
|
- 输入格式验证
|
||||||
|
- Base64解码错误处理
|
||||||
|
- 异常捕获和错误信息输出
|
||||||
|
|
||||||
|
## 应用场景
|
||||||
|
|
||||||
|
- 网站爬虫:绕过阿里云CDN的反爬虫保护
|
||||||
|
- 安全测试:测试网站的反爬虫机制
|
||||||
|
- 学习研究:了解混淆JavaScript代码的逆向工程
|
||||||
|
|
||||||
|
## 技术细节
|
||||||
|
|
||||||
|
### 混淆技术
|
||||||
|
|
||||||
|
原始HTML代码使用了多种混淆技术:
|
||||||
|
|
||||||
|
1. **字符串编码**:Base64编码 + URL编码
|
||||||
|
2. **变量名混淆**:使用无意义的变量名
|
||||||
|
3. **控制流混淆**:使用闭包和条件函数
|
||||||
|
4. **反调试**:通过数学计算和正则表达式检测调试器
|
||||||
|
|
||||||
|
### 逆向工程
|
||||||
|
|
||||||
|
通过分析原始代码,提取出核心算法:
|
||||||
|
|
||||||
|
1. 移除反调试逻辑
|
||||||
|
2. 简化字符串解码
|
||||||
|
3. 提取核心计算逻辑
|
||||||
|
4. 重构为可读代码
|
||||||
|
|
||||||
|
## 性能优化
|
||||||
|
|
||||||
|
- 使用缓存机制避免重复解码
|
||||||
|
- 批量生成功能提高效率
|
||||||
|
- 简化算法减少计算量
|
||||||
|
|
||||||
|
## 安全提示
|
||||||
|
|
||||||
|
本工具仅供学习和研究使用,请勿用于非法用途。使用本工具绕过网站的反爬虫保护可能违反网站的使用条款。
|
||||||
|
|
||||||
|
## 许可证
|
||||||
|
|
||||||
|
本代码仅供学习和研究使用。
|
||||||
|
|
||||||
|
## 联系方式
|
||||||
|
|
||||||
|
如有问题或建议,请通过以下方式联系:
|
||||||
|
|
||||||
|
- 提交Issue
|
||||||
|
- 发送Pull Request
|
||||||
|
- 联系作者
|
||||||
|
|
||||||
|
## 更新日志
|
||||||
|
|
||||||
|
### v1.0.0 (2024-01-01)
|
||||||
|
- 初始版本发布
|
||||||
|
- 支持JavaScript和Python版本
|
||||||
|
- 包含完整的使用示例
|
||||||
|
|
||||||
|
## 致谢
|
||||||
|
|
||||||
|
感谢所有为逆向工程和反爬虫技术研究做出贡献的开发者。
|
||||||
76
domainCheck/add_backlink_count_column.py
Normal file
76
domainCheck/add_backlink_count_column.py
Normal file
@@ -0,0 +1,76 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :add_backlink_count_column.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/10 03:50
|
||||||
|
@explain : 为domains表添加backlink_count字段
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
|
||||||
|
def add_backlink_count_column():
|
||||||
|
"""
|
||||||
|
为domains表添加backlink_count字段
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
print(f"成功连接到数据库: {config.DB_HOST}:{config.DB_PORT}/{config.DB_DATABASE}")
|
||||||
|
|
||||||
|
# 创建游标
|
||||||
|
cur = conn.cursor()
|
||||||
|
|
||||||
|
# 添加backlink_count字段
|
||||||
|
add_column_sql = """
|
||||||
|
ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0
|
||||||
|
"""
|
||||||
|
cur.execute(add_column_sql)
|
||||||
|
print("添加backlink_count字段成功")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
add_comment_sql = """
|
||||||
|
COMMENT ON COLUMN domains.backlink_count IS '友情链接数量'
|
||||||
|
"""
|
||||||
|
cur.execute(add_comment_sql)
|
||||||
|
print("添加字段注释成功")
|
||||||
|
|
||||||
|
# 创建索引
|
||||||
|
create_index_sql = """
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_domains_backlink_count ON domains(backlink_count)
|
||||||
|
"""
|
||||||
|
cur.execute(create_index_sql)
|
||||||
|
print("创建索引成功")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
print("事务提交成功")
|
||||||
|
|
||||||
|
# 关闭游标和连接
|
||||||
|
cur.close()
|
||||||
|
conn.close()
|
||||||
|
print("连接关闭成功")
|
||||||
|
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f"添加backlink_count字段失败: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
print("开始为domains表添加backlink_count字段...")
|
||||||
|
success = add_backlink_count_column()
|
||||||
|
if success:
|
||||||
|
print("backlink_count字段添加成功!")
|
||||||
|
else:
|
||||||
|
print("backlink_count字段添加失败!")
|
||||||
8
domainCheck/add_backlink_count_column.sql
Normal file
8
domainCheck/add_backlink_count_column.sql
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
-- 为domains表添加backlink_count字段
|
||||||
|
ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0;
|
||||||
|
|
||||||
|
-- 添加字段注释
|
||||||
|
COMMENT ON COLUMN domains.backlink_count IS '友情链接数量';
|
||||||
|
|
||||||
|
-- 创建索引
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_domains_backlink_count ON domains(backlink_count);
|
||||||
51
domainCheck/add_expire_date_field.py
Normal file
51
domainCheck/add_expire_date_field.py
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :add_expire_date_field.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/11 22:40
|
||||||
|
@explain : 为domains表添加expire_date字段
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from loguru import logger
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
def add_expire_date_field():
|
||||||
|
"""
|
||||||
|
为domains表添加expire_date字段
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
logger.info("数据库连接成功")
|
||||||
|
|
||||||
|
# 添加expire_date字段
|
||||||
|
try:
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS expire_date DATE")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.expire_date IS '过期时间'")
|
||||||
|
logger.info("添加expire_date字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"添加expire_date字段失败: {e}")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
logger.info("字段添加完成")
|
||||||
|
|
||||||
|
# 关闭连接
|
||||||
|
cur.close()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"执行失败: {e}")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
add_expire_date_field()
|
||||||
59
domainCheck/add_fields_to_domains.py
Normal file
59
domainCheck/add_fields_to_domains.py
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :add_fields_to_domains.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/11 22:40
|
||||||
|
@explain : 为domains表添加缺失的字段
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from loguru import logger
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
def add_fields_to_domains():
|
||||||
|
"""
|
||||||
|
为domains表添加缺失的字段
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
logger.info("数据库连接成功")
|
||||||
|
|
||||||
|
# 添加company_type字段
|
||||||
|
try:
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS company_type VARCHAR(100)")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.company_type IS '单位性质'")
|
||||||
|
logger.info("添加company_type字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"添加company_type字段失败: {e}")
|
||||||
|
|
||||||
|
# 添加website_url字段
|
||||||
|
try:
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS website_url VARCHAR(255)")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.website_url IS '网站首页网址'")
|
||||||
|
logger.info("添加website_url字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"添加website_url字段失败: {e}")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
logger.info("字段添加完成")
|
||||||
|
|
||||||
|
# 关闭连接
|
||||||
|
cur.close()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"执行失败: {e}")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
add_fields_to_domains()
|
||||||
58
domainCheck/add_sensitive_words_table.py
Normal file
58
domainCheck/add_sensitive_words_table.py
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :add_sensitive_words_table.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/12 00:03
|
||||||
|
@explain : 为数据库添加sensitive_words表
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from loguru import logger
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
def add_sensitive_words_table():
|
||||||
|
"""
|
||||||
|
为数据库添加sensitive_words表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
logger.info("数据库连接成功")
|
||||||
|
|
||||||
|
# 创建sensitive_words表
|
||||||
|
create_sensitive_words_table = """
|
||||||
|
CREATE TABLE IF NOT EXISTS sensitive_words (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
word VARCHAR(255) UNIQUE NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_sensitive_words_table)
|
||||||
|
logger.info("创建sensitive_words表成功")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.id IS '主键ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.word IS '敏感词'")
|
||||||
|
logger.info("添加sensitive_words表字段注释成功")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
logger.info("sensitive_words表添加完成")
|
||||||
|
|
||||||
|
# 关闭连接
|
||||||
|
cur.close()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"执行失败: {e}")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
add_sensitive_words_table()
|
||||||
6
domainCheck/app/__init__.py
Normal file
6
domainCheck/app/__init__.py
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
域名库系统
|
||||||
|
'''
|
||||||
|
|
||||||
|
__version__ = "1.0.0"
|
||||||
1257
domainCheck/app/chinaz.js
Normal file
1257
domainCheck/app/chinaz.js
Normal file
File diff suppressed because one or more lines are too long
265
domainCheck/app/config.py
Normal file
265
domainCheck/app/config.py
Normal file
@@ -0,0 +1,265 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :config.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:08
|
||||||
|
@explain : 系统配置
|
||||||
|
'''
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from dotenv import load_dotenv
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_echo(message):
|
||||||
|
"""
|
||||||
|
在不同 Windows 控制台编码下安全输出文本,避免导入阶段因中文打印失败。
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
print(message)
|
||||||
|
except UnicodeEncodeError:
|
||||||
|
try:
|
||||||
|
encoding = sys.stdout.encoding or 'utf-8'
|
||||||
|
sys.stdout.buffer.write((message + '\n').encode(encoding, errors='replace'))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# 确定基础目录
|
||||||
|
if hasattr(sys, '_MEIPASS'):
|
||||||
|
# PyInstaller 打包后的临时目录
|
||||||
|
BASE_DIR = sys._MEIPASS
|
||||||
|
else:
|
||||||
|
# 开发环境目录
|
||||||
|
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
|
||||||
|
# 加载环境变量
|
||||||
|
env_path = os.path.join(BASE_DIR, '.env')
|
||||||
|
if os.path.exists(env_path):
|
||||||
|
load_dotenv(env_path)
|
||||||
|
_safe_echo(f"成功加载环境变量文件: {env_path}")
|
||||||
|
else:
|
||||||
|
load_dotenv()
|
||||||
|
_safe_echo(f"环境变量文件不存在: {env_path},使用默认环境变量")
|
||||||
|
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
"""
|
||||||
|
系统配置
|
||||||
|
"""
|
||||||
|
|
||||||
|
# 数据库配置
|
||||||
|
DB_HOST = os.getenv('DB_HOST', 'localhost')
|
||||||
|
DB_PORT = int(os.getenv('DB_PORT', 5432))
|
||||||
|
DB_DATABASE = os.getenv('DB_DATABASE', 'domain_scan_db')
|
||||||
|
DB_USER = os.getenv('DB_USER', 'postgres')
|
||||||
|
DB_PASSWORD = os.getenv('DB_PASSWORD', 'postgres')
|
||||||
|
DB_POOL_SIZE = int(os.getenv('DB_POOL_SIZE', 5))
|
||||||
|
|
||||||
|
# 消息队列配置
|
||||||
|
RABBITMQ_HOST = os.getenv('RABBITMQ_HOST', 'localhost')
|
||||||
|
RABBITMQ_PORT = int(os.getenv('RABBITMQ_PORT', 5672))
|
||||||
|
RABBITMQ_USER = os.getenv('RABBITMQ_USER', 'guest')
|
||||||
|
RABBITMQ_PASSWORD = os.getenv('RABBITMQ_PASSWORD', 'guest')
|
||||||
|
RABBITMQ_VHOST = os.getenv('RABBITMQ_VHOST', '/')
|
||||||
|
|
||||||
|
# Redis配置
|
||||||
|
REDIS_HOST = os.getenv('REDIS_HOST', 'localhost')
|
||||||
|
REDIS_PORT = int(os.getenv('REDIS_PORT', 6379))
|
||||||
|
REDIS_PASSWORD = os.getenv('REDIS_PASSWORD', '')
|
||||||
|
REDIS_DB = int(os.getenv('REDIS_DB', 0))
|
||||||
|
|
||||||
|
# 聚名网配置
|
||||||
|
JUMING_COOKIE = os.getenv('JUMING_COOKIE', '')
|
||||||
|
JUMING_REFERER = os.getenv('JUMING_REFERER', 'https://www.juming.com/')
|
||||||
|
JUMING_USER_AGENT = os.getenv('JUMING_USER_AGENT', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36')
|
||||||
|
|
||||||
|
# 代理配置
|
||||||
|
PROXY_ENABLED = os.getenv('PROXY_ENABLED', 'false').lower() == 'true'
|
||||||
|
PROXY_URL = os.getenv('PROXY_URL', '')
|
||||||
|
|
||||||
|
# 检测配置
|
||||||
|
DETECT_TIMEOUT = int(os.getenv('DETECT_TIMEOUT', 30))
|
||||||
|
DETECT_RETRY_COUNT = int(os.getenv('DETECT_RETRY_COUNT', 3))
|
||||||
|
DETECT_CONCURRENCY = int(os.getenv('DETECT_CONCURRENCY', 10))
|
||||||
|
WAYBACK_CDX_TIMEOUT = int(os.getenv('WAYBACK_CDX_TIMEOUT', 15))
|
||||||
|
WAYBACK_SNAPSHOT_TIMEOUT = int(os.getenv('WAYBACK_SNAPSHOT_TIMEOUT', 12))
|
||||||
|
WAYBACK_RETRY_COUNT = int(os.getenv('WAYBACK_RETRY_COUNT', 2))
|
||||||
|
WAYBACK_REQUEST_DELAY = float(os.getenv('WAYBACK_REQUEST_DELAY', 0))
|
||||||
|
WAYBACK_PROGRESS_INTERVAL = int(os.getenv('WAYBACK_PROGRESS_INTERVAL', 500))
|
||||||
|
WAYBACK_DOMAIN_CONCURRENCY = int(os.getenv('WAYBACK_DOMAIN_CONCURRENCY', 3))
|
||||||
|
WAYBACK_TITLE_MAX_BYTES = int(os.getenv('WAYBACK_TITLE_MAX_BYTES', 65536))
|
||||||
|
WAYBACK_TIMESTAMP_CACHE_TTL = int(os.getenv('WAYBACK_TIMESTAMP_CACHE_TTL', 86400))
|
||||||
|
WAYBACK_TITLE_CACHE_TTL = int(os.getenv('WAYBACK_TITLE_CACHE_TTL', 2592000))
|
||||||
|
WAYBACK_USER_AGENT = os.getenv(
|
||||||
|
'WAYBACK_USER_AGENT',
|
||||||
|
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
)
|
||||||
|
|
||||||
|
# 域名配置
|
||||||
|
DOMAIN_TLDS = ['com', 'net']
|
||||||
|
DOMAIN_BATCH_SIZE = int(os.getenv('DOMAIN_BATCH_SIZE', 1000))
|
||||||
|
|
||||||
|
# 日志配置
|
||||||
|
LOG_LEVEL = os.getenv('LOG_LEVEL', 'INFO')
|
||||||
|
LOG_FILE = os.getenv('LOG_FILE', 'app.log')
|
||||||
|
|
||||||
|
# 任务配置
|
||||||
|
TASK_PRIORITY = {
|
||||||
|
'rdap': 10,
|
||||||
|
'wayback': 8,
|
||||||
|
'baidu': 6,
|
||||||
|
'qihu360': 5,
|
||||||
|
'google': 5,
|
||||||
|
'chinaz': 4,
|
||||||
|
'aizhan': 4,
|
||||||
|
'juziseo': 3,
|
||||||
|
'jucha': 3
|
||||||
|
}
|
||||||
|
|
||||||
|
# 敏感词配置 - 现在从数据库加载
|
||||||
|
SENSITIVE_WORDS = []
|
||||||
|
|
||||||
|
# 检测项配置
|
||||||
|
DETECT_ITEMS = {
|
||||||
|
'rdap': True,
|
||||||
|
'wayback': True,
|
||||||
|
'baidu': True,
|
||||||
|
'qihu360': True,
|
||||||
|
'google': True,
|
||||||
|
'chinaz': True,
|
||||||
|
'aizhan': True,
|
||||||
|
'juziseo': True,
|
||||||
|
'jucha': True
|
||||||
|
}
|
||||||
|
|
||||||
|
# 目录配置
|
||||||
|
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
DATA_DIR = os.path.join(BASE_DIR, 'data')
|
||||||
|
LOG_DIR = os.path.join(BASE_DIR, 'logs')
|
||||||
|
|
||||||
|
# 确保目录存在
|
||||||
|
os.makedirs(DATA_DIR, exist_ok=True)
|
||||||
|
os.makedirs(LOG_DIR, exist_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
# 从数据库加载敏感词
|
||||||
|
def load_sensitive_words():
|
||||||
|
"""
|
||||||
|
从数据库加载敏感词
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from app.utils.database import Database
|
||||||
|
db = Database()
|
||||||
|
sensitive_words = db.get_sensitive_words()
|
||||||
|
words = [word['word'] for word in sensitive_words]
|
||||||
|
db.close()
|
||||||
|
return words
|
||||||
|
except Exception as e:
|
||||||
|
_safe_echo(f"加载敏感词失败: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
# 从文件加载检测选项
|
||||||
|
def load_detect_options():
|
||||||
|
"""
|
||||||
|
从文件加载检测选项
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
default_order = [
|
||||||
|
'detect_register',
|
||||||
|
'detect_baidu_site',
|
||||||
|
'detect_360_site',
|
||||||
|
'detect_chinaz',
|
||||||
|
'detect_aizhan',
|
||||||
|
'detect_wayback',
|
||||||
|
'detect_jucha',
|
||||||
|
'detect_juziseo',
|
||||||
|
]
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
defaults = {
|
||||||
|
'detect_register': True,
|
||||||
|
'detect_wayback': True,
|
||||||
|
'detect_chinaz': True,
|
||||||
|
'detect_aizhan': True,
|
||||||
|
'detect_baidu_site': True,
|
||||||
|
'detect_360_site': True,
|
||||||
|
'detect_jucha': False,
|
||||||
|
'detect_juziseo': False,
|
||||||
|
'detect_order': default_order,
|
||||||
|
}
|
||||||
|
if os.path.exists('detect_options.json'):
|
||||||
|
with open('detect_options.json', 'r', encoding='utf-8') as f:
|
||||||
|
detect_options = json.load(f)
|
||||||
|
defaults.update(detect_options)
|
||||||
|
if defaults.get('detect_whois') or defaults.get('detect_beian') or defaults.get('detect_intercept'):
|
||||||
|
defaults['detect_jucha'] = True
|
||||||
|
if defaults.get('detect_juziseo_outlink'):
|
||||||
|
defaults['detect_juziseo'] = True
|
||||||
|
order = defaults.get('detect_order') or []
|
||||||
|
normalized_order = [key for key in order if key in default_order]
|
||||||
|
for key in default_order:
|
||||||
|
if key not in normalized_order:
|
||||||
|
normalized_order.append(key)
|
||||||
|
defaults['detect_order'] = normalized_order
|
||||||
|
return defaults
|
||||||
|
except Exception as e:
|
||||||
|
_safe_echo(f"加载检测选项失败: {e}")
|
||||||
|
return {
|
||||||
|
'detect_register': True,
|
||||||
|
'detect_wayback': True,
|
||||||
|
'detect_chinaz': True,
|
||||||
|
'detect_aizhan': True,
|
||||||
|
'detect_baidu_site': True,
|
||||||
|
'detect_360_site': True,
|
||||||
|
'detect_jucha': False,
|
||||||
|
'detect_juziseo': False,
|
||||||
|
}
|
||||||
|
|
||||||
|
# 导出配置
|
||||||
|
config = Config()
|
||||||
|
# 加载检测选项
|
||||||
|
config.DETECT_OPTIONS = load_detect_options()
|
||||||
|
|
||||||
|
# 延迟加载敏感词,避免循环导入
|
||||||
|
def load_sensitive_words_lazy():
|
||||||
|
"""
|
||||||
|
延迟加载敏感词
|
||||||
|
"""
|
||||||
|
return load_sensitive_words()
|
||||||
|
|
||||||
|
# 设置敏感词属性为延迟加载
|
||||||
|
config.load_sensitive_words = load_sensitive_words_lazy
|
||||||
|
|
||||||
|
# 检查检测类型是否应该执行
|
||||||
|
def should_detect(detect_type):
|
||||||
|
"""
|
||||||
|
检查检测类型是否应该执行
|
||||||
|
|
||||||
|
:param detect_type: 检测类型
|
||||||
|
:return: bool - 是否应该执行
|
||||||
|
"""
|
||||||
|
# 检测类型映射
|
||||||
|
type_mapping = {
|
||||||
|
'register': 'detect_register',
|
||||||
|
'wayback': 'detect_wayback',
|
||||||
|
'chinaz': 'detect_chinaz',
|
||||||
|
'aizhan': 'detect_aizhan',
|
||||||
|
'baidu_site': 'detect_baidu_site',
|
||||||
|
'360_site': 'detect_360_site',
|
||||||
|
'whois': 'detect_jucha',
|
||||||
|
'beian': 'detect_jucha',
|
||||||
|
'intercept': 'detect_jucha',
|
||||||
|
'jucha': 'detect_jucha',
|
||||||
|
'juziseo': 'detect_juziseo',
|
||||||
|
'juziseo_outlink': 'detect_juziseo'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 获取对应的配置键
|
||||||
|
config_key = type_mapping.get(detect_type)
|
||||||
|
if not config_key:
|
||||||
|
return True # 默认执行
|
||||||
|
|
||||||
|
# 检查是否在配置中,默认为True
|
||||||
|
return config.DETECT_OPTIONS.get(config_key, True)
|
||||||
0
domainCheck/app/config/sensitive_words.txt
Normal file
0
domainCheck/app/config/sensitive_words.txt
Normal file
4
domainCheck/app/core/__init__.py
Normal file
4
domainCheck/app/core/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
核心功能模块
|
||||||
|
'''
|
||||||
286
domainCheck/app/core/detect_engine.py
Normal file
286
domainCheck/app/core/detect_engine.py
Normal file
@@ -0,0 +1,286 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :detect_engine.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:52
|
||||||
|
@explain : 检测引擎
|
||||||
|
'''
|
||||||
|
|
||||||
|
import time
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.database import Database
|
||||||
|
from app.detectors.rdap_detector import RDAPDetector
|
||||||
|
from app.detectors.wayback_detector import WaybackDetector
|
||||||
|
from app.detectors.baidu_detector import BaiduDetector
|
||||||
|
from app.detectors.qihu360_detector import Qihu360Detector
|
||||||
|
from app.detectors.google_detector import GoogleDetector
|
||||||
|
from app.detectors.chinaz_detector import ChinazDetector
|
||||||
|
from app.detectors.aizhan_detector import AizhanDetector
|
||||||
|
from app.detectors.juziseo_detector import JuziseoDetector
|
||||||
|
from app.detectors.jucha_detector import JuchaDetector
|
||||||
|
from app.utils.status_codes import (
|
||||||
|
DETECT_STATUS_BLACKLISTED,
|
||||||
|
DETECT_STATUS_COMPLETED,
|
||||||
|
DETECT_STATUS_FAILED,
|
||||||
|
DETECT_STATUS_RUNNING,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class DetectEngine:
|
||||||
|
"""
|
||||||
|
检测引擎
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化检测引擎
|
||||||
|
"""
|
||||||
|
self.db = Database()
|
||||||
|
self.rdap_detector = RDAPDetector()
|
||||||
|
self.wayback_detector = WaybackDetector()
|
||||||
|
self.baidu_detector = BaiduDetector()
|
||||||
|
self.qihu360_detector = Qihu360Detector()
|
||||||
|
self.google_detector = GoogleDetector()
|
||||||
|
self.chinaz_detector = ChinazDetector()
|
||||||
|
self.aizhan_detector = AizhanDetector()
|
||||||
|
self.juziseo_detector = JuziseoDetector()
|
||||||
|
self.jucha_detector = JuchaDetector()
|
||||||
|
|
||||||
|
def detect_domain(self, domain_id):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:return: bool - 是否检测成功
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 获取域名信息
|
||||||
|
domain_info = self.db.get_domain_by_id(domain_id)
|
||||||
|
if not domain_info:
|
||||||
|
logger.error(f"域名不存在: {domain_id}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
domain = domain_info['domain']
|
||||||
|
logger.info(f"开始检测域名: {domain}")
|
||||||
|
|
||||||
|
# 更新检测状态为检测中
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_RUNNING)
|
||||||
|
|
||||||
|
# 1. 基础检测
|
||||||
|
if not self._basic_detect(domain_id, domain):
|
||||||
|
logger.info(f"基础检测失败,停止后续检测: {domain}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 2. 深度检测
|
||||||
|
if not self._deep_detect(domain_id, domain):
|
||||||
|
logger.info(f"深度检测失败: {domain}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 更新检测状态为正常
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_COMPLETED)
|
||||||
|
logger.info(f"域名检测完成: {domain}")
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"检测域名出错: {e}")
|
||||||
|
# 更新检测状态为检测失败
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_FAILED)
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _basic_detect(self, domain_id, domain):
|
||||||
|
"""
|
||||||
|
基础检测
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:param domain: 域名
|
||||||
|
:return: bool - 是否检测通过
|
||||||
|
"""
|
||||||
|
# 1. 检查是否为一口价域名
|
||||||
|
is_ykj = self.db.is_ykj_domain(domain_id)
|
||||||
|
|
||||||
|
# 2. 注册状态检测(一口价域名跳过)
|
||||||
|
if not is_ykj:
|
||||||
|
register_status = self.rdap_detector.check_register_status(domain)
|
||||||
|
self.db.update_domain_register_status(domain_id, register_status)
|
||||||
|
|
||||||
|
# 3. 黑名单缓存检查
|
||||||
|
if self.db.is_blacklisted(domain):
|
||||||
|
logger.info(f"域名在黑名单中: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 4. 时光机快照年份采集
|
||||||
|
snapshot_years = self.wayback_detector.get_snapshot_years(domain)
|
||||||
|
if snapshot_years:
|
||||||
|
self.db.update_domain_snapshot_years(domain_id, ','.join(map(str, snapshot_years)))
|
||||||
|
|
||||||
|
# 5. 时光机正文抽样与敏感词匹配
|
||||||
|
if self.wayback_detector.has_sensitive_content(domain):
|
||||||
|
logger.info(f"域名包含敏感词: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
self.db.add_to_blacklist(domain, "快照包含敏感词")
|
||||||
|
return False
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _deep_detect(self, domain_id, domain):
|
||||||
|
"""
|
||||||
|
深度检测
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:param domain: 域名
|
||||||
|
:return: bool - 是否检测通过
|
||||||
|
"""
|
||||||
|
# 1. 百度历史/Site
|
||||||
|
baidu_history = self.baidu_detector.check_history(domain)
|
||||||
|
baidu_site = self.baidu_detector.check_site(domain)
|
||||||
|
|
||||||
|
# 2. 360 Site
|
||||||
|
qihu360_site = self.qihu360_detector.check_site(domain)
|
||||||
|
|
||||||
|
# 3. Google Site
|
||||||
|
google_site = self.google_detector.check_site(domain)
|
||||||
|
|
||||||
|
# 4. 站长之家
|
||||||
|
chinaz_info = self.chinaz_detector.check_domain(domain)
|
||||||
|
|
||||||
|
# 5. 爱站网
|
||||||
|
aizhan_info = self.aizhan_detector.check_domain(domain)
|
||||||
|
|
||||||
|
# 6. 桔子SEO
|
||||||
|
juziseo_info = self.juziseo_detector.check_domain(domain)
|
||||||
|
|
||||||
|
# 7. 聚查
|
||||||
|
jucha_info = self.jucha_detector.check_domain(domain)
|
||||||
|
|
||||||
|
# 检查是否有风险
|
||||||
|
if self._check_risk(domain_id, domain, baidu_history, baidu_site, qihu360_site, google_site, chinaz_info, aizhan_info, juziseo_info, jucha_info):
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 保存检测结果
|
||||||
|
self.db.add_detection_result(domain_id, baidu_history, baidu_site, qihu360_site, google_site, chinaz_info, aizhan_info, juziseo_info, jucha_info)
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _check_risk(self, domain_id, domain, baidu_history, baidu_site, qihu360_site, google_site, chinaz_info, aizhan_info, juziseo_info, jucha_info):
|
||||||
|
"""
|
||||||
|
检查风险
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:param domain: 域名
|
||||||
|
:param baidu_history: 百度历史
|
||||||
|
:param baidu_site: 百度site
|
||||||
|
:param qihu360_site: 360 site
|
||||||
|
:param google_site: Google site
|
||||||
|
:param chinaz_info: 站长之家信息
|
||||||
|
:param aizhan_info: 爱站网信息
|
||||||
|
:param juziseo_info: 桔子SEO信息
|
||||||
|
:param jucha_info: 聚查信息
|
||||||
|
:return: bool - 是否有风险
|
||||||
|
"""
|
||||||
|
# 检查百度历史过灰
|
||||||
|
if baidu_history and '灰' in str(baidu_history):
|
||||||
|
logger.info(f"百度历史过灰: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
self.db.add_to_blacklist(domain, "百度历史过灰")
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查标题敏感词
|
||||||
|
if chinaz_info and 'title' in chinaz_info:
|
||||||
|
if self._contains_sensitive_words(chinaz_info['title']):
|
||||||
|
logger.info(f"标题包含敏感词: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
self.db.add_to_blacklist(domain, "标题包含敏感词")
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查子域名
|
||||||
|
if baidu_site and 'subdomains' in baidu_site:
|
||||||
|
subdomains = baidu_site['subdomains']
|
||||||
|
# 排除 www, @, m
|
||||||
|
valid_subdomains = [sub for sub in subdomains if sub not in ['www', '@', 'm']]
|
||||||
|
if valid_subdomains:
|
||||||
|
logger.info(f"存在子域名: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
self.db.add_to_blacklist(domain, "存在子域名")
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查风险提示
|
||||||
|
if aizhan_info and 'risk' in aizhan_info:
|
||||||
|
if aizhan_info['risk'] in ['低风险', '疑似色情博彩风险', '严重影响权重']:
|
||||||
|
logger.info(f"风险提示: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
self.db.add_to_blacklist(domain, f"风险提示: {aizhan_info['risk']}")
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查WHOIS状态
|
||||||
|
if jucha_info and 'whois' in jucha_info:
|
||||||
|
if jucha_info['whois'].get('status') in ['clientHold', 'serverHold']:
|
||||||
|
logger.info(f"WHOIS状态异常: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
|
||||||
|
self.db.add_to_blacklist(domain, "WHOIS状态异常")
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查拦截检测
|
||||||
|
if jucha_info and 'intercept' in jucha_info:
|
||||||
|
if not jucha_info['intercept'].get('normal', True):
|
||||||
|
logger.info(f"拦截检测异常: {domain}")
|
||||||
|
self.db.update_domain_detect_status(domain_id, 4) # 4 表示黑名单
|
||||||
|
self.db.add_to_blacklist(domain, "拦截检测异常")
|
||||||
|
return True
|
||||||
|
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _contains_sensitive_words(self, text):
|
||||||
|
"""
|
||||||
|
检查文本是否包含敏感词
|
||||||
|
|
||||||
|
:param text: 文本
|
||||||
|
:return: bool - 是否包含敏感词
|
||||||
|
"""
|
||||||
|
sensitive_words = self.db.get_all_sensitive_words()
|
||||||
|
for word in sensitive_words:
|
||||||
|
if word in text:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
def process_task(self, task_id):
|
||||||
|
"""
|
||||||
|
处理检测任务
|
||||||
|
|
||||||
|
:param task_id: 任务ID
|
||||||
|
:return: bool - 是否处理成功
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 获取任务信息
|
||||||
|
task = self.db.get_task_by_id(task_id)
|
||||||
|
if not task:
|
||||||
|
logger.error(f"任务不存在: {task_id}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
domain_id = task['domain_id']
|
||||||
|
|
||||||
|
# 更新任务状态为执行中
|
||||||
|
self.db.update_task_status(task_id, 1) # 1 表示执行中
|
||||||
|
|
||||||
|
# 执行检测
|
||||||
|
success = self.detect_domain(domain_id)
|
||||||
|
|
||||||
|
# 更新任务状态
|
||||||
|
if success:
|
||||||
|
self.db.update_task_status(task_id, 2) # 2 表示完成
|
||||||
|
else:
|
||||||
|
# 增加重试次数
|
||||||
|
retry_count = task.get('retry_count', 0) + 1
|
||||||
|
if retry_count < 3:
|
||||||
|
self.db.update_task_retry_count(task_id, retry_count)
|
||||||
|
self.db.update_task_status(task_id, 0) # 0 表示待执行
|
||||||
|
else:
|
||||||
|
self.db.update_task_status(task_id, 3) # 3 表示失败
|
||||||
|
|
||||||
|
return success
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"处理任务出错: {e}")
|
||||||
|
# 更新任务状态为失败
|
||||||
|
self.db.update_task_status(task_id, 3) # 3 表示失败
|
||||||
|
return False
|
||||||
261
domainCheck/app/core/domain_collector.py
Normal file
261
domainCheck/app/core/domain_collector.py
Normal file
@@ -0,0 +1,261 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :domain_collector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:50
|
||||||
|
@explain : 域名收集器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import re
|
||||||
|
import tldextract
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.database import Database
|
||||||
|
from app.utils.domain_utils import normalize_domain
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
|
||||||
|
class DomainCollector:
|
||||||
|
"""
|
||||||
|
域名收集器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化域名收集器
|
||||||
|
"""
|
||||||
|
self.db = Database(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
|
||||||
|
def add_domain(self, domain, source_type):
|
||||||
|
"""
|
||||||
|
添加域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:param source_type: 来源类型
|
||||||
|
:return: bool - 是否添加成功
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 标准化域名
|
||||||
|
normalized_domain = normalize_domain(domain)
|
||||||
|
if not normalized_domain:
|
||||||
|
logger.warning(f"无效域名: {domain}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 提取顶级域名
|
||||||
|
ext = tldextract.extract(normalized_domain)
|
||||||
|
tld = ext.suffix
|
||||||
|
|
||||||
|
# 只保留 .com 和 .net
|
||||||
|
if tld not in ['com', 'net']:
|
||||||
|
logger.warning(f"不支持的顶级域名: {tld}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 检查是否已存在
|
||||||
|
if self.db.domain_exists(normalized_domain):
|
||||||
|
logger.info(f"域名已存在: {normalized_domain}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# 添加域名
|
||||||
|
domain_id = self.db.add_domain(normalized_domain, tld, source_type)
|
||||||
|
if domain_id:
|
||||||
|
# 创建检测任务
|
||||||
|
self.db.create_detect_task(domain_id, 1) # 1 表示基础检测
|
||||||
|
logger.info(f"成功添加域名: {normalized_domain}")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
logger.error(f"添加域名失败: {normalized_domain}")
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"添加域名出错: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def add_domains_batch(self, domains, source_type, batch_size=1000, dry_run=False):
|
||||||
|
"""
|
||||||
|
批量添加域名
|
||||||
|
|
||||||
|
:param domains: 域名列表
|
||||||
|
:param source_type: 来源类型
|
||||||
|
:param batch_size: 批量大小
|
||||||
|
:param dry_run: 是否仅进行干运行(不实际添加域名)
|
||||||
|
:return: dict - 统计信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 统计信息
|
||||||
|
stats = {
|
||||||
|
'total': len(domains),
|
||||||
|
'valid': 0,
|
||||||
|
'added': 0,
|
||||||
|
'exists': 0,
|
||||||
|
'invalid': 0,
|
||||||
|
'failed': 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# 处理大规模数据时,分批进行标准化和过滤
|
||||||
|
normalized_domains = []
|
||||||
|
batch_domains = []
|
||||||
|
|
||||||
|
for i, domain in enumerate(domains):
|
||||||
|
normalized = normalize_domain(domain)
|
||||||
|
if not normalized:
|
||||||
|
stats['invalid'] += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
# 提取顶级域名
|
||||||
|
ext = tldextract.extract(normalized)
|
||||||
|
tld = ext.suffix
|
||||||
|
|
||||||
|
# 只保留 .com 和 .net
|
||||||
|
if tld not in ['com', 'net']:
|
||||||
|
stats['invalid'] += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
normalized_domains.append((normalized, tld))
|
||||||
|
batch_domains.append(normalized)
|
||||||
|
stats['valid'] += 1
|
||||||
|
|
||||||
|
# 每1000个域名检查一次,避免内存占用过高
|
||||||
|
if (i + 1) % 1000 == 0:
|
||||||
|
logger.info(f"已处理 {i + 1}/{len(domains)} 个域名")
|
||||||
|
|
||||||
|
logger.info(f"域名标准化完成,有效域名: {stats['valid']}")
|
||||||
|
|
||||||
|
# 提取所有域名
|
||||||
|
all_domains = [domain for domain, tld in normalized_domains]
|
||||||
|
|
||||||
|
# 批量检查域名是否存在
|
||||||
|
existing_domains = self.db.check_domains_exist(all_domains)
|
||||||
|
existing_set = set(existing_domains)
|
||||||
|
|
||||||
|
# 准备批量添加数据
|
||||||
|
batch_data = []
|
||||||
|
for domain, tld in normalized_domains:
|
||||||
|
if domain not in existing_set:
|
||||||
|
batch_data.append((domain, tld, source_type))
|
||||||
|
|
||||||
|
stats['exists'] = len(existing_domains)
|
||||||
|
stats['valid'] = len(normalized_domains)
|
||||||
|
|
||||||
|
# 干运行模式下直接返回统计信息
|
||||||
|
if dry_run:
|
||||||
|
stats['added'] = len(batch_data)
|
||||||
|
logger.info(f"干运行模式:准备添加 {len(batch_data)} 个新域名")
|
||||||
|
return stats
|
||||||
|
|
||||||
|
logger.info(f"准备添加 {len(batch_data)} 个新域名")
|
||||||
|
|
||||||
|
# 分批次添加
|
||||||
|
for i in range(0, len(batch_data), batch_size):
|
||||||
|
batch = batch_data[i:i+batch_size]
|
||||||
|
added_count = self.db.add_domains_batch(batch)
|
||||||
|
stats['added'] += added_count
|
||||||
|
stats['failed'] += len(batch) - added_count
|
||||||
|
|
||||||
|
# 每处理一批,记录一次进度
|
||||||
|
if (i + len(batch)) % (batch_size * 10) == 0:
|
||||||
|
logger.info(f"已添加 {i + len(batch)}/{len(batch_data)} 个域名")
|
||||||
|
|
||||||
|
logger.info(f"批量添加域名完成: {stats}")
|
||||||
|
return stats
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"批量添加域名出错: {e}")
|
||||||
|
return {
|
||||||
|
'total': len(domains),
|
||||||
|
'valid': 0,
|
||||||
|
'added': 0,
|
||||||
|
'exists': 0,
|
||||||
|
'invalid': 0,
|
||||||
|
'failed': len(domains)
|
||||||
|
}
|
||||||
|
|
||||||
|
def import_from_file(self, file_path, source_type, batch_size=1000):
|
||||||
|
"""
|
||||||
|
从文件导入域名
|
||||||
|
|
||||||
|
:param file_path: 文件路径
|
||||||
|
:param source_type: 来源类型
|
||||||
|
:param batch_size: 批量大小
|
||||||
|
:return: dict - 导入统计信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with open(file_path, 'r', encoding='utf-8') as f:
|
||||||
|
domains = f.readlines()
|
||||||
|
|
||||||
|
# 提取域名
|
||||||
|
domain_list = []
|
||||||
|
for domain in domains:
|
||||||
|
domain = domain.strip()
|
||||||
|
if domain:
|
||||||
|
domain_list.append(domain)
|
||||||
|
|
||||||
|
# 批量添加域名
|
||||||
|
stats = self.add_domains_batch(domain_list, source_type, batch_size)
|
||||||
|
|
||||||
|
logger.info(f"从文件导入完成: {stats}")
|
||||||
|
return stats
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"从文件导入出错: {e}")
|
||||||
|
return {
|
||||||
|
'total': 0,
|
||||||
|
'valid': 0,
|
||||||
|
'added': 0,
|
||||||
|
'exists': 0,
|
||||||
|
'invalid': 0,
|
||||||
|
'failed': 0
|
||||||
|
}
|
||||||
|
|
||||||
|
def collect_from_juming(self, type_='一口价'):
|
||||||
|
"""
|
||||||
|
从聚名网收集域名
|
||||||
|
|
||||||
|
:param type_: 类型,一口价或过期删除
|
||||||
|
:return: int - 收集到的域名数量
|
||||||
|
"""
|
||||||
|
# 这里可以添加从聚名网收集域名的逻辑
|
||||||
|
logger.info(f"从聚名网收集 {type_} 域名")
|
||||||
|
# 模拟收集结果
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def collect_from_search_engine(self, keyword, limit=100):
|
||||||
|
"""
|
||||||
|
从搜索引擎收集域名
|
||||||
|
|
||||||
|
:param keyword: 关键词
|
||||||
|
:param limit: 限制数量
|
||||||
|
:return: int - 收集到的域名数量
|
||||||
|
"""
|
||||||
|
# 这里可以添加从搜索引擎收集域名的逻辑
|
||||||
|
logger.info(f"从搜索引擎收集域名,关键词: {keyword}, 限制: {limit}")
|
||||||
|
# 模拟收集结果
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def collect_from_enterprise_directory(self, url, limit=100):
|
||||||
|
"""
|
||||||
|
从企业目录收集域名
|
||||||
|
|
||||||
|
:param url: 企业目录URL
|
||||||
|
:param limit: 限制数量
|
||||||
|
:return: int - 收集到的域名数量
|
||||||
|
"""
|
||||||
|
# 这里可以添加从企业目录收集域名的逻辑
|
||||||
|
logger.info(f"从企业目录收集域名,URL: {url}, 限制: {limit}")
|
||||||
|
# 模拟收集结果
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def collect_from_zone_file(self, file_path):
|
||||||
|
"""
|
||||||
|
从Zone File收集域名
|
||||||
|
|
||||||
|
:param file_path: Zone File路径
|
||||||
|
:return: int - 收集到的域名数量
|
||||||
|
"""
|
||||||
|
# 这里可以添加从Zone File收集域名的逻辑
|
||||||
|
logger.info(f"从Zone File收集域名,文件: {file_path}")
|
||||||
|
# 模拟收集结果
|
||||||
|
return 0
|
||||||
150
domainCheck/app/core/domain_processor.py
Normal file
150
domainCheck/app/core/domain_processor.py
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :domain_processor.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:51
|
||||||
|
@explain : 域名处理器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import re
|
||||||
|
import tldextract
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.database import Database
|
||||||
|
from app.utils.domain_utils import normalize_domain
|
||||||
|
|
||||||
|
|
||||||
|
class DomainProcessor:
|
||||||
|
"""
|
||||||
|
域名处理器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化域名处理器
|
||||||
|
"""
|
||||||
|
self.db = Database()
|
||||||
|
|
||||||
|
def process_domain(self, domain):
|
||||||
|
"""
|
||||||
|
处理域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 处理结果
|
||||||
|
"""
|
||||||
|
result = {
|
||||||
|
'original': domain,
|
||||||
|
'normalized': None,
|
||||||
|
'tld': None,
|
||||||
|
'valid': False,
|
||||||
|
'reason': ''
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
# 标准化域名
|
||||||
|
normalized = normalize_domain(domain)
|
||||||
|
if not normalized:
|
||||||
|
result['reason'] = '无效域名格式'
|
||||||
|
return result
|
||||||
|
|
||||||
|
# 提取顶级域名
|
||||||
|
ext = tldextract.extract(normalized)
|
||||||
|
tld = ext.suffix
|
||||||
|
|
||||||
|
# 检查顶级域名
|
||||||
|
if tld not in ['com', 'net']:
|
||||||
|
result['reason'] = '不支持的顶级域名'
|
||||||
|
return result
|
||||||
|
|
||||||
|
result['normalized'] = normalized
|
||||||
|
result['tld'] = tld
|
||||||
|
result['valid'] = True
|
||||||
|
|
||||||
|
return result
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"处理域名出错: {e}")
|
||||||
|
result['reason'] = f"处理出错: {str(e)}"
|
||||||
|
return result
|
||||||
|
|
||||||
|
def batch_process(self, domains):
|
||||||
|
"""
|
||||||
|
批量处理域名
|
||||||
|
|
||||||
|
:param domains: 域名列表
|
||||||
|
:return: list - 处理结果列表
|
||||||
|
"""
|
||||||
|
results = []
|
||||||
|
for domain in domains:
|
||||||
|
result = self.process_domain(domain)
|
||||||
|
results.append(result)
|
||||||
|
return results
|
||||||
|
|
||||||
|
def filter_valid_domains(self, domains):
|
||||||
|
"""
|
||||||
|
过滤有效的域名
|
||||||
|
|
||||||
|
:param domains: 域名列表
|
||||||
|
:return: list - 有效的域名列表
|
||||||
|
"""
|
||||||
|
valid_domains = []
|
||||||
|
for domain in domains:
|
||||||
|
result = self.process_domain(domain)
|
||||||
|
if result['valid']:
|
||||||
|
valid_domains.append(result['normalized'])
|
||||||
|
return valid_domains
|
||||||
|
|
||||||
|
def update_domain_status(self, domain_id, status_type, status_value):
|
||||||
|
"""
|
||||||
|
更新域名状态
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:param status_type: 状态类型
|
||||||
|
:param status_value: 状态值
|
||||||
|
:return: bool - 是否更新成功
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
if status_type == 'use_status':
|
||||||
|
return self.db.update_domain_use_status(domain_id, status_value)
|
||||||
|
elif status_type == 'detect_status':
|
||||||
|
return self.db.update_domain_detect_status(domain_id, status_value)
|
||||||
|
elif status_type == 'register_status':
|
||||||
|
return self.db.update_domain_register_status(domain_id, status_value)
|
||||||
|
else:
|
||||||
|
logger.error(f"未知的状态类型: {status_type}")
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"更新域名状态出错: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def blacklist_domain(self, domain_id, reason):
|
||||||
|
"""
|
||||||
|
将域名加入黑名单
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:param reason: 黑名单原因
|
||||||
|
:return: bool - 是否操作成功
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 更新检测状态为黑名单
|
||||||
|
if self.db.update_domain_detect_status(domain_id, 4): # 4 表示黑名单
|
||||||
|
# 添加到黑名单表
|
||||||
|
domain = self.db.get_domain_by_id(domain_id)
|
||||||
|
if domain:
|
||||||
|
return self.db.add_to_blacklist(domain['domain'], reason)
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"将域名加入黑名单出错: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def get_domain_statistics(self):
|
||||||
|
"""
|
||||||
|
获取域名统计信息
|
||||||
|
|
||||||
|
:return: dict - 统计信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return self.db.get_domain_statistics()
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取域名统计信息出错: {e}")
|
||||||
|
return {}
|
||||||
289
domainCheck/app/core/export_manager.py
Normal file
289
domainCheck/app/core/export_manager.py
Normal file
@@ -0,0 +1,289 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :export_manager.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:54
|
||||||
|
@explain : 导出管理器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import os
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.database import Database
|
||||||
|
|
||||||
|
|
||||||
|
class ExportManager:
|
||||||
|
"""
|
||||||
|
导出管理器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化导出管理器
|
||||||
|
"""
|
||||||
|
self.db = Database()
|
||||||
|
|
||||||
|
def export_domains(self, filter_conditions, output_file):
|
||||||
|
"""
|
||||||
|
导出域名
|
||||||
|
|
||||||
|
:param filter_conditions: 筛选条件
|
||||||
|
:param output_file: 输出文件路径
|
||||||
|
:return: int - 导出的域名数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 查询符合条件的域名
|
||||||
|
domains = self.db.get_domains_by_conditions(filter_conditions)
|
||||||
|
|
||||||
|
if not domains:
|
||||||
|
logger.warning("没有符合条件的域名")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
# 导出到文件
|
||||||
|
with open(output_file, 'w', encoding='utf-8') as f:
|
||||||
|
for domain in domains:
|
||||||
|
f.write(domain['domain'] + '\n')
|
||||||
|
|
||||||
|
logger.info(f"成功导出 {len(domains)} 个域名到 {output_file}")
|
||||||
|
return len(domains)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导出域名出错: {e}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def export_with_details(self, filter_conditions, output_file):
|
||||||
|
"""
|
||||||
|
导出域名及其详细信息
|
||||||
|
|
||||||
|
:param filter_conditions: 筛选条件
|
||||||
|
:param output_file: 输出文件路径
|
||||||
|
:return: int - 导出的域名数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 查询符合条件的域名及其详细信息
|
||||||
|
domains = self.db.get_domains_with_details(filter_conditions)
|
||||||
|
|
||||||
|
if not domains:
|
||||||
|
logger.warning("没有符合条件的域名")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
# 导出到文件
|
||||||
|
with open(output_file, 'w', encoding='utf-8') as f:
|
||||||
|
# 写入表头
|
||||||
|
f.write('域名,注册状态,使用状态,检测状态,备案历史,备案年份,快照年份,友情链接数量\n')
|
||||||
|
|
||||||
|
# 写入数据
|
||||||
|
for domain in domains:
|
||||||
|
line = f"{domain['domain']},{domain['register_status']},{domain['use_status']},{domain['detect_status']},{domain['has_beian']},{domain['beian_year']},{domain['snapshot_years']},{domain['backlink_count']}\n"
|
||||||
|
f.write(line)
|
||||||
|
|
||||||
|
logger.info(f"成功导出 {len(domains)} 个域名及其详细信息到 {output_file}")
|
||||||
|
return len(domains)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导出域名详细信息出错: {e}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def batch_update_status(self, domain_ids, status_type, status_value):
|
||||||
|
"""
|
||||||
|
批量更新域名状态
|
||||||
|
|
||||||
|
:param domain_ids: 域名ID列表
|
||||||
|
:param status_type: 状态类型
|
||||||
|
:param status_value: 状态值
|
||||||
|
:return: int - 更新成功的域名数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
success_count = 0
|
||||||
|
for domain_id in domain_ids:
|
||||||
|
if self.db.update_domain_status(domain_id, status_type, status_value):
|
||||||
|
success_count += 1
|
||||||
|
|
||||||
|
logger.info(f"成功更新 {success_count} 个域名的状态")
|
||||||
|
return success_count
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"批量更新域名状态出错: {e}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def get_filtered_domains(self, filter_conditions, limit=1000):
|
||||||
|
"""
|
||||||
|
获取符合条件的域名
|
||||||
|
|
||||||
|
:param filter_conditions: 筛选条件
|
||||||
|
:param limit: 限制数量
|
||||||
|
:return: list - 域名列表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
domains = self.db.get_domains_by_conditions(filter_conditions, limit)
|
||||||
|
logger.info(f"获取到 {len(domains)} 个符合条件的域名")
|
||||||
|
return domains
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取符合条件的域名出错: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
def generate_report(self, output_file):
|
||||||
|
"""
|
||||||
|
生成统计报告
|
||||||
|
|
||||||
|
:param output_file: 输出文件路径
|
||||||
|
:return: bool - 是否生成成功
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 获取域名统计信息
|
||||||
|
domain_stats = self.db.get_domain_statistics()
|
||||||
|
|
||||||
|
# 获取任务统计信息
|
||||||
|
task_stats = self.db.get_task_statistics()
|
||||||
|
|
||||||
|
# 生成报告
|
||||||
|
with open(output_file, 'w', encoding='utf-8') as f:
|
||||||
|
f.write('域名库系统统计报告\n')
|
||||||
|
f.write('=' * 50 + '\n')
|
||||||
|
|
||||||
|
f.write('\n域名统计:\n')
|
||||||
|
f.write(f'总域名数: {domain_stats.get("total", 0)}\n')
|
||||||
|
f.write(f'可注册域名: {domain_stats.get("available", 0)}\n')
|
||||||
|
f.write(f'已注册域名: {domain_stats.get("registered", 0)}\n')
|
||||||
|
f.write(f'黑名单域名: {domain_stats.get("blacklisted", 0)}\n')
|
||||||
|
|
||||||
|
f.write('\n任务统计:\n')
|
||||||
|
f.write(f'总任务数: {task_stats.get("total", 0)}\n')
|
||||||
|
f.write(f'待执行任务: {task_stats.get("pending", 0)}\n')
|
||||||
|
f.write(f'执行中任务: {task_stats.get("running", 0)}\n')
|
||||||
|
f.write(f'已完成任务: {task_stats.get("completed", 0)}\n')
|
||||||
|
f.write(f'失败任务: {task_stats.get("failed", 0)}\n')
|
||||||
|
|
||||||
|
logger.info(f"成功生成统计报告到 {output_file}")
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"生成统计报告出错: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def export_to_excel(self, domains, output_file):
|
||||||
|
"""
|
||||||
|
导出域名到Excel文件
|
||||||
|
|
||||||
|
:param domains: 域名列表
|
||||||
|
:param output_file: 输出文件路径
|
||||||
|
:return: int - 导出的域名数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 尝试导入openpyxl
|
||||||
|
try:
|
||||||
|
from openpyxl import Workbook
|
||||||
|
except ImportError:
|
||||||
|
# 如果没有安装openpyxl,使用CSV格式作为替代
|
||||||
|
logger.warning("openpyxl库未安装,将使用CSV格式导出")
|
||||||
|
return self.export_to_csv(domains, output_file.replace('.xlsx', '.csv'))
|
||||||
|
|
||||||
|
# 创建工作簿
|
||||||
|
wb = Workbook()
|
||||||
|
ws = wb.active
|
||||||
|
|
||||||
|
# 写入表头
|
||||||
|
headers = ['域名', '注册状态', '使用状态', '检测状态', '人工复核状态', '过期时间', '单位性质', '网站首页网址', '检测时间', '备案历史', '备案年份', '快照年份', '百度历史', '百度Site', '是否中文标题', '360 Site', 'Google Site', '友情链接数量']
|
||||||
|
ws.append(headers)
|
||||||
|
|
||||||
|
# 写入数据
|
||||||
|
for domain in domains:
|
||||||
|
row = [
|
||||||
|
domain.get('domain', ''),
|
||||||
|
domain.get('register_status', ''),
|
||||||
|
domain.get('use_status', ''),
|
||||||
|
domain.get('detect_status', ''),
|
||||||
|
domain.get('review_status', ''),
|
||||||
|
domain.get('expire_date', ''),
|
||||||
|
domain.get('company_type', ''),
|
||||||
|
domain.get('website_url', ''),
|
||||||
|
domain.get('detect_time', ''),
|
||||||
|
domain.get('has_beian', ''),
|
||||||
|
domain.get('beian_year', ''),
|
||||||
|
domain.get('snapshot_years', ''),
|
||||||
|
domain.get('baidu_history', ''),
|
||||||
|
domain.get('baidu_site', ''),
|
||||||
|
domain.get('is_chinese_title', ''),
|
||||||
|
domain.get('qihu360_site', ''),
|
||||||
|
domain.get('google_site', ''),
|
||||||
|
domain.get('backlink_count', '')
|
||||||
|
]
|
||||||
|
ws.append(row)
|
||||||
|
|
||||||
|
# 保存文件
|
||||||
|
wb.save(output_file)
|
||||||
|
|
||||||
|
logger.info(f"成功导出 {len(domains)} 个域名到Excel文件: {output_file}")
|
||||||
|
return len(domains)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导出Excel文件出错: {e}")
|
||||||
|
# 尝试使用CSV格式作为替代
|
||||||
|
try:
|
||||||
|
csv_file = output_file.replace('.xlsx', '.csv')
|
||||||
|
logger.info(f"尝试使用CSV格式导出到: {csv_file}")
|
||||||
|
return self.export_to_csv(domains, csv_file)
|
||||||
|
except Exception as e2:
|
||||||
|
logger.error(f"导出CSV文件也失败: {e2}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def export_to_csv(self, domains, output_file):
|
||||||
|
"""
|
||||||
|
导出域名到CSV文件
|
||||||
|
|
||||||
|
:param domains: 域名列表
|
||||||
|
:param output_file: 输出文件路径
|
||||||
|
:return: int - 导出的域名数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import csv
|
||||||
|
|
||||||
|
# 写入文件
|
||||||
|
with open(output_file, 'w', encoding='utf-8', newline='') as f:
|
||||||
|
writer = csv.writer(f)
|
||||||
|
|
||||||
|
# 写入表头
|
||||||
|
headers = ['域名', '注册状态', '使用状态', '检测状态', '人工复核状态', '过期时间', '单位性质', '网站首页网址', '检测时间', '备案历史', '备案年份', '快照年份', '百度历史', '百度Site', '是否中文标题', '360 Site', 'Google Site', '友情链接数量']
|
||||||
|
writer.writerow(headers)
|
||||||
|
|
||||||
|
# 写入数据
|
||||||
|
for domain in domains:
|
||||||
|
row = [
|
||||||
|
domain.get('domain', ''),
|
||||||
|
domain.get('register_status', ''),
|
||||||
|
domain.get('use_status', ''),
|
||||||
|
domain.get('detect_status', ''),
|
||||||
|
domain.get('review_status', ''),
|
||||||
|
domain.get('expire_date', ''),
|
||||||
|
domain.get('company_type', ''),
|
||||||
|
domain.get('website_url', ''),
|
||||||
|
domain.get('detect_time', ''),
|
||||||
|
domain.get('has_beian', ''),
|
||||||
|
domain.get('beian_year', ''),
|
||||||
|
domain.get('snapshot_years', ''),
|
||||||
|
domain.get('baidu_history', ''),
|
||||||
|
domain.get('baidu_site', ''),
|
||||||
|
domain.get('is_chinese_title', ''),
|
||||||
|
domain.get('qihu360_site', ''),
|
||||||
|
domain.get('google_site', ''),
|
||||||
|
domain.get('backlink_count', '')
|
||||||
|
]
|
||||||
|
writer.writerow(row)
|
||||||
|
|
||||||
|
logger.info(f"成功导出 {len(domains)} 个域名到CSV文件: {output_file}")
|
||||||
|
return len(domains)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导出CSV文件出错: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def export_to_txt(self, domains, output_file):
|
||||||
|
"""
|
||||||
|
导出域名到TXT文件,一行一个域名。
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with open(output_file, 'w', encoding='utf-8') as f:
|
||||||
|
for domain in domains:
|
||||||
|
value = domain.get('domain', '').strip()
|
||||||
|
if value:
|
||||||
|
f.write(value + '\n')
|
||||||
|
logger.info(f"成功导出 {len(domains)} 个域名到TXT文件: {output_file}")
|
||||||
|
return len(domains)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导出TXT文件出错: {e}")
|
||||||
|
raise
|
||||||
162
domainCheck/app/core/task_scheduler.py
Normal file
162
domainCheck/app/core/task_scheduler.py
Normal file
@@ -0,0 +1,162 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :task_scheduler.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:53
|
||||||
|
@explain : 任务调度器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import time
|
||||||
|
import threading
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.database import Database
|
||||||
|
from app.core.detect_engine import DetectEngine
|
||||||
|
|
||||||
|
|
||||||
|
class TaskScheduler:
|
||||||
|
"""
|
||||||
|
任务调度器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化任务调度器
|
||||||
|
"""
|
||||||
|
self.db = Database()
|
||||||
|
self.detect_engine = DetectEngine()
|
||||||
|
self.running = False
|
||||||
|
self.threads = []
|
||||||
|
self.max_threads = 10
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
"""
|
||||||
|
启动任务调度器
|
||||||
|
"""
|
||||||
|
if self.running:
|
||||||
|
logger.info("任务调度器已经在运行中")
|
||||||
|
return
|
||||||
|
|
||||||
|
self.running = True
|
||||||
|
logger.info("启动任务调度器")
|
||||||
|
|
||||||
|
# 启动多个线程处理任务
|
||||||
|
for i in range(self.max_threads):
|
||||||
|
thread = threading.Thread(target=self._process_tasks, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
self.threads.append(thread)
|
||||||
|
logger.info(f"启动任务处理线程 {i+1}")
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
"""
|
||||||
|
停止任务调度器
|
||||||
|
"""
|
||||||
|
self.running = False
|
||||||
|
logger.info("停止任务调度器")
|
||||||
|
|
||||||
|
# 等待线程结束
|
||||||
|
for thread in self.threads:
|
||||||
|
thread.join(timeout=5)
|
||||||
|
|
||||||
|
self.threads.clear()
|
||||||
|
logger.info("任务调度器已停止")
|
||||||
|
|
||||||
|
def _process_tasks(self):
|
||||||
|
"""
|
||||||
|
处理任务
|
||||||
|
"""
|
||||||
|
while self.running:
|
||||||
|
try:
|
||||||
|
# 获取待执行的任务
|
||||||
|
task = self.db.get_pending_task()
|
||||||
|
if not task:
|
||||||
|
# 没有任务,休眠一段时间
|
||||||
|
time.sleep(1)
|
||||||
|
continue
|
||||||
|
|
||||||
|
task_id = task['id']
|
||||||
|
domain_id = task['domain_id']
|
||||||
|
|
||||||
|
logger.info(f"处理任务: {task_id}, 域名ID: {domain_id}")
|
||||||
|
|
||||||
|
# 执行任务
|
||||||
|
success = self.detect_engine.process_task(task_id)
|
||||||
|
|
||||||
|
if success:
|
||||||
|
logger.info(f"任务处理成功: {task_id}")
|
||||||
|
else:
|
||||||
|
logger.warning(f"任务处理失败: {task_id}")
|
||||||
|
|
||||||
|
# 短暂休眠,避免过于频繁的数据库操作
|
||||||
|
time.sleep(0.1)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"处理任务出错: {e}")
|
||||||
|
# 休眠一段时间,避免出错后无限循环
|
||||||
|
time.sleep(5)
|
||||||
|
|
||||||
|
def add_task(self, domain_id, task_type=1, priority=0):
|
||||||
|
"""
|
||||||
|
添加任务
|
||||||
|
|
||||||
|
:param domain_id: 域名ID
|
||||||
|
:param task_type: 任务类型,1-基础检测,2-深度检测
|
||||||
|
:param priority: 优先级,0-低,1-中,2-高
|
||||||
|
:return: int - 任务ID
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
task_id = self.db.create_detect_task(domain_id, task_type, priority)
|
||||||
|
logger.info(f"添加任务成功: {task_id}, 域名ID: {domain_id}")
|
||||||
|
return task_id
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"添加任务失败: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_task_stats(self):
|
||||||
|
"""
|
||||||
|
获取任务统计信息
|
||||||
|
|
||||||
|
:return: dict - 任务统计信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return self.db.get_task_statistics()
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取任务统计信息出错: {e}")
|
||||||
|
return {}
|
||||||
|
|
||||||
|
def retry_failed_tasks(self):
|
||||||
|
"""
|
||||||
|
重试失败的任务
|
||||||
|
|
||||||
|
:return: int - 重试的任务数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
tasks = self.db.get_failed_tasks()
|
||||||
|
retry_count = 0
|
||||||
|
|
||||||
|
for task in tasks:
|
||||||
|
task_id = task['id']
|
||||||
|
self.db.update_task_status(task_id, 0) # 0 表示待执行
|
||||||
|
self.db.update_task_retry_count(task_id, 0) # 重置重试次数
|
||||||
|
retry_count += 1
|
||||||
|
|
||||||
|
logger.info(f"重试 {retry_count} 个失败的任务")
|
||||||
|
return retry_count
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"重试失败任务出错: {e}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def clear_completed_tasks(self, days=7):
|
||||||
|
"""
|
||||||
|
清理已完成的任务
|
||||||
|
|
||||||
|
:param days: 保留天数
|
||||||
|
:return: int - 清理的任务数量
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
count = self.db.clear_completed_tasks(days)
|
||||||
|
logger.info(f"清理 {count} 个已完成的任务")
|
||||||
|
return count
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"清理已完成任务出错: {e}")
|
||||||
|
return 0
|
||||||
10
domainCheck/app/credentials.json
Normal file
10
domainCheck/app/credentials.json
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"juming": {
|
||||||
|
"email": "chaofanai1998@gmail.com",
|
||||||
|
"password": "llzz123,./"
|
||||||
|
},
|
||||||
|
"juziseo": {
|
||||||
|
"email": "mamian",
|
||||||
|
"password": "Abc123456"
|
||||||
|
}
|
||||||
|
}
|
||||||
20
domainCheck/app/detect_options.json
Normal file
20
domainCheck/app/detect_options.json
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
{
|
||||||
|
"detect_register": true,
|
||||||
|
"detect_wayback": true,
|
||||||
|
"detect_chinaz": true,
|
||||||
|
"detect_aizhan": true,
|
||||||
|
"detect_baidu_site": true,
|
||||||
|
"detect_360_site": true,
|
||||||
|
"detect_jucha": false,
|
||||||
|
"detect_juziseo": false,
|
||||||
|
"detect_order": [
|
||||||
|
"detect_register",
|
||||||
|
"detect_baidu_site",
|
||||||
|
"detect_360_site",
|
||||||
|
"detect_chinaz",
|
||||||
|
"detect_aizhan",
|
||||||
|
"detect_wayback",
|
||||||
|
"detect_jucha",
|
||||||
|
"detect_juziseo"
|
||||||
|
]
|
||||||
|
}
|
||||||
4
domainCheck/app/detectors/__init__.py
Normal file
4
domainCheck/app/detectors/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
检测插件模块
|
||||||
|
'''
|
||||||
130
domainCheck/app/detectors/aizhan_detector.py
Normal file
130
domainCheck/app/detectors/aizhan_detector.py
Normal file
@@ -0,0 +1,130 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :aizhan_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:02
|
||||||
|
@explain : 爱站网检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class AizhanDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
爱站网检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化爱站网检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.url = 'https://www.aizhan.com'
|
||||||
|
self.query_url = 'https://www.aizhan.com/cha/{domain}'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.query_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取标题
|
||||||
|
title = self._extract_title(content)
|
||||||
|
|
||||||
|
# 提取风险信息
|
||||||
|
risk = self._extract_risk(content)
|
||||||
|
|
||||||
|
# 检查是否包含敏感词
|
||||||
|
has_sensitive = self._check_sensitive(title, risk)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'title': title,
|
||||||
|
'risk': risk,
|
||||||
|
'has_sensitive': has_sensitive
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"爱站网查询失败: {response.status_code}")
|
||||||
|
return {'title': '', 'risk': '', 'has_sensitive': False}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def _extract_title(self, content):
|
||||||
|
"""
|
||||||
|
提取标题
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: str - 标题
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
pattern = r'<title>(.*?)</title>'
|
||||||
|
match = re.search(pattern, content)
|
||||||
|
if match:
|
||||||
|
return match.group(1).strip()
|
||||||
|
return ''
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_title')
|
||||||
|
return ''
|
||||||
|
|
||||||
|
def _extract_risk(self, content):
|
||||||
|
"""
|
||||||
|
提取风险信息
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: str - 风险信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 这里需要根据实际页面结构调整正则表达式
|
||||||
|
pattern = r'百度网址检测:<span[^>]+>(.*?)</span>'
|
||||||
|
match = re.search(pattern, content)
|
||||||
|
if match:
|
||||||
|
return match.group(1).strip()
|
||||||
|
return ''
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_risk')
|
||||||
|
return ''
|
||||||
|
|
||||||
|
def _check_sensitive(self, title, risk):
|
||||||
|
"""
|
||||||
|
检查是否包含敏感词
|
||||||
|
|
||||||
|
:param title: 标题
|
||||||
|
:param risk: 风险信息
|
||||||
|
:return: bool - 是否包含敏感词
|
||||||
|
"""
|
||||||
|
# 风险类型
|
||||||
|
sensitive_risks = ['低风险', '疑似色情博彩风险', '严重影响权重']
|
||||||
|
|
||||||
|
# 敏感词
|
||||||
|
sensitive_words = ['色情', '赌博', '博彩', '毒品', '暴力', '诈骗']
|
||||||
|
|
||||||
|
# 检查风险
|
||||||
|
for r in sensitive_risks:
|
||||||
|
if r in risk:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查标题
|
||||||
|
for word in sensitive_words:
|
||||||
|
if word in title:
|
||||||
|
return True
|
||||||
|
|
||||||
|
return False
|
||||||
151
domainCheck/app/detectors/baidu_detector.py
Normal file
151
domainCheck/app/detectors/baidu_detector.py
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :baidu_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:58
|
||||||
|
@explain : 百度检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class BaiduDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
百度检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化百度检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.site_url = 'https://www.baidu.com/s'
|
||||||
|
self.history_url = 'https://www.baidu.com/s'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查百度site
|
||||||
|
site_result = self.check_site(domain)
|
||||||
|
|
||||||
|
# 检查百度历史
|
||||||
|
history_result = self.check_history(domain)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'site': site_result,
|
||||||
|
'history': history_result
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def check_site(self, domain):
|
||||||
|
"""
|
||||||
|
检查百度site收录
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
params = {
|
||||||
|
'wd': f'site:{domain}',
|
||||||
|
'rn': '50'
|
||||||
|
}
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(self.site_url, params=params, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取子域名
|
||||||
|
subdomains = self._extract_subdomains(content, domain)
|
||||||
|
|
||||||
|
# 检查是否有收录
|
||||||
|
has_收录 = '没有找到相关结果' not in content
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_收录': has_收录,
|
||||||
|
'subdomains': subdomains
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"百度site查询失败: {response.status_code}")
|
||||||
|
return {'has_收录': False, 'subdomains': []}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_收录': False, 'subdomains': []}
|
||||||
|
|
||||||
|
def check_history(self, domain):
|
||||||
|
"""
|
||||||
|
检查百度历史收录
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
params = {
|
||||||
|
'wd': f'cache:{domain}',
|
||||||
|
'rn': '50'
|
||||||
|
}
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(self.history_url, params=params, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 检查是否有历史收录
|
||||||
|
has_history = '百度快照' in content
|
||||||
|
|
||||||
|
# 检查是否有灰色内容
|
||||||
|
has_gray = '风险提示' in content or '安全警告' in content
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_history': has_history,
|
||||||
|
'has_gray': has_gray
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"百度历史查询失败: {response.status_code}")
|
||||||
|
return {'has_history': False, 'has_gray': False}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_history': False, 'has_gray': False}
|
||||||
|
|
||||||
|
def _extract_subdomains(self, content, domain):
|
||||||
|
"""
|
||||||
|
提取子域名
|
||||||
|
|
||||||
|
:param content: 搜索结果内容
|
||||||
|
:param domain: 主域名
|
||||||
|
:return: list - 子域名列表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 提取所有包含域名的链接
|
||||||
|
pattern = r'https?://([a-zA-Z0-9-]+)\.' + re.escape(domain)
|
||||||
|
matches = re.findall(pattern, content)
|
||||||
|
|
||||||
|
# 去重并过滤空值
|
||||||
|
subdomains = list(set(matches))
|
||||||
|
subdomains = [sub for sub in subdomains if sub]
|
||||||
|
|
||||||
|
return subdomains
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return []
|
||||||
70
domainCheck/app/detectors/base.py
Normal file
70
domainCheck/app/detectors/base.py
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :base.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:55
|
||||||
|
@explain : 基础检测类
|
||||||
|
'''
|
||||||
|
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
|
||||||
|
class BaseDetector(ABC):
|
||||||
|
"""
|
||||||
|
基础检测类
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化检测类
|
||||||
|
"""
|
||||||
|
self.name = self.__class__.__name__
|
||||||
|
logger.info(f"初始化检测器: {self.name}")
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _log_info(self, message):
|
||||||
|
"""
|
||||||
|
记录信息日志
|
||||||
|
|
||||||
|
:param message: 消息
|
||||||
|
"""
|
||||||
|
logger.info(f"[{self.name}] {message}")
|
||||||
|
|
||||||
|
def _log_warning(self, message):
|
||||||
|
"""
|
||||||
|
记录警告日志
|
||||||
|
|
||||||
|
:param message: 消息
|
||||||
|
"""
|
||||||
|
logger.warning(f"[{self.name}] {message}")
|
||||||
|
|
||||||
|
def _log_error(self, message):
|
||||||
|
"""
|
||||||
|
记录错误日志
|
||||||
|
|
||||||
|
:param message: 消息
|
||||||
|
"""
|
||||||
|
logger.error(f"[{self.name}] {message}")
|
||||||
|
|
||||||
|
def _handle_exception(self, e, domain):
|
||||||
|
"""
|
||||||
|
处理异常
|
||||||
|
|
||||||
|
:param e: 异常
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 错误结果
|
||||||
|
"""
|
||||||
|
self._log_error(f"检测域名 {domain} 出错: {e}")
|
||||||
|
return {'error': str(e)}
|
||||||
130
domainCheck/app/detectors/chinaz_detector.py
Normal file
130
domainCheck/app/detectors/chinaz_detector.py
Normal file
@@ -0,0 +1,130 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :chinaz_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:01
|
||||||
|
@explain : 站长之家检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class ChinazDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
站长之家检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化站长之家检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.url = 'https://seo.chinaz.com'
|
||||||
|
self.query_url = 'https://seo.chinaz.com/{domain}'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.query_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取标题
|
||||||
|
title = self._extract_title(content)
|
||||||
|
|
||||||
|
# 提取网站分类
|
||||||
|
category = self._extract_category(content)
|
||||||
|
|
||||||
|
# 检查是否包含敏感词
|
||||||
|
has_sensitive = self._check_sensitive(title, category)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'title': title,
|
||||||
|
'category': category,
|
||||||
|
'has_sensitive': has_sensitive
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"站长之家查询失败: {response.status_code}")
|
||||||
|
return {'title': '', 'category': '', 'has_sensitive': False}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def _extract_title(self, content):
|
||||||
|
"""
|
||||||
|
提取标题
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: str - 标题
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
pattern = r'<title>(.*?)</title>'
|
||||||
|
match = re.search(pattern, content)
|
||||||
|
if match:
|
||||||
|
return match.group(1).strip()
|
||||||
|
return ''
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_title')
|
||||||
|
return ''
|
||||||
|
|
||||||
|
def _extract_category(self, content):
|
||||||
|
"""
|
||||||
|
提取网站分类
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: str - 分类
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 这里需要根据实际页面结构调整正则表达式
|
||||||
|
pattern = r'网站分类:<a[^>]+>(.*?)</a>'
|
||||||
|
match = re.search(pattern, content)
|
||||||
|
if match:
|
||||||
|
return match.group(1).strip()
|
||||||
|
return ''
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_category')
|
||||||
|
return ''
|
||||||
|
|
||||||
|
def _check_sensitive(self, title, category):
|
||||||
|
"""
|
||||||
|
检查是否包含敏感词
|
||||||
|
|
||||||
|
:param title: 标题
|
||||||
|
:param category: 分类
|
||||||
|
:return: bool - 是否包含敏感词
|
||||||
|
"""
|
||||||
|
# 敏感分类
|
||||||
|
sensitive_categories = ['视频电影', '体育运动', '常用查询']
|
||||||
|
|
||||||
|
# 敏感词
|
||||||
|
sensitive_words = ['色情', '赌博', '博彩', '毒品', '暴力', '诈骗']
|
||||||
|
|
||||||
|
# 检查分类
|
||||||
|
for cat in sensitive_categories:
|
||||||
|
if cat in category:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查标题
|
||||||
|
for word in sensitive_words:
|
||||||
|
if word in title:
|
||||||
|
return True
|
||||||
|
|
||||||
|
return False
|
||||||
80
domainCheck/app/detectors/google_detector.py
Normal file
80
domainCheck/app/detectors/google_detector.py
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :google_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:00
|
||||||
|
@explain : Google检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class GoogleDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
Google检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化Google检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.site_url = 'https://www.google.com/search'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查Google site
|
||||||
|
site_result = self.check_site(domain)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'site': site_result
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def check_site(self, domain):
|
||||||
|
"""
|
||||||
|
检查Google site收录
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
params = {
|
||||||
|
'q': f'site:{domain}',
|
||||||
|
'num': '50'
|
||||||
|
}
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(self.site_url, params=params, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 检查是否有收录
|
||||||
|
has_收录 = 'No results found for' not in content
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_收录': has_收录
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"Google site查询失败: {response.status_code}")
|
||||||
|
return {'has_收录': False}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_收录': False}
|
||||||
228
domainCheck/app/detectors/jucha_detector.py
Normal file
228
domainCheck/app/detectors/jucha_detector.py
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :jucha_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:04
|
||||||
|
@explain : 聚查检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class JuchaDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
聚查检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化聚查检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.url = 'https://www.jucha.com'
|
||||||
|
self.whois_url = 'https://www.jucha.com/whois/{domain}'
|
||||||
|
self.beian_url = 'https://www.jucha.com/beian/{domain}'
|
||||||
|
self.intercept_url = 'https://www.jucha.com/intercept/{domain}'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查WHOIS
|
||||||
|
whois_result = self.check_whois(domain)
|
||||||
|
|
||||||
|
# 检查备案
|
||||||
|
beian_result = self.check_beian(domain)
|
||||||
|
|
||||||
|
# 检查拦截
|
||||||
|
intercept_result = self.check_intercept(domain)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'whois': whois_result,
|
||||||
|
'beian': beian_result,
|
||||||
|
'intercept': intercept_result
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def check_whois(self, domain):
|
||||||
|
"""
|
||||||
|
检查WHOIS
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.whois_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取WHOIS信息
|
||||||
|
whois_info = self._extract_whois_info(content)
|
||||||
|
|
||||||
|
return whois_info
|
||||||
|
else:
|
||||||
|
self._log_warning(f"聚查WHOIS查询失败: {response.status_code}")
|
||||||
|
return {'status': ''}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'status': ''}
|
||||||
|
|
||||||
|
def check_beian(self, domain):
|
||||||
|
"""
|
||||||
|
检查备案
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.beian_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取备案信息
|
||||||
|
beian_info = self._extract_beian_info(content)
|
||||||
|
|
||||||
|
return beian_info
|
||||||
|
else:
|
||||||
|
self._log_warning(f"聚查备案查询失败: {response.status_code}")
|
||||||
|
return {'has_beian': False, 'beian_year': '', 'is_enterprise': False, 'beian_match': False}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_beian': False, 'beian_year': '', 'is_enterprise': False, 'beian_match': False}
|
||||||
|
|
||||||
|
def check_intercept(self, domain):
|
||||||
|
"""
|
||||||
|
检查拦截
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.intercept_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 检查是否被拦截
|
||||||
|
is_normal = self._check_intercept_status(content)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'normal': is_normal
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"聚查拦截查询失败: {response.status_code}")
|
||||||
|
return {'normal': False}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'normal': False}
|
||||||
|
|
||||||
|
def _extract_whois_info(self, content):
|
||||||
|
"""
|
||||||
|
提取WHOIS信息
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: dict - WHOIS信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 提取状态信息
|
||||||
|
pattern = r'域名状态:<span[^>]+>(.*?)</span>'
|
||||||
|
match = re.search(pattern, content)
|
||||||
|
status = match.group(1).strip() if match else ''
|
||||||
|
|
||||||
|
return {
|
||||||
|
'status': status
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_whois_info')
|
||||||
|
return {'status': ''}
|
||||||
|
|
||||||
|
def _extract_beian_info(self, content):
|
||||||
|
"""
|
||||||
|
提取备案信息
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: dict - 备案信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查是否有备案
|
||||||
|
has_beian = '备案信息' in content
|
||||||
|
|
||||||
|
# 提取备案年份
|
||||||
|
beian_year = ''
|
||||||
|
pattern = r'审核时间:(\d{4})-\d{2}-\d{2}'
|
||||||
|
match = re.search(pattern, content)
|
||||||
|
if match:
|
||||||
|
beian_year = match.group(1)
|
||||||
|
|
||||||
|
# 提取单位性质
|
||||||
|
is_enterprise = '企业' in content
|
||||||
|
|
||||||
|
# 检查首网址和备案网址是否一致
|
||||||
|
beian_match = '网站首页网址' in content
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_beian': has_beian,
|
||||||
|
'beian_year': beian_year,
|
||||||
|
'is_enterprise': is_enterprise,
|
||||||
|
'beian_match': beian_match
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_beian_info')
|
||||||
|
return {'has_beian': False, 'beian_year': '', 'is_enterprise': False, 'beian_match': False}
|
||||||
|
|
||||||
|
def _check_intercept_status(self, content):
|
||||||
|
"""
|
||||||
|
检查拦截状态
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: bool - 是否正常
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查是否包含正常标识
|
||||||
|
if '正常' in content:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查是否包含拦截标识
|
||||||
|
if '拦截' in content:
|
||||||
|
return False
|
||||||
|
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'check_intercept_status')
|
||||||
|
return False
|
||||||
214
domainCheck/app/detectors/juziseo_detector.py
Normal file
214
domainCheck/app/detectors/juziseo_detector.py
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :juziseo_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:03
|
||||||
|
@explain : 桔子SEO检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class JuziseoDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
桔子SEO检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化桔子SEO检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.url = 'https://seo.juziseo.com'
|
||||||
|
self.history_url = 'https://seo.juziseo.com/history/{domain}'
|
||||||
|
self.backlink_url = 'https://seo.juziseo.com/backlink/{domain}'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查历史信息
|
||||||
|
history_result = self.check_history(domain)
|
||||||
|
|
||||||
|
# 检查外链
|
||||||
|
backlink_result = self.check_backlink(domain)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'history': history_result,
|
||||||
|
'backlink': backlink_result
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def check_history(self, domain):
|
||||||
|
"""
|
||||||
|
检查历史信息
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.history_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取历史信息
|
||||||
|
history_info = self._extract_history_info(content)
|
||||||
|
|
||||||
|
# 检查是否包含敏感词
|
||||||
|
has_sensitive = self._check_sensitive(history_info)
|
||||||
|
|
||||||
|
# 检查是否有百度历史收录
|
||||||
|
has_baidu_history = '百度历史收录' in content
|
||||||
|
|
||||||
|
# 检查是否有子域名
|
||||||
|
has_subdomains = '子域名' in content
|
||||||
|
|
||||||
|
# 检查是否为简体中文
|
||||||
|
is_simplified = self._check_simplified(content)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_sensitive': has_sensitive,
|
||||||
|
'has_baidu_history': has_baidu_history,
|
||||||
|
'has_subdomains': has_subdomains,
|
||||||
|
'is_simplified': is_simplified
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"桔子SEO历史查询失败: {response.status_code}")
|
||||||
|
return {'has_sensitive': False, 'has_baidu_history': False, 'has_subdomains': False, 'is_simplified': True}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_sensitive': False, 'has_baidu_history': False, 'has_subdomains': False, 'is_simplified': True}
|
||||||
|
|
||||||
|
def check_backlink(self, domain):
|
||||||
|
"""
|
||||||
|
检查外链
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 构建查询URL
|
||||||
|
query_url = self.backlink_url.format(domain=domain)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 检查是否包含敏感词
|
||||||
|
has_sensitive = self._check_backlink_sensitive(content)
|
||||||
|
|
||||||
|
# 检查是否有子域名
|
||||||
|
has_subdomains = '子域名' in content
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_sensitive': has_sensitive,
|
||||||
|
'has_subdomains': has_subdomains
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"桔子SEO外链查询失败: {response.status_code}")
|
||||||
|
return {'has_sensitive': False, 'has_subdomains': False}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_sensitive': False, 'has_subdomains': False}
|
||||||
|
|
||||||
|
def _extract_history_info(self, content):
|
||||||
|
"""
|
||||||
|
提取历史信息
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: str - 历史信息
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 这里需要根据实际页面结构调整正则表达式
|
||||||
|
pattern = r'<div class="history-info">(.*?)</div>'
|
||||||
|
match = re.search(pattern, content, re.DOTALL)
|
||||||
|
if match:
|
||||||
|
return match.group(1).strip()
|
||||||
|
return ''
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, 'extract_history_info')
|
||||||
|
return ''
|
||||||
|
|
||||||
|
def _check_sensitive(self, history_info):
|
||||||
|
"""
|
||||||
|
检查是否包含敏感词
|
||||||
|
|
||||||
|
:param history_info: 历史信息
|
||||||
|
:return: bool - 是否包含敏感词
|
||||||
|
"""
|
||||||
|
# 敏感词
|
||||||
|
sensitive_words = [
|
||||||
|
'色情', '赌博', '博彩', '毒品', '暴力', '诈骗',
|
||||||
|
'足球', '直播', '证券', '配资', '软件',
|
||||||
|
'体育', '商行', '下载', '影视', '网络',
|
||||||
|
'计算', 'app', 'HTML SiteMap', '模拟器', '传媒',
|
||||||
|
'二次元', '成人', '米乐', '小说', '凯发',
|
||||||
|
'人才', '华体', '娱乐', '开户'
|
||||||
|
]
|
||||||
|
|
||||||
|
for word in sensitive_words:
|
||||||
|
if word in history_info:
|
||||||
|
return True
|
||||||
|
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _check_simplified(self, content):
|
||||||
|
"""
|
||||||
|
检查是否为简体中文
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: bool - 是否为简体中文
|
||||||
|
"""
|
||||||
|
# 检查是否包含简体中文标识
|
||||||
|
if '简体中文' in content:
|
||||||
|
return True
|
||||||
|
|
||||||
|
# 检查是否包含繁体中文标识
|
||||||
|
if '繁体中文' in content:
|
||||||
|
return False
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _check_backlink_sensitive(self, content):
|
||||||
|
"""
|
||||||
|
检查外链是否包含敏感词
|
||||||
|
|
||||||
|
:param content: 页面内容
|
||||||
|
:return: bool - 是否包含敏感词
|
||||||
|
"""
|
||||||
|
# 敏感词
|
||||||
|
sensitive_words = [
|
||||||
|
'内幕', '猛料', '精料', '高手', '绝杀',
|
||||||
|
'权威', '澳门', '色情', '赌博', '博彩'
|
||||||
|
]
|
||||||
|
|
||||||
|
for word in sensitive_words:
|
||||||
|
if word in content:
|
||||||
|
return True
|
||||||
|
|
||||||
|
return False
|
||||||
107
domainCheck/app/detectors/qihu360_detector.py
Normal file
107
domainCheck/app/detectors/qihu360_detector.py
Normal file
@@ -0,0 +1,107 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :qihu360_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:59
|
||||||
|
@explain : 360检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
import re
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class Qihu360Detector(BaseDetector):
|
||||||
|
"""
|
||||||
|
360检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化360检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.site_url = 'https://www.so.com/s'
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 检查360 site
|
||||||
|
site_result = self.check_site(domain)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'site': site_result
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def check_site(self, domain):
|
||||||
|
"""
|
||||||
|
检查360 site收录
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
params = {
|
||||||
|
'q': f'site:{domain}',
|
||||||
|
'pn': '1',
|
||||||
|
'rn': '50'
|
||||||
|
}
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
|
||||||
|
}
|
||||||
|
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(self.site_url, params=params, headers=headers, impersonate='chrome', timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
content = response.text
|
||||||
|
|
||||||
|
# 提取子域名
|
||||||
|
subdomains = self._extract_subdomains(content, domain)
|
||||||
|
|
||||||
|
# 检查是否有收录
|
||||||
|
has_收录 = '没有找到相关结果' not in content
|
||||||
|
|
||||||
|
return {
|
||||||
|
'has_收录': has_收录,
|
||||||
|
'subdomains': subdomains
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
self._log_warning(f"360 site查询失败: {response.status_code}")
|
||||||
|
return {'has_收录': False, 'subdomains': []}
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return {'has_收录': False, 'subdomains': []}
|
||||||
|
|
||||||
|
def _extract_subdomains(self, content, domain):
|
||||||
|
"""
|
||||||
|
提取子域名
|
||||||
|
|
||||||
|
:param content: 搜索结果内容
|
||||||
|
:param domain: 主域名
|
||||||
|
:return: list - 子域名列表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 提取所有包含域名的链接
|
||||||
|
pattern = r'https?://([a-zA-Z0-9-]+)\.' + re.escape(domain)
|
||||||
|
matches = re.findall(pattern, content)
|
||||||
|
|
||||||
|
# 去重并过滤空值
|
||||||
|
subdomains = list(set(matches))
|
||||||
|
subdomains = [sub for sub in subdomains if sub]
|
||||||
|
|
||||||
|
return subdomains
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return []
|
||||||
128
domainCheck/app/detectors/rdap_detector.py
Normal file
128
domainCheck/app/detectors/rdap_detector.py
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :rdap_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:56
|
||||||
|
@explain : RDAP检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class RDAPDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
RDAP检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化RDAP检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.rdap_urls = {
|
||||||
|
'com': 'https://rdap.verisign.com/com/v1/domain/',
|
||||||
|
'net': 'https://rdap.verisign.com/net/v1/domain/'
|
||||||
|
}
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 提取顶级域名
|
||||||
|
tld = domain.split('.')[-1]
|
||||||
|
if tld not in self.rdap_urls:
|
||||||
|
return {'error': '不支持的顶级域名'}
|
||||||
|
|
||||||
|
# 构建RDAP查询URL
|
||||||
|
url = self.rdap_urls[tld] + domain
|
||||||
|
|
||||||
|
# 发送请求
|
||||||
|
response = requests.get(url, timeout=10)
|
||||||
|
|
||||||
|
if response.status_code == 200:
|
||||||
|
data = response.json()
|
||||||
|
return self._parse_rdap_response(data)
|
||||||
|
elif response.status_code == 404:
|
||||||
|
return {'status': 'available'}
|
||||||
|
else:
|
||||||
|
return {'error': f'RDAP查询失败: {response.status_code}'}
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def check_register_status(self, domain):
|
||||||
|
"""
|
||||||
|
检查注册状态
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: int - 注册状态码
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
result = self.check_domain(domain)
|
||||||
|
|
||||||
|
if 'error' in result:
|
||||||
|
return 9 # 状态未知
|
||||||
|
|
||||||
|
if result.get('status') == 'available':
|
||||||
|
return 2 # 可注册
|
||||||
|
|
||||||
|
# 检查域名状态
|
||||||
|
statuses = result.get('status', [])
|
||||||
|
if 'clientHold' in statuses:
|
||||||
|
return 7 # clientHold
|
||||||
|
elif 'serverHold' in statuses:
|
||||||
|
return 8 # serverHold
|
||||||
|
elif 'autoRenewPeriod' in statuses:
|
||||||
|
return 4 # 宽限期
|
||||||
|
elif 'redemptionPeriod' in statuses:
|
||||||
|
return 5 # 赎回期
|
||||||
|
elif 'pendingDelete' in statuses:
|
||||||
|
return 6 # 删除期
|
||||||
|
else:
|
||||||
|
return 3 # 已注册
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return 10 # 检测失败
|
||||||
|
|
||||||
|
def _parse_rdap_response(self, data):
|
||||||
|
"""
|
||||||
|
解析RDAP响应
|
||||||
|
|
||||||
|
:param data: RDAP响应数据
|
||||||
|
:return: dict - 解析结果
|
||||||
|
"""
|
||||||
|
result = {
|
||||||
|
'status': 'registered',
|
||||||
|
'domain': data.get('ldhName'),
|
||||||
|
'statuses': data.get('status', []),
|
||||||
|
'registrar': None,
|
||||||
|
'creation_date': None,
|
||||||
|
'expiration_date': None,
|
||||||
|
'last_update': None
|
||||||
|
}
|
||||||
|
|
||||||
|
# 解析注册商信息
|
||||||
|
for entity in data.get('entities', []):
|
||||||
|
if 'registrar' in entity.get('roles', []):
|
||||||
|
result['registrar'] = entity.get('vcardArray', [[], []])[1][1][3]
|
||||||
|
break
|
||||||
|
|
||||||
|
# 解析时间信息
|
||||||
|
for event in data.get('events', []):
|
||||||
|
event_action = event.get('eventAction')
|
||||||
|
event_date = event.get('eventDate')
|
||||||
|
|
||||||
|
if event_action == 'registration':
|
||||||
|
result['creation_date'] = event_date
|
||||||
|
elif event_action == 'expiration':
|
||||||
|
result['expiration_date'] = event_date
|
||||||
|
elif event_action == 'last update':
|
||||||
|
result['last_update'] = event_date
|
||||||
|
|
||||||
|
return result
|
||||||
562
domainCheck/app/detectors/wayback_detector.py
Normal file
562
domainCheck/app/detectors/wayback_detector.py
Normal file
@@ -0,0 +1,562 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :wayback_detector.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:57
|
||||||
|
@explain : Wayback检测器
|
||||||
|
'''
|
||||||
|
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import zlib
|
||||||
|
from base64 import b64decode, b64encode
|
||||||
|
from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
|
||||||
|
|
||||||
|
import redis
|
||||||
|
import requests
|
||||||
|
from requests.adapters import HTTPAdapter
|
||||||
|
from urllib3.util.retry import Retry
|
||||||
|
|
||||||
|
from app.config import config
|
||||||
|
from app.detectors.base import BaseDetector
|
||||||
|
|
||||||
|
|
||||||
|
class WaybackDetector(BaseDetector):
|
||||||
|
"""
|
||||||
|
Wayback检测器
|
||||||
|
"""
|
||||||
|
|
||||||
|
TITLE_PATTERN = re.compile(r'<title[^>]*>(.*?)</title>', re.IGNORECASE | re.DOTALL)
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化Wayback检测器
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.cdx_api_url = 'https://web.archive.org/cdx/search/cdx'
|
||||||
|
# 使用 id_ 直接取快照内容,避免回放页面额外壳层干扰正文匹配。
|
||||||
|
self.snapshot_url = 'https://web.archive.org/web/{timestamp}id_/{domain}'
|
||||||
|
self._timestamp_cache = {}
|
||||||
|
self._title_cache = {}
|
||||||
|
self._cache_lock = threading.Lock()
|
||||||
|
self.session = self._build_session()
|
||||||
|
self.redis_client = self._build_redis_client()
|
||||||
|
|
||||||
|
def _build_session(self):
|
||||||
|
session = requests.Session()
|
||||||
|
retry = Retry(
|
||||||
|
total=max(0, config.WAYBACK_RETRY_COUNT),
|
||||||
|
backoff_factor=0.5,
|
||||||
|
status_forcelist=(429, 500, 502, 503, 504),
|
||||||
|
allowed_methods=frozenset(["GET"]),
|
||||||
|
raise_on_status=False,
|
||||||
|
)
|
||||||
|
adapter = HTTPAdapter(max_retries=retry, pool_connections=10, pool_maxsize=10)
|
||||||
|
session.mount('http://', adapter)
|
||||||
|
session.mount('https://', adapter)
|
||||||
|
session.headers.update({
|
||||||
|
'User-Agent': config.WAYBACK_USER_AGENT,
|
||||||
|
'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
|
||||||
|
})
|
||||||
|
return session
|
||||||
|
|
||||||
|
def _build_redis_client(self):
|
||||||
|
try:
|
||||||
|
client = redis.Redis(
|
||||||
|
host=config.REDIS_HOST,
|
||||||
|
port=config.REDIS_PORT,
|
||||||
|
password=config.REDIS_PASSWORD,
|
||||||
|
db=config.REDIS_DB,
|
||||||
|
decode_responses=True,
|
||||||
|
socket_connect_timeout=5,
|
||||||
|
socket_timeout=5,
|
||||||
|
)
|
||||||
|
client.ping()
|
||||||
|
return client
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _title_cache_key(self, domain, timestamp):
|
||||||
|
return f"domain_tool:wayback_title:{domain}:{timestamp}"
|
||||||
|
|
||||||
|
def _timestamp_cache_key(self, domain):
|
||||||
|
return f"domain_tool:wayback_timestamps:{domain}"
|
||||||
|
|
||||||
|
def _record_cache_key(self, domain):
|
||||||
|
return f"domain_tool:wayback_records:{domain}"
|
||||||
|
|
||||||
|
def _normalize_title(self, title):
|
||||||
|
normalized = html.unescape(title or '')
|
||||||
|
normalized = re.sub(r'\s+', ' ', normalized, flags=re.DOTALL).strip().lower()
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _extract_title(self, content):
|
||||||
|
match = self.TITLE_PATTERN.search(content or '')
|
||||||
|
if not match:
|
||||||
|
return ''
|
||||||
|
return html.unescape(match.group(1)).strip()
|
||||||
|
|
||||||
|
def _load_cached_title(self, domain, timestamp):
|
||||||
|
cache_key = self._title_cache_key(domain, timestamp)
|
||||||
|
with self._cache_lock:
|
||||||
|
if cache_key in self._title_cache:
|
||||||
|
return self._title_cache[cache_key]
|
||||||
|
if self.redis_client:
|
||||||
|
try:
|
||||||
|
raw_value = self.redis_client.get(cache_key)
|
||||||
|
if raw_value:
|
||||||
|
data = json.loads(raw_value)
|
||||||
|
with self._cache_lock:
|
||||||
|
self._title_cache[cache_key] = data
|
||||||
|
return data
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _save_cached_title(self, domain, timestamp, data):
|
||||||
|
cache_key = self._title_cache_key(domain, timestamp)
|
||||||
|
with self._cache_lock:
|
||||||
|
self._title_cache[cache_key] = data
|
||||||
|
if self.redis_client:
|
||||||
|
try:
|
||||||
|
self.redis_client.set(
|
||||||
|
cache_key,
|
||||||
|
json.dumps(data, ensure_ascii=False),
|
||||||
|
ex=max(0, config.WAYBACK_TITLE_CACHE_TTL) or None,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _load_cached_timestamps(self, domain):
|
||||||
|
if domain in self._timestamp_cache:
|
||||||
|
return list(self._timestamp_cache[domain])
|
||||||
|
if not self.redis_client:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
raw_value = self.redis_client.get(self._timestamp_cache_key(domain))
|
||||||
|
if not raw_value:
|
||||||
|
return None
|
||||||
|
compressed = b64decode(raw_value.encode('ascii'))
|
||||||
|
timestamps = json.loads(zlib.decompress(compressed).decode('utf-8'))
|
||||||
|
if isinstance(timestamps, list):
|
||||||
|
self._timestamp_cache[domain] = tuple(timestamps)
|
||||||
|
return list(timestamps)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _save_cached_timestamps(self, domain, timestamps):
|
||||||
|
self._timestamp_cache[domain] = tuple(timestamps)
|
||||||
|
if not self.redis_client:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
payload = json.dumps(timestamps, separators=(',', ':')).encode('utf-8')
|
||||||
|
compressed = zlib.compress(payload, level=6)
|
||||||
|
self.redis_client.set(
|
||||||
|
self._timestamp_cache_key(domain),
|
||||||
|
b64encode(compressed).decode('ascii'),
|
||||||
|
ex=max(0, config.WAYBACK_TIMESTAMP_CACHE_TTL) or None,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _load_cached_records(self, domain):
|
||||||
|
cache_key = self._record_cache_key(domain)
|
||||||
|
with self._cache_lock:
|
||||||
|
if cache_key in self._timestamp_cache:
|
||||||
|
return [
|
||||||
|
{'timestamp': item[0], 'digest': item[1]}
|
||||||
|
for item in self._timestamp_cache[cache_key]
|
||||||
|
]
|
||||||
|
if not self.redis_client:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
raw_value = self.redis_client.get(cache_key)
|
||||||
|
if not raw_value:
|
||||||
|
return None
|
||||||
|
compressed = b64decode(raw_value.encode('ascii'))
|
||||||
|
records = json.loads(zlib.decompress(compressed).decode('utf-8'))
|
||||||
|
if isinstance(records, list):
|
||||||
|
normalized = tuple(
|
||||||
|
(item.get('timestamp', ''), item.get('digest', ''))
|
||||||
|
for item in records if isinstance(item, dict)
|
||||||
|
)
|
||||||
|
with self._cache_lock:
|
||||||
|
self._timestamp_cache[cache_key] = normalized
|
||||||
|
return [
|
||||||
|
{'timestamp': item[0], 'digest': item[1]}
|
||||||
|
for item in normalized if item[0]
|
||||||
|
]
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _save_cached_records(self, domain, records):
|
||||||
|
cache_key = self._record_cache_key(domain)
|
||||||
|
normalized = tuple(
|
||||||
|
(item.get('timestamp', ''), item.get('digest', ''))
|
||||||
|
for item in (records or [])
|
||||||
|
if item and item.get('timestamp')
|
||||||
|
)
|
||||||
|
with self._cache_lock:
|
||||||
|
self._timestamp_cache[cache_key] = normalized
|
||||||
|
if not self.redis_client:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
payload = json.dumps(
|
||||||
|
[{'timestamp': item[0], 'digest': item[1]} for item in normalized],
|
||||||
|
separators=(',', ':')
|
||||||
|
).encode('utf-8')
|
||||||
|
compressed = zlib.compress(payload, level=6)
|
||||||
|
self.redis_client.set(
|
||||||
|
cache_key,
|
||||||
|
b64encode(compressed).decode('ascii'),
|
||||||
|
ex=max(0, config.WAYBACK_TIMESTAMP_CACHE_TTL) or None,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _fetch_cdx_records(self, domain, limit=None, fast_latest=False):
|
||||||
|
response = None
|
||||||
|
try:
|
||||||
|
params = {
|
||||||
|
'url': domain,
|
||||||
|
'output': 'txt',
|
||||||
|
'fl': 'timestamp,digest',
|
||||||
|
'filter': ['statuscode:200', 'mimetype:text/html'],
|
||||||
|
}
|
||||||
|
if limit is not None:
|
||||||
|
params['limit'] = str(limit)
|
||||||
|
if fast_latest:
|
||||||
|
params['fastLatest'] = 'true'
|
||||||
|
response = self.session.get(
|
||||||
|
self.cdx_api_url,
|
||||||
|
params=params,
|
||||||
|
timeout=config.WAYBACK_CDX_TIMEOUT,
|
||||||
|
stream=True,
|
||||||
|
)
|
||||||
|
if response.status_code != 200:
|
||||||
|
self._log_warning(f"获取快照记录失败: {response.status_code}")
|
||||||
|
return []
|
||||||
|
records = []
|
||||||
|
seen = set()
|
||||||
|
for raw_line in response.iter_lines(decode_unicode=True):
|
||||||
|
line = (raw_line or '').strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
parts = line.split()
|
||||||
|
timestamp = parts[0].strip() if parts else ''
|
||||||
|
digest = parts[1].strip() if len(parts) > 1 else ''
|
||||||
|
if not timestamp or timestamp in seen:
|
||||||
|
continue
|
||||||
|
seen.add(timestamp)
|
||||||
|
records.append({'timestamp': timestamp, 'digest': digest})
|
||||||
|
return records
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return []
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
if response is not None:
|
||||||
|
response.close()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_latest_snapshot_record(self, domain):
|
||||||
|
records = self._fetch_cdx_records(domain, limit=-1, fast_latest=True)
|
||||||
|
return records[0] if records else None
|
||||||
|
|
||||||
|
def get_snapshot_records(self, domain):
|
||||||
|
cached_records = self._load_cached_records(domain)
|
||||||
|
if cached_records is not None:
|
||||||
|
return cached_records
|
||||||
|
records = self._fetch_cdx_records(domain)
|
||||||
|
if records:
|
||||||
|
self._save_cached_records(domain, records)
|
||||||
|
self._save_cached_timestamps(domain, [item['timestamp'] for item in records])
|
||||||
|
return records
|
||||||
|
|
||||||
|
def check_domain(self, domain):
|
||||||
|
"""
|
||||||
|
检测域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: dict - 检测结果
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return self.scan_snapshots(domain)
|
||||||
|
except Exception as e:
|
||||||
|
return self._handle_exception(e, domain)
|
||||||
|
|
||||||
|
def get_snapshot_timestamps(self, domain):
|
||||||
|
cached_records = self._load_cached_records(domain)
|
||||||
|
if cached_records is not None:
|
||||||
|
return [item['timestamp'] for item in cached_records if item.get('timestamp')]
|
||||||
|
cached_timestamps = self._load_cached_timestamps(domain)
|
||||||
|
if cached_timestamps is not None:
|
||||||
|
return cached_timestamps
|
||||||
|
records = self.get_snapshot_records(domain)
|
||||||
|
return [item['timestamp'] for item in records if item.get('timestamp')]
|
||||||
|
|
||||||
|
def _fetch_snapshot_title(self, domain, timestamp):
|
||||||
|
cached = self._load_cached_title(domain, timestamp)
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
|
||||||
|
snapshot_url = self.snapshot_url.format(timestamp=timestamp, domain=domain)
|
||||||
|
response = None
|
||||||
|
try:
|
||||||
|
response = self.session.get(
|
||||||
|
snapshot_url,
|
||||||
|
timeout=config.WAYBACK_SNAPSHOT_TIMEOUT,
|
||||||
|
stream=True,
|
||||||
|
)
|
||||||
|
if response.status_code != 200:
|
||||||
|
data = {'timestamp': timestamp, 'title': '', 'ok': False}
|
||||||
|
self._save_cached_title(domain, timestamp, data)
|
||||||
|
return data
|
||||||
|
|
||||||
|
content_type = (response.headers.get('Content-Type') or '').lower()
|
||||||
|
if content_type and 'text/html' not in content_type and 'application/xhtml+xml' not in content_type:
|
||||||
|
data = {'timestamp': timestamp, 'title': '', 'ok': False}
|
||||||
|
self._save_cached_title(domain, timestamp, data)
|
||||||
|
return data
|
||||||
|
|
||||||
|
chunks = []
|
||||||
|
total_bytes = 0
|
||||||
|
found_title = False
|
||||||
|
for chunk in response.iter_content(chunk_size=4096, decode_unicode=True):
|
||||||
|
if not chunk:
|
||||||
|
continue
|
||||||
|
chunks.append(chunk)
|
||||||
|
total_bytes += len(chunk.encode('utf-8', errors='ignore'))
|
||||||
|
current_text = ''.join(chunks)
|
||||||
|
if '</title>' in current_text.lower():
|
||||||
|
found_title = True
|
||||||
|
break
|
||||||
|
if total_bytes >= config.WAYBACK_TITLE_MAX_BYTES:
|
||||||
|
break
|
||||||
|
|
||||||
|
content = ''.join(chunks)
|
||||||
|
title = self._extract_title(content) if found_title or content else ''
|
||||||
|
data = {'timestamp': timestamp, 'title': title, 'ok': True}
|
||||||
|
self._save_cached_title(domain, timestamp, data)
|
||||||
|
return data
|
||||||
|
except Exception:
|
||||||
|
data = {'timestamp': timestamp, 'title': '', 'ok': False}
|
||||||
|
self._save_cached_title(domain, timestamp, data)
|
||||||
|
return data
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
if response is not None:
|
||||||
|
response.close()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_snapshot_years(self, domain):
|
||||||
|
"""
|
||||||
|
获取快照年份
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: list - 快照年份列表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
years = set()
|
||||||
|
for timestamp in self.get_snapshot_timestamps(domain):
|
||||||
|
if len(timestamp) >= 4:
|
||||||
|
years.add(int(timestamp[:4]))
|
||||||
|
return sorted(years)
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return []
|
||||||
|
|
||||||
|
def has_sensitive_content(self, domain):
|
||||||
|
"""
|
||||||
|
检查是否包含敏感内容
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: bool - 是否包含敏感内容
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
result = self.scan_snapshots(domain)
|
||||||
|
return result.get('has_sensitive_content', False)
|
||||||
|
except Exception as e:
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
return False
|
||||||
|
|
||||||
|
def scan_snapshots(self, domain, sensitive_words=None, stop_on_first_hit=True):
|
||||||
|
sensitive_words = sensitive_words or config.load_sensitive_words()
|
||||||
|
latest_record = self.get_latest_snapshot_record(domain)
|
||||||
|
latest_timestamp = (latest_record or {}).get('timestamp')
|
||||||
|
latest_digest = (latest_record or {}).get('digest', '')
|
||||||
|
matched_word = None
|
||||||
|
matched_timestamp = None
|
||||||
|
matched_title = None
|
||||||
|
fetched_snapshot_count = 0
|
||||||
|
failed_snapshot_count = 0
|
||||||
|
unique_title_count = 0
|
||||||
|
duplicate_title_skipped = 0
|
||||||
|
digest_duplicate_skipped = 0
|
||||||
|
started_at = time.time()
|
||||||
|
progress_interval = max(1, config.WAYBACK_PROGRESS_INTERVAL)
|
||||||
|
title_seen = set()
|
||||||
|
digest_seen = set()
|
||||||
|
checked_snapshot_count = 0
|
||||||
|
domain_concurrency = max(1, config.WAYBACK_DOMAIN_CONCURRENCY)
|
||||||
|
|
||||||
|
if latest_timestamp:
|
||||||
|
latest_result = self._fetch_snapshot_title(domain, latest_timestamp)
|
||||||
|
checked_snapshot_count = 1
|
||||||
|
if latest_result and latest_result.get('ok'):
|
||||||
|
fetched_snapshot_count = 1
|
||||||
|
title = latest_result.get('title', '')
|
||||||
|
normalized_title = self._normalize_title(title)
|
||||||
|
if latest_digest:
|
||||||
|
digest_seen.add(latest_digest)
|
||||||
|
if normalized_title:
|
||||||
|
title_seen.add(normalized_title)
|
||||||
|
unique_title_count = 1
|
||||||
|
matched_word = self._find_sensitive_word(title, sensitive_words)
|
||||||
|
if matched_word and stop_on_first_hit:
|
||||||
|
matched_timestamp = latest_timestamp
|
||||||
|
matched_title = title
|
||||||
|
return {
|
||||||
|
'snapshot_years': [int(latest_timestamp[:4])] if len(latest_timestamp) >= 4 else [],
|
||||||
|
'has_sensitive_content': True,
|
||||||
|
'matched_word': matched_word,
|
||||||
|
'matched_timestamp': matched_timestamp,
|
||||||
|
'matched_title': matched_title,
|
||||||
|
'backlink_count': 0,
|
||||||
|
'backlink_count_gt_10': False,
|
||||||
|
'checked_snapshot_count': checked_snapshot_count,
|
||||||
|
'fetched_snapshot_count': fetched_snapshot_count,
|
||||||
|
'failed_snapshot_count': failed_snapshot_count,
|
||||||
|
'unique_title_count': unique_title_count,
|
||||||
|
'duplicate_title_skipped': duplicate_title_skipped,
|
||||||
|
'digest_duplicate_skipped': digest_duplicate_skipped,
|
||||||
|
'elapsed_seconds': round(time.time() - started_at, 2),
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
failed_snapshot_count += 1
|
||||||
|
|
||||||
|
records = sorted(self.get_snapshot_records(domain), key=lambda item: item.get('timestamp', ''), reverse=True)
|
||||||
|
years = sorted({int(item['timestamp'][:4]) for item in records if len(item.get('timestamp', '')) >= 4})
|
||||||
|
checked_snapshot_count = len(records)
|
||||||
|
pending_records = []
|
||||||
|
for item in records:
|
||||||
|
timestamp = item.get('timestamp', '')
|
||||||
|
digest = item.get('digest', '')
|
||||||
|
if not timestamp:
|
||||||
|
continue
|
||||||
|
if latest_timestamp and timestamp == latest_timestamp:
|
||||||
|
continue
|
||||||
|
if digest and digest in digest_seen:
|
||||||
|
digest_duplicate_skipped += 1
|
||||||
|
continue
|
||||||
|
if digest:
|
||||||
|
digest_seen.add(digest)
|
||||||
|
pending_records.append(item)
|
||||||
|
|
||||||
|
with ThreadPoolExecutor(max_workers=domain_concurrency) as executor:
|
||||||
|
pending = {}
|
||||||
|
index = 0
|
||||||
|
finished_count = 1 if latest_timestamp else 0
|
||||||
|
stop_requested = False
|
||||||
|
|
||||||
|
while (index < len(pending_records) or pending) and not stop_requested:
|
||||||
|
while index < len(pending_records) and len(pending) < domain_concurrency and not stop_requested:
|
||||||
|
timestamp = pending_records[index]['timestamp']
|
||||||
|
future = executor.submit(self._fetch_snapshot_title, domain, timestamp)
|
||||||
|
pending[future] = timestamp
|
||||||
|
index += 1
|
||||||
|
|
||||||
|
if not pending:
|
||||||
|
break
|
||||||
|
|
||||||
|
done, _ = wait(list(pending.keys()), return_when=FIRST_COMPLETED)
|
||||||
|
for future in done:
|
||||||
|
timestamp = pending.pop(future, None)
|
||||||
|
finished_count += 1
|
||||||
|
try:
|
||||||
|
result = future.result()
|
||||||
|
except Exception as e:
|
||||||
|
failed_snapshot_count += 1
|
||||||
|
self._handle_exception(e, domain)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if not result or not result.get('ok'):
|
||||||
|
failed_snapshot_count += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
fetched_snapshot_count += 1
|
||||||
|
title = result.get('title', '')
|
||||||
|
normalized_title = self._normalize_title(title)
|
||||||
|
if normalized_title:
|
||||||
|
if normalized_title in title_seen:
|
||||||
|
duplicate_title_skipped += 1
|
||||||
|
else:
|
||||||
|
title_seen.add(normalized_title)
|
||||||
|
unique_title_count += 1
|
||||||
|
matched_word = self._find_sensitive_word(title, sensitive_words)
|
||||||
|
if matched_word:
|
||||||
|
matched_timestamp = timestamp
|
||||||
|
matched_title = title
|
||||||
|
if stop_on_first_hit:
|
||||||
|
stop_requested = True
|
||||||
|
if finished_count % progress_interval == 0:
|
||||||
|
elapsed = round(time.time() - started_at, 2)
|
||||||
|
self._log_info(
|
||||||
|
f"{domain} 时光机进度: {finished_count}/{checked_snapshot_count},成功 {fetched_snapshot_count},失败 {failed_snapshot_count},唯一标题 {unique_title_count},标题重复跳过 {duplicate_title_skipped},digest 重复跳过 {digest_duplicate_skipped},耗时 {elapsed}s"
|
||||||
|
)
|
||||||
|
if config.WAYBACK_REQUEST_DELAY > 0:
|
||||||
|
time.sleep(config.WAYBACK_REQUEST_DELAY)
|
||||||
|
|
||||||
|
if stop_requested:
|
||||||
|
for future in pending:
|
||||||
|
future.cancel()
|
||||||
|
|
||||||
|
return {
|
||||||
|
'snapshot_years': years,
|
||||||
|
'has_sensitive_content': matched_word is not None,
|
||||||
|
'matched_word': matched_word,
|
||||||
|
'matched_timestamp': matched_timestamp,
|
||||||
|
'matched_title': matched_title,
|
||||||
|
'backlink_count': 0,
|
||||||
|
'backlink_count_gt_10': False,
|
||||||
|
'checked_snapshot_count': checked_snapshot_count,
|
||||||
|
'fetched_snapshot_count': fetched_snapshot_count,
|
||||||
|
'failed_snapshot_count': failed_snapshot_count,
|
||||||
|
'unique_title_count': unique_title_count,
|
||||||
|
'duplicate_title_skipped': duplicate_title_skipped,
|
||||||
|
'digest_duplicate_skipped': digest_duplicate_skipped,
|
||||||
|
'elapsed_seconds': round(time.time() - started_at, 2),
|
||||||
|
}
|
||||||
|
|
||||||
|
def _check_sensitive_words(self, content):
|
||||||
|
"""
|
||||||
|
检查敏感词
|
||||||
|
|
||||||
|
:param content: 内容
|
||||||
|
:return: bool - 是否包含敏感词
|
||||||
|
"""
|
||||||
|
return self._find_sensitive_word(content, config.load_sensitive_words()) is not None
|
||||||
|
|
||||||
|
def _find_sensitive_word(self, content, sensitive_words):
|
||||||
|
for word in sensitive_words or []:
|
||||||
|
if word and word in (content or ''):
|
||||||
|
return word
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_backlink_count(self, domain):
|
||||||
|
"""
|
||||||
|
当前策略仅扫描标题,不再抓取正文,友链数量默认返回 0。
|
||||||
|
"""
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def _count_backlinks(self, content):
|
||||||
|
return 0
|
||||||
3
domainCheck/app/domain_suffixes.json
Normal file
3
domainCheck/app/domain_suffixes.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"suffixes": ".com,.net"
|
||||||
|
}
|
||||||
BIN
domainCheck/app/jucha_cookies.pkl
Normal file
BIN
domainCheck/app/jucha_cookies.pkl
Normal file
Binary file not shown.
BIN
domainCheck/app/juming_cookies.pkl
Normal file
BIN
domainCheck/app/juming_cookies.pkl
Normal file
Binary file not shown.
BIN
domainCheck/app/juziseo_cookies.pkl
Normal file
BIN
domainCheck/app/juziseo_cookies.pkl
Normal file
Binary file not shown.
68
domainCheck/app/main.py
Normal file
68
domainCheck/app/main.py
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :main.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:09
|
||||||
|
@explain : 系统主入口
|
||||||
|
'''
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
# 添加项目根目录到 sys.path
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
from PySide6.QtWidgets import QApplication
|
||||||
|
from app.ui.main_window import MainWindow
|
||||||
|
from app.utils.database import Database
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
# 配置日志
|
||||||
|
logger.add(
|
||||||
|
os.path.join(config.LOG_DIR, config.LOG_FILE),
|
||||||
|
level=config.LOG_LEVEL,
|
||||||
|
rotation="10 MB",
|
||||||
|
compression="zip"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""
|
||||||
|
主函数
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 初始化数据库连接
|
||||||
|
db = Database(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
|
||||||
|
# 测试数据库连接
|
||||||
|
db.execute("SELECT 1")
|
||||||
|
logger.info("数据库连接成功")
|
||||||
|
|
||||||
|
# 初始化应用程序
|
||||||
|
app = QApplication(sys.argv)
|
||||||
|
window = MainWindow()
|
||||||
|
window.show()
|
||||||
|
|
||||||
|
# 运行应用程序
|
||||||
|
sys.exit(app.exec())
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"启动应用程序失败: {e}")
|
||||||
|
sys.exit(1)
|
||||||
|
finally:
|
||||||
|
# 关闭数据库连接
|
||||||
|
if 'db' in locals():
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
|
||||||
4
domainCheck/app/proxy_config.json
Normal file
4
domainCheck/app/proxy_config.json
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"proxy_enable": false,
|
||||||
|
"proxy_url": ""
|
||||||
|
}
|
||||||
15108
domainCheck/app/sdk_leg.js
Normal file
15108
domainCheck/app/sdk_leg.js
Normal file
File diff suppressed because one or more lines are too long
897
domainCheck/app/sdk_leg_env.js
Normal file
897
domainCheck/app/sdk_leg_env.js
Normal file
@@ -0,0 +1,897 @@
|
|||||||
|
process_ = process;
|
||||||
|
require_ = require;
|
||||||
|
delete Buffer;
|
||||||
|
// delete process;
|
||||||
|
delete require;
|
||||||
|
delete global;
|
||||||
|
delete module;
|
||||||
|
delete exports;
|
||||||
|
delete __filename;
|
||||||
|
delete __dirname;
|
||||||
|
delete SharedArrayBuffer;
|
||||||
|
|
||||||
|
AsObj = {
|
||||||
|
// print: console.log,
|
||||||
|
print: function () { },
|
||||||
|
// print_:console.log,
|
||||||
|
}
|
||||||
|
|
||||||
|
no_print = ['Boolean','String','parseFloat','Array','Object','prepareStackTrace_'];
|
||||||
|
function watch(object, WatchName) {
|
||||||
|
const handler = {
|
||||||
|
get(target, property, receiver) {
|
||||||
|
if (
|
||||||
|
property !== 'isNaN' &&
|
||||||
|
property !== 'encodeURI' &&
|
||||||
|
property !== "Uint8Array" &&
|
||||||
|
property !== 'undefined' &&
|
||||||
|
property !== 'JSON' &&
|
||||||
|
property !== 'Number' &&
|
||||||
|
!no_print.includes(property) &&
|
||||||
|
property !== Symbol.for('nodejs.util.inspect.custom') &&
|
||||||
|
typeof property !== 'symbol'
|
||||||
|
) {
|
||||||
|
|
||||||
|
if (property === 'global') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (property === 'Buffer') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (property === 'process') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (WatchName === 'config_data') {
|
||||||
|
debugger
|
||||||
|
}
|
||||||
|
if (WatchName.indexOf('.prototype') != -1 && target[property] != undefined) {
|
||||||
|
return Reflect.get(target, property, receiver);
|
||||||
|
}
|
||||||
|
|
||||||
|
AsObj.print(
|
||||||
|
"方法:", "get",
|
||||||
|
"对象:", WatchName,
|
||||||
|
"属性:", property,
|
||||||
|
"属性类型:", typeof property,
|
||||||
|
"属性值:", typeof target[property] == 'object' ? "object" : target[property],
|
||||||
|
"属性值类型:", typeof target[property]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (WatchName === 'top') {
|
||||||
|
return window;
|
||||||
|
}
|
||||||
|
|
||||||
|
return Reflect.get(target, property, receiver);
|
||||||
|
},
|
||||||
|
|
||||||
|
set(target, property, value, receiver) {
|
||||||
|
if (WatchName.indexOf('.prototype') != -1 && value != undefined) {
|
||||||
|
return Reflect.set(target, property, value, receiver);
|
||||||
|
}
|
||||||
|
AsObj.print(
|
||||||
|
"方法:", "set",
|
||||||
|
"对象:", WatchName,
|
||||||
|
"属性:", property,
|
||||||
|
"属性类型:", typeof property,
|
||||||
|
"属性值:", typeof value == 'object' ? "object" : value,
|
||||||
|
"属性值类型:", typeof target[property]
|
||||||
|
);
|
||||||
|
return Reflect.set(target, property, value, receiver);
|
||||||
|
},
|
||||||
|
// in操作 检测
|
||||||
|
has(target, property) {
|
||||||
|
AsObj.print(
|
||||||
|
"代理对象:", WatchName,
|
||||||
|
"方法:", "has",
|
||||||
|
"检查属性:", property,
|
||||||
|
"结果:", typeof target[property] == 'object' ? "object" : target[property],
|
||||||
|
);
|
||||||
|
return Reflect.has(target, property);
|
||||||
|
},
|
||||||
|
// Object.key 检测
|
||||||
|
ownKeys(target) {
|
||||||
|
AsObj.print(
|
||||||
|
"方法:", "ownKeys",
|
||||||
|
"对象:", target+''
|
||||||
|
);
|
||||||
|
return Reflect.ownKeys(target);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
return new Proxy(object, handler);
|
||||||
|
}
|
||||||
|
// function watch(object, WatchName) {
|
||||||
|
// return object
|
||||||
|
// }
|
||||||
|
|
||||||
|
// 保护函数,toString检测
|
||||||
|
const safeFunction = function safeFunction(func) {
|
||||||
|
//处理安全函数
|
||||||
|
Function.prototype.$call = Function.prototype.call;
|
||||||
|
const $toString = Function.toString;
|
||||||
|
const myFunction_toString_symbol = Symbol('('.concat('', ')'));
|
||||||
|
|
||||||
|
const myToString = function myToString() {
|
||||||
|
return typeof this === 'function' && this[myFunction_toString_symbol] || $toString.$call(this);
|
||||||
|
}
|
||||||
|
|
||||||
|
const set_native = function set_native(func, key, value) {
|
||||||
|
Object.defineProperty(func, key, {
|
||||||
|
"enumerable": false,
|
||||||
|
"configurable": true,
|
||||||
|
"writable": true,
|
||||||
|
"value": value
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
delete Function.prototype['toString'];
|
||||||
|
set_native(Function.prototype, "toString", myToString);
|
||||||
|
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
|
||||||
|
|
||||||
|
const safe_Function = function safe_Function(func) {
|
||||||
|
set_native(func, myFunction_toString_symbol, "function" + (func.name ? " " + func.name : "") + "() { [native code] }");
|
||||||
|
}
|
||||||
|
|
||||||
|
return safe_Function(func)
|
||||||
|
}
|
||||||
|
|
||||||
|
//创建函数,并代理上
|
||||||
|
const makeFunction = function makeFunction(name) {
|
||||||
|
v_log = AsObj.print;
|
||||||
|
// 使用 Function 保留函数名
|
||||||
|
func = new Function("v_log", `
|
||||||
|
return function ${name}() {
|
||||||
|
v_log('函数${name}传参-->', arguments);
|
||||||
|
};
|
||||||
|
`)(v_log); // 传递 v_log 到动态函数
|
||||||
|
|
||||||
|
safeFunction(func);
|
||||||
|
func = watch(func,`${name}`);
|
||||||
|
func.prototype = watch(func.prototype, `${name}.prototype`);
|
||||||
|
return func;
|
||||||
|
}
|
||||||
|
|
||||||
|
!(function () {
|
||||||
|
"use strict";
|
||||||
|
const $toString = Function.toString;
|
||||||
|
const myFunction_toString_symbol = Symbol('('.concat('', ')_', (Math.random() + '').toString(36)));
|
||||||
|
const mytoString = function () {
|
||||||
|
return typeof this == 'function' && this[myFunction_toString_symbol] || $toString.call(this);
|
||||||
|
};
|
||||||
|
|
||||||
|
function set_native(func, key, value) {
|
||||||
|
Object.defineProperty(func, key, {
|
||||||
|
"enumerable": false,
|
||||||
|
"configurable": true,
|
||||||
|
"writable": true,
|
||||||
|
"value": value
|
||||||
|
})
|
||||||
|
};
|
||||||
|
delete Function.prototype['toString'];
|
||||||
|
set_native(Function.prototype, "toString", mytoString);
|
||||||
|
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
|
||||||
|
this.func_set_native = function (func) {
|
||||||
|
set_native(func, myFunction_toString_symbol, `function ${myFunction_toString_symbol, func.name || ''}() { [native code] }`)
|
||||||
|
}
|
||||||
|
}).call(globalThis);
|
||||||
|
|
||||||
|
// 重写全局对象原型链
|
||||||
|
function setTostringAndstringTag(obj) {
|
||||||
|
Object.defineProperties(obj.prototype, {
|
||||||
|
[Symbol.toStringTag]: {
|
||||||
|
configurable: true,
|
||||||
|
value: obj.name
|
||||||
|
}
|
||||||
|
});
|
||||||
|
safeFunction(obj);
|
||||||
|
};
|
||||||
|
|
||||||
|
// 创建标签原型
|
||||||
|
function createTagProto(propObj,portotypeObj) {
|
||||||
|
let res = propObj + ' = ' + 'function ' + propObj + '() { throw new TypeError("Illegal constructor"); };\n';
|
||||||
|
res += 'setTostringAndstringTag(' + propObj + ',null);\n';
|
||||||
|
if (portotypeObj) {
|
||||||
|
for (let key in portotypeObj) {
|
||||||
|
res += propObj + '.prototype.' + portotypeObj[key] + '= function ' + portotypeObj[key] + '() {AsObj.print("'+propObj+'.prototype.' + portotypeObj[key] + '原型方法(需在实例对象上补该方法)::",arguments)};\n';
|
||||||
|
res += 'globalThis.func_set_native(' + propObj + '.prototype.' + portotypeObj[key] + ');\n';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
eval(res);
|
||||||
|
}
|
||||||
|
|
||||||
|
Object.defineProperties(globalThis, {
|
||||||
|
[Symbol.toStringTag]: {
|
||||||
|
configurable: true,
|
||||||
|
value: 'Window'
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
for (let key in globalThis) {
|
||||||
|
if (typeof globalThis[key] === 'function') {
|
||||||
|
safeFunction(globalThis[key])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (let key in console) {
|
||||||
|
if (typeof console[key] === 'function') {
|
||||||
|
safeFunction(console[key])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
createTagProto('EventTarget',['addEventListener']);
|
||||||
|
createTagProto('WindowProperties');
|
||||||
|
createTagProto('Window');
|
||||||
|
|
||||||
|
window = globalThis;
|
||||||
|
window.__proto__ = Window.prototype;
|
||||||
|
window.__proto__.__proto__ = WindowProperties.prototype;
|
||||||
|
window.__proto__.__proto__.__proto__ = EventTarget.prototype;
|
||||||
|
Window.__proto__ = EventTarget;
|
||||||
|
|
||||||
|
Object.defineProperty(window, 'WindowProperties', {
|
||||||
|
get: function () {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
function randoms(min, max) {
|
||||||
|
return Math.floor(Math.random() * (max - min + 1) + min)
|
||||||
|
}
|
||||||
|
|
||||||
|
function getRandomValues(buf) {
|
||||||
|
var min = 0,
|
||||||
|
max = 255;
|
||||||
|
if (buf instanceof Uint16Array) {
|
||||||
|
max = 65535;
|
||||||
|
} else if (buf instanceof Uint32Array) {
|
||||||
|
max = 4294967295;
|
||||||
|
}
|
||||||
|
for (var element in buf) {
|
||||||
|
buf[element] = randoms(min, max);
|
||||||
|
}
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
self = window.self = window;
|
||||||
|
frames = window.frames = window;
|
||||||
|
top = window.top = window;
|
||||||
|
parent = window.parent = window;
|
||||||
|
global = window.global = window;
|
||||||
|
|
||||||
|
Object.defineProperty(window, "global", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function global(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "top", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function top(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "self", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function self(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "parent", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function parent(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "frames", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function frames(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
innerWidth = 1536
|
||||||
|
innerHeight = 715
|
||||||
|
outerWidth = 1536
|
||||||
|
outerHeight = 824
|
||||||
|
devicePixelRatio = 1.25;
|
||||||
|
screenLeft = 0;
|
||||||
|
screenX = 0;
|
||||||
|
screenTop = 0;
|
||||||
|
screenY = 0;
|
||||||
|
opener = null;
|
||||||
|
isSecureContext = true;
|
||||||
|
crypto = {
|
||||||
|
getRandomValues:getRandomValues
|
||||||
|
};
|
||||||
|
|
||||||
|
createTagProto('DOMStringMap')
|
||||||
|
createTagProto('HTMLHeadElement',['insertBefore','removeChild'])
|
||||||
|
createTagProto('HTMLBodyElement',['addEventListener','appendChild','removeChild'])
|
||||||
|
createTagProto('HTMLHtmlElement',['getAttribute'])
|
||||||
|
createTagProto('HTMLDocument')
|
||||||
|
createTagProto('Document',['browsingTopics','appendChild','querySelector','evaluate','querySelectorAll','removeChild','requestStorageAccess','requestStorageAccessFor','hasStorageAccess','getElementsByTagName','hasPrivateToken','createElement','hasRedemptionRecord','hasFocus'])
|
||||||
|
createTagProto('Node')
|
||||||
|
document = {};
|
||||||
|
document.__proto__ = HTMLDocument.prototype;
|
||||||
|
document.__proto__.__proto__ = Document.prototype;
|
||||||
|
document.__proto__.__proto__.__proto__ = Node.prototype;
|
||||||
|
document.__proto__.__proto__.__proto__.__proto__ = EventTarget.prototype;
|
||||||
|
HTMLDocument.__proto__ = Document;
|
||||||
|
HTMLDocument.__proto__.__proto__ = Node;
|
||||||
|
HTMLDocument.__proto__.__proto__.__proto__ = EventTarget;
|
||||||
|
Document.__proto__ = Node;
|
||||||
|
Document.__proto__.__proto__ = EventTarget;
|
||||||
|
Node.__proto__ = EventTarget;
|
||||||
|
|
||||||
|
createTagProto('Plugin');
|
||||||
|
createTagProto('PluginArray');
|
||||||
|
plugins0 = {
|
||||||
|
name: 'PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type: 'application/pdf',
|
||||||
|
},
|
||||||
|
'1':{
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins0['0'].enabledPlugin = plugins0;
|
||||||
|
plugins0['1'].enabledPlugin = plugins0;
|
||||||
|
plugins1 = {
|
||||||
|
name: 'Chrome PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1': {
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins1['0'].enabledPlugin = plugins1;
|
||||||
|
plugins1['1'].enabledPlugin = plugins1;
|
||||||
|
plugins2 = {
|
||||||
|
name: 'Chromium PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1': {
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins2['0'].enabledPlugin = plugins2;
|
||||||
|
plugins2['1'].enabledPlugin = plugins2;
|
||||||
|
plugins3 = {
|
||||||
|
name: 'Microsoft Edge PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0':{
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1': {
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins3['0'].enabledPlugin = plugins3;
|
||||||
|
plugins3['1'].enabledPlugin = plugins3;
|
||||||
|
plugins4 = {
|
||||||
|
name: 'WebKit built-in PDF',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1':{
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins4['0'].enabledPlugin = plugins4;
|
||||||
|
plugins4['1'].enabledPlugin = plugins4;
|
||||||
|
plugins = {
|
||||||
|
length: 5,
|
||||||
|
'0': plugins0,
|
||||||
|
'1': plugins1,
|
||||||
|
'2': plugins2,
|
||||||
|
'3': plugins3,
|
||||||
|
'4': plugins4,
|
||||||
|
namedItem : function (name) {
|
||||||
|
AsObj.print('Plugin-namedItem:', name)
|
||||||
|
},
|
||||||
|
item: function (index) {
|
||||||
|
AsObj.print('Plugin-item:', index)
|
||||||
|
return watch(plugins0,'item-'+index);
|
||||||
|
},
|
||||||
|
refresh: function () {
|
||||||
|
AsObj.print('Plugin-refresh:',arguments)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
plugins.__proto__ = PluginArray.prototype;
|
||||||
|
|
||||||
|
MimeTypeArray = function MimeTypeArray() {
|
||||||
|
this.length = 2;
|
||||||
|
this['0'] = {
|
||||||
|
suffixes: 'pdf',
|
||||||
|
type: 'application/pdf',
|
||||||
|
description:"Portable Document Format",
|
||||||
|
enabledPlugin: plugins0
|
||||||
|
};
|
||||||
|
this['1'] = {
|
||||||
|
suffixes: 'pdf',
|
||||||
|
type: 'text/pdf',
|
||||||
|
description:"Portable Document Format",
|
||||||
|
enabledPlugin: plugins0
|
||||||
|
};
|
||||||
|
};
|
||||||
|
MimeTypeArray.prototype.toString = function () { return '[object MimeTypeArray]'; }
|
||||||
|
MimeTypeArray.toString = function () { return 'function MimeTypeArray() { [native code] }'; }
|
||||||
|
Object.defineProperties(MimeTypeArray.prototype, { [Symbol.toStringTag]: { value: 'MimeTypeArray' } })
|
||||||
|
MimeTypeArrayc = new MimeTypeArray();
|
||||||
|
MimeTypeArrayc[Symbol.iterator] = function* () {
|
||||||
|
for (let key in this) {
|
||||||
|
yield this[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 创建电池管理器对象原型
|
||||||
|
const BatteryManager = {
|
||||||
|
level: 1,
|
||||||
|
charging: true,
|
||||||
|
chargingTime: 0,
|
||||||
|
dischargingTime: null,
|
||||||
|
onchargingchange: null,
|
||||||
|
onlevelchange: null,
|
||||||
|
toString: function toString() {
|
||||||
|
return `BatteryManager {
|
||||||
|
charging: ${this.charging},
|
||||||
|
level: ${this.level},
|
||||||
|
chargingTime: ${this.chargingTime},
|
||||||
|
dischargingTime: ${this.dischargingTime}
|
||||||
|
}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
window.BatteryManager = BatteryManager;
|
||||||
|
|
||||||
|
Promise2 = {
|
||||||
|
then: function () {
|
||||||
|
return this;
|
||||||
|
},
|
||||||
|
catch: function (){},
|
||||||
|
};
|
||||||
|
|
||||||
|
createTagProto('Bluetooth');
|
||||||
|
createTagProto('Navigator');
|
||||||
|
Navigator.prototype.hardwareConcurrency = 8;
|
||||||
|
Navigator.prototype.userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36';
|
||||||
|
Navigator.prototype.appVersion = '5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36'
|
||||||
|
Navigator.prototype.appName = 'Netscape';
|
||||||
|
Navigator.prototype.appCodeName = 'Mozilla';
|
||||||
|
Navigator.prototype.vendor = 'Google Inc.';
|
||||||
|
Navigator.prototype.maxTouchPoints = 10;
|
||||||
|
Navigator.prototype.platform = 'Win32';
|
||||||
|
Navigator.prototype.adAuctionComponents = function adAuctionComponents() {
|
||||||
|
AsObj.print('adAuctionComponents:::', arguments)
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.adAuctionComponents)
|
||||||
|
Navigator.prototype.runAdAuction = function runAdAuction() {
|
||||||
|
AsObj.print('runAdAuction:::', arguments)
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.runAdAuction)
|
||||||
|
Navigator.prototype.canLoadAdAuctionFencedFrame = makeFunction('canLoadAdAuctionFencedFrame')
|
||||||
|
Navigator.prototype.deprecatedReplaceInURN = makeFunction('deprecatedReplaceInURN')
|
||||||
|
Navigator.prototype.deprecatedURNToURL = makeFunction('deprecatedURNToURL')
|
||||||
|
Navigator.prototype.joinAdInterestGroup = makeFunction('joinAdInterestGroup')
|
||||||
|
Navigator.prototype.leaveAdInterestGroup = makeFunction('leaveAdInterestGroup')
|
||||||
|
Navigator.prototype.updateAdInterestGroups = makeFunction('updateAdInterestGroups')
|
||||||
|
Navigator.prototype.connection = watch({
|
||||||
|
downlink: 9.1,
|
||||||
|
effectiveType: '4g',
|
||||||
|
rtt: 0,
|
||||||
|
saveData: false,
|
||||||
|
},'connection')
|
||||||
|
Navigator.prototype.language = 'zh-CN';
|
||||||
|
Navigator.prototype.languages = ["zh-CN"];
|
||||||
|
Navigator.prototype.plugins = plugins;
|
||||||
|
Navigator.prototype.webdriver = false;
|
||||||
|
Navigator.prototype.cookieEnabled = true;
|
||||||
|
Navigator.prototype.onLine = true;
|
||||||
|
Navigator.prototype.doNotTrack = null;
|
||||||
|
Navigator.prototype.bluetooth = {};
|
||||||
|
Navigator.prototype.product = 'Gecko'
|
||||||
|
Navigator.prototype.deviceMemory = 8
|
||||||
|
Navigator.prototype.mediaDevices = watch({
|
||||||
|
enumerateDevices: function enumerateDevices() {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = [
|
||||||
|
{deviceId: '', kind: 'audioinput', label: '', groupId: ''},
|
||||||
|
{deviceId: '', kind: 'videoinput', label: '', groupId: ''},
|
||||||
|
{deviceId: '', kind: 'audiooutput', label: '', groupId: ''},
|
||||||
|
]
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
getUserMedia: function getUserMedia() {
|
||||||
|
AsObj.print('getUserMedia:::', arguments)
|
||||||
|
}
|
||||||
|
},'mediaDevices')
|
||||||
|
Navigator.prototype.storage = {
|
||||||
|
estimate: function estimate() {
|
||||||
|
AsObj.print('estimate:::', arguments)
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
usage: 0, // 1GB
|
||||||
|
quota: 2147483648, // 1GB,
|
||||||
|
usageDetails: {caches: 512, indexedDB: 2855}
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Navigator.prototype.webkitPersistentStorage = watch({},'webkitPersistentStorage')
|
||||||
|
Navigator.prototype.webkitTemporaryStorage = watch({
|
||||||
|
queryUsageAndQuota: function queryUsageAndQuota() {
|
||||||
|
AsObj.print('queryUsageAndQuota:::', arguments)
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
usage: 1024 * 1024 * 1024, // 1GB
|
||||||
|
quota: 1024 * 1024 * 1024, // 1GB
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
},'webkitTemporaryStorage')
|
||||||
|
Navigator.prototype.bluetooth.__proto__ = Bluetooth.prototype;
|
||||||
|
Navigator.prototype.javaEnabled = function javaEnabled() {
|
||||||
|
return false
|
||||||
|
};
|
||||||
|
safeFunction(Navigator.prototype.javaEnabled)
|
||||||
|
Navigator.prototype.getBattery = function getBattery() {
|
||||||
|
AsObj.print('getBattery:::', arguments)
|
||||||
|
return Promise.resolve({
|
||||||
|
__proto__: BatteryManager,
|
||||||
|
// 动态参数配置(示例值)
|
||||||
|
level: 1,
|
||||||
|
charging: true,
|
||||||
|
dischargingTime: null // 2小时放电时间
|
||||||
|
})
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.getBattery)
|
||||||
|
Navigator.prototype.registerProtocolHandler = function registerProtocolHandler() {
|
||||||
|
AsObj.print('registerProtocolHandler:::',arguments)
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.registerProtocolHandler)
|
||||||
|
Navigator.prototype.mimeTypes = watch(MimeTypeArrayc,'mimeTypes');
|
||||||
|
Navigator.prototype.geolocation = {
|
||||||
|
getCurrentPosition: function getCurrentPosition() {
|
||||||
|
return Promise2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Navigator.prototype.pdfViewerEnabled = true;
|
||||||
|
Navigator.prototype.doNotTrack = null;
|
||||||
|
Navigator.prototype.keyboard = watch({
|
||||||
|
getLayoutMap: function getLayoutMap() {
|
||||||
|
AsObj.print('Navigator.prototype.keyboard:', arguments)
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
// arguments[0](watch({
|
||||||
|
// size: 48,
|
||||||
|
// values: function () {
|
||||||
|
// return ['k', 'g', '2', '0', 'v', 'a', '`', 'l', '\\', "'", 'w', '8', 'm', 'h', '.', '7', '1', 'p', 'd', 'f', 'o', 'q', 'c', 'n', '[', 'z', 'y', '3', '6', '5', 'x', '/', '\\', ',', '-', '4', 'b', 't', '9', 's', 'i', 'u', '=', 'j', ';', 'r', ']', 'e']
|
||||||
|
// }
|
||||||
|
// }, 'navigator.keyboard.getLayoutMap.then'))
|
||||||
|
return {
|
||||||
|
catch: function () {
|
||||||
|
arguments[0]({
|
||||||
|
message:'getLayoutMap() must be called from a top-level browsing context or allowed by the permission policy.'
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},'navigator.keyboard')
|
||||||
|
Navigator.prototype.permissions = watch({
|
||||||
|
query: function query() {
|
||||||
|
arg_obj = arguments[0];
|
||||||
|
if (arg_obj.name === 'audio_capture') {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
state: 'prompt',
|
||||||
|
onchange: null,
|
||||||
|
name:arg_obj.name
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (arg_obj.name === 'microphone') {
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0](watch({
|
||||||
|
state: 'denied',
|
||||||
|
onchange: null,
|
||||||
|
name: 'audio_capture'
|
||||||
|
},'permissions.query.microphone'));
|
||||||
|
return {catch:function(){}}
|
||||||
|
},
|
||||||
|
catch:function(){}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (arg_obj.name === 'camera') {
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0](watch({
|
||||||
|
state: 'prompt',
|
||||||
|
onchange: null,
|
||||||
|
name: 'video_capture'
|
||||||
|
},'permissions.query.camera'));
|
||||||
|
return {catch:function(){}}
|
||||||
|
},
|
||||||
|
catch:function(){}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
AsObj.print('permissions.query:::', arguments)
|
||||||
|
|
||||||
|
}
|
||||||
|
},'permissions')
|
||||||
|
Navigator.prototype.productSub = '20030107'
|
||||||
|
Navigator.prototype.getGamepads = function getGamepads() {
|
||||||
|
AsObj.print('getGamepads:::', arguments)
|
||||||
|
return [null,null,null,null]
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.getGamepads)
|
||||||
|
|
||||||
|
Navigator.prototype.sendBeacon = makeFunction('sendBeacon')
|
||||||
|
|
||||||
|
Navigator.prototype.deprecatedRunAdAuctionEnforcesKAnonymity = false
|
||||||
|
Navigator.prototype.gpu = watch({
|
||||||
|
getPreferredCanvasFormat: function getPreferredCanvasFormat() {
|
||||||
|
AsObj.print('gpu.getPreferredCanvasFormat:', arguments)
|
||||||
|
return 'bgra8unorm'
|
||||||
|
},
|
||||||
|
wgslLanguageFeatures: watch({
|
||||||
|
size: 7,
|
||||||
|
values: function values() {
|
||||||
|
debugger
|
||||||
|
AsObj.print('wgslLanguageFeatures.values')
|
||||||
|
return ['packed_4x8_integer_dot_product', 'unrestricted_pointer_parameters', 'subgroup_uniformity', 'subgroup_id', 'pointer_composite_access', 'readonly_and_readwrite_storage_textures', 'uniform_buffer_standard_layout']
|
||||||
|
},
|
||||||
|
}, 'gpu.wgslLanguageFeatures'),
|
||||||
|
requestAdapter: function requestAdapter() {
|
||||||
|
AsObj.print('gpu.requestAdapter:', arguments)
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0](watch({
|
||||||
|
features: watch({
|
||||||
|
size: 19,
|
||||||
|
values: function () {
|
||||||
|
return ['depth32float-stencil8', 'rg11b10ufloat-renderable', 'bgra8unorm-storage', 'texture-formats-tier1', 'texture-compression-bc', 'dual-source-blending', 'core-features-and-limits', 'float32-filterable', 'indirect-first-instance', 'float32-blendable', 'depth-clip-control', 'texture-compression-bc-sliced-3d', 'timestamp-query', 'texture-formats-tier2', 'clip-distances', 'shader-f16', 'primitive-index', 'texture-component-swizzle', 'subgroups']
|
||||||
|
}
|
||||||
|
}, 'gpu.requestAdapter.features'),
|
||||||
|
info: watch({ vendor: 'intel', architecture: 'gen-11', device: '', description: '', subgroupMinSize: 16 }, 'gpu.requestAdapter.info'),
|
||||||
|
limits: watch({
|
||||||
|
maxBufferSize: 2147483648,
|
||||||
|
maxStorageBufferBindingSize:2147483644
|
||||||
|
}, 'gpu.requestAdapter.limits'),
|
||||||
|
catch:function(){}
|
||||||
|
}, 'gpu.requestAdapter'));
|
||||||
|
return {
|
||||||
|
catch: function () {
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0]()
|
||||||
|
return {catch:function(){}}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
},
|
||||||
|
catch: function () {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},'navigator.gpu')
|
||||||
|
Navigator.prototype.userAgentData = watch({
|
||||||
|
brands:[
|
||||||
|
{
|
||||||
|
"brand": "Google Chrome",
|
||||||
|
"version": "143"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Chromium",
|
||||||
|
"version": "143"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Not A(Brand",
|
||||||
|
"version": "24"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
mobile: false,
|
||||||
|
platform: "Windows",
|
||||||
|
getHighEntropyValues: function getHighEntropyValues() {
|
||||||
|
if (arguments[0] + '' === 'architecture,bitness,model,platformVersion,uaFullVersion,wow64') {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
"architecture": "x86",
|
||||||
|
"bitness": "64",
|
||||||
|
"brands": [
|
||||||
|
{
|
||||||
|
"brand": "Not:A-Brand",
|
||||||
|
"version": "99"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Google Chrome",
|
||||||
|
"version": "145"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Chromium",
|
||||||
|
"version": "145"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"mobile": false,
|
||||||
|
"model": "",
|
||||||
|
"platform": "Windows",
|
||||||
|
"platformVersion": "10.0.0",
|
||||||
|
"uaFullVersion": "145.0.7632.117",
|
||||||
|
"wow64": false
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
AsObj.print('getHighEntropyValues:::', arguments)
|
||||||
|
}
|
||||||
|
},'userAgentData')
|
||||||
|
|
||||||
|
navigator = {};
|
||||||
|
navigator.__proto__ = Navigator.prototype;
|
||||||
|
|
||||||
|
createTagProto('Location');
|
||||||
|
location = {
|
||||||
|
"ancestorOrigins": {},
|
||||||
|
"href": "https://www.neimanmarcus.com/",
|
||||||
|
"origin": "https://www.neimanmarcus.com",
|
||||||
|
"protocol": "https:",
|
||||||
|
"host": "www.neimanmarcus.com",
|
||||||
|
"hostname": "www.neimanmarcus.com",
|
||||||
|
"port": "",
|
||||||
|
"pathname": "/",
|
||||||
|
"search": "",
|
||||||
|
"hash": ""
|
||||||
|
};
|
||||||
|
|
||||||
|
location.__proto__ = Location.prototype;
|
||||||
|
location.toString = function toString() {
|
||||||
|
return this.href;
|
||||||
|
}
|
||||||
|
|
||||||
|
createTagProto('Screen');
|
||||||
|
Screen.prototype = Object.assign(Screen.prototype, {
|
||||||
|
availHeight: 824,
|
||||||
|
availLeft: 0,
|
||||||
|
availTop: 0,
|
||||||
|
availWidth: 1536,
|
||||||
|
colorDepth: 32,
|
||||||
|
height: 864,
|
||||||
|
isExtended: true,
|
||||||
|
onchange: null,
|
||||||
|
pixelDepth: 24,
|
||||||
|
width: 1536,
|
||||||
|
orientation: {
|
||||||
|
angle: 0,
|
||||||
|
type: "landscape-primary",
|
||||||
|
onchange: null
|
||||||
|
}
|
||||||
|
})
|
||||||
|
screen = {};
|
||||||
|
screen.__proto__ = Screen.prototype;
|
||||||
|
|
||||||
|
createTagProto('History',['replaceState']);
|
||||||
|
history = {};
|
||||||
|
history.__proto__ = History.prototype;
|
||||||
|
|
||||||
|
chrome = {
|
||||||
|
loadTimes: function loadTimes() { },
|
||||||
|
csi: function csi() { },
|
||||||
|
app: {
|
||||||
|
InstallState: { DISABLED: 'disabled', INSTALLED: 'installed', NOT_INSTALLED: 'not_installed' },
|
||||||
|
RunningState: { CANNOT_RUN: 'cannot_run', READY_TO_RUN: 'ready_to_run', RUNNING: 'running' },
|
||||||
|
getDetails:function getDetails(){},
|
||||||
|
getIsInstalled:function getIsInstalled(){},
|
||||||
|
installState:function installState(){},
|
||||||
|
isInstalled: false,
|
||||||
|
runningState: function runningState(){}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
createTagProto('Storage');
|
||||||
|
local = {
|
||||||
|
};
|
||||||
|
localStorage = {
|
||||||
|
getItem: function getItem(key) {
|
||||||
|
AsObj.print("localStorage.getItem::", arguments);
|
||||||
|
if (!local[key]) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return local[key];
|
||||||
|
},
|
||||||
|
setItem: function setItem(key, value) {
|
||||||
|
AsObj.print("localStorage.setItem::", arguments);
|
||||||
|
local[key] = value;
|
||||||
|
},
|
||||||
|
clear: function clear() {
|
||||||
|
local = {};
|
||||||
|
},
|
||||||
|
removeItem: function removeItem(key) {
|
||||||
|
AsObj.print("localStorage.removeItem::", arguments);
|
||||||
|
delete local[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
localStorage.__proto__ = Storage.prototype;
|
||||||
|
sessionStorage = {
|
||||||
|
getItem: function getItem(key) {
|
||||||
|
AsObj.print("sessionStorage.getItem::", arguments);
|
||||||
|
if (!local[key]) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return local[key];
|
||||||
|
},
|
||||||
|
setItem: function setItem(key, value) {
|
||||||
|
AsObj.print("sessionStorage.setItem::", arguments);
|
||||||
|
local[key] = value;
|
||||||
|
},
|
||||||
|
clear: function clear() {
|
||||||
|
local = {};
|
||||||
|
},
|
||||||
|
removeItem: function removeItem(key) {
|
||||||
|
AsObj.print("sessionStorage.removeItem::", arguments);
|
||||||
|
delete local[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sessionStorage.__proto__ = Storage.prototype;
|
||||||
|
|
||||||
|
// window = watch(window, 'window');
|
||||||
|
// global = watch(global, 'global');
|
||||||
|
// globalThis = watch(globalThis, 'globalThis');
|
||||||
|
// self = watch(self, 'self');
|
||||||
|
// crypto = watch(crypto, 'crypto');
|
||||||
|
// performance = watch(performance, 'performance');
|
||||||
|
// document = watch(document, 'document');
|
||||||
|
// navigator = watch(navigator, 'navigator');
|
||||||
|
// location = watch(location, 'location');
|
||||||
|
// screen = watch(screen, 'screen');
|
||||||
|
// history = watch(history, 'history');
|
||||||
|
// localStorage = watch(localStorage, 'localStorage');
|
||||||
|
// sessionStorage = watch(sessionStorage, 'sessionStorage');
|
||||||
|
// chrome = watch(chrome, 'chrome');
|
||||||
|
|
||||||
|
require_('./sdk_leg.js');
|
||||||
|
|
||||||
|
let input = '';
|
||||||
|
// 收集数据
|
||||||
|
process.stdin.on('data', chunk => {
|
||||||
|
input += chunk;
|
||||||
|
});
|
||||||
|
process.stdin.on('end', async () => {
|
||||||
|
var config_data = JSON.parse(input);
|
||||||
|
var cryptoManager = await CaptchaSDKCorecc();
|
||||||
|
var encryptData = await buildEncryptedVerifyRequestcc(config_data, cryptoManager);
|
||||||
|
console.log(JSON.stringify(encryptData));
|
||||||
|
process.exit(0);
|
||||||
|
})
|
||||||
3
domainCheck/app/thread_count.json
Normal file
3
domainCheck/app/thread_count.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"thread_count": "1"
|
||||||
|
}
|
||||||
4
domainCheck/app/ui/__init__.py
Normal file
4
domainCheck/app/ui/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
UI 模块
|
||||||
|
'''
|
||||||
1423
domainCheck/app/ui/domain_filter.py
Normal file
1423
domainCheck/app/ui/domain_filter.py
Normal file
File diff suppressed because it is too large
Load Diff
510
domainCheck/app/ui/domain_import.py
Normal file
510
domainCheck/app/ui/domain_import.py
Normal file
@@ -0,0 +1,510 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :domain_import.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:47
|
||||||
|
@explain : 域名导入界面
|
||||||
|
'''
|
||||||
|
|
||||||
|
from PySide6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QTextEdit, QFileDialog, QProgressBar
|
||||||
|
from PySide6.QtCore import Qt, QThread, Signal
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from app.core.domain_collector import DomainCollector
|
||||||
|
|
||||||
|
|
||||||
|
class ImportThread(QThread):
|
||||||
|
"""
|
||||||
|
导入线程
|
||||||
|
"""
|
||||||
|
progress_updated = Signal(int)
|
||||||
|
finished = Signal(bool, str)
|
||||||
|
|
||||||
|
def __init__(self, domain_list, source_type):
|
||||||
|
"""
|
||||||
|
初始化导入线程
|
||||||
|
|
||||||
|
:param domain_list: 域名列表
|
||||||
|
:param source_type: 来源类型
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.domain_list = domain_list
|
||||||
|
self.source_type = source_type
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
"""
|
||||||
|
运行导入线程
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
collector = DomainCollector()
|
||||||
|
total = len(self.domain_list)
|
||||||
|
|
||||||
|
# 实时更新进度:开始
|
||||||
|
self.progress_updated.emit(0)
|
||||||
|
|
||||||
|
# 标准化域名和检查是否存在(占30%进度)
|
||||||
|
normalized_domains = []
|
||||||
|
for i, domain in enumerate(self.domain_list):
|
||||||
|
from app.utils.domain_utils import normalize_domain
|
||||||
|
import tldextract
|
||||||
|
normalized = normalize_domain(domain)
|
||||||
|
if normalized:
|
||||||
|
# 提取顶级域名
|
||||||
|
ext = tldextract.extract(normalized)
|
||||||
|
tld = ext.suffix
|
||||||
|
normalized_domains.append((normalized, tld))
|
||||||
|
|
||||||
|
# 更新进度
|
||||||
|
progress = int((i + 1) / total * 30)
|
||||||
|
self.progress_updated.emit(progress)
|
||||||
|
|
||||||
|
# 批量检查域名是否存在
|
||||||
|
batch_data = []
|
||||||
|
existing_domains = []
|
||||||
|
if normalized_domains:
|
||||||
|
all_domains = [domain for domain, tld in normalized_domains]
|
||||||
|
existing_domains = collector.db.check_domains_exist(all_domains)
|
||||||
|
existing_set = set(existing_domains)
|
||||||
|
|
||||||
|
# 准备批量添加数据
|
||||||
|
for domain, tld in normalized_domains:
|
||||||
|
if domain not in existing_set:
|
||||||
|
batch_data.append((domain, tld, self.source_type))
|
||||||
|
|
||||||
|
# 分批次添加域名(占70%进度)
|
||||||
|
batch_size = 1000
|
||||||
|
total_batches = len(batch_data)
|
||||||
|
for i in range(0, len(batch_data), batch_size):
|
||||||
|
batch = batch_data[i:i+batch_size]
|
||||||
|
collector.db.add_domains_batch(batch)
|
||||||
|
|
||||||
|
# 更新进度
|
||||||
|
processed = min(i + len(batch), total_batches)
|
||||||
|
progress = 30 + int(processed / total_batches * 70)
|
||||||
|
self.progress_updated.emit(progress)
|
||||||
|
|
||||||
|
# 完成导入
|
||||||
|
self.progress_updated.emit(100)
|
||||||
|
|
||||||
|
# 计算统计信息
|
||||||
|
stats = {
|
||||||
|
'total': total,
|
||||||
|
'valid': len(normalized_domains),
|
||||||
|
'added': len(batch_data),
|
||||||
|
'exists': len(existing_domains),
|
||||||
|
'invalid': total - len(normalized_domains),
|
||||||
|
'failed': 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# 根据统计信息生成消息
|
||||||
|
if stats['added'] > 0:
|
||||||
|
message = f"导入完成: 总域名数 {stats['total']}, 有效域名数 {stats['valid']}, 新增域名数 {stats['added']}, 已存在域名数 {stats['exists']}, 无效域名数 {stats['invalid']}"
|
||||||
|
else:
|
||||||
|
message = f"导入完成: 所有域名已存在,未添加新域名"
|
||||||
|
|
||||||
|
self.finished.emit(True, message)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导入失败: {e}")
|
||||||
|
self.finished.emit(False, f"导入失败: {str(e)}")
|
||||||
|
|
||||||
|
|
||||||
|
class ImportFileThread(QThread):
|
||||||
|
"""
|
||||||
|
文件导入线程,用于处理大文件
|
||||||
|
"""
|
||||||
|
progress_updated = Signal(int)
|
||||||
|
finished = Signal(bool, str)
|
||||||
|
|
||||||
|
def __init__(self, file_path, source_type):
|
||||||
|
"""
|
||||||
|
初始化文件导入线程
|
||||||
|
|
||||||
|
:param file_path: 文件路径
|
||||||
|
:param source_type: 来源类型
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.file_path = file_path
|
||||||
|
self.source_type = source_type
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
"""
|
||||||
|
运行文件导入线程
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
collector = DomainCollector()
|
||||||
|
|
||||||
|
# 首先计算文件中的域名数量
|
||||||
|
total = 0
|
||||||
|
encodings = ['utf-8', 'utf-8-sig', 'gbk', 'gb2312', 'cp936', 'latin-1', 'ascii']
|
||||||
|
encoding = 'utf-8' # 默认编码
|
||||||
|
|
||||||
|
# 尝试不同的编码格式计算域名数量
|
||||||
|
for enc in encodings:
|
||||||
|
try:
|
||||||
|
with open(self.file_path, 'r', encoding=enc) as f:
|
||||||
|
total = sum(1 for line in f if line.strip())
|
||||||
|
encoding = enc
|
||||||
|
break
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if total == 0:
|
||||||
|
# 尝试使用二进制模式读取
|
||||||
|
try:
|
||||||
|
import chardet
|
||||||
|
with open(self.file_path, 'rb') as f:
|
||||||
|
raw_data = f.read()
|
||||||
|
result = chardet.detect(raw_data)
|
||||||
|
encoding = result['encoding']
|
||||||
|
if encoding:
|
||||||
|
total = sum(1 for line in raw_data.decode(encoding).split('\n') if line.strip())
|
||||||
|
else:
|
||||||
|
# 最后尝试使用 replace 模式读取
|
||||||
|
with open(self.file_path, 'r', encoding='utf-8', errors='replace') as f:
|
||||||
|
total = sum(1 for line in f if line.strip())
|
||||||
|
encoding = 'utf-8'
|
||||||
|
except Exception:
|
||||||
|
# 最后尝试使用 replace 模式读取
|
||||||
|
with open(self.file_path, 'r', encoding='utf-8', errors='replace') as f:
|
||||||
|
total = sum(1 for line in f if line.strip())
|
||||||
|
encoding = 'utf-8'
|
||||||
|
|
||||||
|
# 实时更新进度:开始
|
||||||
|
self.progress_updated.emit(0)
|
||||||
|
|
||||||
|
# 逐行读取文件并处理域名
|
||||||
|
normalized_domains = []
|
||||||
|
processed = 0
|
||||||
|
|
||||||
|
with open(self.file_path, 'r', encoding=encoding, errors='replace') as f:
|
||||||
|
for line in f:
|
||||||
|
domain = line.strip()
|
||||||
|
if domain:
|
||||||
|
from app.utils.domain_utils import normalize_domain
|
||||||
|
import tldextract
|
||||||
|
normalized = normalize_domain(domain)
|
||||||
|
if normalized:
|
||||||
|
# 提取顶级域名
|
||||||
|
ext = tldextract.extract(normalized)
|
||||||
|
tld = ext.suffix
|
||||||
|
normalized_domains.append((normalized, tld))
|
||||||
|
|
||||||
|
processed += 1
|
||||||
|
# 更新进度(占30%)
|
||||||
|
progress = int(processed / total * 30)
|
||||||
|
self.progress_updated.emit(progress)
|
||||||
|
|
||||||
|
# 批量检查域名是否存在
|
||||||
|
batch_data = []
|
||||||
|
existing_domains = []
|
||||||
|
if normalized_domains:
|
||||||
|
all_domains = [domain for domain, tld in normalized_domains]
|
||||||
|
existing_domains = collector.db.check_domains_exist(all_domains)
|
||||||
|
existing_set = set(existing_domains)
|
||||||
|
|
||||||
|
# 准备批量添加数据
|
||||||
|
for domain, tld in normalized_domains:
|
||||||
|
if domain not in existing_set:
|
||||||
|
batch_data.append((domain, tld, self.source_type))
|
||||||
|
|
||||||
|
# 分批次添加域名(占70%进度)
|
||||||
|
batch_size = 1000
|
||||||
|
total_batches = len(batch_data)
|
||||||
|
for i in range(0, len(batch_data), batch_size):
|
||||||
|
batch = batch_data[i:i+batch_size]
|
||||||
|
collector.db.add_domains_batch(batch)
|
||||||
|
|
||||||
|
# 更新进度
|
||||||
|
processed_batches = min(i + len(batch), total_batches)
|
||||||
|
progress = 30 + int(processed_batches / total_batches * 70)
|
||||||
|
self.progress_updated.emit(progress)
|
||||||
|
|
||||||
|
# 完成导入
|
||||||
|
self.progress_updated.emit(100)
|
||||||
|
|
||||||
|
# 计算统计信息
|
||||||
|
stats = {
|
||||||
|
'total': total,
|
||||||
|
'valid': len(normalized_domains),
|
||||||
|
'added': len(batch_data),
|
||||||
|
'exists': len(existing_domains),
|
||||||
|
'invalid': total - len(normalized_domains),
|
||||||
|
'failed': 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# 根据统计信息生成消息
|
||||||
|
if stats['added'] > 0:
|
||||||
|
message = f"导入完成: 总域名数 {stats['total']}, 有效域名数 {stats['valid']}, 新增域名数 {stats['added']}, 已存在域名数 {stats['exists']}, 无效域名数 {stats['invalid']}"
|
||||||
|
else:
|
||||||
|
message = f"导入完成: 所有域名已存在,未添加新域名"
|
||||||
|
|
||||||
|
self.finished.emit(True, message)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导入失败: {e}")
|
||||||
|
self.finished.emit(False, f"导入失败: {str(e)}")
|
||||||
|
|
||||||
|
|
||||||
|
class DomainImportWidget(QWidget):
|
||||||
|
"""
|
||||||
|
域名导入界面
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化域名导入界面
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
# 创建布局
|
||||||
|
layout = QVBoxLayout(self)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# 创建文本编辑框
|
||||||
|
self.text_edit = QTextEdit()
|
||||||
|
self.text_edit.setPlaceholderText("请输入域名,一行一个")
|
||||||
|
self.text_edit.setStyleSheet("""
|
||||||
|
QTextEdit {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 10px;
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-radius: 4px;
|
||||||
|
background-color: #f9f9f9;
|
||||||
|
min-height: 300px;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
layout.addWidget(self.text_edit)
|
||||||
|
|
||||||
|
# 创建按钮布局
|
||||||
|
button_layout = QHBoxLayout()
|
||||||
|
|
||||||
|
# 导入文件按钮
|
||||||
|
self.import_file_btn = QPushButton("导入文件")
|
||||||
|
self.import_file_btn.clicked.connect(self.import_file)
|
||||||
|
self.import_file_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #2196F3;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #0b7dda;
|
||||||
|
}
|
||||||
|
QPushButton:disabled {
|
||||||
|
background-color: #cccccc;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.import_file_btn)
|
||||||
|
|
||||||
|
# 开始导入按钮
|
||||||
|
self.start_import_btn = QPushButton("开始导入")
|
||||||
|
self.start_import_btn.clicked.connect(self.start_import)
|
||||||
|
self.start_import_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #4CAF50;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
margin-left: 10px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #45a049;
|
||||||
|
}
|
||||||
|
QPushButton:disabled {
|
||||||
|
background-color: #cccccc;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.start_import_btn)
|
||||||
|
button_layout.setContentsMargins(0, 15, 0, 15)
|
||||||
|
layout.addLayout(button_layout)
|
||||||
|
|
||||||
|
# 创建进度条
|
||||||
|
self.progress_bar = QProgressBar()
|
||||||
|
self.progress_bar.setVisible(False)
|
||||||
|
self.progress_bar.setStyleSheet("""
|
||||||
|
QProgressBar {
|
||||||
|
height: 20px;
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-radius: 10px;
|
||||||
|
background-color: #f0f0f0;
|
||||||
|
margin-bottom: 10px;
|
||||||
|
}
|
||||||
|
QProgressBar::chunk {
|
||||||
|
background-color: #4CAF50;
|
||||||
|
border-radius: 10px;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
layout.addWidget(self.progress_bar)
|
||||||
|
|
||||||
|
# 创建状态标签
|
||||||
|
self.status_label = QLabel("")
|
||||||
|
self.status_label.setAlignment(Qt.AlignCenter)
|
||||||
|
self.status_label.setStyleSheet("font-size: 14px; color: #333; padding: 10px; background-color: #f0f8ff; border-radius: 4px;")
|
||||||
|
layout.addWidget(self.status_label)
|
||||||
|
|
||||||
|
logger.info("域名导入界面创建完成")
|
||||||
|
|
||||||
|
def import_file(self):
|
||||||
|
"""
|
||||||
|
导入文件
|
||||||
|
"""
|
||||||
|
file_path, _ = QFileDialog.getOpenFileName(self, "选择文件", "", "文本文件 (*.txt)")
|
||||||
|
if file_path:
|
||||||
|
try:
|
||||||
|
# 尝试不同的编码格式
|
||||||
|
encodings = ['utf-8', 'utf-8-sig', 'gbk', 'gb2312', 'cp936', 'latin-1', 'ascii']
|
||||||
|
domain_count = 0
|
||||||
|
|
||||||
|
# 尝试使用不同编码读取并计数
|
||||||
|
for encoding in encodings:
|
||||||
|
try:
|
||||||
|
with open(file_path, 'r', encoding=encoding) as f:
|
||||||
|
domain_count = sum(1 for line in f if line.strip())
|
||||||
|
logger.info(f"使用编码 {encoding} 成功读取文件")
|
||||||
|
break
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# 如果仍然失败,尝试使用二进制模式读取并猜测编码
|
||||||
|
if domain_count == 0:
|
||||||
|
try:
|
||||||
|
import chardet
|
||||||
|
with open(file_path, 'rb') as f:
|
||||||
|
raw_data = f.read()
|
||||||
|
result = chardet.detect(raw_data)
|
||||||
|
encoding = result['encoding']
|
||||||
|
if encoding:
|
||||||
|
domain_count = sum(1 for line in raw_data.decode(encoding).split('\n') if line.strip())
|
||||||
|
logger.info(f"使用 chardet 检测到编码 {encoding} 并成功读取文件")
|
||||||
|
else:
|
||||||
|
raise Exception("无法识别文件编码")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"chardet 检测失败: {e}")
|
||||||
|
# 最后尝试使用 replace 模式读取
|
||||||
|
with open(file_path, 'r', encoding='utf-8', errors='replace') as f:
|
||||||
|
domain_count = sum(1 for line in f if line.strip())
|
||||||
|
logger.info("使用 utf-8 replace 模式读取文件")
|
||||||
|
|
||||||
|
# 对于大文件,不显示所有域名,只显示文件路径和域名数量
|
||||||
|
if domain_count > 1000:
|
||||||
|
self.text_edit.setText(f"文件路径: {file_path}\n域名数量: {domain_count}\n\n提示: 由于文件较大,仅显示文件信息,不显示具体域名。")
|
||||||
|
# 保存文件路径,用于后续导入
|
||||||
|
self.imported_file_path = file_path
|
||||||
|
else:
|
||||||
|
# 对于小文件,显示所有域名
|
||||||
|
domains = []
|
||||||
|
for encoding in encodings:
|
||||||
|
try:
|
||||||
|
with open(file_path, 'r', encoding=encoding) as f:
|
||||||
|
domains = f.readlines()
|
||||||
|
break
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
continue
|
||||||
|
if not domains:
|
||||||
|
# 尝试使用二进制模式读取
|
||||||
|
try:
|
||||||
|
import chardet
|
||||||
|
with open(file_path, 'rb') as f:
|
||||||
|
raw_data = f.read()
|
||||||
|
result = chardet.detect(raw_data)
|
||||||
|
encoding = result['encoding']
|
||||||
|
if encoding:
|
||||||
|
domains = raw_data.decode(encoding).split('\n')
|
||||||
|
else:
|
||||||
|
with open(file_path, 'r', encoding='utf-8', errors='replace') as f:
|
||||||
|
domains = f.readlines()
|
||||||
|
except Exception:
|
||||||
|
with open(file_path, 'r', encoding='utf-8', errors='replace') as f:
|
||||||
|
domains = f.readlines()
|
||||||
|
domains = [domain.strip() for domain in domains if domain.strip()]
|
||||||
|
self.text_edit.setText('\n'.join(domains))
|
||||||
|
# 清除文件路径,使用文本框中的域名
|
||||||
|
self.imported_file_path = None
|
||||||
|
|
||||||
|
self.status_label.setText(f"成功读取 {domain_count} 个域名")
|
||||||
|
logger.info(f"成功读取文件: {file_path}, 共 {domain_count} 个域名")
|
||||||
|
except Exception as e:
|
||||||
|
self.status_label.setText(f"读取文件失败: {str(e)}")
|
||||||
|
logger.error(f"读取文件失败: {e}")
|
||||||
|
|
||||||
|
def start_import(self):
|
||||||
|
"""
|
||||||
|
开始导入
|
||||||
|
"""
|
||||||
|
# 检查是否有导入的文件路径
|
||||||
|
if hasattr(self, 'imported_file_path') and self.imported_file_path:
|
||||||
|
# 大文件导入,使用文件路径
|
||||||
|
file_path = self.imported_file_path
|
||||||
|
|
||||||
|
# 显示进度条
|
||||||
|
self.progress_bar.setVisible(True)
|
||||||
|
self.progress_bar.setValue(0)
|
||||||
|
self.status_label.setText("正在导入...")
|
||||||
|
|
||||||
|
# 禁用按钮
|
||||||
|
self.import_file_btn.setEnabled(False)
|
||||||
|
self.start_import_btn.setEnabled(False)
|
||||||
|
|
||||||
|
# 创建并启动导入线程
|
||||||
|
self.import_thread = ImportFileThread(file_path, 7) # 7 表示 TXT 导入
|
||||||
|
self.import_thread.progress_updated.connect(self.update_progress)
|
||||||
|
self.import_thread.finished.connect(self.import_finished)
|
||||||
|
self.import_thread.start()
|
||||||
|
|
||||||
|
logger.info(f"开始从文件导入: {file_path}")
|
||||||
|
else:
|
||||||
|
# 小文件或手动输入的域名
|
||||||
|
domains = self.text_edit.toPlainText().split('\n')
|
||||||
|
domains = [domain.strip() for domain in domains if domain.strip()]
|
||||||
|
|
||||||
|
if not domains:
|
||||||
|
self.status_label.setText("请输入域名")
|
||||||
|
return
|
||||||
|
|
||||||
|
# 显示进度条
|
||||||
|
self.progress_bar.setVisible(True)
|
||||||
|
self.progress_bar.setValue(0)
|
||||||
|
self.status_label.setText("正在导入...")
|
||||||
|
|
||||||
|
# 禁用按钮
|
||||||
|
self.import_file_btn.setEnabled(False)
|
||||||
|
self.start_import_btn.setEnabled(False)
|
||||||
|
|
||||||
|
# 创建并启动导入线程
|
||||||
|
self.import_thread = ImportThread(domains, 7) # 7 表示 TXT 导入
|
||||||
|
self.import_thread.progress_updated.connect(self.update_progress)
|
||||||
|
self.import_thread.finished.connect(self.import_finished)
|
||||||
|
self.import_thread.start()
|
||||||
|
|
||||||
|
logger.info(f"开始导入 {len(domains)} 个域名")
|
||||||
|
|
||||||
|
def update_progress(self, progress):
|
||||||
|
"""
|
||||||
|
更新进度
|
||||||
|
|
||||||
|
:param progress: 进度值
|
||||||
|
"""
|
||||||
|
self.progress_bar.setValue(progress)
|
||||||
|
|
||||||
|
def import_finished(self, success, message):
|
||||||
|
"""
|
||||||
|
导入完成
|
||||||
|
|
||||||
|
:param success: 是否成功
|
||||||
|
:param message: 消息
|
||||||
|
"""
|
||||||
|
self.status_label.setText(message)
|
||||||
|
self.progress_bar.setVisible(False)
|
||||||
|
|
||||||
|
# 启用按钮
|
||||||
|
self.import_file_btn.setEnabled(True)
|
||||||
|
self.start_import_btn.setEnabled(True)
|
||||||
|
|
||||||
|
logger.info(f"导入完成: {message}")
|
||||||
577
domainCheck/app/ui/juming_crawler.py
Normal file
577
domainCheck/app/ui/juming_crawler.py
Normal file
@@ -0,0 +1,577 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :juming_crawler.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 15:00
|
||||||
|
@explain : 聚名网爬取页面
|
||||||
|
'''
|
||||||
|
|
||||||
|
from PySide6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QTextEdit, QProgressBar, QLineEdit, QComboBox, QDateEdit, QCheckBox
|
||||||
|
from PySide6.QtCore import Qt, QThread, Signal, QDate
|
||||||
|
from PySide6.QtGui import QIntValidator
|
||||||
|
from loguru import logger
|
||||||
|
import re
|
||||||
|
import datetime
|
||||||
|
import time
|
||||||
|
|
||||||
|
from app.core.domain_collector import DomainCollector
|
||||||
|
from detect.juming import JM
|
||||||
|
|
||||||
|
|
||||||
|
class JumingCrawlThread(QThread):
|
||||||
|
"""
|
||||||
|
聚名爬取线程
|
||||||
|
"""
|
||||||
|
progress_updated = Signal(int)
|
||||||
|
status_updated = Signal(str)
|
||||||
|
finished = Signal(bool, str)
|
||||||
|
|
||||||
|
def __init__(self, crawl_type, page_start=1, page_size=50, crawl_date=None, auto_date=True):
|
||||||
|
"""
|
||||||
|
初始化聚名爬取线程
|
||||||
|
|
||||||
|
:param crawl_type: 爬取类型 (1: 一口价, 2: 删除列表)
|
||||||
|
:param page_start: 起始页码
|
||||||
|
:param page_size: 每页数量
|
||||||
|
:param crawl_date: 爬取日期(删除列表用)
|
||||||
|
:param auto_date: 是否自动新增日期
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.crawl_type = crawl_type
|
||||||
|
self.page_start = page_start
|
||||||
|
self.page_size = page_size
|
||||||
|
self.crawl_date = crawl_date
|
||||||
|
self.auto_date = auto_date
|
||||||
|
self.is_paused = False
|
||||||
|
self.is_stopped = False
|
||||||
|
self.current_page = 0
|
||||||
|
self.total_count = 0
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
"""
|
||||||
|
运行聚名爬取线程
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 初始化聚名客户端
|
||||||
|
jm = JM()
|
||||||
|
|
||||||
|
# 加载 Cookie
|
||||||
|
jm.load_cookies()
|
||||||
|
logger.info("已加载 Cookie")
|
||||||
|
self.status_updated.emit("已加载 Cookie")
|
||||||
|
|
||||||
|
# 直接开始爬取,不需要登录,因为 Cookie 已经在系统设置页面加载了
|
||||||
|
self.progress_updated.emit(10)
|
||||||
|
|
||||||
|
if self.crawl_type == 1: # 一口价
|
||||||
|
self.progress_updated.emit(30)
|
||||||
|
logger.info("开始获取一口价域名")
|
||||||
|
self.status_updated.emit("开始获取一口价域名")
|
||||||
|
|
||||||
|
# 自动爬取多页
|
||||||
|
page = self.page_start
|
||||||
|
while not self.is_stopped:
|
||||||
|
if self.is_paused:
|
||||||
|
time.sleep(0.1)
|
||||||
|
continue
|
||||||
|
|
||||||
|
self.current_page = page
|
||||||
|
self.status_updated.emit(f"正在爬取第 {page} 页")
|
||||||
|
logger.info(f"正在爬取第 {page} 页")
|
||||||
|
|
||||||
|
# 获取当前页
|
||||||
|
success, html = jm.ykj_get_list(page=page, page_size=self.page_size)
|
||||||
|
if success:
|
||||||
|
pattern_ym = r"<a class='yda1 ydz' ym='([^']*)'"
|
||||||
|
results = re.findall(pattern_ym, html)
|
||||||
|
domains = [domain.strip() for domain in results if domain.strip()]
|
||||||
|
domain_count = len(domains)
|
||||||
|
self.total_count += domain_count
|
||||||
|
|
||||||
|
self.status_updated.emit(f"第 {page} 页找到 {domain_count} 个域名,累计 {self.total_count} 个")
|
||||||
|
logger.info(f"第 {page} 页找到 {domain_count} 个域名,累计 {self.total_count} 个")
|
||||||
|
|
||||||
|
# 自动入库
|
||||||
|
if domains:
|
||||||
|
collector = DomainCollector()
|
||||||
|
stats = collector.add_domains_batch(domains, 1) # 1 表示一口价
|
||||||
|
logger.info(f"自动入库完成: {stats}")
|
||||||
|
self.status_updated.emit(f"自动入库完成: 成功添加 {stats['added']} 个域名")
|
||||||
|
|
||||||
|
# 如果返回的数量小于指定的数量,停止爬取
|
||||||
|
if domain_count < self.page_size:
|
||||||
|
logger.info(f"返回数量小于指定数量,停止爬取")
|
||||||
|
self.status_updated.emit("返回数量小于指定数量,停止爬取")
|
||||||
|
break
|
||||||
|
|
||||||
|
# 增加页码
|
||||||
|
page += 1
|
||||||
|
|
||||||
|
# 模拟网络延迟
|
||||||
|
time.sleep(1)
|
||||||
|
else:
|
||||||
|
logger.error(f"获取一口价域名失败: {html}")
|
||||||
|
self.status_updated.emit(f"获取一口价域名失败: {html}")
|
||||||
|
break
|
||||||
|
|
||||||
|
elif self.crawl_type == 2: # 删除列表
|
||||||
|
self.progress_updated.emit(30)
|
||||||
|
logger.info("开始获取删除域名列表")
|
||||||
|
self.status_updated.emit("开始获取删除域名列表")
|
||||||
|
|
||||||
|
# 使用传入的日期或默认今天
|
||||||
|
start_date_str = self.crawl_date if self.crawl_date else datetime.date.today().strftime("%Y-%m-%d")
|
||||||
|
start_date = datetime.datetime.strptime(start_date_str, "%Y-%m-%d").date()
|
||||||
|
# 计算结束日期:今天 + 4天
|
||||||
|
end_date = datetime.date.today() + datetime.timedelta(days=4)
|
||||||
|
|
||||||
|
if self.auto_date:
|
||||||
|
# 自动新增日期,从起始日期到今天+4天
|
||||||
|
current_date = start_date
|
||||||
|
while current_date <= end_date and not self.is_stopped:
|
||||||
|
if self.is_paused:
|
||||||
|
time.sleep(0.1)
|
||||||
|
continue
|
||||||
|
|
||||||
|
crawl_date = current_date.strftime("%Y-%m-%d")
|
||||||
|
self.status_updated.emit(f"正在爬取 {crawl_date} 的删除域名")
|
||||||
|
logger.info(f"正在爬取 {crawl_date} 的删除域名")
|
||||||
|
|
||||||
|
deleted_domains = jm.new_cha_del(crawl_date)
|
||||||
|
domains = [domain.strip() for domain in deleted_domains if domain.strip()]
|
||||||
|
domain_count = len(domains)
|
||||||
|
self.total_count += domain_count
|
||||||
|
|
||||||
|
self.status_updated.emit(f"{crawl_date} 找到 {domain_count} 个删除域名,累计 {self.total_count} 个")
|
||||||
|
logger.info(f"{crawl_date} 找到 {domain_count} 个删除域名,累计 {self.total_count} 个")
|
||||||
|
|
||||||
|
# 自动入库
|
||||||
|
if domains:
|
||||||
|
collector = DomainCollector()
|
||||||
|
stats = collector.add_domains_batch(domains, 2) # 2 表示删除列表
|
||||||
|
logger.info(f"{crawl_date} 自动入库完成: {stats}")
|
||||||
|
self.status_updated.emit(f"{crawl_date} 自动入库完成: 成功添加 {stats['added']} 个域名")
|
||||||
|
|
||||||
|
# 增加日期
|
||||||
|
current_date = current_date + datetime.timedelta(days=1)
|
||||||
|
|
||||||
|
# 模拟网络延迟
|
||||||
|
time.sleep(1)
|
||||||
|
else:
|
||||||
|
# 只爬取指定日期
|
||||||
|
crawl_date = start_date_str
|
||||||
|
self.status_updated.emit(f"正在爬取 {crawl_date} 的删除域名")
|
||||||
|
logger.info(f"正在爬取 {crawl_date} 的删除域名")
|
||||||
|
|
||||||
|
deleted_domains = jm.new_cha_del(crawl_date)
|
||||||
|
domains = [domain.strip() for domain in deleted_domains if domain.strip()]
|
||||||
|
domain_count = len(domains)
|
||||||
|
self.total_count = domain_count
|
||||||
|
|
||||||
|
self.status_updated.emit(f"找到 {domain_count} 个删除域名")
|
||||||
|
logger.info(f"找到 {domain_count} 个删除域名")
|
||||||
|
|
||||||
|
# 自动入库
|
||||||
|
if domains:
|
||||||
|
collector = DomainCollector()
|
||||||
|
stats = collector.add_domains_batch(domains, 2) # 2 表示删除列表
|
||||||
|
logger.info(f"自动入库完成: {stats}")
|
||||||
|
self.status_updated.emit(f"自动入库完成: 成功添加 {stats['added']} 个域名")
|
||||||
|
|
||||||
|
self.progress_updated.emit(90)
|
||||||
|
|
||||||
|
self.progress_updated.emit(100)
|
||||||
|
self.finished.emit(True, f"成功获取 {self.total_count} 个域名")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"从聚名网爬取失败: {e}")
|
||||||
|
self.status_updated.emit(f"爬取失败: {str(e)}")
|
||||||
|
self.finished.emit(False, f"爬取失败: {str(e)}")
|
||||||
|
|
||||||
|
def pause(self):
|
||||||
|
"""
|
||||||
|
暂停爬取
|
||||||
|
"""
|
||||||
|
self.is_paused = True
|
||||||
|
logger.info("爬取已暂停")
|
||||||
|
self.status_updated.emit("爬取已暂停")
|
||||||
|
|
||||||
|
def resume(self):
|
||||||
|
"""
|
||||||
|
恢复爬取
|
||||||
|
"""
|
||||||
|
self.is_paused = False
|
||||||
|
logger.info("爬取已恢复")
|
||||||
|
self.status_updated.emit("爬取已恢复")
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
"""
|
||||||
|
停止爬取
|
||||||
|
"""
|
||||||
|
self.is_stopped = True
|
||||||
|
logger.info("爬取已停止")
|
||||||
|
self.status_updated.emit("爬取已停止")
|
||||||
|
|
||||||
|
|
||||||
|
class JumingCrawlerWidget(QWidget):
|
||||||
|
"""
|
||||||
|
聚名网爬取页面
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化聚名网爬取页面
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
# 创建布局
|
||||||
|
layout = QVBoxLayout(self)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# 爬取类型选择
|
||||||
|
type_layout = QHBoxLayout()
|
||||||
|
type_label = QLabel("爬取类型:")
|
||||||
|
type_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px;")
|
||||||
|
self.type_combo = QComboBox()
|
||||||
|
self.type_combo.addItem("一口价域名", 1)
|
||||||
|
self.type_combo.addItem("删除列表域名", 2)
|
||||||
|
self.type_combo.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px;")
|
||||||
|
# 监听类型变化
|
||||||
|
self.type_combo.currentIndexChanged.connect(self.on_type_changed)
|
||||||
|
type_layout.addWidget(type_label)
|
||||||
|
type_layout.addWidget(self.type_combo)
|
||||||
|
type_layout.setContentsMargins(0, 0, 0, 15)
|
||||||
|
layout.addLayout(type_layout)
|
||||||
|
|
||||||
|
# 页码和每页数量设置
|
||||||
|
page_layout = QHBoxLayout()
|
||||||
|
|
||||||
|
# 起始页码
|
||||||
|
page_start_label = QLabel("起始页码:")
|
||||||
|
page_start_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px;")
|
||||||
|
self.page_start_edit = QLineEdit("1")
|
||||||
|
# 移除所有限制,允许输入任意正整数
|
||||||
|
self.page_start_edit.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px; width: 120px;")
|
||||||
|
page_layout.addWidget(page_start_label)
|
||||||
|
page_layout.addWidget(self.page_start_edit)
|
||||||
|
|
||||||
|
# 每页数量
|
||||||
|
page_size_label = QLabel("每页数量:")
|
||||||
|
page_size_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px; margin-left: 20px;")
|
||||||
|
self.page_size_edit = QLineEdit("500")
|
||||||
|
self.page_size_edit.setValidator(QIntValidator(1, 1000))
|
||||||
|
self.page_size_edit.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px; width: 80px;")
|
||||||
|
page_layout.addWidget(page_size_label)
|
||||||
|
page_layout.addWidget(self.page_size_edit)
|
||||||
|
page_layout.setContentsMargins(0, 0, 0, 20)
|
||||||
|
layout.addLayout(page_layout)
|
||||||
|
|
||||||
|
# 日期设置(删除列表用)
|
||||||
|
date_container = QWidget()
|
||||||
|
date_layout = QHBoxLayout(date_container)
|
||||||
|
|
||||||
|
# 起始日期
|
||||||
|
date_label = QLabel("起始日期:")
|
||||||
|
date_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px;")
|
||||||
|
self.date_edit = QDateEdit()
|
||||||
|
self.date_edit.setDate(QDate.currentDate())
|
||||||
|
self.date_edit.setCalendarPopup(True)
|
||||||
|
self.date_edit.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px; width: 150px;")
|
||||||
|
# 设置最大日期为今天+4天,最小日期为今天的前4天
|
||||||
|
max_date = QDate.currentDate().addDays(4)
|
||||||
|
min_date = QDate.currentDate().addDays(-4)
|
||||||
|
self.date_edit.setMinimumDate(min_date)
|
||||||
|
self.date_edit.setMaximumDate(max_date)
|
||||||
|
date_layout.addWidget(date_label)
|
||||||
|
date_layout.addWidget(self.date_edit)
|
||||||
|
|
||||||
|
# 自动新增日期选项
|
||||||
|
auto_date_checkbox = QCheckBox("自动新增日期")
|
||||||
|
auto_date_checkbox.setChecked(True)
|
||||||
|
auto_date_checkbox.setStyleSheet("font-size: 14px; margin-left: 20px;")
|
||||||
|
self.auto_date_checkbox = auto_date_checkbox
|
||||||
|
date_layout.addWidget(auto_date_checkbox)
|
||||||
|
|
||||||
|
date_layout.setContentsMargins(0, 0, 0, 20)
|
||||||
|
self.date_container = date_container
|
||||||
|
layout.addWidget(date_container)
|
||||||
|
# 默认隐藏日期输入框
|
||||||
|
self.date_container.setVisible(False)
|
||||||
|
|
||||||
|
# 按钮布局
|
||||||
|
button_layout = QHBoxLayout()
|
||||||
|
|
||||||
|
# 创建开始按钮
|
||||||
|
self.start_btn = QPushButton("开始爬取")
|
||||||
|
self.start_btn.clicked.connect(self.start_crawl)
|
||||||
|
self.start_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #4CAF50;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #45a049;
|
||||||
|
}
|
||||||
|
QPushButton:disabled {
|
||||||
|
background-color: #cccccc;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.start_btn)
|
||||||
|
|
||||||
|
# 创建暂停按钮
|
||||||
|
self.pause_btn = QPushButton("暂停爬取")
|
||||||
|
self.pause_btn.clicked.connect(self.pause_crawl)
|
||||||
|
self.pause_btn.setEnabled(False)
|
||||||
|
self.pause_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #ff9800;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
margin-left: 10px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #f57c00;
|
||||||
|
}
|
||||||
|
QPushButton:disabled {
|
||||||
|
background-color: #cccccc;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.pause_btn)
|
||||||
|
|
||||||
|
# 创建停止按钮
|
||||||
|
self.stop_btn = QPushButton("停止爬取")
|
||||||
|
self.stop_btn.clicked.connect(self.stop_crawl)
|
||||||
|
self.stop_btn.setEnabled(False)
|
||||||
|
self.stop_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #f44336;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
margin-left: 10px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #d32f2f;
|
||||||
|
}
|
||||||
|
QPushButton:disabled {
|
||||||
|
background-color: #cccccc;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.stop_btn)
|
||||||
|
button_layout.setContentsMargins(0, 0, 0, 20)
|
||||||
|
layout.addLayout(button_layout)
|
||||||
|
|
||||||
|
# 创建日志显示区域
|
||||||
|
self.log_edit = QTextEdit()
|
||||||
|
self.log_edit.setPlaceholderText("爬取日志将显示在这里")
|
||||||
|
self.log_edit.setReadOnly(True)
|
||||||
|
self.log_edit.setStyleSheet("""
|
||||||
|
QTextEdit {
|
||||||
|
font-size: 13px;
|
||||||
|
font-family: Consolas, Monaco, monospace;
|
||||||
|
padding: 10px;
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-radius: 4px;
|
||||||
|
background-color: #f9f9f9;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
layout.addWidget(self.log_edit)
|
||||||
|
|
||||||
|
# 创建进度条
|
||||||
|
self.progress_bar = QProgressBar()
|
||||||
|
self.progress_bar.setVisible(False)
|
||||||
|
self.progress_bar.setStyleSheet("""
|
||||||
|
QProgressBar {
|
||||||
|
height: 20px;
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-radius: 10px;
|
||||||
|
background-color: #f0f0f0;
|
||||||
|
margin-top: 10px;
|
||||||
|
margin-bottom: 10px;
|
||||||
|
}
|
||||||
|
QProgressBar::chunk {
|
||||||
|
background-color: #4CAF50;
|
||||||
|
border-radius: 10px;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
layout.addWidget(self.progress_bar)
|
||||||
|
|
||||||
|
# 创建状态标签
|
||||||
|
self.status_label = QLabel("")
|
||||||
|
self.status_label.setAlignment(Qt.AlignCenter)
|
||||||
|
self.status_label.setStyleSheet("font-size: 14px; color: #333; margin-top: 10px; padding: 8px; background-color: #f0f8ff; border-radius: 4px;")
|
||||||
|
layout.addWidget(self.status_label)
|
||||||
|
|
||||||
|
# 存储爬取线程
|
||||||
|
self.crawl_thread = None
|
||||||
|
|
||||||
|
logger.info("聚名网爬取页面创建完成")
|
||||||
|
|
||||||
|
def start_crawl(self):
|
||||||
|
"""
|
||||||
|
开始爬取
|
||||||
|
"""
|
||||||
|
crawl_type = self.type_combo.currentData()
|
||||||
|
|
||||||
|
# 获取起始页码和每页数量
|
||||||
|
try:
|
||||||
|
page_start = int(self.page_start_edit.text())
|
||||||
|
page_size = int(self.page_size_edit.text())
|
||||||
|
if page_start < 1:
|
||||||
|
self.status_label.setText("起始页码必须大于0")
|
||||||
|
return
|
||||||
|
if page_size < 1:
|
||||||
|
self.status_label.setText("每页数量必须大于0")
|
||||||
|
return
|
||||||
|
except ValueError:
|
||||||
|
self.status_label.setText("请输入有效的页码和每页数量")
|
||||||
|
return
|
||||||
|
|
||||||
|
# 显示进度条
|
||||||
|
self.progress_bar.setVisible(True)
|
||||||
|
self.progress_bar.setValue(0)
|
||||||
|
self.status_label.setText("正在爬取域名...")
|
||||||
|
|
||||||
|
# 清空日志
|
||||||
|
self.log_edit.clear()
|
||||||
|
|
||||||
|
# 启用/禁用按钮
|
||||||
|
self.start_btn.setEnabled(False)
|
||||||
|
self.pause_btn.setEnabled(True)
|
||||||
|
self.stop_btn.setEnabled(True)
|
||||||
|
|
||||||
|
# 获取爬取日期和自动新增日期选项(删除列表用)
|
||||||
|
crawl_date = None
|
||||||
|
auto_date = False
|
||||||
|
if crawl_type == 2: # 删除列表
|
||||||
|
crawl_date = self.date_edit.date().toString("yyyy-MM-dd")
|
||||||
|
auto_date = self.auto_date_checkbox.isChecked()
|
||||||
|
|
||||||
|
# 创建并启动爬取线程
|
||||||
|
self.crawl_thread = JumingCrawlThread(crawl_type, page_start, page_size, crawl_date, auto_date)
|
||||||
|
self.crawl_thread.progress_updated.connect(self.update_progress)
|
||||||
|
self.crawl_thread.status_updated.connect(self.update_status)
|
||||||
|
self.crawl_thread.finished.connect(self.crawl_finished)
|
||||||
|
self.crawl_thread.start()
|
||||||
|
|
||||||
|
if crawl_type == 2: # 删除列表
|
||||||
|
auto_date_str = "是" if auto_date else "否"
|
||||||
|
logger.info(f"开始爬取聚名网域名, 类型: {crawl_type}, 爬取日期: {crawl_date}, 自动新增日期: {auto_date_str}")
|
||||||
|
self.log_edit.append(f"开始爬取聚名网域名, 类型: {crawl_type}, 爬取日期: {crawl_date}, 自动新增日期: {auto_date_str}")
|
||||||
|
else: # 一口价
|
||||||
|
logger.info(f"开始爬取聚名网域名, 类型: {crawl_type}, 起始页码: {page_start}, 每页数量: {page_size}")
|
||||||
|
self.log_edit.append(f"开始爬取聚名网域名, 类型: {crawl_type}, 起始页码: {page_start}, 每页数量: {page_size}")
|
||||||
|
|
||||||
|
def pause_crawl(self):
|
||||||
|
"""
|
||||||
|
暂停爬取
|
||||||
|
"""
|
||||||
|
if self.crawl_thread:
|
||||||
|
if self.crawl_thread.is_paused:
|
||||||
|
self.crawl_thread.resume()
|
||||||
|
self.pause_btn.setText("暂停爬取")
|
||||||
|
else:
|
||||||
|
self.crawl_thread.pause()
|
||||||
|
self.pause_btn.setText("恢复爬取")
|
||||||
|
|
||||||
|
def stop_crawl(self):
|
||||||
|
"""
|
||||||
|
停止爬取
|
||||||
|
"""
|
||||||
|
if self.crawl_thread:
|
||||||
|
self.crawl_thread.stop()
|
||||||
|
|
||||||
|
def update_progress(self, progress):
|
||||||
|
"""
|
||||||
|
更新进度
|
||||||
|
|
||||||
|
:param progress: 进度值
|
||||||
|
"""
|
||||||
|
self.progress_bar.setValue(progress)
|
||||||
|
|
||||||
|
def update_status(self, status):
|
||||||
|
"""
|
||||||
|
更新状态
|
||||||
|
|
||||||
|
:param status: 状态消息
|
||||||
|
"""
|
||||||
|
self.status_label.setText(status)
|
||||||
|
self.log_edit.append(status)
|
||||||
|
|
||||||
|
def crawl_finished(self, success, message):
|
||||||
|
"""
|
||||||
|
爬取完成
|
||||||
|
|
||||||
|
:param success: 是否成功
|
||||||
|
:param message: 消息
|
||||||
|
"""
|
||||||
|
self.status_label.setText(message)
|
||||||
|
self.log_edit.append(message)
|
||||||
|
|
||||||
|
# 启用/禁用按钮
|
||||||
|
self.start_btn.setEnabled(True)
|
||||||
|
self.pause_btn.setEnabled(False)
|
||||||
|
self.pause_btn.setText("暂停爬取")
|
||||||
|
self.stop_btn.setEnabled(False)
|
||||||
|
|
||||||
|
self.progress_bar.setVisible(False)
|
||||||
|
|
||||||
|
logger.info(f"聚名网爬取完成: {message}")
|
||||||
|
|
||||||
|
def on_type_changed(self, index):
|
||||||
|
"""
|
||||||
|
爬取类型变化时的处理
|
||||||
|
|
||||||
|
:param index: 选择的索引
|
||||||
|
"""
|
||||||
|
crawl_type = self.type_combo.currentData()
|
||||||
|
if crawl_type == 2: # 删除列表
|
||||||
|
self.date_container.setVisible(True)
|
||||||
|
else: # 一口价
|
||||||
|
self.date_container.setVisible(False)
|
||||||
|
|
||||||
|
|
||||||
|
class ImportThread(QThread):
|
||||||
|
"""
|
||||||
|
导入线程
|
||||||
|
"""
|
||||||
|
progress_updated = Signal(int)
|
||||||
|
finished = Signal(bool, str)
|
||||||
|
|
||||||
|
def __init__(self, domain_list, source_type):
|
||||||
|
"""
|
||||||
|
初始化导入线程
|
||||||
|
|
||||||
|
:param domain_list: 域名列表
|
||||||
|
:param source_type: 来源类型
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
self.domain_list = domain_list
|
||||||
|
self.source_type = source_type
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
"""
|
||||||
|
运行导入线程
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
collector = DomainCollector()
|
||||||
|
total = len(self.domain_list)
|
||||||
|
for i, domain in enumerate(self.domain_list):
|
||||||
|
collector.add_domain(domain, self.source_type)
|
||||||
|
progress = int((i + 1) / total * 100)
|
||||||
|
self.progress_updated.emit(progress)
|
||||||
|
self.finished.emit(True, f"成功导入 {total} 个域名")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"导入失败: {e}")
|
||||||
|
self.finished.emit(False, f"导入失败: {str(e)}")
|
||||||
137
domainCheck/app/ui/main_window.py
Normal file
137
domainCheck/app/ui/main_window.py
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :main_window.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:46
|
||||||
|
@explain : 主窗口
|
||||||
|
'''
|
||||||
|
|
||||||
|
from PySide6.QtWidgets import QMainWindow, QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QTabWidget, QLabel, QScrollArea, QFrame
|
||||||
|
from PySide6.QtCore import Qt
|
||||||
|
from PySide6.QtGui import QIcon
|
||||||
|
import os
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from app.ui.domain_import import DomainImportWidget
|
||||||
|
from app.ui.domain_filter import DomainFilterWidget
|
||||||
|
from app.ui.sensitive_words import SensitiveWordsWidget
|
||||||
|
from app.ui.juming_crawler import JumingCrawlerWidget
|
||||||
|
from app.ui.system_settings import SystemSettingsWidget
|
||||||
|
|
||||||
|
|
||||||
|
class MainWindow(QMainWindow):
|
||||||
|
"""
|
||||||
|
主窗口
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化主窗口
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
# 设置窗口标题和大小
|
||||||
|
self.setWindowTitle("域名工具")
|
||||||
|
self.setGeometry(100, 100, 2000, 800)
|
||||||
|
self.setMinimumSize(1200, 720)
|
||||||
|
|
||||||
|
# 设置窗口图标
|
||||||
|
icon_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "favicon.ico")
|
||||||
|
if os.path.exists(icon_path):
|
||||||
|
self.setWindowIcon(QIcon(icon_path))
|
||||||
|
logger.info(f"设置窗口图标成功: {icon_path}")
|
||||||
|
else:
|
||||||
|
logger.warning(f"窗口图标文件不存在: {icon_path}")
|
||||||
|
|
||||||
|
# 创建中央部件
|
||||||
|
central_widget = QWidget()
|
||||||
|
self.setCentralWidget(central_widget)
|
||||||
|
|
||||||
|
# 创建主布局
|
||||||
|
main_layout = QVBoxLayout(central_widget)
|
||||||
|
|
||||||
|
# 创建标签页
|
||||||
|
self.tab_widget = QTabWidget()
|
||||||
|
self.tab_widget.setStyleSheet("""
|
||||||
|
QTabWidget {
|
||||||
|
font-size: 14px;
|
||||||
|
}
|
||||||
|
QTabBar::tab {
|
||||||
|
padding: 10px 20px;
|
||||||
|
background-color: #f0f0f0;
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-bottom: none;
|
||||||
|
border-top-left-radius: 4px;
|
||||||
|
border-top-right-radius: 4px;
|
||||||
|
margin-right: 2px;
|
||||||
|
}
|
||||||
|
QTabBar::tab:hover {
|
||||||
|
background-color: #e0e0e0;
|
||||||
|
}
|
||||||
|
QTabBar::tab:selected {
|
||||||
|
background-color: white;
|
||||||
|
color: #4CAF50;
|
||||||
|
font-weight: bold;
|
||||||
|
border-color: #4CAF50;
|
||||||
|
}
|
||||||
|
QTabWidget::pane {
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-top: none;
|
||||||
|
border-radius: 0 0 4px 4px;
|
||||||
|
padding: 10px;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
main_layout.addWidget(self.tab_widget)
|
||||||
|
|
||||||
|
# 创建标签页内容
|
||||||
|
self.create_tabs()
|
||||||
|
|
||||||
|
# 记录日志
|
||||||
|
logger.info("主窗口创建完成")
|
||||||
|
|
||||||
|
def create_tabs(self):
|
||||||
|
"""
|
||||||
|
创建标签页
|
||||||
|
"""
|
||||||
|
# 聚名爬取标签页
|
||||||
|
juming_widget = JumingCrawlerWidget()
|
||||||
|
self.tab_widget.addTab(self.wrap_scrollable_tab(juming_widget), "聚名爬取")
|
||||||
|
|
||||||
|
# 域名筛选标签页
|
||||||
|
filter_widget = DomainFilterWidget()
|
||||||
|
self.tab_widget.addTab(self.wrap_scrollable_tab(filter_widget), "域名筛选")
|
||||||
|
|
||||||
|
# 域名导入标签页
|
||||||
|
import_widget = DomainImportWidget()
|
||||||
|
self.tab_widget.addTab(self.wrap_scrollable_tab(import_widget), "域名导入")
|
||||||
|
|
||||||
|
# 敏感词配置标签页
|
||||||
|
sensitive_widget = SensitiveWordsWidget()
|
||||||
|
self.tab_widget.addTab(self.wrap_scrollable_tab(sensitive_widget), "敏感词配置")
|
||||||
|
|
||||||
|
# 系统设置标签页
|
||||||
|
settings_widget = SystemSettingsWidget()
|
||||||
|
self.tab_widget.addTab(self.wrap_scrollable_tab(settings_widget), "系统设置")
|
||||||
|
|
||||||
|
logger.info("标签页创建完成")
|
||||||
|
|
||||||
|
def wrap_scrollable_tab(self, widget):
|
||||||
|
"""
|
||||||
|
给标签页统一包一层滚动区域,保证右侧滚动条始终可用
|
||||||
|
"""
|
||||||
|
container = QWidget()
|
||||||
|
container_layout = QVBoxLayout(container)
|
||||||
|
container_layout.setContentsMargins(0, 0, 0, 0)
|
||||||
|
container_layout.setSpacing(0)
|
||||||
|
|
||||||
|
scroll_area = QScrollArea()
|
||||||
|
scroll_area.setWidgetResizable(True)
|
||||||
|
scroll_area.setFrameShape(QFrame.NoFrame)
|
||||||
|
scroll_area.setVerticalScrollBarPolicy(Qt.ScrollBarAsNeeded)
|
||||||
|
scroll_area.setHorizontalScrollBarPolicy(Qt.ScrollBarAsNeeded)
|
||||||
|
scroll_area.setWidget(widget)
|
||||||
|
|
||||||
|
container_layout.addWidget(scroll_area)
|
||||||
|
return container
|
||||||
284
domainCheck/app/ui/sensitive_words.py
Normal file
284
domainCheck/app/ui/sensitive_words.py
Normal file
@@ -0,0 +1,284 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :sensitive_words.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/8 23:49
|
||||||
|
@explain : 敏感词配置界面
|
||||||
|
'''
|
||||||
|
|
||||||
|
from PySide6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QTextEdit, QFileDialog
|
||||||
|
from PySide6.QtCore import Qt, QThread, Signal
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.database import Database
|
||||||
|
|
||||||
|
|
||||||
|
class SaveWordsThread(QThread):
|
||||||
|
"""
|
||||||
|
保存敏感词线程
|
||||||
|
"""
|
||||||
|
finished = Signal(bool, str, int)
|
||||||
|
|
||||||
|
def __init__(self, words):
|
||||||
|
super().__init__()
|
||||||
|
self.words = words
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
try:
|
||||||
|
db = Database()
|
||||||
|
|
||||||
|
# 先清空现有敏感词
|
||||||
|
db.execute("DELETE FROM sensitive_words")
|
||||||
|
|
||||||
|
# 批量添加敏感词
|
||||||
|
word_tuples = [(word, 'default', 1) for word in self.words]
|
||||||
|
if word_tuples:
|
||||||
|
db.batch_add_sensitive_words(word_tuples)
|
||||||
|
|
||||||
|
db.close()
|
||||||
|
self.finished.emit(True, "成功保存敏感词", len(self.words))
|
||||||
|
except Exception as e:
|
||||||
|
self.finished.emit(False, str(e), 0)
|
||||||
|
|
||||||
|
|
||||||
|
class LoadWordsThread(QThread):
|
||||||
|
"""
|
||||||
|
加载敏感词线程
|
||||||
|
"""
|
||||||
|
finished = Signal(bool, list, str)
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
try:
|
||||||
|
db = Database()
|
||||||
|
sensitive_words = db.get_sensitive_words()
|
||||||
|
words = [word['word'] for word in sensitive_words]
|
||||||
|
db.close()
|
||||||
|
self.finished.emit(True, words, f"成功加载 {len(words)} 个敏感词")
|
||||||
|
except Exception as e:
|
||||||
|
self.finished.emit(False, [], str(e))
|
||||||
|
|
||||||
|
|
||||||
|
class SensitiveWordsWidget(QWidget):
|
||||||
|
"""
|
||||||
|
敏感词配置界面
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
"""
|
||||||
|
初始化敏感词配置界面
|
||||||
|
"""
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
# 创建布局
|
||||||
|
layout = QVBoxLayout(self)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# 创建文本编辑框
|
||||||
|
self.text_edit = QTextEdit()
|
||||||
|
self.text_edit.setPlaceholderText("请输入敏感词,一行一个")
|
||||||
|
self.text_edit.setStyleSheet("""
|
||||||
|
QTextEdit {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 10px;
|
||||||
|
border: 1px solid #ddd;
|
||||||
|
border-radius: 4px;
|
||||||
|
background-color: #f9f9f9;
|
||||||
|
min-height: 300px;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
layout.addWidget(self.text_edit)
|
||||||
|
|
||||||
|
# 创建按钮布局
|
||||||
|
button_layout = QHBoxLayout()
|
||||||
|
|
||||||
|
# 导入按钮
|
||||||
|
self.import_btn = QPushButton("导入")
|
||||||
|
self.import_btn.clicked.connect(self.import_words)
|
||||||
|
self.import_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #2196F3;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #0b7dda;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.import_btn)
|
||||||
|
|
||||||
|
# 导出按钮
|
||||||
|
self.export_btn = QPushButton("导出")
|
||||||
|
self.export_btn.clicked.connect(self.export_words)
|
||||||
|
self.export_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #ff9800;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
margin-left: 10px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #f57c00;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.export_btn)
|
||||||
|
|
||||||
|
# 保存按钮
|
||||||
|
self.save_btn = QPushButton("保存")
|
||||||
|
self.save_btn.clicked.connect(self.save_words)
|
||||||
|
self.save_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #4CAF50;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
margin-left: 10px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #45a049;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.save_btn)
|
||||||
|
|
||||||
|
# 加载按钮
|
||||||
|
self.load_btn = QPushButton("加载")
|
||||||
|
self.load_btn.clicked.connect(self.load_words)
|
||||||
|
self.load_btn.setStyleSheet("""
|
||||||
|
QPushButton {
|
||||||
|
font-size: 14px;
|
||||||
|
padding: 8px 16px;
|
||||||
|
background-color: #9c27b0;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
margin-left: 10px;
|
||||||
|
}
|
||||||
|
QPushButton:hover {
|
||||||
|
background-color: #7b1fa2;
|
||||||
|
}
|
||||||
|
""")
|
||||||
|
button_layout.addWidget(self.load_btn)
|
||||||
|
button_layout.setContentsMargins(0, 15, 0, 15)
|
||||||
|
layout.addLayout(button_layout)
|
||||||
|
|
||||||
|
# 创建状态标签
|
||||||
|
self.status_label = QLabel("")
|
||||||
|
self.status_label.setAlignment(Qt.AlignCenter)
|
||||||
|
self.status_label.setStyleSheet("font-size: 14px; color: #333; padding: 10px; background-color: #f0f8ff; border-radius: 4px;")
|
||||||
|
layout.addWidget(self.status_label)
|
||||||
|
|
||||||
|
# 初始化线程
|
||||||
|
self.save_thread = None
|
||||||
|
self.load_thread = None
|
||||||
|
|
||||||
|
# 加载敏感词
|
||||||
|
self.load_words()
|
||||||
|
|
||||||
|
logger.info("敏感词配置界面创建完成")
|
||||||
|
|
||||||
|
def import_words(self):
|
||||||
|
"""
|
||||||
|
导入敏感词
|
||||||
|
"""
|
||||||
|
file_path, _ = QFileDialog.getOpenFileName(self, "选择文件", "", "文本文件 (*.txt)")
|
||||||
|
if file_path:
|
||||||
|
try:
|
||||||
|
with open(file_path, 'r', encoding='utf-8') as f:
|
||||||
|
words = f.readlines()
|
||||||
|
words = [word.strip() for word in words if word.strip()]
|
||||||
|
self.text_edit.setText('\n'.join(words))
|
||||||
|
self.status_label.setText(f"成功导入 {len(words)} 个敏感词")
|
||||||
|
logger.info(f"成功导入敏感词文件: {file_path}, 共 {len(words)} 个敏感词")
|
||||||
|
except Exception as e:
|
||||||
|
self.status_label.setText(f"导入失败: {str(e)}")
|
||||||
|
logger.error(f"导入敏感词失败: {e}")
|
||||||
|
|
||||||
|
def export_words(self):
|
||||||
|
"""
|
||||||
|
导出敏感词
|
||||||
|
"""
|
||||||
|
words = self.text_edit.toPlainText().split('\n')
|
||||||
|
words = [word.strip() for word in words if word.strip()]
|
||||||
|
|
||||||
|
if not words:
|
||||||
|
self.status_label.setText("没有敏感词可导出")
|
||||||
|
return
|
||||||
|
|
||||||
|
file_path, _ = QFileDialog.getSaveFileName(self, "保存文件", "", "文本文件 (*.txt)")
|
||||||
|
if file_path:
|
||||||
|
try:
|
||||||
|
with open(file_path, 'w', encoding='utf-8') as f:
|
||||||
|
for word in words:
|
||||||
|
f.write(word + '\n')
|
||||||
|
self.status_label.setText(f"成功导出 {len(words)} 个敏感词")
|
||||||
|
logger.info(f"成功导出 {len(words)} 个敏感词到 {file_path}")
|
||||||
|
except Exception as e:
|
||||||
|
self.status_label.setText(f"导出失败: {str(e)}")
|
||||||
|
logger.error(f"导出敏感词失败: {e}")
|
||||||
|
|
||||||
|
def save_words(self):
|
||||||
|
"""
|
||||||
|
保存敏感词
|
||||||
|
"""
|
||||||
|
words = self.text_edit.toPlainText().split('\n')
|
||||||
|
words = [word.strip() for word in words if word.strip()]
|
||||||
|
|
||||||
|
# 禁用按钮,防止重复点击
|
||||||
|
self.save_btn.setEnabled(False)
|
||||||
|
self.status_label.setText("正在保存敏感词...")
|
||||||
|
|
||||||
|
# 创建并启动保存线程
|
||||||
|
self.save_thread = SaveWordsThread(words)
|
||||||
|
self.save_thread.finished.connect(self.on_save_finished)
|
||||||
|
self.save_thread.start()
|
||||||
|
|
||||||
|
def on_save_finished(self, success, message, count):
|
||||||
|
"""
|
||||||
|
保存完成的回调函数
|
||||||
|
"""
|
||||||
|
if success:
|
||||||
|
self.status_label.setText(f"成功保存 {count} 个敏感词")
|
||||||
|
logger.info(f"成功保存 {count} 个敏感词到数据库")
|
||||||
|
else:
|
||||||
|
self.status_label.setText(f"保存失败: {message}")
|
||||||
|
logger.error(f"保存敏感词失败: {message}")
|
||||||
|
|
||||||
|
# 重新启用按钮
|
||||||
|
self.save_btn.setEnabled(True)
|
||||||
|
|
||||||
|
def load_words(self):
|
||||||
|
"""
|
||||||
|
加载敏感词
|
||||||
|
"""
|
||||||
|
# 禁用按钮,防止重复点击
|
||||||
|
self.load_btn.setEnabled(False)
|
||||||
|
self.status_label.setText("正在加载敏感词...")
|
||||||
|
|
||||||
|
# 创建并启动加载线程
|
||||||
|
self.load_thread = LoadWordsThread()
|
||||||
|
self.load_thread.finished.connect(self.on_load_finished)
|
||||||
|
self.load_thread.start()
|
||||||
|
|
||||||
|
def on_load_finished(self, success, words, message):
|
||||||
|
"""
|
||||||
|
加载完成的回调函数
|
||||||
|
"""
|
||||||
|
if success:
|
||||||
|
self.text_edit.setText('\n'.join(words))
|
||||||
|
self.status_label.setText(message)
|
||||||
|
logger.info(message)
|
||||||
|
else:
|
||||||
|
self.status_label.setText(f"加载失败: {message}")
|
||||||
|
logger.error(f"加载敏感词失败: {message}")
|
||||||
|
|
||||||
|
# 重新启用按钮
|
||||||
|
self.load_btn.setEnabled(True)
|
||||||
1402
domainCheck/app/ui/system_settings.py
Normal file
1402
domainCheck/app/ui/system_settings.py
Normal file
File diff suppressed because it is too large
Load Diff
4
domainCheck/app/utils/__init__.py
Normal file
4
domainCheck/app/utils/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
工具类模块
|
||||||
|
'''
|
||||||
1499
domainCheck/app/utils/database.py
Normal file
1499
domainCheck/app/utils/database.py
Normal file
File diff suppressed because it is too large
Load Diff
195
domainCheck/app/utils/domain_utils.py
Normal file
195
domainCheck/app/utils/domain_utils.py
Normal file
@@ -0,0 +1,195 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :domain_utils.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:05
|
||||||
|
@explain : 域名工具类
|
||||||
|
'''
|
||||||
|
|
||||||
|
import re
|
||||||
|
import tldextract
|
||||||
|
from loguru import logger
|
||||||
|
from app.utils.status_codes import DETECT_STATUS_LABELS, REGISTER_STATUS_LABELS, USE_STATUS_LABELS
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_domain(domain):
|
||||||
|
"""
|
||||||
|
标准化域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: str - 标准化后的域名
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 转换为小写
|
||||||
|
domain = domain.lower()
|
||||||
|
|
||||||
|
# 去除空格
|
||||||
|
domain = domain.strip()
|
||||||
|
|
||||||
|
# 去除协议
|
||||||
|
domain = re.sub(r'^https?://', '', domain)
|
||||||
|
|
||||||
|
# 去除路径和查询参数
|
||||||
|
domain = domain.split('/')[0]
|
||||||
|
domain = domain.split('?')[0]
|
||||||
|
|
||||||
|
# 去除端口
|
||||||
|
domain = domain.split(':')[0]
|
||||||
|
|
||||||
|
# 只保留主域
|
||||||
|
ext = tldextract.extract(domain)
|
||||||
|
if ext.domain and ext.suffix:
|
||||||
|
domain = f"{ext.domain}.{ext.suffix}"
|
||||||
|
|
||||||
|
# 验证域名格式
|
||||||
|
if not is_valid_domain(domain):
|
||||||
|
return None
|
||||||
|
|
||||||
|
return domain
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"标准化域名出错: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def is_valid_domain(domain):
|
||||||
|
"""
|
||||||
|
验证域名格式
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: bool - 是否有效
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 域名格式正则
|
||||||
|
pattern = r'^[a-zA-Z0-9][a-zA-Z0-9-]{0,61}[a-zA-Z0-9]\.[a-zA-Z]{2,}$'
|
||||||
|
return bool(re.match(pattern, domain))
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"验证域名格式出错: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def extract_tld(domain):
|
||||||
|
"""
|
||||||
|
提取顶级域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: str - 顶级域名
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
ext = tldextract.extract(domain)
|
||||||
|
return ext.suffix
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"提取顶级域名出错: {e}")
|
||||||
|
return ''
|
||||||
|
|
||||||
|
|
||||||
|
def extract_domain(domain):
|
||||||
|
"""
|
||||||
|
提取主域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: str - 主域名
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
ext = tldextract.extract(domain)
|
||||||
|
if ext.domain and ext.suffix:
|
||||||
|
return f"{ext.domain}.{ext.suffix}"
|
||||||
|
return domain
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"提取主域名出错: {e}")
|
||||||
|
return domain
|
||||||
|
|
||||||
|
|
||||||
|
def is_com_or_net(domain):
|
||||||
|
"""
|
||||||
|
检查是否为 .com 或 .net 域名
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: bool - 是否为 .com 或 .net 域名
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
tld = extract_tld(domain)
|
||||||
|
return tld in ['com', 'net']
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"检查域名后缀出错: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def generate_domain_variants(domain):
|
||||||
|
"""
|
||||||
|
生成域名变体
|
||||||
|
|
||||||
|
:param domain: 域名
|
||||||
|
:return: list - 域名变体列表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
variants = []
|
||||||
|
|
||||||
|
# 原始域名
|
||||||
|
variants.append(domain)
|
||||||
|
|
||||||
|
# 添加 www
|
||||||
|
if not domain.startswith('www.'):
|
||||||
|
variants.append(f"www.{domain}")
|
||||||
|
|
||||||
|
# 移除 www
|
||||||
|
if domain.startswith('www.'):
|
||||||
|
variants.append(domain[4:])
|
||||||
|
|
||||||
|
return variants
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"生成域名变体出错: {e}")
|
||||||
|
return [domain]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_domain_status(status_code):
|
||||||
|
"""
|
||||||
|
解析域名状态码
|
||||||
|
|
||||||
|
:param status_code: 状态码
|
||||||
|
:return: str - 状态描述
|
||||||
|
"""
|
||||||
|
return REGISTER_STATUS_LABELS.get(status_code, '未知')
|
||||||
|
|
||||||
|
|
||||||
|
def parse_use_status(status_code):
|
||||||
|
"""
|
||||||
|
解析使用状态码
|
||||||
|
|
||||||
|
:param status_code: 状态码
|
||||||
|
:return: str - 状态描述
|
||||||
|
"""
|
||||||
|
return USE_STATUS_LABELS.get(status_code, '未知')
|
||||||
|
|
||||||
|
|
||||||
|
def parse_detect_status(status_code):
|
||||||
|
"""
|
||||||
|
解析检测状态码
|
||||||
|
|
||||||
|
:param status_code: 状态码
|
||||||
|
:return: str - 状态描述
|
||||||
|
"""
|
||||||
|
return DETECT_STATUS_LABELS.get(status_code, '未知')
|
||||||
|
|
||||||
|
|
||||||
|
def parse_source_type(source_type):
|
||||||
|
"""
|
||||||
|
解析来源类型
|
||||||
|
|
||||||
|
:param source_type: 来源类型
|
||||||
|
:return: str - 来源描述
|
||||||
|
"""
|
||||||
|
source_map = {
|
||||||
|
1: '聚名一口价',
|
||||||
|
2: '聚名过期删除',
|
||||||
|
3: 'zone file',
|
||||||
|
4: '搜索引擎采集',
|
||||||
|
5: '企业目录采集',
|
||||||
|
6: '手工录入',
|
||||||
|
7: 'TXT 导入',
|
||||||
|
8: '第三方接口',
|
||||||
|
9: '其它'
|
||||||
|
}
|
||||||
|
|
||||||
|
return source_map.get(source_type, '未知')
|
||||||
157
domainCheck/app/utils/http_utils.py
Normal file
157
domainCheck/app/utils/http_utils.py
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :http_utils.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 0:06
|
||||||
|
@explain : HTTP工具类
|
||||||
|
'''
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from curl_cffi import requests as curl_requests
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
|
||||||
|
class HTTPUtils:
|
||||||
|
"""
|
||||||
|
HTTP工具类
|
||||||
|
"""
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get(url, headers=None, params=None, timeout=10, proxies=None, use_curl=False):
|
||||||
|
"""
|
||||||
|
发送GET请求
|
||||||
|
|
||||||
|
:param url: 请求URL
|
||||||
|
:param headers: 请求头
|
||||||
|
:param params: 查询参数
|
||||||
|
:param timeout: 超时时间
|
||||||
|
:param proxies: 代理
|
||||||
|
:param use_curl: 是否使用curl_cffi
|
||||||
|
:return: requests.Response - 响应对象
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
if use_curl:
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.get(url, headers=headers, params=params, timeout=timeout, proxies=proxies, impersonate='chrome')
|
||||||
|
else:
|
||||||
|
# 使用requests
|
||||||
|
response = requests.get(url, headers=headers, params=params, timeout=timeout, proxies=proxies)
|
||||||
|
|
||||||
|
response.raise_for_status() # 检查状态码
|
||||||
|
return response
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"GET请求失败: {url}, 错误: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def post(url, headers=None, data=None, json=None, timeout=10, proxies=None, use_curl=False):
|
||||||
|
"""
|
||||||
|
发送POST请求
|
||||||
|
|
||||||
|
:param url: 请求URL
|
||||||
|
:param headers: 请求头
|
||||||
|
:param data: 表单数据
|
||||||
|
:param json: JSON数据
|
||||||
|
:param timeout: 超时时间
|
||||||
|
:param proxies: 代理
|
||||||
|
:param use_curl: 是否使用curl_cffi
|
||||||
|
:return: requests.Response - 响应对象
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
if use_curl:
|
||||||
|
# 使用curl_cffi模拟浏览器
|
||||||
|
response = curl_requests.post(url, headers=headers, data=data, json=json, timeout=timeout, proxies=proxies, impersonate='chrome')
|
||||||
|
else:
|
||||||
|
# 使用requests
|
||||||
|
response = requests.post(url, headers=headers, data=data, json=json, timeout=timeout, proxies=proxies)
|
||||||
|
|
||||||
|
response.raise_for_status() # 检查状态码
|
||||||
|
return response
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"POST请求失败: {url}, 错误: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_random_user_agent():
|
||||||
|
"""
|
||||||
|
获取随机用户代理
|
||||||
|
|
||||||
|
:return: str - 用户代理
|
||||||
|
"""
|
||||||
|
user_agents = [
|
||||||
|
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36',
|
||||||
|
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36',
|
||||||
|
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Firefox/138.0',
|
||||||
|
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/146.0.0.0',
|
||||||
|
'Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.6 Safari/605.1.15'
|
||||||
|
]
|
||||||
|
|
||||||
|
import random
|
||||||
|
return random.choice(user_agents)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_default_headers():
|
||||||
|
"""
|
||||||
|
获取默认请求头
|
||||||
|
|
||||||
|
:return: dict - 请求头
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
'User-Agent': HTTPUtils.get_random_user_agent(),
|
||||||
|
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
|
||||||
|
'Accept-Language': 'zh-CN,zh;q=0.9',
|
||||||
|
'Accept-Encoding': 'gzip, deflate, br',
|
||||||
|
'Connection': 'keep-alive',
|
||||||
|
'Upgrade-Insecure-Requests': '1',
|
||||||
|
'Sec-Fetch-Dest': 'document',
|
||||||
|
'Sec-Fetch-Mode': 'navigate',
|
||||||
|
'Sec-Fetch-Site': 'none',
|
||||||
|
'Sec-Fetch-User': '?1'
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def retry_request(func, max_retries=3, delay=1):
|
||||||
|
"""
|
||||||
|
重试请求
|
||||||
|
|
||||||
|
:param func: 请求函数
|
||||||
|
:param max_retries: 最大重试次数
|
||||||
|
:param delay: 重试延迟
|
||||||
|
:return: 函数返回值
|
||||||
|
"""
|
||||||
|
import time
|
||||||
|
|
||||||
|
for i in range(max_retries):
|
||||||
|
try:
|
||||||
|
result = func()
|
||||||
|
if result:
|
||||||
|
return result
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"请求失败,第 {i+1} 次重试: {e}")
|
||||||
|
|
||||||
|
if i < max_retries - 1:
|
||||||
|
time.sleep(delay)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def check_proxy(proxy):
|
||||||
|
"""
|
||||||
|
检查代理是否可用
|
||||||
|
|
||||||
|
:param proxy: 代理URL
|
||||||
|
:return: bool - 是否可用
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
proxies = {
|
||||||
|
'http': proxy,
|
||||||
|
'https': proxy
|
||||||
|
}
|
||||||
|
|
||||||
|
response = requests.get('https://www.baidu.com', proxies=proxies, timeout=5)
|
||||||
|
return response.status_code == 200
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"代理检查失败: {proxy}, 错误: {e}")
|
||||||
|
return False
|
||||||
66
domainCheck/app/utils/status_codes.py
Normal file
66
domainCheck/app/utils/status_codes.py
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
"""Centralized status codes and labels used across detection, database and UI."""
|
||||||
|
|
||||||
|
REGISTER_STATUS_PENDING = 0
|
||||||
|
REGISTER_STATUS_AVAILABLE = 2
|
||||||
|
REGISTER_STATUS_REGISTERED = 3
|
||||||
|
REGISTER_STATUS_GRACE = 4
|
||||||
|
REGISTER_STATUS_REDEMPTION = 5
|
||||||
|
REGISTER_STATUS_PENDING_DELETE = 6
|
||||||
|
REGISTER_STATUS_CLIENT_HOLD = 7
|
||||||
|
REGISTER_STATUS_SERVER_HOLD = 8
|
||||||
|
REGISTER_STATUS_UNKNOWN = 9
|
||||||
|
REGISTER_STATUS_FAILED = 10
|
||||||
|
|
||||||
|
DETECT_STATUS_PENDING = 0
|
||||||
|
DETECT_STATUS_COMPLETED = 1
|
||||||
|
DETECT_STATUS_RUNNING = 2
|
||||||
|
DETECT_STATUS_BLACKLISTED = 3
|
||||||
|
DETECT_STATUS_FAILED = 4
|
||||||
|
|
||||||
|
USE_STATUS_UNUSED = 0
|
||||||
|
USE_STATUS_USED = 1
|
||||||
|
USE_STATUS_SOLD = 2
|
||||||
|
USE_STATUS_RESERVED = 3
|
||||||
|
|
||||||
|
REVIEW_STATUS_NONE = 0
|
||||||
|
REVIEW_STATUS_PENDING = 1
|
||||||
|
REVIEW_STATUS_APPROVED = 2
|
||||||
|
REVIEW_STATUS_REJECTED = 3
|
||||||
|
|
||||||
|
THIRD_PARTY_STATUS_PENDING = 0
|
||||||
|
THIRD_PARTY_STATUS_DONE = 1
|
||||||
|
|
||||||
|
REGISTER_STATUS_LABELS = {
|
||||||
|
REGISTER_STATUS_PENDING: '待检测',
|
||||||
|
REGISTER_STATUS_AVAILABLE: '可注册',
|
||||||
|
REGISTER_STATUS_REGISTERED: '已注册',
|
||||||
|
REGISTER_STATUS_GRACE: '宽限期',
|
||||||
|
REGISTER_STATUS_REDEMPTION: '赎回期',
|
||||||
|
REGISTER_STATUS_PENDING_DELETE: '删除期',
|
||||||
|
REGISTER_STATUS_CLIENT_HOLD: 'clientHold',
|
||||||
|
REGISTER_STATUS_SERVER_HOLD: 'serverHold',
|
||||||
|
REGISTER_STATUS_UNKNOWN: '状态未知',
|
||||||
|
REGISTER_STATUS_FAILED: '检测失败',
|
||||||
|
}
|
||||||
|
|
||||||
|
DETECT_STATUS_LABELS = {
|
||||||
|
DETECT_STATUS_PENDING: '待检测',
|
||||||
|
DETECT_STATUS_COMPLETED: '检测完成',
|
||||||
|
DETECT_STATUS_RUNNING: '检测中',
|
||||||
|
DETECT_STATUS_BLACKLISTED: '黑名单',
|
||||||
|
DETECT_STATUS_FAILED: '检测失败',
|
||||||
|
}
|
||||||
|
|
||||||
|
USE_STATUS_LABELS = {
|
||||||
|
USE_STATUS_UNUSED: '未使用',
|
||||||
|
USE_STATUS_USED: '已经使用',
|
||||||
|
USE_STATUS_SOLD: '已经卖出',
|
||||||
|
USE_STATUS_RESERVED: '已经预定',
|
||||||
|
}
|
||||||
|
|
||||||
|
REVIEW_STATUS_LABELS = {
|
||||||
|
REVIEW_STATUS_NONE: '无需复核',
|
||||||
|
REVIEW_STATUS_PENDING: '待人工复核',
|
||||||
|
REVIEW_STATUS_APPROVED: '人工通过',
|
||||||
|
REVIEW_STATUS_REJECTED: '人工拒绝',
|
||||||
|
}
|
||||||
74
domainCheck/check_database.py
Normal file
74
domainCheck/check_database.py
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :check_database.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 21:40
|
||||||
|
@explain : 检查数据库表结构和注释
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
|
||||||
|
def check_database():
|
||||||
|
"""
|
||||||
|
检查数据库表结构和注释
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
print("数据库连接成功")
|
||||||
|
|
||||||
|
# 检查所有表
|
||||||
|
print("\n=== 所有表 ===")
|
||||||
|
cur.execute("SELECT table_name FROM information_schema.tables WHERE table_schema = 'public'")
|
||||||
|
tables = cur.fetchall()
|
||||||
|
for table in tables:
|
||||||
|
print(f"表名: {table[0]}")
|
||||||
|
|
||||||
|
# 检查domains表结构
|
||||||
|
print("\n=== domains表结构 ===")
|
||||||
|
cur.execute("\d+ domains.txt")
|
||||||
|
result = cur.fetchall()
|
||||||
|
for row in result:
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
# 检查其他表结构
|
||||||
|
print("\n=== detect_tasks表结构 ===")
|
||||||
|
cur.execute("\d+ detect_tasks")
|
||||||
|
result = cur.fetchall()
|
||||||
|
for row in result:
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
print("\n=== domain_blacklist表结构 ===")
|
||||||
|
cur.execute("\d+ domain_blacklist")
|
||||||
|
result = cur.fetchall()
|
||||||
|
for row in result:
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
print("\n=== domain_detections表结构 ===")
|
||||||
|
cur.execute("\d+ domain_detections")
|
||||||
|
result = cur.fetchall()
|
||||||
|
for row in result:
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"检查数据库失败: {e}")
|
||||||
|
finally:
|
||||||
|
if cur:
|
||||||
|
cur.close()
|
||||||
|
if conn:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
check_database()
|
||||||
69
domainCheck/check_db_structure.py
Normal file
69
domainCheck/check_db_structure.py
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :check_db_structure.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 21:50
|
||||||
|
@explain : 检查数据库表结构和索引
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
|
||||||
|
def check_db_structure():
|
||||||
|
"""
|
||||||
|
检查数据库表结构和索引
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
print("数据库连接成功")
|
||||||
|
|
||||||
|
# 检查domains表结构
|
||||||
|
print("\n=== domains表结构 ===")
|
||||||
|
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'domains.txt'")
|
||||||
|
for row in cur.fetchall():
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
# 检查domains表索引
|
||||||
|
print("\n=== domains表索引 ===")
|
||||||
|
cur.execute("SELECT indexname, indexdef FROM pg_indexes WHERE tablename = 'domains.txt'")
|
||||||
|
for row in cur.fetchall():
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
# 检查其他表结构
|
||||||
|
print("\n=== detect_tasks表结构 ===")
|
||||||
|
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'detect_tasks'")
|
||||||
|
for row in cur.fetchall():
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
print("\n=== domain_blacklist表结构 ===")
|
||||||
|
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'domain_blacklist'")
|
||||||
|
for row in cur.fetchall():
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
print("\n=== domain_detections表结构 ===")
|
||||||
|
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'domain_detections'")
|
||||||
|
for row in cur.fetchall():
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"检查数据库失败: {e}")
|
||||||
|
finally:
|
||||||
|
if cur:
|
||||||
|
cur.close()
|
||||||
|
if conn:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
check_db_structure()
|
||||||
128
domainCheck/create_sensitive_words_table.py
Normal file
128
domainCheck/create_sensitive_words_table.py
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :create_sensitive_words_table.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/10 03:30
|
||||||
|
@explain : 创建敏感词表
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
|
||||||
|
def create_sensitive_words_table():
|
||||||
|
"""
|
||||||
|
创建敏感词表
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
print(f"成功连接到数据库: {config.DB_HOST}:{config.DB_PORT}/{config.DB_DATABASE}")
|
||||||
|
|
||||||
|
# 创建游标
|
||||||
|
cur = conn.cursor()
|
||||||
|
|
||||||
|
# 创建敏感词表
|
||||||
|
create_table_sql = """
|
||||||
|
CREATE TABLE IF NOT EXISTS sensitive_words (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
word VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
category VARCHAR(50) DEFAULT 'default',
|
||||||
|
priority INTEGER DEFAULT 1,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_table_sql)
|
||||||
|
print("创建敏感词表成功")
|
||||||
|
|
||||||
|
# 添加表注释
|
||||||
|
add_table_comment_sql = """
|
||||||
|
COMMENT ON TABLE sensitive_words IS '敏感词表,存储需要过滤的敏感词'
|
||||||
|
"""
|
||||||
|
cur.execute(add_table_comment_sql)
|
||||||
|
print("添加表注释成功")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
add_column_comments_sql = """
|
||||||
|
COMMENT ON COLUMN sensitive_words.id IS '主键ID';
|
||||||
|
COMMENT ON COLUMN sensitive_words.word IS '敏感词';
|
||||||
|
COMMENT ON COLUMN sensitive_words.category IS '敏感词分类';
|
||||||
|
COMMENT ON COLUMN sensitive_words.priority IS '优先级,数字越大优先级越高';
|
||||||
|
COMMENT ON COLUMN sensitive_words.created_at IS '创建时间';
|
||||||
|
COMMENT ON COLUMN sensitive_words.updated_at IS '更新时间';
|
||||||
|
"""
|
||||||
|
cur.execute(add_column_comments_sql)
|
||||||
|
print("添加字段注释成功")
|
||||||
|
|
||||||
|
# 创建索引
|
||||||
|
create_index_sql = """
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sensitive_words_word ON sensitive_words(word);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sensitive_words_category ON sensitive_words(category);
|
||||||
|
"""
|
||||||
|
cur.execute(create_index_sql)
|
||||||
|
print("创建索引成功")
|
||||||
|
|
||||||
|
# 创建更新触发器
|
||||||
|
create_function_sql = """
|
||||||
|
CREATE OR REPLACE FUNCTION update_updated_at_column()
|
||||||
|
RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
NEW.updated_at = CURRENT_TIMESTAMP;
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
"""
|
||||||
|
cur.execute(create_function_sql)
|
||||||
|
print("创建更新函数成功")
|
||||||
|
|
||||||
|
# 检查触发器是否存在
|
||||||
|
check_trigger_sql = """
|
||||||
|
SELECT COUNT(*) FROM pg_trigger WHERE tgname = 'update_sensitive_words_updated_at'
|
||||||
|
"""
|
||||||
|
cur.execute(check_trigger_sql)
|
||||||
|
trigger_exists = cur.fetchone()[0] > 0
|
||||||
|
|
||||||
|
if not trigger_exists:
|
||||||
|
create_trigger_sql = """
|
||||||
|
CREATE TRIGGER update_sensitive_words_updated_at
|
||||||
|
BEFORE UPDATE ON sensitive_words
|
||||||
|
FOR EACH ROW
|
||||||
|
EXECUTE FUNCTION update_updated_at_column();
|
||||||
|
"""
|
||||||
|
cur.execute(create_trigger_sql)
|
||||||
|
print("创建触发器成功")
|
||||||
|
else:
|
||||||
|
print("触发器已存在,跳过创建")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
print("事务提交成功")
|
||||||
|
|
||||||
|
# 关闭游标和连接
|
||||||
|
cur.close()
|
||||||
|
conn.close()
|
||||||
|
print("连接关闭成功")
|
||||||
|
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f"创建敏感词表失败: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
print("开始创建敏感词表...")
|
||||||
|
success = create_sensitive_words_table()
|
||||||
|
if success:
|
||||||
|
print("敏感词表创建成功!")
|
||||||
|
else:
|
||||||
|
print("敏感词表创建失败!")
|
||||||
27
domainCheck/create_sensitive_words_table.sql
Normal file
27
domainCheck/create_sensitive_words_table.sql
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
-- 创建敏感词表
|
||||||
|
CREATE TABLE IF NOT EXISTS sensitive_words (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
word VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
category VARCHAR(50) DEFAULT 'default',
|
||||||
|
priority INTEGER DEFAULT 1,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 创建索引
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sensitive_words_word ON sensitive_words(word);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_sensitive_words_category ON sensitive_words(category);
|
||||||
|
|
||||||
|
-- 更新updated_at触发器
|
||||||
|
CREATE OR REPLACE FUNCTION update_updated_at_column()
|
||||||
|
RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
NEW.updated_at = CURRENT_TIMESTAMP;
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER update_sensitive_words_updated_at
|
||||||
|
BEFORE UPDATE ON sensitive_words
|
||||||
|
FOR EACH ROW
|
||||||
|
EXECUTE FUNCTION update_updated_at_column();
|
||||||
10
domainCheck/credentials.json
Normal file
10
domainCheck/credentials.json
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"juming": {
|
||||||
|
"email": "chaofanai1998@gmail.com",
|
||||||
|
"password": "dpapi:AQAAANCMnd8BFdERjHoAwE/Cl+sBAAAA75Mg9f3V6US9VHIqKDeFygAAAAACAAAAAAAQZgAAAAEAACAAAABWF/Vfy5UuhJJCDfZysE6bKwCAYElSlyA9DNYer/f33gAAAAAOgAAAAAIAACAAAAC5SFpZMi3xCmsyafDYXG3Y6xgBQYBp4cbLVjOw/WzVyhAAAABvn1gHcy2q1P7N3EQwllBOQAAAANIVDlnB3xQo4T74pBRoI/15OZB6StRQMGT8OLjXIXzEcQBbsjOETUBP0mJF3Cx5eR5G9kZdAEc7GzW0QlVa5uA="
|
||||||
|
},
|
||||||
|
"juziseo": {
|
||||||
|
"email": "",
|
||||||
|
"password": ""
|
||||||
|
}
|
||||||
|
}
|
||||||
155
domainCheck/detect/aizhan.py
Normal file
155
domainCheck/detect/aizhan.py
Normal file
@@ -0,0 +1,155 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :aizhan.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/3/29 20:59
|
||||||
|
@explain : 爱站网域名查询工具 - 获取域名的网站标题信息
|
||||||
|
'''
|
||||||
|
|
||||||
|
import re # 正则表达式模块,用于从HTML中提取网站标题
|
||||||
|
|
||||||
|
import requests # HTTP请求库,用于发送网络请求
|
||||||
|
from loguru import logger # 日志记录模块,用于记录程序运行日志
|
||||||
|
|
||||||
|
from typing import List, Optional # 类型提示
|
||||||
|
|
||||||
|
|
||||||
|
def check_aizhan(domain: str, sensitive_words: Optional[List[str]] = None, proxies: dict = None):
|
||||||
|
'''
|
||||||
|
查询域名的网站标题信息是否存在敏感词
|
||||||
|
|
||||||
|
通过爱站网(aizhan.com)查询域名的网站标题
|
||||||
|
|
||||||
|
:param domain: 待查询的域名(如:www.baidu.com,不带协议头)
|
||||||
|
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
|
||||||
|
:return: str - 网站标题字符串,如果查询失败则返回空字符串
|
||||||
|
'''
|
||||||
|
# 初始化敏感词列表
|
||||||
|
if sensitive_words is None:
|
||||||
|
sensitive_words = []
|
||||||
|
|
||||||
|
# 构建爱站网查询URL
|
||||||
|
# 格式:https://www.aizhan.com/cha/{域名}/
|
||||||
|
url = f'https://www.aizhan.com/cha/{domain}/'
|
||||||
|
|
||||||
|
# 构建HTTP请求头(模拟浏览器访问)
|
||||||
|
headers = {
|
||||||
|
'Accept': '*/*', # 接受所有类型的内容
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0',
|
||||||
|
# 用户代理(浏览器标识)
|
||||||
|
'Referer': url, # 来源页面(防盗链)
|
||||||
|
'host': 'www.aizhan.com', # 目标主机地址
|
||||||
|
}
|
||||||
|
|
||||||
|
# 添加重试机制
|
||||||
|
max_retries = 3
|
||||||
|
for retry in range(max_retries):
|
||||||
|
try:
|
||||||
|
# 发送GET请求获取页面内容
|
||||||
|
# url: 请求URL
|
||||||
|
# headers: HTTP请求头
|
||||||
|
# timeout=10: 设置超时时间10秒
|
||||||
|
# proxies: 代理配置(如需使用代理)
|
||||||
|
response = requests.get(url, headers=headers, timeout=10, proxies=proxies)
|
||||||
|
|
||||||
|
# 判断请求是否成功(HTTP 200表示成功)
|
||||||
|
if response.status_code == 200:
|
||||||
|
# 使用正则表达式提取网站标题
|
||||||
|
# 正则解释:匹配 id="webpage_title"> 开头,</div> 结尾,中间的内容
|
||||||
|
# 匹配模式:id="webpage_title">标题内容</div>
|
||||||
|
# ([^<]+) 表示匹配一个或多个非<字符,作为捕获组
|
||||||
|
match = re.search(r'id="webpage_title">([^<]+)</div>', response.text)
|
||||||
|
|
||||||
|
# 如果找到匹配则返回标题文本,否则返回空字符串
|
||||||
|
title = match.group(1) if match else ''
|
||||||
|
if title:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
# 使用正则表达式检查是否包含敏感词
|
||||||
|
if re.search(sensitive_word, title, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}'
|
||||||
|
return True, title
|
||||||
|
|
||||||
|
# 请求失败,继续重试
|
||||||
|
logger.warning(f"爱站网检测第{retry+1}次失败: {domain}, 状态码: {response.status_code}")
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,尝试不使用代理
|
||||||
|
if proxies:
|
||||||
|
logger.info('爱站网检测失败,尝试不使用代理')
|
||||||
|
try:
|
||||||
|
response = requests.get(url, headers=headers, timeout=10, proxies=None)
|
||||||
|
if response.status_code == 200:
|
||||||
|
match = re.search(r'id="webpage_title">([^<]+)</div>', response.text)
|
||||||
|
title = match.group(1) if match else ''
|
||||||
|
if title:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
if re.search(sensitive_word, title, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}'
|
||||||
|
return True, title
|
||||||
|
return True, ''
|
||||||
|
except Exception as e2:
|
||||||
|
logger.error(f"不使用代理的爱站网检测也失败: {domain}, 错误: {e2}")
|
||||||
|
return False, str(e2)
|
||||||
|
return True, ''
|
||||||
|
|
||||||
|
except Exception as e: # 捕获所有异常(网络错误、超时等)
|
||||||
|
logger.warning(f"爱站网检测第{retry+1}次失败: {domain}, 错误: {e}")
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,尝试不使用代理
|
||||||
|
if proxies:
|
||||||
|
logger.info('爱站网检测失败,尝试不使用代理')
|
||||||
|
try:
|
||||||
|
response = requests.get(url, headers=headers, timeout=10, proxies=None)
|
||||||
|
if response.status_code == 200:
|
||||||
|
match = re.search(r'id="webpage_title">([^<]+)</div>', response.text)
|
||||||
|
title = match.group(1) if match else ''
|
||||||
|
if title:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
if re.search(sensitive_word, title, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}'
|
||||||
|
return True, title
|
||||||
|
return True, ''
|
||||||
|
except Exception as e2:
|
||||||
|
logger.error(f"不使用代理的爱站网检测也失败: {domain}, 错误: {e2}")
|
||||||
|
return False, str(e2)
|
||||||
|
return False, str(e)
|
||||||
|
|
||||||
|
# # ========================= 【测试代码】 =========================
|
||||||
|
# if __name__ == '__main__':
|
||||||
|
# # 测试用例1:查询常见域名
|
||||||
|
# logger.info("=== 测试爱站网查询 ===")
|
||||||
|
# test_domain_1 = 'baidu.com'
|
||||||
|
# logger.info(f"查询域名: {test_domain_1}")
|
||||||
|
#
|
||||||
|
# result_1 = check_aizhan(test_domain_1)
|
||||||
|
# logger.info(f"网站标题: {result_1}")
|
||||||
|
# logger.info("")
|
||||||
|
#
|
||||||
|
# # 测试用例2:查询另一个域名
|
||||||
|
# logger.info("=== 测试查询QQ域名 ===")
|
||||||
|
# test_domain_2 = 'qq.com'
|
||||||
|
# logger.info(f"查询域名: {test_domain_2}")
|
||||||
|
#
|
||||||
|
# result_2 = check_aizhan(test_domain_2)
|
||||||
|
# logger.info(f"网站标题: {result_2}")
|
||||||
|
# logger.info("")
|
||||||
|
#
|
||||||
|
#
|
||||||
|
#
|
||||||
|
# # 测试用例4:查询可能不存在的域名
|
||||||
|
# logger.info("=== 测试不存在域名 ===")
|
||||||
|
# test_domain_4 = 'nonexistentdomain123456.com'
|
||||||
|
# logger.info(f"查询域名: {test_domain_4}")
|
||||||
|
#
|
||||||
|
# result_4 = check_aizhan(test_domain_4)
|
||||||
|
# logger.info(f"网站标题: {result_4 if result_4 else '(无结果)'}")
|
||||||
263
domainCheck/detect/baidu.py
Normal file
263
domainCheck/detect/baidu.py
Normal file
@@ -0,0 +1,263 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :baidu.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/3/29 21:54
|
||||||
|
@explain : 百度域名检测工具 - 包含百度搜索结果查询和百度安全API检测功能
|
||||||
|
'''
|
||||||
|
|
||||||
|
import random # 随机数生成模块,用于随机选择浏览器类型
|
||||||
|
import re # 正则表达式模块,用于从HTML中提取搜索结果
|
||||||
|
import time
|
||||||
|
|
||||||
|
from loguru import logger # 日志记录模块,用于记录程序运行日志
|
||||||
|
from curl_cffi import requests # curl_cffi库,支持浏览器模拟的HTTP请求
|
||||||
|
from typing import List, Optional, Union, Any # 类型提示
|
||||||
|
|
||||||
|
def check_site(domain: str, sensitive_words: Optional[List[str]] = None, proxies: dict = None) -> Union[
|
||||||
|
None, tuple[bool, str], list[Any]]:
|
||||||
|
'''
|
||||||
|
通过百度搜索查询域名的相关信息
|
||||||
|
|
||||||
|
使用curl_cffi模拟浏览器访问百度搜索,获取site:domain的搜索结果标题
|
||||||
|
|
||||||
|
:param domain: 待查询的域名(如:www.baidu.com)
|
||||||
|
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
|
||||||
|
:return: list - 搜索结果标题列表,如果查询失败则返回空列表
|
||||||
|
'''
|
||||||
|
# 初始化敏感词列表
|
||||||
|
if sensitive_words is None:
|
||||||
|
sensitive_words = []
|
||||||
|
|
||||||
|
# 添加重试机制
|
||||||
|
max_retries = 3
|
||||||
|
for retry in range(max_retries):
|
||||||
|
try: # try-except块,用于捕获网络请求中的异常
|
||||||
|
# 构建搜索查询参数
|
||||||
|
# wd: 搜索关键词,格式为site:域名
|
||||||
|
params = {
|
||||||
|
'wd': f'site:{domain}' # 搜索site:domain,限制搜索结果为指定域名
|
||||||
|
}
|
||||||
|
|
||||||
|
cookies = {
|
||||||
|
'BIDUPSID': '053DBE4D820C6EFB729DC7B13B1F82B2',
|
||||||
|
'PSTM': '1775657119',
|
||||||
|
'H_PS_PSSID': '63148_67862_67986_68002_68142_68148_68152_68141_68165_68189_68226_68267_68296_68336_68369_68453_68438_68464_68541_68546_68558_68520_68589_68621_68615_68606_68601_68682_68671_68735_68544_68733_68766_68807_68901_68918_68836_68921_68955_68976_68997_69007_69010_69018_69024_69014',
|
||||||
|
'BAIDUID': '053DBE4D820C6EFB729DC7B13B1F82B2:FG=1',
|
||||||
|
'BDORZ': 'B490B5EBF6F3CD402E515D22BCDA1598',
|
||||||
|
'BAIDUID_BFESS': '053DBE4D820C6EFB729DC7B13B1F82B2:FG=1',
|
||||||
|
'BA_HECTOR': 'ah0l24210la4050l0g0k0g2g0h20071ktcocj27',
|
||||||
|
'ZFY': 'Qxyr75Xm7o8zUxYzuFnYoW7cnS:AnVp:BpnrNVkr3usno:C',
|
||||||
|
'delPer': '0',
|
||||||
|
'BAIDUID_REF': '053DBE4D820C6EFB729DC7B13B1F82B2:FG=1',
|
||||||
|
'H_WISE_SIDS': '110085_661771_667681_673676_683089_682564_685373_660925_687556_686285_690306_690303_690478_690576_690334_691545_685595_690883_692754_692777_692118_692910_693221_693364_693403_693656_693941_694124_693392_694171_694236_693996_694316_694324_693886_694409_694379_694696_694780_694783_692006_694884_694918_694996_694991_695032_695118_694840_695140_695193_695110_695262_688610_694933_695278_694865_695214_695374_690651_694577_692378_695452_695457_695480_695603_695642_695631_695668_695824_695843_695866_695888_695892_695897_695975_694177_695900_695939_696145_696150_696153_696069_696078_696066_694359_696288_696308_694985_696297_695715_696128_696313_696457_696431_696473_696492_696590_693385_696610_696682_696679_696662_696651_696655_696658_696673_696643_696110_696729_696733_696772_8000116_8000133_8000138_8000159_8000163_8000167_8000176_8000186_8000190_8000204',
|
||||||
|
'H_WISE_SIDS_BFESS': '110085_661771_667681_673676_683089_682564_685373_660925_687556_686285_690306_690303_690478_690576_690334_691545_685595_690883_692754_692777_692118_692910_693221_693364_693403_693656_693941_694124_693392_694171_694236_693996_694316_694324_693886_694409_694379_694696_694780_694783_692006_694884_694918_694996_694991_695032_695118_694840_695140_695193_695110_695262_688610_694933_695278_694865_695214_695374_690651_694577_692378_695452_695457_695480_695603_695642_695631_695668_695824_695843_695866_695888_695892_695897_695975_694177_695900_695939_696145_696150_696153_696069_696078_696066_694359_696288_696308_694985_696297_695715_696128_696313_696457_696431_696473_696492_696590_693385_696610_696682_696679_696662_696651_696655_696658_696673_696643_696110_696729_696733_696772_8000116_8000133_8000138_8000159_8000163_8000167_8000176_8000186_8000190_8000204',
|
||||||
|
'MSA_PBT': '147',
|
||||||
|
'MSA_ZOOM': '1000',
|
||||||
|
'wpr': '0',
|
||||||
|
'COOKIE_SESSION': '0_0_0_0_0_0_0_0_0_0_0_0_0_1775657396%7C1%230_0_0_0_0_0_0_0_1775657396%7C1',
|
||||||
|
'MSA_PHY_WH': '1440_3440',
|
||||||
|
'POLYFILL': '0',
|
||||||
|
'MSA_WH': '1254_940',
|
||||||
|
'kleck': '7ce2abac229d2e8ba435a8bcc6256f57f53e9d08954443bf',
|
||||||
|
'PSCBD': '16%3A1%3A3',
|
||||||
|
'SE_LAUNCH': '5%3A1775657396_16%3A29594323%3A3',
|
||||||
|
'BDSVRTM': '3',
|
||||||
|
'PSINO': '6',
|
||||||
|
'__bsi': '17976785662214065461_00_7_R_R_6_0303_c02f_Y',
|
||||||
|
}
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
|
||||||
|
'Accept-Language': 'zh-CN,zh;q=0.9',
|
||||||
|
'Connection': 'keep-alive',
|
||||||
|
'Sec-Fetch-Dest': 'document',
|
||||||
|
'Sec-Fetch-Mode': 'navigate',
|
||||||
|
'Sec-Fetch-Site': 'none',
|
||||||
|
'Sec-Fetch-User': '?1',
|
||||||
|
'Upgrade-Insecure-Requests': '1',
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0',
|
||||||
|
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Microsoft Edge";v="146"',
|
||||||
|
'sec-ch-ua-mobile': '?0',
|
||||||
|
'sec-ch-ua-platform': '"Windows"',
|
||||||
|
}
|
||||||
|
|
||||||
|
# 百度搜索URL
|
||||||
|
url = 'https://m.baidu.com/s'
|
||||||
|
|
||||||
|
# 发送GET请求获取搜索结果
|
||||||
|
# url: 请求URL
|
||||||
|
# params: 查询参数
|
||||||
|
# headers: HTTP请求头
|
||||||
|
# timeout=15: 超时时间15秒
|
||||||
|
# proxies: 代理配置
|
||||||
|
# impersonate: 模拟的浏览器类型
|
||||||
|
# .content: 获取响应内容(字节流)
|
||||||
|
# .decode('utf-8'): 解码为UTF-8字符串
|
||||||
|
response_text = requests.get(url, params=params, headers=headers, cookies=cookies, timeout=15,
|
||||||
|
proxies=proxies).content.decode('utf-8')
|
||||||
|
# 使用正则表达式提取搜索结果标题
|
||||||
|
# re.findall()返回所有匹配的结果列表
|
||||||
|
# 正则解释:匹配 <!--s-text--> 和 <!--/s-text--> 之间的内容
|
||||||
|
match = re.findall(r'<!--s-text-->([^<]+)<!--/s-text-->', response_text,re.S)
|
||||||
|
|
||||||
|
# 返回匹配结果列表,如果没有匹配则返回空列表
|
||||||
|
search_results= match if match else []
|
||||||
|
|
||||||
|
# 如果没有搜索结果,直接返回通过
|
||||||
|
if not search_results:
|
||||||
|
return True, ''
|
||||||
|
|
||||||
|
# 检查每个搜索结果是否包含敏感词
|
||||||
|
for result in search_results:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
# 使用正则表达式检查是否包含敏感词
|
||||||
|
if re.search(sensitive_word, result, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
|
||||||
|
|
||||||
|
# 所有搜索结果都不包含敏感词,返回通过
|
||||||
|
return True, ''
|
||||||
|
except Exception as e: # 捕获所有异常
|
||||||
|
# logger.warning(f"百度site检测第{retry+1}次失败: {domain}, 错误: {e}")
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
proxies=None
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,返回检测失败
|
||||||
|
logger.error(f"百度site检测失败: {domain}, 已达到最大重试次数")
|
||||||
|
return False, str(e) # 返回错误信息
|
||||||
|
|
||||||
|
|
||||||
|
def baidu(domain: str, proxies: dict = None) -> str:
|
||||||
|
'''
|
||||||
|
通过百度安全API检测域名安全状态
|
||||||
|
|
||||||
|
调用百度移动安全API检测域名是否为风险网站
|
||||||
|
|
||||||
|
:param domain: 待检测的域名(如:www.baidu.com)
|
||||||
|
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
|
||||||
|
:return: str - 安全状态描述("风险网站提示"、"高危网站提示"或"正常"),如果检测失败则返回错误信息
|
||||||
|
'''
|
||||||
|
# 构建HTTP请求头
|
||||||
|
headers = {
|
||||||
|
'Cache-Control': 'no-cache', # 不使用缓存
|
||||||
|
'Content-Type': 'application/json; charset=utf-8', # 内容类型为JSON
|
||||||
|
'Host': 'mobsec-sec.baidu.com', # 目标主机
|
||||||
|
'User-Agent': 'okhttp/3.12.12', # 模拟OKHttp客户端
|
||||||
|
}
|
||||||
|
|
||||||
|
# 构建请求参数
|
||||||
|
params = {
|
||||||
|
'auth_ver': '2', # 认证版本
|
||||||
|
'appkey': '4665fd2c6b0922a551e8ae74', # API应用密钥
|
||||||
|
'nonce': '1751309255340', # 随机数(注意:实际使用时应该动态生成)
|
||||||
|
'lc': '77qHTv4VtmRiXYtd', # 本地配置参数
|
||||||
|
'pkg': 'com.baidu.searchbox', # 应用包名
|
||||||
|
'vc': '-1', # 版本码
|
||||||
|
'cuid': 'CF0E6FCCD824D146605C16AC1C8BAC95%7CVFWAO56TC', # 设备ID(注意:实际使用时应该动态生成)
|
||||||
|
'tk': '', # 令牌(空)
|
||||||
|
'type': '3', # 检测类型
|
||||||
|
'vn': '2.6.0', # 版本号
|
||||||
|
's': 'fdfef2ce96c1c7193f9b80425020f63e', # 签名(注意:实际使用时应该动态生成)
|
||||||
|
}
|
||||||
|
|
||||||
|
# 构建JSON请求数据
|
||||||
|
json_data = {
|
||||||
|
'url': f'https://{domain}/', # 待检测的URL(添加https协议头)
|
||||||
|
}
|
||||||
|
|
||||||
|
# 提取URL作为后续获取结果的键
|
||||||
|
key = json_data['url']
|
||||||
|
|
||||||
|
# 添加重试机制
|
||||||
|
max_retries = 3
|
||||||
|
for retry in range(max_retries):
|
||||||
|
try: # try-except块,用于捕获API调用中的异常
|
||||||
|
# 发送POST请求到百度安全API
|
||||||
|
# url: API接口地址
|
||||||
|
# params: 查询参数
|
||||||
|
# headers: HTTP请求头
|
||||||
|
# json: JSON请求数据
|
||||||
|
# proxies: 代理配置
|
||||||
|
# .json(): 解析JSON响应
|
||||||
|
response = requests.post('https://mobsec-sec.baidu.com/3.1/scanurl', params=params, headers=headers,
|
||||||
|
json=json_data, proxies=proxies, timeout=15).json()
|
||||||
|
|
||||||
|
# # 记录API响应(调试用)
|
||||||
|
# logger.info(response)
|
||||||
|
|
||||||
|
# 安全状态码映射表
|
||||||
|
msgs = {
|
||||||
|
2193: '风险网站提示', # 风险网站
|
||||||
|
2243: '高危网站提示', # 高危网站
|
||||||
|
'_': '正常' # 默认正常
|
||||||
|
}
|
||||||
|
|
||||||
|
# 从响应中获取安全等级并映射为状态描述
|
||||||
|
# response['response']['datas'][key].get('grand_level', 'level'): 获取安全等级,默认值为'level'
|
||||||
|
# msgs.get(): 根据安全等级获取对应的状态描述,默认值为'正常'
|
||||||
|
return msgs.get(response['response']['datas'][key].get('grand_level', 'level'), '正常')
|
||||||
|
|
||||||
|
except Exception as e: # 捕获所有异常
|
||||||
|
logger.warning(f"百度网页安全中心检测第{retry+1}次失败: {domain}, 错误: {e}")
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,尝试不使用代理
|
||||||
|
if proxies:
|
||||||
|
logger.info('百度网页安全中心检测失败,尝试不使用代理')
|
||||||
|
try:
|
||||||
|
response = requests.post('https://mobsec-sec.baidu.com/3.1/scanurl', params=params, headers=headers,
|
||||||
|
json=json_data, proxies=None, timeout=15).json()
|
||||||
|
msgs = {
|
||||||
|
2193: '风险网站提示', # 风险网站
|
||||||
|
2243: '高危网站提示', # 高危网站
|
||||||
|
'_': '正常' # 默认正常
|
||||||
|
}
|
||||||
|
return msgs.get(response['response']['datas'][key].get('grand_level', 'level'), '正常')
|
||||||
|
except Exception as e2:
|
||||||
|
logger.error(f"不使用代理的百度网页安全中心检测也失败: {domain}, 错误: {e2}")
|
||||||
|
return str(e2) # 返回错误信息
|
||||||
|
return str(e) # 返回错误信息
|
||||||
|
|
||||||
|
|
||||||
|
# # ========================= 【测试代码】 =========================
|
||||||
|
# if __name__ == '__main__': # 程序入口,当直接运行此文件时执行以下代码
|
||||||
|
# # proxies = get_proxies()
|
||||||
|
# proxies = None
|
||||||
|
# # 测试用例1:查询域名搜索结果
|
||||||
|
# logger.info("=== 测试百度搜索结果查询 ===")
|
||||||
|
# test_domain_1 = '920zl.com'
|
||||||
|
# logger.info(f"查询域名: {test_domain_1}")
|
||||||
|
#
|
||||||
|
# result_1 = check_site(test_domain_1, proxies=proxies)
|
||||||
|
# logger.info(result_1)
|
||||||
|
|
||||||
|
#
|
||||||
|
# # 测试用例3:检测域名安全状态
|
||||||
|
# logger.info("=== 测试域名安全检测 ===")
|
||||||
|
# test_domain_3 = 'baidu.com'
|
||||||
|
# logger.info(f"检测域名: {test_domain_3}")
|
||||||
|
#
|
||||||
|
# result_3 = baidu(test_domain_3, proxies=proxies)
|
||||||
|
# logger.info(f"安全状态: {result_3}")
|
||||||
|
# logger.debug(proxies)
|
||||||
|
|
||||||
|
# domains = ['Lqyingye.com', 'tjjinLikeji.com', 'guokangLxs.com', 'zhuoyijz.com', 'jiandanrongyi.com', 'fromhhc.com',
|
||||||
|
# 'djxow.com', '725game.com', 'gcLpchd.cn', '6rdb.com.cn', 'sunmantech.com', 'hroxa.com', 'guLhu65.cn',
|
||||||
|
# 'xxjdch.com', '7e7fm1.cn', 'kaoxueb.com', 'jabas.cn', 'shzhongyi.com', 'figuangze.com', 'eztw9k.cn',
|
||||||
|
# 'foxok.com', 'oLLbmr.com', 'vwpwwkp.cn', 'cbmrs.com', 'zaozhuang56.com', 'hengxiangracing.com',
|
||||||
|
# 'jdyfbpq.cn', 'czca.cn', 'shibingxiongdi.com', 'ubxbm.cn', 'ydbaoshi.com', '2046hd.cn', 'mffmcp.com',
|
||||||
|
# 'hgLdhzz.com', 'okswmi.cn', 'caidaozg.com', 'dkdywx.cn', 'qkjps.com', 'Lwxgk.cn', 'drtechnoLogy.cn',
|
||||||
|
# 'akcie.cn', '588uu.com', 'kuaiyijian.net', 'c5b.cn', 'faka866.cn', 'wiraf03.cn', 'bobjibar.com',
|
||||||
|
# '166233.cn', 'adLfjcjq.cn', 'whrsom.com']
|
||||||
|
# for domain in domains:
|
||||||
|
# result_1 = check_site(domain)
|
||||||
|
# logger.info(f"搜索结果标题数: {len(result_1)}")
|
||||||
|
# for i, title in enumerate(result_1, 1):
|
||||||
|
# logger.info(f" 结果{i}: {title}")
|
||||||
|
# time.sleep(1)
|
||||||
191
domainCheck/detect/c360.py
Normal file
191
domainCheck/detect/c360.py
Normal file
@@ -0,0 +1,191 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :c360.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/2 15:57
|
||||||
|
@explain : 360搜索敏感词检测工具 - 检测域名是否包含敏感词
|
||||||
|
'''
|
||||||
|
|
||||||
|
import re # 正则表达式模块
|
||||||
|
import requests # HTTP请求库
|
||||||
|
from loguru import logger # 日志记录
|
||||||
|
from typing import List, Tuple, Optional, Dict # 类型提示
|
||||||
|
|
||||||
|
|
||||||
|
# 常量定义
|
||||||
|
SO_SEARCH_URL = 'https://www.so.com/s' # 360搜索URL
|
||||||
|
SO_REFERER_TEMPLATE = 'https://www.so.com/s?ie=utf-8&q=site%3A{domain}' # Referer模板
|
||||||
|
SEARCH_PATTERN = r'target="_blank">([^<]+)</a></h3>' # 搜索结果匹配模式
|
||||||
|
BLOCKED_CODE = 3 # 拦截状态码
|
||||||
|
|
||||||
|
|
||||||
|
def check_domain(
|
||||||
|
domain: str,
|
||||||
|
sensitive_words: Optional[List[str]] = None,
|
||||||
|
proxies: Optional[Dict] = None
|
||||||
|
) -> Tuple[bool, str]:
|
||||||
|
"""
|
||||||
|
检查域名是否包含敏感词
|
||||||
|
|
||||||
|
通过360搜索获取域名的搜索结果,然后检查搜索结果中是否包含敏感词
|
||||||
|
|
||||||
|
Args:
|
||||||
|
domain: 要检测的域名
|
||||||
|
sensitive_words: 敏感词列表,如果为None则初始化为空列表
|
||||||
|
proxies: 代理配置
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple[bool, str]: (是否通过, 错误信息)
|
||||||
|
- True, '': 通过检测,不包含敏感词
|
||||||
|
- False, '敏感词:xxx': 包含敏感词,返回具体的敏感词
|
||||||
|
"""
|
||||||
|
# 初始化敏感词列表
|
||||||
|
if sensitive_words is None:
|
||||||
|
sensitive_words = []
|
||||||
|
|
||||||
|
# 构建请求头
|
||||||
|
headers = {
|
||||||
|
'Accept': '*/*', # 接受所有类型
|
||||||
|
'Accept-Language': 'zh-cn', # 中文语言
|
||||||
|
'Referer': SO_REFERER_TEMPLATE.format(domain=domain), # 来源页面
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.87 Safari/537.36', # 用户代理
|
||||||
|
'Host': 'www.so.com', # 主机名
|
||||||
|
}
|
||||||
|
|
||||||
|
# 构建查询参数
|
||||||
|
params = {
|
||||||
|
'ie': 'utf-8', # 编码格式
|
||||||
|
'q': f'site:{domain}', # 搜索查询
|
||||||
|
}
|
||||||
|
|
||||||
|
# 添加重试机制
|
||||||
|
max_retries = 3
|
||||||
|
for retry in range(max_retries):
|
||||||
|
try:
|
||||||
|
# 发送GET请求获取搜索结果
|
||||||
|
response = requests.get(
|
||||||
|
SO_SEARCH_URL,
|
||||||
|
params=params,
|
||||||
|
headers=headers,
|
||||||
|
proxies=proxies,
|
||||||
|
timeout=15 # 增加超时时间到15秒
|
||||||
|
)
|
||||||
|
|
||||||
|
# 解析响应内容
|
||||||
|
response_html = response.content.decode('utf-8', errors='ignore')
|
||||||
|
|
||||||
|
# 使用正则表达式提取搜索结果
|
||||||
|
search_results = re.findall(SEARCH_PATTERN, response_html)
|
||||||
|
|
||||||
|
# 记录搜索结果数量
|
||||||
|
logger.info(f'360搜索结果数量: {len(search_results)}')
|
||||||
|
|
||||||
|
# 如果没有搜索结果,直接返回通过
|
||||||
|
if not search_results:
|
||||||
|
return True, ''
|
||||||
|
|
||||||
|
# 检查每个搜索结果是否包含敏感词
|
||||||
|
for result in search_results:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
# 使用正则表达式检查是否包含敏感词
|
||||||
|
if re.search(sensitive_word, result, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
|
||||||
|
|
||||||
|
# 所有搜索结果都不包含敏感词,返回通过
|
||||||
|
return True, ''
|
||||||
|
|
||||||
|
except requests.exceptions.Timeout:
|
||||||
|
logger.warning(f'360搜索请求第{retry+1}次超时')
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,尝试不使用代理
|
||||||
|
if proxies:
|
||||||
|
logger.info('360搜索请求超时,尝试不使用代理')
|
||||||
|
try:
|
||||||
|
response = requests.get(
|
||||||
|
SO_SEARCH_URL,
|
||||||
|
params=params,
|
||||||
|
headers=headers,
|
||||||
|
proxies=None,
|
||||||
|
timeout=15
|
||||||
|
)
|
||||||
|
response_html = response.content.decode('utf-8', errors='ignore')
|
||||||
|
search_results = re.findall(SEARCH_PATTERN, response_html)
|
||||||
|
logger.info(f'不使用代理的360搜索结果数量: {len(search_results)}')
|
||||||
|
if not search_results:
|
||||||
|
return True, ''
|
||||||
|
for result in search_results:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
if re.search(sensitive_word, result, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
|
||||||
|
return True, ''
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f'不使用代理的360搜索请求也失败: {str(e)}')
|
||||||
|
return False, f'360搜索请求超时'
|
||||||
|
return False, '360搜索请求超时'
|
||||||
|
except requests.exceptions.RequestException as e:
|
||||||
|
error_str = str(e)
|
||||||
|
logger.warning(f'360搜索请求第{retry+1}次失败: {error_str}')
|
||||||
|
# 检查是否是"Too many open files"错误
|
||||||
|
if 'Too many open files' in error_str:
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,返回failure标记,不拉黑域名
|
||||||
|
logger.error(f'360搜索请求失败,已达到最大重试次数: {error_str}')
|
||||||
|
return True, 'failure'
|
||||||
|
else:
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,尝试不使用代理
|
||||||
|
if proxies:
|
||||||
|
logger.info('360搜索请求失败,尝试不使用代理')
|
||||||
|
try:
|
||||||
|
response = requests.get(
|
||||||
|
SO_SEARCH_URL,
|
||||||
|
params=params,
|
||||||
|
headers=headers,
|
||||||
|
proxies=None,
|
||||||
|
timeout=15
|
||||||
|
)
|
||||||
|
response_html = response.content.decode('utf-8', errors='ignore')
|
||||||
|
search_results = re.findall(SEARCH_PATTERN, response_html)
|
||||||
|
logger.info(f'不使用代理的360搜索结果数量: {len(search_results)}')
|
||||||
|
if not search_results:
|
||||||
|
return True, ''
|
||||||
|
for result in search_results:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
if re.search(sensitive_word, result, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
|
||||||
|
return True, ''
|
||||||
|
except Exception as e2:
|
||||||
|
logger.error(f'不使用代理的360搜索请求也失败: {str(e2)}')
|
||||||
|
return False, f'360搜索请求失败: {str(e)}'
|
||||||
|
return False, f'360搜索请求失败: {str(e)}'
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f'360搜索第{retry+1}次未知错误: {str(e)}')
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
else:
|
||||||
|
return False, f'未知错误: {str(e)}'
|
||||||
|
|
||||||
|
|
||||||
|
# if __name__ == '__main__':
|
||||||
|
# # 测试用例
|
||||||
|
# test_domain = 'niuniushouka.com'
|
||||||
|
# test_sensitive_words = ['赌博', '色情', '暴力','收卡']
|
||||||
|
#
|
||||||
|
# logger.info(f'开始检测域名: {test_domain}')
|
||||||
|
# result = check_domain(test_domain, test_sensitive_words)
|
||||||
|
# logger.info(f'检测结果: {result}')
|
||||||
1257
domainCheck/detect/chinaz.js
Normal file
1257
domainCheck/detect/chinaz.js
Normal file
File diff suppressed because one or more lines are too long
242
domainCheck/detect/chinaz.py
Normal file
242
domainCheck/detect/chinaz.py
Normal file
@@ -0,0 +1,242 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :chinaz.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/3/27 23:48
|
||||||
|
@explain : 站长工具SEO查询工具 - 获取域名网站标题、SEO排名等信息
|
||||||
|
'''
|
||||||
|
|
||||||
|
import json # JSON处理模块,用于解析API响应数据
|
||||||
|
import os # 操作系统接口模块,用于获取脚本所在目录路径
|
||||||
|
import re # 正则表达式模块,用于提取网页中的标题和加密密钥
|
||||||
|
import time # 时间处理模块,用于生成时间戳
|
||||||
|
from typing import List, Optional # 类型提示
|
||||||
|
|
||||||
|
import quickjs # JavaScript执行引擎,用于执行JS加密代码生成签名
|
||||||
|
import requests # HTTP请求库,用于发送网络请求
|
||||||
|
from loguru import logger # 日志记录模块,用于记录程序运行日志
|
||||||
|
|
||||||
|
|
||||||
|
def check_title(domain: str,sensitive_words: Optional[List[str]] = None, proxies: dict = None):
|
||||||
|
'''
|
||||||
|
检测域名标题并获取SEO数据
|
||||||
|
|
||||||
|
:param domain: 待检测域名(如:www.baidu.com,不带协议头)
|
||||||
|
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
|
||||||
|
:return: Tuple[bool, str, Tuple[bool, Any]] - (是否成功, 消息, (SEO查询是否成功, SEO数据或错误信息))
|
||||||
|
'''
|
||||||
|
# 初始化敏感词列表
|
||||||
|
if sensitive_words is None:
|
||||||
|
sensitive_words = []
|
||||||
|
|
||||||
|
# 添加重试机制
|
||||||
|
max_retries = 3
|
||||||
|
for retry in range(max_retries):
|
||||||
|
try:
|
||||||
|
# 构建SEO查询URL(站长工具网站)
|
||||||
|
url = f'https://seo.chinaz.com/{domain}'
|
||||||
|
|
||||||
|
# 发送GET请求获取页面内容,设置超时10秒
|
||||||
|
response = requests.get(url, timeout=10, proxies=proxies)
|
||||||
|
|
||||||
|
# 判断请求是否成功(HTTP 200表示成功)
|
||||||
|
if response.status_code == 200:
|
||||||
|
# 使用正则表达式提取网站标题(从id="site_title"的div标签中提取文本内容)
|
||||||
|
# 正则解释:匹配 id="site_title"> 开头,</div> 结尾,中间的内容
|
||||||
|
match = re.search(r'id="site_title">([^<]+)</div>', response.text)
|
||||||
|
|
||||||
|
# 使用正则表达式提取加密密钥enkey(用于后续API请求的身份验证)
|
||||||
|
# 正则解释:匹配 var enkey = '...' 中的单引号内容
|
||||||
|
pattern = r"var enkey = '([^']+)''"
|
||||||
|
match_enkey = re.search(pattern, response.text)
|
||||||
|
|
||||||
|
# 如果上面的正则匹配失败,尝试匹配正确的单引号版本
|
||||||
|
if not match_enkey:
|
||||||
|
pattern = r"var enkey = '([^']+)" # 匹配 var enkey = '...' 格式
|
||||||
|
match_enkey = re.search(pattern, response.text)
|
||||||
|
|
||||||
|
# 如果找到enkey则提取并去除首尾空格,否则为空字符串
|
||||||
|
enkey = match_enkey.group(1).strip() if match_enkey else ''
|
||||||
|
|
||||||
|
# 提取网站标题,如果找到则返回,否则返回空字符串
|
||||||
|
title = match.group(1).strip() if match else ''
|
||||||
|
|
||||||
|
if title:
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
# 使用正则表达式检查是否包含敏感词
|
||||||
|
if re.search(sensitive_word, title, re.IGNORECASE):
|
||||||
|
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
|
||||||
|
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}', ()
|
||||||
|
|
||||||
|
# 调用get_site_data获取详细SEO数据
|
||||||
|
success, result = get_site_data(domain, enkey, proxies=proxies)
|
||||||
|
|
||||||
|
# 检查网站分类
|
||||||
|
if success and result:
|
||||||
|
# 定义需要拉黑的分类
|
||||||
|
blacklist_categories = ['视频电影', '体育运动', '常用查询', '游戏网站', '游戏', '视频', '体育']
|
||||||
|
|
||||||
|
# 检查分类是否在黑名单中
|
||||||
|
if isinstance(result, dict):
|
||||||
|
# 检查可能的分类字段
|
||||||
|
category_fields = ['Category', 'category', '分类', '网站分类']
|
||||||
|
for field in category_fields:
|
||||||
|
if field in result:
|
||||||
|
category = result[field]
|
||||||
|
if isinstance(category, str):
|
||||||
|
for blacklist_category in blacklist_categories:
|
||||||
|
if blacklist_category in category:
|
||||||
|
logger.warning(f'检测到黑名单分类: {category} 在域名: {domain}')
|
||||||
|
return False, f'网站分类: {category} 属于黑名单分类', ()
|
||||||
|
|
||||||
|
# 检查Result是否为列表
|
||||||
|
elif isinstance(result, list):
|
||||||
|
for item in result:
|
||||||
|
if isinstance(item, dict):
|
||||||
|
category_fields = ['Category', 'category', '分类', '网站分类']
|
||||||
|
for field in category_fields:
|
||||||
|
if field in item:
|
||||||
|
category = item[field]
|
||||||
|
if isinstance(category, str):
|
||||||
|
for blacklist_category in blacklist_categories:
|
||||||
|
if blacklist_category in category:
|
||||||
|
logger.warning(f'检测到黑名单分类: {category} 在域名: {domain}')
|
||||||
|
return False, f'网站分类: {category} 属于黑名单分类', ()
|
||||||
|
|
||||||
|
return True, 'success', (success, result)
|
||||||
|
|
||||||
|
# 请求失败返回空标题和空元组
|
||||||
|
return True, '', ()
|
||||||
|
|
||||||
|
except Exception as e: # 捕获所有异常
|
||||||
|
error_str = str(e)
|
||||||
|
# 检查是否是需要重试的错误类型
|
||||||
|
if any(error_type in error_str for error_type in ['Too many open files', 'EOF occurred in violation of protocol', 'Max retries exceeded', 'Read timed out']):
|
||||||
|
logger.warning(f"站长工具检测第{retry+1}次失败: {domain}, 错误: {e}")
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
continue
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,返回失败但不拉黑
|
||||||
|
logger.error(f"站长工具检测失败,已达到最大重试次数: {domain}, 错误: {e}")
|
||||||
|
return True, 'failure', (False, str(e))
|
||||||
|
else:
|
||||||
|
# 其他错误直接返回
|
||||||
|
logger.error(f"站长工具检测失败: {domain}, 错误: {e}")
|
||||||
|
return False, str(e), ()
|
||||||
|
|
||||||
|
|
||||||
|
# ========================= 【最终请求】 =========================
|
||||||
|
def get_site_data(domain, enkey: str, proxies: dict = None):
|
||||||
|
'''
|
||||||
|
获取站点的详细SEO数据
|
||||||
|
|
||||||
|
通过站长工具API获取域名的SEO排名、权重等信息
|
||||||
|
|
||||||
|
:param domain: 待检测域名(如:www.baidu.com)
|
||||||
|
:param enkey: 加密密钥(从check_title函数获取)
|
||||||
|
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
|
||||||
|
:return: Tuple[bool, Any] - (是否成功, 结果数据或错误信息)
|
||||||
|
- (True, Result数据): 查询成功
|
||||||
|
- (False, 错误信息): 查询失败
|
||||||
|
'''
|
||||||
|
try:
|
||||||
|
# 获取当前脚本所在目录(用于构建JS文件的完整路径)
|
||||||
|
current_dir = os.path.dirname(__file__)
|
||||||
|
|
||||||
|
# 构建JS加密代码文件的完整路径
|
||||||
|
js_path = os.path.join(current_dir, "chinaz.js")
|
||||||
|
|
||||||
|
# 读取JS加密代码文件(包含生成签名的函数)
|
||||||
|
with open(js_path, "r", encoding="utf-8") as f:
|
||||||
|
js_code = f.read()
|
||||||
|
|
||||||
|
# 创建JavaScript执行上下文(QuickJS引擎)
|
||||||
|
ctx = quickjs.Context()
|
||||||
|
|
||||||
|
# 【关键步骤】加载JS代码到上下文,使其可以被调用
|
||||||
|
# 这一步必须先执行,否则后续的函数调用会失败
|
||||||
|
ctx.eval(js_code)
|
||||||
|
|
||||||
|
# 调用JS函数生成host_key(主机密钥,是生成签名的基础)
|
||||||
|
# 格式:generateHostKey("domain")
|
||||||
|
host_key = ctx.eval(f'generateHostKey("{domain}")')
|
||||||
|
|
||||||
|
# 获取当前时间戳(毫秒),用于防止缓存和重放攻击
|
||||||
|
ts = str(int(time.time() * 1000))
|
||||||
|
|
||||||
|
# 构建请求参数字典(包含API所需的全部参数)
|
||||||
|
params = {
|
||||||
|
# jQuery回调函数名(格式:jQuery + 随机数字 + 时间戳)
|
||||||
|
"callback": f"jQuery11130943805094955342_{int(time.time() * 1000) - 1000}",
|
||||||
|
"action": "GetCategory", # 动作类型:获取分类/排名数据
|
||||||
|
"host": domain, # 主机域名
|
||||||
|
"secretkey": enkey, # 加密密钥(从页面获取)
|
||||||
|
|
||||||
|
# 随机数字(调用JS函数生成,与host_key相关)
|
||||||
|
"rd": ctx.eval(f'getRandomNum("{host_key}")'),
|
||||||
|
"ts": ts, # 时间戳(毫秒)
|
||||||
|
|
||||||
|
# MD5令牌(调用JS函数生成,用于请求签名验证)
|
||||||
|
"token": ctx.eval(f'generateMD5Token("{host_key}","{ts}")'),
|
||||||
|
|
||||||
|
"_": int(time.time() * 1000) # 额外时间戳参数(用于缓存破坏)
|
||||||
|
}
|
||||||
|
|
||||||
|
# 构建HTTP请求头(模拟浏览器访问)
|
||||||
|
headers = {
|
||||||
|
'Accept': '*/*', # 接受所有类型的内容
|
||||||
|
'Accept-Language': 'zh-cn', # 接受的语言:简体中文
|
||||||
|
'Referer': 'https://othertool.chinaz.com/GetTopRanked.ashx', # 来源页面(防盗链)
|
||||||
|
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.87 Safari/537.36', # 用户代理(浏览器标识)
|
||||||
|
'Host': 'othertool.chinaz.com', # 目标主机地址
|
||||||
|
}
|
||||||
|
|
||||||
|
# API请求URL(站长工具SEO数据查询接口)
|
||||||
|
url = 'https://othertool.chinaz.com/GetTopRanked.ashx'
|
||||||
|
|
||||||
|
# 发送GET请求获取SEO数据
|
||||||
|
resp = requests.get(
|
||||||
|
url, # 请求URL
|
||||||
|
params=params, # 查询参数(自动拼接为URL参数)
|
||||||
|
headers=headers, # 请求头
|
||||||
|
timeout=10, # 超时时间10秒
|
||||||
|
proxies=proxies # 代理配置
|
||||||
|
)
|
||||||
|
|
||||||
|
# 判断请求是否成功
|
||||||
|
if resp.status_code == 200:
|
||||||
|
# 使用正则表达式提取JSON数据(API返回格式:callback(json_data))
|
||||||
|
# 正则解释:匹配括号内的JSON内容
|
||||||
|
match = re.search(r'\((.*?)\)', resp.text)
|
||||||
|
|
||||||
|
# 提取JSON字符串
|
||||||
|
json_str = match.group(1)
|
||||||
|
|
||||||
|
# 解析JSON数据为Python字典
|
||||||
|
data = json.loads(json_str)
|
||||||
|
|
||||||
|
# 返回状态码和结果(StateCode为1表示成功)
|
||||||
|
return data['StateCode'] == 1, data['Result']
|
||||||
|
|
||||||
|
except Exception as e: # 捕获所有异常
|
||||||
|
logger.error(e) # 记录错误日志
|
||||||
|
return False, str(e) # 返回失败状态和错误信息
|
||||||
|
|
||||||
|
|
||||||
|
# # ========================= 【测试代码】 =========================
|
||||||
|
# if __name__ == '__main__':
|
||||||
|
#
|
||||||
|
#
|
||||||
|
# # 测试用例3:使用代理查询
|
||||||
|
# logger.info("=== 测试使用代理 ===")
|
||||||
|
# test_domain_3 = 'tt.com'
|
||||||
|
#
|
||||||
|
# logger.info(f"查询域名: {test_domain_3}")
|
||||||
|
#
|
||||||
|
#
|
||||||
|
# title_3, seo_data_3 = check_title(test_domain_3)
|
||||||
|
# logger.info(f"域名标题: {title_3}")
|
||||||
|
# logger.info(f"SEO数据: {seo_data_3}")
|
||||||
253
domainCheck/detect/geetest2.py
Normal file
253
domainCheck/detect/geetest2.py
Normal file
@@ -0,0 +1,253 @@
|
|||||||
|
# -*- coding:utf-8 -*-
|
||||||
|
import requests, re, json, base64
|
||||||
|
import io, os, random
|
||||||
|
import time, cv2, json
|
||||||
|
from PIL import Image
|
||||||
|
from functools import partial
|
||||||
|
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
from urllib3.util import proxy
|
||||||
|
|
||||||
|
from detect.module.use_ua import randomUA
|
||||||
|
from detect.module.gap import quekou
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8')
|
||||||
|
import execjs
|
||||||
|
|
||||||
|
|
||||||
|
class slide():
|
||||||
|
def __init__(self):
|
||||||
|
self.headers = randomUA()
|
||||||
|
self.t = round(time.time() * 1000)
|
||||||
|
|
||||||
|
def __ease_out_expo(self, sep):
|
||||||
|
if sep == 1:
|
||||||
|
return 1
|
||||||
|
else:
|
||||||
|
return 1 - pow(2, -10 * sep)
|
||||||
|
|
||||||
|
def get_slide_track(self, vdict):
|
||||||
|
distance = self.huak(vdict)
|
||||||
|
slide_track = [
|
||||||
|
[random.randint(-50, -10), random.randint(-50, -10), 0],
|
||||||
|
[0, 0, 0],
|
||||||
|
]
|
||||||
|
count = 30 + int(distance / 2)
|
||||||
|
t = random.randint(50, 100)
|
||||||
|
_x = 0
|
||||||
|
_y = 0
|
||||||
|
for i in range(count):
|
||||||
|
x = round(self.__ease_out_expo(i / count) * distance)
|
||||||
|
t += random.randint(10, 20)
|
||||||
|
if x == _x:
|
||||||
|
continue
|
||||||
|
slide_track.append([x, _y, t])
|
||||||
|
_x = x
|
||||||
|
slide_track.append(slide_track[-1])
|
||||||
|
return slide_track
|
||||||
|
|
||||||
|
def tp_huanyuan(self, content):
|
||||||
|
_img = Image.open(BytesIO(content))
|
||||||
|
_Ge = [{"x": -157, "y": -58}, {"x": -145, "y": -58}, {"x": -265, "y": -58}, {"x": -277, "y": -58},
|
||||||
|
{"x": -181, "y": -58}, {"x": -169, "y": -58}, {"x": -241, "y": -58}, {"x": -253, "y": -58},
|
||||||
|
{"x": -109, "y": -58}, {"x": -97, "y": -58}, {"x": -289, "y": -58}, {"x": -301, "y": -58},
|
||||||
|
{"x": -85, "y": -58}, {"x": -73, "y": -58}, {"x": -25, "y": -58}, {"x": -37, "y": -58},
|
||||||
|
{"x": -13, "y": -58}, {"x": -1, "y": -58}, {"x": -121, "y": -58}, {"x": -133, "y": -58},
|
||||||
|
{"x": -61, "y": -58}, {"x": -49, "y": -58}, {"x": -217, "y": -58}, {"x": -229, "y": -58},
|
||||||
|
{"x": -205, "y": -58}, {"x": -193, "y": -58}, {"x": -145, "y": 0}, {"x": -157, "y": 0},
|
||||||
|
{"x": -277, "y": 0}, {"x": -265, "y": 0}, {"x": -169, "y": 0}, {"x": -181, "y": 0},
|
||||||
|
{"x": -253, "y": 0}, {"x": -241, "y": 0}, {"x": -97, "y": 0}, {"x": -109, "y": 0},
|
||||||
|
{"x": -301, "y": 0}, {"x": -289, "y": 0}, {"x": -73, "y": 0}, {"x": -85, "y": 0},
|
||||||
|
{"x": -37, "y": 0}, {"x": -25, "y": 0}, {"x": -1, "y": 0}, {"x": -13, "y": 0},
|
||||||
|
{"x": -133, "y": 0}, {"x": -121, "y": 0}, {"x": -49, "y": 0}, {"x": -61, "y": 0},
|
||||||
|
{"x": -229, "y": 0}, {"x": -217, "y": 0}, {"x": -193, "y": 0}, {"x": -205, "y": 0}]
|
||||||
|
w_sep, h_sep = 10, 58
|
||||||
|
new_img = Image.new('RGB', (260, 116))
|
||||||
|
|
||||||
|
for idx in range(len(_Ge)):
|
||||||
|
x = abs(_Ge[idx]['x'])
|
||||||
|
y = 58 if _Ge[idx]['y'] == -58 else 0
|
||||||
|
img_cut = _img.crop((x, y, x + w_sep, y + h_sep))
|
||||||
|
new_x = idx % 26 * 10
|
||||||
|
new_y = 0 if idx < 26 else 58
|
||||||
|
new_img.paste(img_cut, (new_x, new_y))
|
||||||
|
|
||||||
|
img_byte = BytesIO()
|
||||||
|
new_img.save(img_byte, 'png')
|
||||||
|
return img_byte.getvalue()
|
||||||
|
|
||||||
|
def huak(self, vdict):
|
||||||
|
count = 1
|
||||||
|
bgbase64 = ""
|
||||||
|
tpbase64 = ""
|
||||||
|
for idv, p_url in vdict.items():
|
||||||
|
p_url = 'http://static.geetest.com/' + p_url
|
||||||
|
# print(p_url)
|
||||||
|
vcode = requests.get(p_url, headers=self.headers)
|
||||||
|
text = vcode.content
|
||||||
|
if idv == 'bg':
|
||||||
|
text = self.tp_huanyuan(text)
|
||||||
|
bgbase64 = base64.encodebytes(text).decode()
|
||||||
|
else:
|
||||||
|
tpbase64 = base64.encodebytes(text).decode()
|
||||||
|
|
||||||
|
count += 1
|
||||||
|
if bgbase64 and tpbase64:
|
||||||
|
dis = quekou().get_distance(bgbase64, tpbase64)
|
||||||
|
else:
|
||||||
|
dis = 0
|
||||||
|
return dis
|
||||||
|
|
||||||
|
|
||||||
|
class Geetest2():
|
||||||
|
def __init__(self):
|
||||||
|
self.header = {
|
||||||
|
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36",
|
||||||
|
}
|
||||||
|
# 获取当前文件的目录
|
||||||
|
current_dir = os.path.dirname(__file__)
|
||||||
|
|
||||||
|
with open(os.path.join(current_dir, 'module', 'crack_geetest2x.js'), "r", encoding="gb2312",
|
||||||
|
errors='ignore') as f:
|
||||||
|
js_encrypt = f.read()
|
||||||
|
self.js = execjs.compile(js_encrypt)
|
||||||
|
|
||||||
|
self.t = round(time.time() * 1000)
|
||||||
|
self.gct_vdi = {}
|
||||||
|
self.gct_volue = ['', ''] # 初始值
|
||||||
|
self.token_vdi = {}
|
||||||
|
self.timeout = (7, 9.05)
|
||||||
|
self.geetest_type = 'slide'
|
||||||
|
self.geetest_path = "/static/js/geetest.6.0.9.js"
|
||||||
|
|
||||||
|
def new_requests(self):
|
||||||
|
self.session = requests.session()
|
||||||
|
self.session.headers = self.header
|
||||||
|
|
||||||
|
def get_vsion(self, http, gt, proxies, pparmas):
|
||||||
|
url = "https://api.geetest.com/gettype.php"
|
||||||
|
params = {
|
||||||
|
"gt": gt,
|
||||||
|
"callback": f"geetest_{self.t}"
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
respe = http.get(url=url, params=params, proxies=proxies).text
|
||||||
|
except Exception as e:
|
||||||
|
if proxy:
|
||||||
|
if pparmas:
|
||||||
|
vdict = {'result': '错误', 'message': f'代理IP超时!,请重新更换代理IP,{e}', 'proxy': proxies,
|
||||||
|
'pparmas': pparmas}
|
||||||
|
else:
|
||||||
|
vdict = {'result': '错误', 'message': f'代理IP超时!,请重新更换代理IP,{e}', 'proxy': proxies}
|
||||||
|
else:
|
||||||
|
vdict = {'result': '错误', 'message': f'请求错误,{e}'}
|
||||||
|
return vdict
|
||||||
|
|
||||||
|
try:
|
||||||
|
data = json.loads(respe[22:-1])["data"]
|
||||||
|
except:
|
||||||
|
vdict = {'result': '错误', 'type': 'auto', 'message': respe}
|
||||||
|
return vdict
|
||||||
|
self.geetest_type = data["type"]
|
||||||
|
self.geetest_path = data['path']
|
||||||
|
self.token_vdi[gt] = 1
|
||||||
|
|
||||||
|
def get_tp(self, gt, challenge, type_, proxies=None, pparmas=None):
|
||||||
|
self.new_requests()
|
||||||
|
self.t = round(time.time() * 1000)
|
||||||
|
if pparmas:
|
||||||
|
h_list = re.findall('([\w-]+):(.+)', pparmas)
|
||||||
|
for idx in h_list:
|
||||||
|
print('新增协议头' + str(idx))
|
||||||
|
self.session.headers[idx[0]] = idx[1].replace('\r', '').replace('\n', '').replace('\r\n', '')
|
||||||
|
print(self.session.headers)
|
||||||
|
|
||||||
|
if not self.token_vdi.get('gt', ''):
|
||||||
|
self.get_vsion(self.session, gt, proxies, pparmas)
|
||||||
|
url = f"https://api.geetest.com/get.php"
|
||||||
|
params = {
|
||||||
|
"gt": gt,
|
||||||
|
"challenge": challenge,
|
||||||
|
"product": "popup",
|
||||||
|
"offline": "false",
|
||||||
|
"protocol": "https://",
|
||||||
|
"type": self.geetest_type,
|
||||||
|
"path": self.geetest_path,
|
||||||
|
"callback": f"geetest_{self.t}"
|
||||||
|
}
|
||||||
|
response = self.session.get(url, params=params, timeout=self.timeout).text
|
||||||
|
geetest_type = self.geetest_type
|
||||||
|
if geetest_type == 'slide':
|
||||||
|
data = json.loads(response[22:-1])
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
data = json.loads(response[22:-1])["data"]
|
||||||
|
except:
|
||||||
|
vdict = {'result': '错误', 'type': type_, 'message': response}
|
||||||
|
return vdict
|
||||||
|
##print(data)
|
||||||
|
nc, ns = data['c'], data['s']
|
||||||
|
|
||||||
|
if geetest_type == 'slide':
|
||||||
|
data = json.loads(response[22:-1])
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
data = json.loads(response[22:-1])["data"]
|
||||||
|
except:
|
||||||
|
vdict = {'result': '错误', 'type': type_, 'message': json.loads(response)}
|
||||||
|
return vdict
|
||||||
|
##print(data)
|
||||||
|
nc, ns = data['c'], data['s']
|
||||||
|
gct_url = 'http://static.geetest.com' + data['gct_path']
|
||||||
|
##print('gct地址:', gct_url)
|
||||||
|
if geetest_type == 'click':
|
||||||
|
return {'result': '请求类型不是滑块', 'type': type_, 'gt': gt, 'challenge': challenge,
|
||||||
|
'当前类型': '点选'}
|
||||||
|
if geetest_type == 'slide': # slide
|
||||||
|
|
||||||
|
challenge = data['challenge'] # 这里challenge改变了
|
||||||
|
bg = data['bg'] # 背景图片
|
||||||
|
slice = data['slice'] # 缺口图片
|
||||||
|
vdict = {
|
||||||
|
"bg": bg,
|
||||||
|
"slice": slice,
|
||||||
|
}
|
||||||
|
slide_track = slide().get_slide_track(vdict)
|
||||||
|
imgload = 37
|
||||||
|
w = self.js.call('get_slide_w', nc, ns, gt, challenge, slide_track, imgload, self.gct_volue)
|
||||||
|
params = (
|
||||||
|
('gt', gt),
|
||||||
|
('challenge', challenge),
|
||||||
|
('w', w),
|
||||||
|
('callback', f"geetest_{self.t}"),
|
||||||
|
)
|
||||||
|
response = self.session.get('https://api.geetest.com/ajax.php', params=params, proxies=proxies,
|
||||||
|
timeout=self.timeout).text
|
||||||
|
data = json.loads(response[22:-1])
|
||||||
|
validate = data.get('validate')
|
||||||
|
vdict = {'result': data.get('message'), 'type': geetest_type, 'gt': gt, 'challenge': challenge,
|
||||||
|
'validate': validate}
|
||||||
|
|
||||||
|
return vdict
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
a = Geetest2()
|
||||||
|
headers = {
|
||||||
|
|
||||||
|
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36",
|
||||||
|
}
|
||||||
|
url = "https://seo.juziseo.com/class/gtcode/msg/StartMsgCaptchaServlet.php"
|
||||||
|
url = "https://gsxt.hlj.gov.cn/registerValidate.jspx"
|
||||||
|
|
||||||
|
for i in range(1):
|
||||||
|
index = requests.get(url, headers=headers)
|
||||||
|
# print(index.text)
|
||||||
|
html = index.json()
|
||||||
|
challenge = html["challenge"]
|
||||||
|
gt = html["gt"]
|
||||||
|
print('gt', gt, 'challenge', challenge)
|
||||||
|
r = a.get_tp(gt, challenge, 'auto', '')
|
||||||
|
print(r)
|
||||||
557
domainCheck/detect/jucha.py
Normal file
557
domainCheck/detect/jucha.py
Normal file
@@ -0,0 +1,557 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :jucha.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/2 22:49
|
||||||
|
@explain : 聚查网API封装类 - 优化版,减少冗余代码,添加详细注释
|
||||||
|
'''
|
||||||
|
|
||||||
|
# 导入标准库
|
||||||
|
import base64 # Base64编码解码
|
||||||
|
import hashlib # 哈希加密
|
||||||
|
import json # JSON数据处理
|
||||||
|
import os # 操作系统接口
|
||||||
|
import pickle # 序列化反序列化
|
||||||
|
import random # 随机数生成
|
||||||
|
import subprocess # 子进程管理
|
||||||
|
import time # 时间处理
|
||||||
|
from functools import partial
|
||||||
|
|
||||||
|
# 导入第三方库
|
||||||
|
import requests # HTTP请求库
|
||||||
|
from loguru import logger # 日志记录
|
||||||
|
from requests.cookies import RequestsCookieJar # Cookie管理
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# # 移除自定义子进程类的替换,避免影响其他模块
|
||||||
|
# class MySubprocessPopen(subprocess.Popen): # 自定义子进程类
|
||||||
|
# def __init__(self, *args, **kwargs): # 初始化方法
|
||||||
|
# kwargs['encoding'] = "UTF-8" # 设置默认编码为UTF-8
|
||||||
|
# super().__init__(*args, **kwargs) # 调用父类初始化方法
|
||||||
|
#
|
||||||
|
#
|
||||||
|
# subprocess.Popen = MySubprocessPopen # 替换subprocess.Popen为自定义类
|
||||||
|
os.environ["EXECJS_RUNTIME"] = "Node" # 设置JavaScript运行时环境为Node.js
|
||||||
|
startupinfo = subprocess.STARTUPINFO()
|
||||||
|
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
|
||||||
|
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8', startupinfo=startupinfo)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_node_executable() -> str:
|
||||||
|
current_dir = os.path.dirname(__file__)
|
||||||
|
project_dir = os.path.dirname(current_dir)
|
||||||
|
bundled_node = os.path.join(project_dir, "tools", "node-v20.19.4-win-x64", "node.exe")
|
||||||
|
if os.path.exists(bundled_node):
|
||||||
|
return bundled_node
|
||||||
|
return "node"
|
||||||
|
|
||||||
|
def calculate_seed(t: int) -> str: # 计算验证码种子值
|
||||||
|
now = int(time.time()) # 获取当前时间戳
|
||||||
|
c = now - t # 计算时间差
|
||||||
|
seed = base64.b64encode(str(c).encode()).decode() # Base64编码
|
||||||
|
return seed # 返回种子字符串
|
||||||
|
|
||||||
|
|
||||||
|
def random_fingerprint() -> str: # 生成随机设备指纹
|
||||||
|
data = str(random.random()) + str(time.time()) # 组合随机数和时间戳
|
||||||
|
return hashlib.sha256(data.encode()).hexdigest() # 返回SHA256哈希值
|
||||||
|
|
||||||
|
|
||||||
|
class JC(object): # 聚查网API封装类
|
||||||
|
token: str # 验证码token
|
||||||
|
session_id: str # 会话ID
|
||||||
|
fingerprint: str # 设备指纹
|
||||||
|
captchaId: str # 验证码ID
|
||||||
|
encryptionPublicKey: str # 加密公钥
|
||||||
|
cookie: RequestsCookieJar = {} # 聚查网Cookie
|
||||||
|
juming_cookie: RequestsCookieJar = {} # 聚名网Cookie
|
||||||
|
|
||||||
|
# 通用请求头配置
|
||||||
|
headers = {
|
||||||
|
'accept': 'application/json, text/javascript, */*; q=0.01', # 接受的内容类型
|
||||||
|
'accept-language': 'zh-CN,zh;q=0.9', # 接受的语言
|
||||||
|
'content-type': 'application/x-www-form-urlencoded', # 内容类型
|
||||||
|
'origin': 'https://www.jucha.com', # 请求源
|
||||||
|
'priority': 'u=1, i', # 请求优先级
|
||||||
|
'referer': 'https://www.jucha.com/login', # 来源页面
|
||||||
|
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"', # 浏览器标识
|
||||||
|
'sec-ch-ua-mobile': '?0', # 是否移动端
|
||||||
|
'sec-ch-ua-platform': '"Windows"', # 操作系统平台
|
||||||
|
'sec-fetch-dest': 'empty', # 请求目标
|
||||||
|
'sec-fetch-mode': 'cors', # 请求模式
|
||||||
|
'sec-fetch-site': 'same-origin', # 请求站点
|
||||||
|
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36',
|
||||||
|
# 用户代理
|
||||||
|
'x-requested-with': 'XMLHttpRequest', # AJAX请求标识
|
||||||
|
}
|
||||||
|
|
||||||
|
def __init__(self, proxies: dict = None): # 初始化JC类
|
||||||
|
self.session = requests.Session() # 创建会话对象
|
||||||
|
self.session.timeout = 10 # 设置超时时间
|
||||||
|
self.session.proxies = proxies # 设置代理
|
||||||
|
self.base_url = "https://www.jucha.com" # 设置基础URL
|
||||||
|
self.cookie = requests.cookies.RequestsCookieJar() # 初始化CookieJar
|
||||||
|
self.juming_cookie = requests.cookies.RequestsCookieJar() # 初始化聚名网CookieJar
|
||||||
|
|
||||||
|
def _get_headers(self, referer: str = None) -> dict: # 获取请求头(可自定义referer)
|
||||||
|
headers = self.headers.copy() # 复制默认请求头
|
||||||
|
if referer: # 如果指定了referer
|
||||||
|
headers['referer'] = referer # 更新referer
|
||||||
|
return headers # 返回请求头
|
||||||
|
|
||||||
|
def _get_route_headers(self, route: str) -> dict: # 根据路由获取对应的请求头
|
||||||
|
referer_map = { # 路由到referer的映射
|
||||||
|
'whois': f'{self.base_url}/whois/', # WHOIS查询的referer
|
||||||
|
'beian': f'{self.base_url}/baian/', # 备案查询的referer
|
||||||
|
'safe': f'{self.base_url}/safe/', # 安全检测的referer
|
||||||
|
}
|
||||||
|
return self._get_headers(referer_map.get(route)) # 返回对应的请求头
|
||||||
|
|
||||||
|
def _handle_captcha(self, retry_times: int = 5) -> tuple: # 处理滑块验证码
|
||||||
|
for _ in range(retry_times): # 循环重试
|
||||||
|
init_result = self.captcha_init() # 初始化验证码
|
||||||
|
if init_result[0]: # 初始化成功
|
||||||
|
verify_result = self.captcha_verify() # 验证验证码
|
||||||
|
if verify_result[0]: # 验证成功
|
||||||
|
return True, '验证码验证成功' # 返回成功
|
||||||
|
return False, f'滑块验证码失败,{retry_times}次内未成功' # 返回失败
|
||||||
|
|
||||||
|
def _check_request(self, url: str, data: dict, headers: dict, cookies=None) -> dict: # 检查请求并处理验证码
|
||||||
|
res = self.session.post(url=url, data=data, headers=headers, cookies=cookies) # 发送POST请求
|
||||||
|
try:
|
||||||
|
response = res.json() # 尝试解析JSON
|
||||||
|
except Exception as e: # JSON解析失败
|
||||||
|
logger.error(f"JSON解析失败: {str(e)}") # 记录错误
|
||||||
|
logger.error(f"原始响应内容(前500字符): {res.text[:500]}") # 记录原始响应
|
||||||
|
return {'code': -1, 'msg': f'JSON解析失败: {str(e)}'} # 返回错误
|
||||||
|
return response # 返回响应
|
||||||
|
|
||||||
|
def _check_and_handle_captcha(self, response: dict, domain: str, route: str, xm_codes,
|
||||||
|
data: dict = None) -> tuple: # 检查并处理验证码
|
||||||
|
# 检查response是否为字典
|
||||||
|
if not isinstance(response, dict):
|
||||||
|
return False, None, response # 返回失败
|
||||||
|
|
||||||
|
if response.get('code') == 1001: # 需要验证码
|
||||||
|
captcha_result = self._handle_captcha() # 处理验证码
|
||||||
|
if captcha_result[0]: # 验证码验证成功
|
||||||
|
new_data = data.copy() if data else {} # 复制数据
|
||||||
|
new_data.update({ # 更新验证码参数
|
||||||
|
'_csrf': '', # CSRF令牌
|
||||||
|
'ymlb': domain, # 域名
|
||||||
|
'xm_codes[]': xm_codes, # 检测代码
|
||||||
|
'type': '2' if route in ['beian', 'safe'] else '1', # 类型:备案和安全检测为2,WHOIS为1
|
||||||
|
'route': route, # 路由
|
||||||
|
'captcha_verify_param': self.token, # 验证码token
|
||||||
|
'sessionId': self.session_id, # 会话ID
|
||||||
|
})
|
||||||
|
return True, new_data, None # 返回需要重试
|
||||||
|
return False, None, captcha_result # 返回失败
|
||||||
|
return False, None, response # 不需要验证码
|
||||||
|
|
||||||
|
def captcha_init(self): # 初始化滑块验证码
|
||||||
|
self.fingerprint = random_fingerprint() # 生成随机指纹
|
||||||
|
data = { # 构建请求数据
|
||||||
|
"request_id": self.fingerprint, # 请求ID
|
||||||
|
"scene": "default", # 场景
|
||||||
|
"seed": calculate_seed(286) # 种子值
|
||||||
|
}
|
||||||
|
url = f"{self.base_url}/captcha/init" # 初始化URL
|
||||||
|
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json() # 发送请求
|
||||||
|
if response['code'] == 1: # 初始化成功
|
||||||
|
self.captchaId = response['data']['captchaId'] # 保存验证码ID
|
||||||
|
self.encryptionPublicKey = response['data']['encryptionPublicKey'] # 保存加密公钥
|
||||||
|
return response['code'] == 1, response['msg'] # 返回结果
|
||||||
|
|
||||||
|
def captcha_verify(self): # 验证滑块验证码
|
||||||
|
data = { # 构建验证数据
|
||||||
|
"offset": 290, # 滑块偏移量
|
||||||
|
"duration": 611, # 滑动持续时间
|
||||||
|
"trail": [ # 滑动轨迹
|
||||||
|
{"x": 0, "y": 0, "time": 0}, {"x": 0, "y": 0, "time": 17}, {"x": 1, "y": 0, "time": 89},
|
||||||
|
{"x": 5, "y": 0, "time": 106}, {"x": 14, "y": 0, "time": 123}, {"x": 28, "y": 0, "time": 139},
|
||||||
|
{"x": 47, "y": 0, "time": 156}, {"x": 74, "y": 0, "time": 173}, {"x": 102, "y": 0, "time": 189},
|
||||||
|
{"x": 131, "y": 0, "time": 206}, {"x": 159, "y": 0, "time": 223}, {"x": 184, "y": 0, "time": 239},
|
||||||
|
{"x": 205, "y": 0, "time": 256}, {"x": 227, "y": 1, "time": 273}, {"x": 248, "y": 0, "time": 289},
|
||||||
|
{"x": 263, "y": 0, "time": 306}, {"x": 276, "y": 0, "time": 323}, {"x": 286, "y": 0, "time": 339},
|
||||||
|
{"x": 290, "y": 0, "time": 356}, {"x": 290, "y": 0, "time": 373}, {"x": 290, "y": 0, "time": 389},
|
||||||
|
{"x": 290, "y": 0, "time": 406}, {"x": 290, "y": 0, "time": 423},
|
||||||
|
{"x": 290, "y": 0, "time": 478}
|
||||||
|
],
|
||||||
|
"fingerprint": self.fingerprint, # 设备指纹
|
||||||
|
"captchaId": self.captchaId, # 验证码ID
|
||||||
|
"serverPublicKey": self.encryptionPublicKey # 服务器公钥
|
||||||
|
}
|
||||||
|
data_str = json.dumps(data, separators=(",", ":")) # 转换为JSON字符串
|
||||||
|
current_dir = os.path.dirname(__file__)
|
||||||
|
|
||||||
|
jsFile_path = current_dir + "/sdk_leg_env.js" # JavaScript脚本路径
|
||||||
|
|
||||||
|
if not os.path.exists(jsFile_path): # 检查脚本文件是否存在
|
||||||
|
logger.error(f"Node.js脚本文件不存在: {jsFile_path}") # 记录错误
|
||||||
|
return False, f"缺少签名脚本: {jsFile_path}" # 返回失败
|
||||||
|
|
||||||
|
try: # 尝试执行Node.js脚本
|
||||||
|
with subprocess.Popen( # 创建子进程
|
||||||
|
[resolve_node_executable(), jsFile_path], # 执行node命令
|
||||||
|
stdin=subprocess.PIPE, # 标准输入管道
|
||||||
|
stdout=subprocess.PIPE, # 标准输出管道
|
||||||
|
stderr=subprocess.PIPE, # 标准错误管道
|
||||||
|
text=True, # 文本模式
|
||||||
|
encoding="utf-8", # UTF-8编码
|
||||||
|
errors="ignore" # 忽略编码错误
|
||||||
|
) as proc: # 进程上下文
|
||||||
|
stdout, stderr = proc.communicate(data_str, timeout=10) # 传入参数并获取输出
|
||||||
|
verify_data = stdout.strip() # 去除空白字符
|
||||||
|
if stderr: # 如果有错误输出
|
||||||
|
logger.error(f'genData_stderr->[{stderr}]') # 记录错误
|
||||||
|
|
||||||
|
if proc.returncode != 0: # 检查退出码
|
||||||
|
return False, f"Node.js脚本执行失败,退出码: {proc.returncode}, 错误信息: {stderr}" # 返回失败
|
||||||
|
|
||||||
|
url = f"{self.base_url}/captcha/verify" # 验证URL
|
||||||
|
verify_headers = self.headers.copy() # 复制请求头
|
||||||
|
verify_headers['Content-Type'] = 'application/json' # 设置内容类型(仅用于验证请求)
|
||||||
|
verify_response = self.session.post(url, headers=verify_headers, data=verify_data,
|
||||||
|
cookies=self.cookie) # 发送验证请求
|
||||||
|
if verify_response.cookies: # 如果响应中有新的Cookie
|
||||||
|
self.cookie.update(verify_response.cookies) # 合并Cookie(而不是替换)
|
||||||
|
response = verify_response.json() # 解析JSON响应
|
||||||
|
if response['code'] == 1: # 验证成功
|
||||||
|
self.token = response['data']['token'] # 保存token
|
||||||
|
self.session_id = response['data']['session_id'] # 保存会话ID
|
||||||
|
return response['code'] == 1, response['msg'] # 返回结果
|
||||||
|
except subprocess.TimeoutExpired: # 超时异常
|
||||||
|
logger.error("Node.js脚本执行超时(超过10秒)") # 记录错误
|
||||||
|
return False, "Node.js脚本执行超时" # 返回失败
|
||||||
|
except Exception as e: # 其他异常
|
||||||
|
logger.error(f"执行Node.js脚本异常: {str(e)}") # 记录错误
|
||||||
|
return False, f"执行Node.js脚本异常: {str(e)}" # 返回失败
|
||||||
|
|
||||||
|
if not verify_data: # 检查输出是否为空
|
||||||
|
return False, "Node.js脚本未输出任何内容" # 返回失败
|
||||||
|
|
||||||
|
def save_cookies(self, filepath="jucha_cookies.pkl"): # 保存Cookie到文件
|
||||||
|
with open(filepath, "wb") as f: # 以二进制写入模式打开文件
|
||||||
|
pickle.dump(self.cookie, f) # 序列化保存Cookie
|
||||||
|
logger.info(f"已保存Cookie到文件: {filepath}")
|
||||||
|
# 保存到Redis
|
||||||
|
try:
|
||||||
|
import redis
|
||||||
|
from app.config import config
|
||||||
|
redis_client = redis.Redis(
|
||||||
|
host=config.REDIS_HOST,
|
||||||
|
port=config.REDIS_PORT,
|
||||||
|
password=config.REDIS_PASSWORD,
|
||||||
|
db=config.REDIS_DB,
|
||||||
|
decode_responses=True
|
||||||
|
)
|
||||||
|
# 将cookie转换为字典
|
||||||
|
cookie_dict = {}
|
||||||
|
# 检查self.cookie的类型
|
||||||
|
if isinstance(self.cookie, dict):
|
||||||
|
# 如果是字典,直接使用
|
||||||
|
cookie_dict = self.cookie
|
||||||
|
logger.info(f"Cookie是字典类型,直接使用")
|
||||||
|
elif hasattr(self.cookie, '__iter__'):
|
||||||
|
# 如果是可迭代对象,遍历处理
|
||||||
|
for cookie in self.cookie:
|
||||||
|
# 检查cookie对象是否有name和value属性
|
||||||
|
if hasattr(cookie, 'name') and hasattr(cookie, 'value'):
|
||||||
|
cookie_dict[cookie.name] = cookie.value
|
||||||
|
logger.info(f"Cookie是可迭代对象,处理后得到: {cookie_dict}")
|
||||||
|
else:
|
||||||
|
logger.warning(f"Cookie类型不支持: {type(self.cookie)}")
|
||||||
|
|
||||||
|
redis_client.set('domain_tool:jucha_cookies', str(cookie_dict))
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"保存Cookie到Redis异常: {str(e)}")
|
||||||
|
pass
|
||||||
|
|
||||||
|
def load_cookies(self, filepath="jucha_cookies.pkl"): # 从文件加载Cookie
|
||||||
|
try: # 尝试加载
|
||||||
|
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
|
||||||
|
loaded_cookie = pickle.load(f) # 反序列化加载Cookie
|
||||||
|
# 检查加载的cookie类型
|
||||||
|
if isinstance(loaded_cookie, dict):
|
||||||
|
# 如果是字典,转换为RequestsCookieJar
|
||||||
|
cookie_jar = requests.cookies.RequestsCookieJar()
|
||||||
|
for name, value in loaded_cookie.items():
|
||||||
|
cookie_jar.set(name, value)
|
||||||
|
self.cookie = cookie_jar
|
||||||
|
else:
|
||||||
|
self.cookie = loaded_cookie
|
||||||
|
except Exception as e: # 加载失败
|
||||||
|
logger.error(f"加载Cookie失败: {e}")
|
||||||
|
self.cookie = requests.cookies.RequestsCookieJar() # 创建空的CookieJar
|
||||||
|
|
||||||
|
def load_juming_cookies(self, filepath="juming_cookies.pkl"): # 从文件加载聚名网Cookie
|
||||||
|
try: # 尝试加载
|
||||||
|
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
|
||||||
|
self.juming_cookie = pickle.load(f) # 反序列化加载Cookie
|
||||||
|
except: # 加载失败
|
||||||
|
self.juming_cookie = requests.cookies.RequestsCookieJar() # 创建空的CookieJar
|
||||||
|
|
||||||
|
def auth_login(self): # 使用聚名网账号登录聚查网
|
||||||
|
params = { # 请求参数
|
||||||
|
'platform': 'juming', # 平台标识
|
||||||
|
}
|
||||||
|
res = self.session.get( # 发送GET请求
|
||||||
|
url=f'{self.base_url}/home/login/get_auth_url', # 获取授权URL
|
||||||
|
params=params, # 请求参数
|
||||||
|
headers=self.headers, # 请求头
|
||||||
|
cookies=self.juming_cookie # 聚名网Cookie
|
||||||
|
)
|
||||||
|
if res.cookies: # 如果响应中有Cookie
|
||||||
|
self.cookie.update(res.cookies) # 更新Cookie
|
||||||
|
response = res.json() # 解析JSON响应
|
||||||
|
if response['code'] == 1: # 获取授权URL成功
|
||||||
|
url = f'%s&tiao=1' % response['data']['auth_url'] # 构建跳转URL
|
||||||
|
# 合并聚名网Cookie和聚查网Cookie
|
||||||
|
combined_cookies = requests.cookies.RequestsCookieJar()
|
||||||
|
combined_cookies.update(self.juming_cookie)
|
||||||
|
combined_cookies.update(self.cookie)
|
||||||
|
response = self.session.get( # 发送GET请求
|
||||||
|
url=url, # 跳转URL
|
||||||
|
headers=self.headers, # 请求头
|
||||||
|
cookies=combined_cookies, # 合并后的Cookie
|
||||||
|
allow_redirects=False # 不自动重定向
|
||||||
|
)
|
||||||
|
if response.cookies: # 如果响应中有Cookie
|
||||||
|
# 安全更新Cookie,避免冲突
|
||||||
|
for name, value in response.cookies.items():
|
||||||
|
self.cookie[name] = value # 直接赋值,覆盖同名Cookie
|
||||||
|
if response.status_code == 302: # 重定向状态码
|
||||||
|
url = response.headers['location'] # 获取重定向URL
|
||||||
|
response = self.session.get( # 发送GET请求
|
||||||
|
url=url, # 重定向URL
|
||||||
|
headers=self.headers, # 请求头
|
||||||
|
allow_redirects=False # 不自动重定向
|
||||||
|
)
|
||||||
|
if response.cookies: # 如果响应中有Cookie
|
||||||
|
# 安全更新Cookie,避免冲突
|
||||||
|
for name, value in response.cookies.items():
|
||||||
|
self.cookie[name] = value # 直接赋值,覆盖同名Cookie
|
||||||
|
if response.status_code == 302: # 再次重定向
|
||||||
|
return True, '登录成功' # 返回成功
|
||||||
|
return False, '聚名登录过期' # 返回失败
|
||||||
|
return False, response['msg'] # 返回失败
|
||||||
|
|
||||||
|
def _build_check_data(self, domain: str, route: str, xm_codes, data: dict = None) -> dict: # 构建检测请求数据
|
||||||
|
if data is not None: # 如果data不为空
|
||||||
|
return data # 直接返回data
|
||||||
|
return { # 构建新的请求数据
|
||||||
|
'_csrf': '', # CSRF令牌
|
||||||
|
'ymlb': domain, # 域名
|
||||||
|
'xm_codes[]': xm_codes, # 检测代码
|
||||||
|
'type': '2' if route in ['beian', 'safe'] else '1', # 类型:备案和安全检测为2,WHOIS为1
|
||||||
|
'route': route, # 路由
|
||||||
|
}
|
||||||
|
|
||||||
|
def _make_check_request(self, domain: str, route: str, xm_codes, data: dict = None) -> tuple: # 发起检测请求
|
||||||
|
headers = self._get_route_headers(route) # 获取对应的请求头
|
||||||
|
data = self._build_check_data(domain, route, xm_codes, data) # 构建请求数据
|
||||||
|
url = f'{self.base_url}/home_item/check' # 检测URL
|
||||||
|
response = self._check_request(url, data, headers, self.cookie) # 发送请求
|
||||||
|
return response, data # 返回响应和数据
|
||||||
|
|
||||||
|
def _handle_check_response(self, response: dict, domain: str, route: str, xm_codes, data: dict,
|
||||||
|
callback) -> tuple: # 处理检测响应
|
||||||
|
need_retry, retry_data, final_response = self._check_and_handle_captcha( # 检查并处理验证码
|
||||||
|
response, domain, route, xm_codes, data
|
||||||
|
)
|
||||||
|
if need_retry: # 需要重试
|
||||||
|
return callback(domain=domain, data=retry_data) # 递归调用回调
|
||||||
|
|
||||||
|
# 检查final_response是否为字典
|
||||||
|
if not isinstance(final_response, dict):
|
||||||
|
return False, f'响应格式错误: {final_response}', '' # 返回失败
|
||||||
|
|
||||||
|
if final_response['code'] == 1: # 检测成功
|
||||||
|
return self._get_search_result(final_response, domain, route) # 获取查询结果(传入domain)
|
||||||
|
return False, final_response.get('msg', '未知错误'), '' # 返回失败
|
||||||
|
|
||||||
|
def _get_search_result(self, response: dict, domain: str, route: str) -> tuple: # 获取查询结果
|
||||||
|
data = { # 构建查询数据
|
||||||
|
'_csrf': '', # CSRF令牌
|
||||||
|
'domain': domain, # 域名(注意:使用domain而不是ymlb)
|
||||||
|
'rwid': response['data']['rwid'], # 任务ID
|
||||||
|
'type': '2' if route in ['beian', 'safe'] else '1', # 类型
|
||||||
|
'route': 'baian' if route == 'beian' else route, # 路由
|
||||||
|
}
|
||||||
|
headers = self._get_route_headers(route) # 获取对应的请求头
|
||||||
|
url = f'{self.base_url}/home/item/search' if route != 'whois' else f'{self.base_url}/home/item/search_one' # 查询URL
|
||||||
|
search_response = self._check_request(url, data, headers, self.cookie) # 发送查询请求
|
||||||
|
|
||||||
|
# 检查search_response是否为字典
|
||||||
|
if not isinstance(search_response, dict):
|
||||||
|
return False, f'响应格式错误: {search_response}', '' # 返回失败
|
||||||
|
|
||||||
|
if search_response.get('code', -1) != 1: # 查询失败
|
||||||
|
return False, search_response.get('msg', '查询失败'), '' # 返回失败
|
||||||
|
|
||||||
|
if route == 'whois': # WHOIS查询
|
||||||
|
# 安全获取WHOIS状态
|
||||||
|
whois_zt = ''
|
||||||
|
try:
|
||||||
|
data1 = search_response.get('data', {})
|
||||||
|
if isinstance(data1, dict):
|
||||||
|
data2 = data1.get('data', {})
|
||||||
|
if isinstance(data2, dict):
|
||||||
|
whois = data2.get('whois', {})
|
||||||
|
if isinstance(whois, dict):
|
||||||
|
data3 = whois.get('data', {})
|
||||||
|
if isinstance(data3, dict):
|
||||||
|
data4 = data3.get('data', {})
|
||||||
|
if isinstance(data4, dict):
|
||||||
|
whois_zt = data4.get('zt', '')
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取WHOIS状态失败: {e}")
|
||||||
|
|
||||||
|
return ( # 返回WHOIS结果
|
||||||
|
True, # 成功
|
||||||
|
search_response['msg'], # 消息
|
||||||
|
whois_zt # WHOIS状态
|
||||||
|
)
|
||||||
|
elif route == 'beian': # 备案查询
|
||||||
|
# 安全获取备案数据
|
||||||
|
beian_data = {}
|
||||||
|
beian_msg = ''
|
||||||
|
try:
|
||||||
|
data1 = search_response.get('data', {})
|
||||||
|
if isinstance(data1, dict):
|
||||||
|
data2 = data1.get('data', {})
|
||||||
|
if isinstance(data2, dict):
|
||||||
|
beian = data2.get('beian', {})
|
||||||
|
if isinstance(beian, dict):
|
||||||
|
data3 = beian.get('data', {})
|
||||||
|
if isinstance(data3, dict):
|
||||||
|
beian_data = data3.get('data', {})
|
||||||
|
beian_msg = data3.get('msg', '')
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取备案数据失败: {e}")
|
||||||
|
|
||||||
|
is_dict = isinstance(beian_data, dict) # 是否为字典
|
||||||
|
return ( # 返回备案结果
|
||||||
|
True, # 成功
|
||||||
|
search_response['msg'], # 消息
|
||||||
|
( # 备案信息元组
|
||||||
|
beian_data.get('sj', '') if is_dict else '', # 备案时间
|
||||||
|
beian_data.get('lx', '') if is_dict else '', # 备案类型
|
||||||
|
beian_data.get('sy', '') if is_dict else '', # 备案首页地址
|
||||||
|
beian_msg, # 备案状态
|
||||||
|
)
|
||||||
|
)
|
||||||
|
elif route == 'safe': # 安全检测
|
||||||
|
# 安全获取安全检测数据
|
||||||
|
safe_data = {}
|
||||||
|
try:
|
||||||
|
data1 = search_response.get('data', {})
|
||||||
|
if isinstance(data1, dict):
|
||||||
|
safe_data = data1.get('data', {})
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取安全检测数据失败: {e}")
|
||||||
|
|
||||||
|
check_items = [ # 检测项配置列表
|
||||||
|
('qqjc', 'QQ检测'),
|
||||||
|
('weixin', '微信检测'),
|
||||||
|
('qiang', '被墙检测'),
|
||||||
|
('dyjc', '抖音检测'),
|
||||||
|
('bdjc', '百度检测'),
|
||||||
|
('llqjcgg', '谷歌检测'),
|
||||||
|
('llqjchh', '火狐检测'),
|
||||||
|
]
|
||||||
|
|
||||||
|
results = [] # 结果列表
|
||||||
|
for item_key, _ in check_items: # 遍历检测项
|
||||||
|
try:
|
||||||
|
code = 1
|
||||||
|
msg = ''
|
||||||
|
item_data = safe_data.get(item_key, {})
|
||||||
|
if isinstance(item_data, dict):
|
||||||
|
data1 = item_data.get('data', {})
|
||||||
|
if isinstance(data1, dict):
|
||||||
|
code = int(data1.get('data', 1)) # 获取检测码
|
||||||
|
msg = data1.get('msg', '') # 获取消息
|
||||||
|
if code == 3: # 如果检测码为3
|
||||||
|
msg = '拦截' # 设置为拦截
|
||||||
|
results.append((code, msg)) # 添加到结果列表
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"获取安全检测项 {item_key} 失败: {e}")
|
||||||
|
results.append((1, '查询失败')) # 添加失败结果
|
||||||
|
|
||||||
|
return ( # 返回安全检测结果
|
||||||
|
True, # 成功
|
||||||
|
search_response['msg'], # 消息
|
||||||
|
tuple(results) # 安全检测结果元组
|
||||||
|
)
|
||||||
|
return False, '未知路由', '' # 返回失败
|
||||||
|
|
||||||
|
def check_whois_domain(self, domain: str, data=None): # 查询域名WHOIS信息
|
||||||
|
try: # 异常处理
|
||||||
|
response, check_data = self._make_check_request(domain, 'whois', 'whois', data) # 发起检测请求
|
||||||
|
return self._handle_check_response(response, domain, 'whois', 'whois', check_data,
|
||||||
|
self.check_whois_domain) # 处理响应
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
logger.error(f"check_domain异常: {e}") # 记录错误
|
||||||
|
return False, str(e), '' # 返回失败
|
||||||
|
|
||||||
|
def beian_check_domain(self, domain: str, data=None): # 查询域名备案信息
|
||||||
|
try: # 异常处理
|
||||||
|
response, check_data = self._make_check_request(domain, 'beian', 'beian', data) # 发起检测请求
|
||||||
|
return self._handle_check_response(response, domain, 'beian', 'beian', check_data,
|
||||||
|
self.beian_check_domain) # 处理响应
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
logger.error(f"beian_check_domain异常: {e}") # 记录错误
|
||||||
|
return False, str(e), [] # 返回失败
|
||||||
|
|
||||||
|
def safe_check_domain(self, domain: str, data=None): # 查询域名安全信息
|
||||||
|
try: # 异常处理
|
||||||
|
xm_codes = [ # 检测代码列表
|
||||||
|
'qqjc', # QQ检测
|
||||||
|
'weixin', # 微信检测
|
||||||
|
'dyjc', # 抖音检测
|
||||||
|
'qiang', # 被墙检测
|
||||||
|
'bdjc', # 百度检测
|
||||||
|
'llqjcgg', # 谷歌检测
|
||||||
|
'llqjchh', # 火狐检测
|
||||||
|
]
|
||||||
|
response, check_data = self._make_check_request(domain, 'safe', xm_codes, data) # 发起检测请求
|
||||||
|
return self._handle_check_response(response, domain, 'safe', xm_codes, check_data,
|
||||||
|
self.safe_check_domain) # 处理响应
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
logger.error(f"safe_check_domain异常: {e}") # 记录错误
|
||||||
|
return False, str(e), [] # 返回失败
|
||||||
|
|
||||||
|
|
||||||
|
# # 测试代码
|
||||||
|
# if __name__ == '__main__':
|
||||||
|
# j = JC() # 创建JC实例
|
||||||
|
# j.load_cookies()
|
||||||
|
# j.load_juming_cookies() # 加载聚名网Cookie
|
||||||
|
# logger.info("开始登录...")
|
||||||
|
# login_result = j.auth_login() # 使用聚名网账号登录聚查网
|
||||||
|
# logger.info(f"登录结果: {login_result}")
|
||||||
|
# if not login_result[0]:
|
||||||
|
# logger.error("登录失败,无法继续测试")
|
||||||
|
# exit(1)
|
||||||
|
# # 测试WHOIS查询
|
||||||
|
# logger.info("测试WHOIS查询:")
|
||||||
|
# result = j.check_whois_domain('921229.com')
|
||||||
|
# logger.info(f"WHOIS查询结果: {result}")
|
||||||
|
# logger.info(j.cookie)
|
||||||
|
|
||||||
|
# # # 测试备案查询
|
||||||
|
# # logger.info("测试备案查询:")
|
||||||
|
# # result = j.beian_check_domain('baidu.com')
|
||||||
|
# # logger.info(f"备案查询结果: {result}")
|
||||||
|
# #
|
||||||
|
# # 测试安全检测
|
||||||
|
# logger.info("测试安全检测:")
|
||||||
|
# result = j.safe_check_domain('576777.com')
|
||||||
|
# logger.info(f"安全检测结果: {result}")
|
||||||
|
#
|
||||||
|
# # j.save_cookies() # 保存Cookie
|
||||||
497
domainCheck/detect/juming.py
Normal file
497
domainCheck/detect/juming.py
Normal file
@@ -0,0 +1,497 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :juming.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/3/19 15:51
|
||||||
|
@explain : 聚名网API封装类 - 提供登录、域名查询、删除域名列表等功能
|
||||||
|
'''
|
||||||
|
|
||||||
|
import base64 # Base64编码模块
|
||||||
|
import hashlib # 哈希算法模块
|
||||||
|
import json # JSON处理模块
|
||||||
|
import os # 操作系统接口模块
|
||||||
|
import pickle # 序列化模块
|
||||||
|
import random # 随机数生成模块
|
||||||
|
import subprocess # 子进程管理模块
|
||||||
|
import time # 时间处理模块
|
||||||
|
from functools import partial
|
||||||
|
from typing import Optional, Tuple, Dict, List, Any # 类型提示
|
||||||
|
|
||||||
|
import requests # HTTP请求库
|
||||||
|
from loguru import logger # 日志记录库
|
||||||
|
from requests.cookies import RequestsCookieJar # Cookie处理
|
||||||
|
|
||||||
|
|
||||||
|
# 常量定义
|
||||||
|
BASE_URL = "https://www.juming.com" # 聚名网基础URL
|
||||||
|
CAPTCHA_MAX_RETRY = 5 # 验证码最大重试次数
|
||||||
|
REQUEST_TIMEOUT = 10 # 请求超时时间(秒)
|
||||||
|
DOWNLOAD_TIMEOUT = 60 # 下载超时时间(秒)
|
||||||
|
SEED_TIME_OFFSET = 286 # 种子时间偏移量
|
||||||
|
SLIDE_OFFSET = 290 # 滑块偏移量
|
||||||
|
SLIDE_DURATION = 611 # 滑动持续时间
|
||||||
|
|
||||||
|
|
||||||
|
class MySubprocessPopen(subprocess.Popen): # 自定义子进程类
|
||||||
|
def __init__(self, *args, **kwargs): # 初始化方法
|
||||||
|
kwargs['encoding'] = "UTF-8" # 设置编码为UTF-8
|
||||||
|
super().__init__(*args, **kwargs) # 调用父类初始化
|
||||||
|
|
||||||
|
|
||||||
|
subprocess.Popen = MySubprocessPopen # 替换默认的Popen类
|
||||||
|
os.environ["EXECJS_RUNTIME"] = "Node" # 设置JavaScript运行时为Node.js
|
||||||
|
startupinfo = subprocess.STARTUPINFO()
|
||||||
|
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
|
||||||
|
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8', startupinfo=startupinfo)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_node_executable() -> str:
|
||||||
|
current_dir = os.path.dirname(__file__)
|
||||||
|
project_dir = os.path.dirname(current_dir)
|
||||||
|
bundled_node = os.path.join(project_dir, "tools", "node-v20.19.4-win-x64", "node.exe")
|
||||||
|
if os.path.exists(bundled_node):
|
||||||
|
return bundled_node
|
||||||
|
return "node"
|
||||||
|
|
||||||
|
def calculate_seed(t: int) -> str: # 计算验证码种子值
|
||||||
|
now = int(time.time()) # 获取当前时间戳
|
||||||
|
c = now - t # 计算时间差
|
||||||
|
seed = base64.b64encode(str(c).encode()).decode() # Base64编码
|
||||||
|
return seed # 返回种子字符串
|
||||||
|
|
||||||
|
|
||||||
|
def random_fingerprint() -> str: # 生成随机设备指纹
|
||||||
|
data = str(random.random()) + str(time.time()) # 组合随机数和时间戳
|
||||||
|
return hashlib.sha256(data.encode()).hexdigest() # 返回SHA256哈希值
|
||||||
|
|
||||||
|
|
||||||
|
def glwb(s: Optional[str]) -> str: # JS文本过滤函数
|
||||||
|
"""
|
||||||
|
过滤特殊字符,防止XSS攻击
|
||||||
|
Args:
|
||||||
|
s: 待过滤的字符串
|
||||||
|
Returns:
|
||||||
|
过滤后的字符串
|
||||||
|
"""
|
||||||
|
if s is None or not isinstance(s, str): # 检查输入是否有效
|
||||||
|
return '' # 返回空字符串
|
||||||
|
|
||||||
|
a_nr = s # 复制字符串
|
||||||
|
a_nr = a_nr.replace('"', '"') # 替换双引号
|
||||||
|
a_nr = a_nr.replace("'", ''') # 替换单引号
|
||||||
|
a_nr = a_nr.replace('<', '<') # 替换小于号
|
||||||
|
a_nr = a_nr.replace('>', '>') # 替换大于号
|
||||||
|
a_nr = a_nr.replace('\\', '\') # 替换反斜杠
|
||||||
|
|
||||||
|
return a_nr # 返回过滤后的字符串
|
||||||
|
|
||||||
|
|
||||||
|
def md5_19(text: str) -> str: # 计算MD5并返回前19位
|
||||||
|
"""
|
||||||
|
计算MD5哈希值并截取前19位
|
||||||
|
Args:
|
||||||
|
text: 待哈希的文本
|
||||||
|
Returns:
|
||||||
|
MD5哈希值的前19位
|
||||||
|
"""
|
||||||
|
if not text: # 检查输入是否为空
|
||||||
|
return '' # 返回空字符串
|
||||||
|
md5 = hashlib.md5(text.encode('utf-8')).hexdigest() # 生成32位MD5
|
||||||
|
return md5[:19] # 返回前19位
|
||||||
|
|
||||||
|
|
||||||
|
def encrypt_password(loginToken: str, password: str) -> str: # 加密密码
|
||||||
|
"""
|
||||||
|
使用双重MD5加密密码
|
||||||
|
Args:
|
||||||
|
loginToken: 登录令牌
|
||||||
|
password: 明文密码
|
||||||
|
Returns:
|
||||||
|
加密后的密码
|
||||||
|
"""
|
||||||
|
filtered_pwd = glwb(password) # 过滤密码中的特殊字符
|
||||||
|
step1 = f'[jiami{filtered_pwd}mima]' # 拼接固定盐值
|
||||||
|
md5_step1 = md5_19(step1) # 第一次MD5加密
|
||||||
|
step2 = loginToken + md5_step1 # 拼接登录令牌
|
||||||
|
final_result = md5_19(step2) # 第二次MD5加密
|
||||||
|
return final_result # 返回最终加密结果
|
||||||
|
|
||||||
|
|
||||||
|
class JM(object): # 聚名网API封装类
|
||||||
|
token: str # 验证码token
|
||||||
|
loginToken: str = '' # 登录令牌
|
||||||
|
session_id: str # 会话ID
|
||||||
|
fingerprint: str # 设备指纹
|
||||||
|
captchaId: str # 验证码ID
|
||||||
|
encryptionPublicKey: str # 加密公钥
|
||||||
|
cookie: RequestsCookieJar = {} # Cookie存储
|
||||||
|
|
||||||
|
# 默认请求头
|
||||||
|
headers = {
|
||||||
|
'Host': 'www.juming.com', # 主机名
|
||||||
|
'sec-ch-ua-platform': '"Windows"', # 平台标识
|
||||||
|
'x-requested-with': 'XMLHttpRequest', # AJAX请求标识
|
||||||
|
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0', # 用户代理
|
||||||
|
'accept': '*/*', # 接受所有类型
|
||||||
|
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Microsoft Edge";v="146"', # 浏览器标识
|
||||||
|
'sec-ch-ua-mobile': '?0', # 是否移动端
|
||||||
|
'origin': 'https://www.juming.com', # 请求源
|
||||||
|
'sec-fetch-site': 'same-origin', # 请求站点
|
||||||
|
'sec-fetch-mode': 'cors', # 请求模式
|
||||||
|
'sec-fetch-dest': 'empty', # 请求目标
|
||||||
|
'referer': 'https://www.juming.com/', # 来源页面
|
||||||
|
'accept-language': 'zh-CN,zh;q=0.9', # 接受的语言
|
||||||
|
'priority': 'u=1, i', # 请求优先级
|
||||||
|
'Content-Type': 'application/x-www-form-urlencoded', # 内容类型
|
||||||
|
}
|
||||||
|
|
||||||
|
def __init__(self, proxies: Optional[Dict] = None): # 初始化方法
|
||||||
|
self.session = requests.Session() # 创建会话对象
|
||||||
|
self.session.timeout = REQUEST_TIMEOUT # 设置超时时间
|
||||||
|
self.session.proxies = proxies # 设置代理
|
||||||
|
self.base_url = BASE_URL # 设置基础URL
|
||||||
|
logger.debug(proxies)
|
||||||
|
|
||||||
|
def captcha_init(self) -> Tuple[bool, str]: # 初始化滑块验证码
|
||||||
|
"""
|
||||||
|
初始化滑块验证码
|
||||||
|
Returns:
|
||||||
|
Tuple[bool, str]: (是否成功, 消息)
|
||||||
|
"""
|
||||||
|
self.fingerprint = random_fingerprint() # 生成随机指纹
|
||||||
|
data = { # 构建请求数据
|
||||||
|
"request_id": self.fingerprint, # 请求ID
|
||||||
|
"scene": "default", # 场景
|
||||||
|
"seed": calculate_seed(SEED_TIME_OFFSET) # 种子值
|
||||||
|
}
|
||||||
|
logger.info(data)
|
||||||
|
self.headers['Content-Type'] = 'application/x-www-form-urlencoded' # 设置内容类型
|
||||||
|
|
||||||
|
url = f"{self.base_url}/captcha/init" # 初始化URL
|
||||||
|
# self.cookie.clear()
|
||||||
|
# self.cookie.update({'acw_sc__v2': '69d948ed49117d89433d4316a79ada6f00d68f26'})
|
||||||
|
# logger.info(self.cookie)
|
||||||
|
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json() # 发送请求
|
||||||
|
logger.info(response)
|
||||||
|
|
||||||
|
if response['code'] == 1: # 初始化成功
|
||||||
|
self.captchaId = response['data']['captchaId'] # 保存验证码ID
|
||||||
|
self.encryptionPublicKey = response['data']['encryptionPublicKey'] # 保存加密公钥
|
||||||
|
return response['code'] == 1, response['msg'] # 返回结果
|
||||||
|
|
||||||
|
def captcha_verify(self) -> Tuple[bool, str]: # 验证滑块验证码
|
||||||
|
"""
|
||||||
|
验证滑块验证码
|
||||||
|
Returns:
|
||||||
|
Tuple[bool, str]: (是否成功, 消息)
|
||||||
|
"""
|
||||||
|
data = { # 构建验证数据
|
||||||
|
"offset": SLIDE_OFFSET, # 滑块偏移量
|
||||||
|
"duration": SLIDE_DURATION, # 滑动持续时间
|
||||||
|
"trail": [ # 滑动轨迹
|
||||||
|
{"x": 0, "y": 0, "time": 0}, # 起点
|
||||||
|
{"x": 0, "y": 0, "time": 17}, # 第1个点
|
||||||
|
{"x": 1, "y": 0, "time": 89}, # 第2个点
|
||||||
|
{"x": 5, "y": 0, "time": 106}, # 第3个点
|
||||||
|
{"x": 14, "y": 0, "time": 123}, # 第4个点
|
||||||
|
{"x": 28, "y": 0, "time": 139}, # 第5个点
|
||||||
|
{"x": 47, "y": 0, "time": 156}, # 第6个点
|
||||||
|
{"x": 74, "y": 0, "time": 173}, # 第7个点
|
||||||
|
{"x": 102, "y": 0, "time": 189}, # 第8个点
|
||||||
|
{"x": 131, "y": 0, "time": 206}, # 第9个点
|
||||||
|
{"x": 159, "y": 0, "time": 223}, # 第10个点
|
||||||
|
{"x": 184, "y": 0, "time": 239}, # 第11个点
|
||||||
|
{"x": 205, "y": 0, "time": 256}, # 第12个点
|
||||||
|
{"x": 227, "y": 1, "time": 273}, # 第13个点
|
||||||
|
{"x": 248, "y": 0, "time": 289}, # 第14个点
|
||||||
|
{"x": 263, "y": 0, "time": 306}, # 第15个点
|
||||||
|
{"x": 276, "y": 0, "time": 323}, # 第16个点
|
||||||
|
{"x": 286, "y": 0, "time": 339}, # 第17个点
|
||||||
|
{"x": 290, "y": 0, "time": 356}, # 第18个点
|
||||||
|
{"x": 290, "y": 0, "time": 373}, # 第19个点
|
||||||
|
{"x": 290, "y": 0, "time": 389}, # 第20个点
|
||||||
|
{"x": 290, "y": 0, "time": 406}, # 第21个点
|
||||||
|
{"x": 290, "y": 0, "time": 423}, # 第22个点
|
||||||
|
{"x": 290, "y": 0, "time": 478}, # 终点
|
||||||
|
],
|
||||||
|
"fingerprint": self.fingerprint, # 设备指纹
|
||||||
|
"captchaId": self.captchaId, # 验证码ID
|
||||||
|
"serverPublicKey": self.encryptionPublicKey # 服务器公钥
|
||||||
|
}
|
||||||
|
data_str = json.dumps(data, separators=(",", ":")) # 转换为JSON字符串(压缩格式)
|
||||||
|
current_dir = os.path.dirname(__file__)
|
||||||
|
|
||||||
|
jsFile_path = current_dir+"/sdk_leg_env.js" # JavaScript脚本路径
|
||||||
|
|
||||||
|
if not os.path.exists(jsFile_path): # 检查脚本文件是否存在
|
||||||
|
logger.error(f"Node.js脚本文件不存在: {jsFile_path}") # 记录错误
|
||||||
|
return False, f"缺少签名脚本: {jsFile_path}" # 返回失败
|
||||||
|
|
||||||
|
try: # 异常处理
|
||||||
|
with subprocess.Popen( # 执行Node.js脚本
|
||||||
|
[resolve_node_executable(), jsFile_path], # 命令和参数
|
||||||
|
stdin=subprocess.PIPE, # 标准输入
|
||||||
|
stdout=subprocess.PIPE, # 标准输出
|
||||||
|
stderr=subprocess.PIPE, # 标准错误
|
||||||
|
text=True, # 文本模式
|
||||||
|
encoding="utf-8", # UTF-8编码
|
||||||
|
errors="ignore" # 忽略编码错误
|
||||||
|
) as proc: # 进程上下文
|
||||||
|
stdout, stderr = proc.communicate(data_str, timeout=REQUEST_TIMEOUT) # 传入参数并获取输出
|
||||||
|
verify_data = stdout.strip() # 去除空白字符
|
||||||
|
|
||||||
|
if stderr: # 如果有错误输出
|
||||||
|
logger.error(f'genData_stderr->[{stderr}]') # 记录错误
|
||||||
|
|
||||||
|
if proc.returncode != 0: # 检查退出码
|
||||||
|
return False, f"Node.js脚本执行失败,退出码: {proc.returncode}, 错误信息: {stderr}" # 返回失败
|
||||||
|
|
||||||
|
url = f"{self.base_url}/captcha/verify" # 验证URL
|
||||||
|
self.headers['Content-Type'] = 'application/json' # 设置内容类型
|
||||||
|
response = self.session.post(url, headers=self.headers, data=verify_data, cookies=self.cookie).json() # 发送验证请求
|
||||||
|
if response['code'] == 1: # 验证成功
|
||||||
|
self.token = response['data']['token'] # 保存token
|
||||||
|
self.session_id = response['data']['session_id'] # 保存会话ID
|
||||||
|
return response['code'] == 1, response['msg'] # 返回结果
|
||||||
|
except subprocess.TimeoutExpired: # 超时异常
|
||||||
|
logger.error("Node.js脚本执行超时") # 记录错误
|
||||||
|
return False, "Node.js脚本执行超时" # 返回失败
|
||||||
|
except Exception as e: # 其他异常
|
||||||
|
logger.error(f"执行Node.js脚本异常: {str(e)}") # 记录错误
|
||||||
|
return False, f"执行Node.js脚本异常: {str(e)}" # 返回失败
|
||||||
|
|
||||||
|
if not verify_data: # 检查输出是否为空
|
||||||
|
return False, "Node.js脚本未输出任何内容" # 返回失败
|
||||||
|
|
||||||
|
def user_zh_wxdl_ewm(self) -> Tuple[bool, str, Optional[str]]: # 获取微信登录二维码
|
||||||
|
"""
|
||||||
|
获取微信登录二维码
|
||||||
|
Returns:
|
||||||
|
Tuple[bool, str, Optional[str]]: (是否成功, 消息, 二维码URL)
|
||||||
|
"""
|
||||||
|
url = f"{self.base_url}/user_zh/wxdl_ewm" # 二维码URL
|
||||||
|
try: # 异常处理
|
||||||
|
response = self.session.post(url, headers=self.headers).json() # 发送请求
|
||||||
|
self.token = response['data']["token"] # 保存token
|
||||||
|
return response['code'] == 1, response['msg'], response['data']['url'] if response['code'] == 1 else None # 返回结果
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
return False, f"请求失败: {str(e)}", None # 返回失败
|
||||||
|
|
||||||
|
def user_zh_p_login(self, email: str, password: str, data: Optional[Dict] = None) -> Tuple[bool, str]: # 账号密码登录
|
||||||
|
"""
|
||||||
|
使用账号密码登录
|
||||||
|
Args:
|
||||||
|
email: 邮箱账号
|
||||||
|
password: 明文密码
|
||||||
|
data: 携带的请求数据(用于递归刷新token时传参)
|
||||||
|
Returns:
|
||||||
|
Tuple[bool, str]: (是否成功, 消息)
|
||||||
|
"""
|
||||||
|
try: # 异常处理
|
||||||
|
captcha_success = False # 验证码成功标志
|
||||||
|
captcha_msg = f"滑块验证码失败,{CAPTCHA_MAX_RETRY}次内未成功" # 默认失败消息
|
||||||
|
|
||||||
|
for _ in range(CAPTCHA_MAX_RETRY): # 循环尝试验证滑块
|
||||||
|
if data is None: # 如果没有传入data
|
||||||
|
captcha_success, captcha_msg = self.captcha_init() # 初始化验证码
|
||||||
|
else: # 如果已有data
|
||||||
|
captcha_success = True # 直接视为验证通过
|
||||||
|
|
||||||
|
if captcha_success: # 验证码初始化成功
|
||||||
|
if data is None: # 只有第一次需要验证滑块
|
||||||
|
captcha_success, captcha_msg = self.captcha_verify() # 验证滑块
|
||||||
|
|
||||||
|
if captcha_success: # 滑块验证成功
|
||||||
|
if data is None: # 首次登录
|
||||||
|
data = { # 构建登录数据
|
||||||
|
"re_mm": encrypt_password(self.loginToken, password), # 加密密码
|
||||||
|
"re_yx": email, # 邮箱
|
||||||
|
"fs": "tl", # 登录方式
|
||||||
|
"dltoken": self.loginToken, # 登录令牌
|
||||||
|
"token": json.dumps( # token(压缩格式JSON)
|
||||||
|
{"code": self.token, "sessionId": self.session_id},
|
||||||
|
separators=(",", ":")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
login_url = f"{self.base_url}/user_zh/p_login" # 登录URL
|
||||||
|
response = self.session.post( # 发送登录请求
|
||||||
|
url=login_url,
|
||||||
|
headers=self.headers,
|
||||||
|
json=data,
|
||||||
|
cookies=self.cookie,
|
||||||
|
)
|
||||||
|
logger.info(response.text)
|
||||||
|
result = response.json() # 解析响应
|
||||||
|
code = result.get("code") # 获取状态码
|
||||||
|
msg = result.get("msg", "登录接口未返回消息") # 获取消息
|
||||||
|
|
||||||
|
if code == 1: # 登录成功
|
||||||
|
self.cookie = response.cookies # 保存Cookie
|
||||||
|
return True, msg # 返回成功
|
||||||
|
|
||||||
|
elif code == -118: # token过期
|
||||||
|
self.loginToken = result.get("token", "") # 更新登录令牌
|
||||||
|
data["re_mm"] = encrypt_password(self.loginToken, password) # 重新加密密码
|
||||||
|
data["dltoken"] = self.loginToken # 更新令牌
|
||||||
|
return self.user_zh_p_login(email, password, data) # 递归重新登录
|
||||||
|
|
||||||
|
else: # 其他错误
|
||||||
|
return False, msg # 返回失败
|
||||||
|
|
||||||
|
return captcha_success, captcha_msg # 返回验证码结果
|
||||||
|
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
return False, f"请求失败: {str(e)}" # 返回失败
|
||||||
|
|
||||||
|
def ykj_get_list(self, page: int = 1, page_size: int = 50, data: Optional[Dict] = None) -> Tuple[bool, Any]: # 获取一口价域名列表
|
||||||
|
"""
|
||||||
|
获取一口价域名列表
|
||||||
|
Args:
|
||||||
|
page: 页码
|
||||||
|
page_size: 每页数量
|
||||||
|
data: 额外数据
|
||||||
|
Returns:
|
||||||
|
Tuple[bool, Any]: (是否成功, HTML内容或错误消息)
|
||||||
|
"""
|
||||||
|
data = { # 构建请求数据
|
||||||
|
'psize': page_size, # 每页数量
|
||||||
|
'page': page, # 页码
|
||||||
|
} if data is None else data # 如果有data则使用data
|
||||||
|
url = f"{self.base_url}/ykj/get_list" # 列表URL
|
||||||
|
try: # 异常处理
|
||||||
|
self.headers['Content-Type'] = 'application/x-www-form-urlencoded' # 设置内容类型
|
||||||
|
res = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie) # 发送请求
|
||||||
|
|
||||||
|
response = res.json() # 解析响应
|
||||||
|
if response['code'] == -401: # 需要验证码
|
||||||
|
captcha_response = False, f'滑块验证码失败,{CAPTCHA_MAX_RETRY}次内未成功' # 默认失败
|
||||||
|
for _ in range(CAPTCHA_MAX_RETRY): # 循环尝试验证
|
||||||
|
captcha_response = self.captcha_init() # 初始化验证码
|
||||||
|
if captcha_response[0]: # 初始化成功
|
||||||
|
captcha_response = self.captcha_verify() # 验证验证码
|
||||||
|
if captcha_response[0]: # 验证成功
|
||||||
|
data = {
|
||||||
|
'psize': page_size, # 每页数量
|
||||||
|
'page': page, # 页码
|
||||||
|
# 构建token数据
|
||||||
|
'token': json.dumps({ # token(压缩格式JSON)
|
||||||
|
'code': self.token,
|
||||||
|
'sessionId': self.session_id,
|
||||||
|
}, separators=(",", ":"))
|
||||||
|
}
|
||||||
|
return self.ykj_get_list(page, page_size,data=data) # 递归调用
|
||||||
|
return captcha_response[0], captcha_response[1] # 返回验证码结果
|
||||||
|
|
||||||
|
elif response['code'] == 1 and response.get('data') == 'yzmhuaok': # 验证码通过
|
||||||
|
self.cookie = res.cookies # 更新Cookie
|
||||||
|
return self.ykj_get_list(page, page_size) # 递归调用
|
||||||
|
|
||||||
|
elif response['code'] == 1 and response.get('html'): # 成功获取HTML
|
||||||
|
return response['code'] == 1, response['html'] # 返回HTML
|
||||||
|
|
||||||
|
return False, response['msg'] # 返回失败
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
return False, f"请求失败: {str(e)}" # 返回失败
|
||||||
|
|
||||||
|
def new_cha_del(self, date: str) -> List[str]: # 获取删除域名列表
|
||||||
|
"""
|
||||||
|
获取指定日期的删除域名列表
|
||||||
|
Args:
|
||||||
|
date: 日期(格式:YYYY-MM-DD)
|
||||||
|
Returns:
|
||||||
|
List[str]: 域名列表
|
||||||
|
"""
|
||||||
|
url = f"{self.base_url}/newcha/del_down?scsj={date}" # 下载URL
|
||||||
|
try: # 异常处理
|
||||||
|
response = self.session.get(url, headers=self.headers, cookies=self.cookie, allow_redirects=False) # 发送请求
|
||||||
|
url = response.headers['Location'] # 获取重定向URL
|
||||||
|
response = self.session.get(url, timeout=DOWNLOAD_TIMEOUT) # 下载文件
|
||||||
|
response_text = response.content.decode('utf-8') # 解码内容
|
||||||
|
lines = response_text.strip().splitlines() # 按行分割
|
||||||
|
return lines # 返回域名列表
|
||||||
|
except Exception as e: # 异常处理
|
||||||
|
logger.error(f"<获取删除域名列表错误>: {str(e)}") # 记录错误
|
||||||
|
return [] # 返回空列表
|
||||||
|
|
||||||
|
def save_cookies(self, filepath: str = "juming_cookies.pkl") -> None: # 保存Cookie到文件
|
||||||
|
"""
|
||||||
|
保存Cookie到文件和Redis
|
||||||
|
Args:
|
||||||
|
filepath: 文件路径
|
||||||
|
"""
|
||||||
|
with open(filepath, "wb") as f: # 以二进制写入模式打开文件
|
||||||
|
pickle.dump(self.cookie, f) # 序列化保存
|
||||||
|
|
||||||
|
# 保存到Redis
|
||||||
|
try:
|
||||||
|
import redis
|
||||||
|
from app.config import config
|
||||||
|
redis_client = redis.Redis(
|
||||||
|
host=config.REDIS_HOST,
|
||||||
|
port=config.REDIS_PORT,
|
||||||
|
password=config.REDIS_PASSWORD,
|
||||||
|
db=config.REDIS_DB,
|
||||||
|
decode_responses=True
|
||||||
|
)
|
||||||
|
# 将cookie转换为字典
|
||||||
|
cookie_dict = {}
|
||||||
|
for cookie in self.cookie:
|
||||||
|
cookie_dict[cookie.name] = cookie.value
|
||||||
|
redis_client.set('domain_tool:juming_cookies', str(cookie_dict))
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"保存Cookie到Redis失败: {str(e)}")
|
||||||
|
pass
|
||||||
|
|
||||||
|
def load_cookies(self, filepath: str = "juming_cookies.pkl") -> None: # 从文件加载Cookie
|
||||||
|
"""
|
||||||
|
从文件加载Cookie
|
||||||
|
Args:
|
||||||
|
filepath: 文件路径
|
||||||
|
"""
|
||||||
|
try: # 异常处理
|
||||||
|
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
|
||||||
|
self.cookie = pickle.load(f) # 反序列化加载
|
||||||
|
except: # 异常处理
|
||||||
|
self.cookie = requests.cookies.RequestsCookieJar() # 创建空Cookie
|
||||||
|
|
||||||
|
|
||||||
|
# if __name__ == '__main__': # 主程序入口
|
||||||
|
# m = JM() # 创建JM实例
|
||||||
|
#
|
||||||
|
# # 账号密码登录(请替换为您的账号密码)
|
||||||
|
# email = 'chaofanai1998@gmail.com' # 邮箱账号
|
||||||
|
# password = 'llzz123,./' # 密码
|
||||||
|
#
|
||||||
|
# logger.info(f"开始登录: {email}") # 记录日志
|
||||||
|
# login_result = m.user_zh_p_login(email, password) # 执行登录
|
||||||
|
# logger.info(f"登录结果: {login_result}") # 记录结果
|
||||||
|
#
|
||||||
|
# if not login_result[0]: # 登录失败
|
||||||
|
# logger.error("登录失败,程序退出") # 记录错误
|
||||||
|
# exit(1) # 退出程序
|
||||||
|
#
|
||||||
|
# m.load_cookies() # 加载Cookie
|
||||||
|
#
|
||||||
|
# # 获取一口价域名
|
||||||
|
# import re # 导入正则表达式模块
|
||||||
|
#
|
||||||
|
# for page in range(1, 2): # 循环获取页面
|
||||||
|
# success, html = m.ykj_get_list(page) # 获取列表
|
||||||
|
# if success: # 获取成功
|
||||||
|
# pattern_ym = r"<a class='yda1 ydz' ym='([^']*)'" # 匹配域名
|
||||||
|
# results = re.findall(pattern_ym, html) # 查找所有域名
|
||||||
|
# logger.info(results)
|
||||||
|
# logger.info(f"第{page}页,找到{len(results)}个域名") # 记录结果
|
||||||
|
# if not results: # 如果没有域名
|
||||||
|
# logger.info("没有更多域名,停止获取") # 记录日志
|
||||||
|
# break # 退出循环
|
||||||
|
# time.sleep(5) # 等待5秒
|
||||||
|
#
|
||||||
|
# # 获取删除域名
|
||||||
|
# deleted_domains = m.new_cha_del("2026-03-11") # 获取删除域名
|
||||||
|
# logger.info(f"找到{len(deleted_domains)}个删除域名") # 记录结果
|
||||||
|
#
|
||||||
|
# m.save_cookies() # 保存Cookie
|
||||||
BIN
domainCheck/detect/juming_cookies.pkl
Normal file
BIN
domainCheck/detect/juming_cookies.pkl
Normal file
Binary file not shown.
236
domainCheck/detect/juziseo.py
Normal file
236
domainCheck/detect/juziseo.py
Normal file
@@ -0,0 +1,236 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :juziseo.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/10 15:00
|
||||||
|
@explain : 桔子SEO API封装类 - 提供登录、域名查询等功能
|
||||||
|
'''
|
||||||
|
|
||||||
|
# 导入标准库
|
||||||
|
import os # 操作系统接口
|
||||||
|
import pickle # 序列化反序列化
|
||||||
|
import re
|
||||||
|
import time # 时间处理
|
||||||
|
from Crypto.Cipher import AES
|
||||||
|
from Crypto.Util.Padding import pad
|
||||||
|
import base64
|
||||||
|
# 导入第三方库
|
||||||
|
import requests # HTTP请求库
|
||||||
|
from loguru import logger # 日志记录
|
||||||
|
from requests.cookies import RequestsCookieJar # Cookie管理
|
||||||
|
|
||||||
|
from detect.geetest2 import Geetest2
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# AES-CBC 加密
|
||||||
|
def aes_cbc_encrypt(data, key=b'pvjxzjmzwawfscft', iv=b'qvibva1wg0uxwjeu'):
|
||||||
|
cipher = AES.new(key, AES.MODE_CBC, iv)
|
||||||
|
padded_data = pad(data, AES.block_size) # PKCS7填充
|
||||||
|
encrypted = cipher.encrypt(padded_data)
|
||||||
|
return base64.b64encode(encrypted).decode()
|
||||||
|
|
||||||
|
|
||||||
|
class Juziseo:
|
||||||
|
"""
|
||||||
|
桔子SEO API封装类
|
||||||
|
"""
|
||||||
|
cookie: RequestsCookieJar = {} # 桔子SEO Cookie
|
||||||
|
|
||||||
|
# 通用请求头配置
|
||||||
|
headers = {
|
||||||
|
'accept': 'application/json, text/javascript, */*; q=0.01', # 接受的内容类型
|
||||||
|
'accept-language': 'zh-CN,zh;q=0.9', # 接受的语言
|
||||||
|
'content-type': 'application/x-www-form-urlencoded', # 内容类型
|
||||||
|
'origin': 'https://seo.juziseo.com', # 请求源
|
||||||
|
'priority': 'u=1, i', # 请求优先级
|
||||||
|
'referer': 'https://seo.juziseo.com/login', # 来源页面
|
||||||
|
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"', # 浏览器标识
|
||||||
|
'sec-ch-ua-mobile': '?0', # 是否移动端
|
||||||
|
'sec-ch-ua-platform': '"Windows"', # 操作系统平台
|
||||||
|
'sec-fetch-dest': 'empty', # 请求目标
|
||||||
|
'sec-fetch-mode': 'cors', # 请求模式
|
||||||
|
'sec-fetch-site': 'same-origin', # 请求站点
|
||||||
|
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36',
|
||||||
|
# 用户代理
|
||||||
|
'x-requested-with': 'XMLHttpRequest', # AJAX请求标识
|
||||||
|
}
|
||||||
|
|
||||||
|
def __init__(self, proxies: dict = None):
|
||||||
|
"""
|
||||||
|
初始化桔子SEO类
|
||||||
|
|
||||||
|
:param proxies: 代理配置
|
||||||
|
"""
|
||||||
|
self.validate = None
|
||||||
|
self.challenge = None
|
||||||
|
self.gt = None
|
||||||
|
self.session = requests.Session() # 创建会话对象
|
||||||
|
self.session.proxies = proxies # 设置代理
|
||||||
|
self.session.timeout = 10 # 设置超时时间10秒
|
||||||
|
self.base_url = "https://seo.juziseo.com" # 设置基础URL
|
||||||
|
|
||||||
|
def save_cookies(self, filepath="juziseo_cookies.pkl"):
|
||||||
|
"""
|
||||||
|
保存Cookie到文件和Redis
|
||||||
|
|
||||||
|
:param filepath: 文件路径
|
||||||
|
"""
|
||||||
|
with open(filepath, "wb") as f: # 以二进制写入模式打开文件
|
||||||
|
pickle.dump(self.cookie, f) # 序列化保存Cookie
|
||||||
|
logger.info(f"已保存桔子SEO Cookie到 {filepath}")
|
||||||
|
|
||||||
|
# 保存到Redis
|
||||||
|
try:
|
||||||
|
import redis
|
||||||
|
from app.config import config
|
||||||
|
redis_client = redis.Redis(
|
||||||
|
host=config.REDIS_HOST,
|
||||||
|
port=config.REDIS_PORT,
|
||||||
|
password=config.REDIS_PASSWORD,
|
||||||
|
db=config.REDIS_DB,
|
||||||
|
decode_responses=True
|
||||||
|
)
|
||||||
|
# 将cookie转换为字典
|
||||||
|
cookie_dict = {}
|
||||||
|
for cookie in self.cookie:
|
||||||
|
cookie_dict[cookie.name] = cookie.value
|
||||||
|
redis_client.set('domain_tool:juziseo_cookies', str(cookie_dict))
|
||||||
|
except Exception as e:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def load_cookies(self, filepath="juziseo_cookies.pkl"):
|
||||||
|
"""
|
||||||
|
从文件加载Cookie
|
||||||
|
|
||||||
|
:param filepath: 文件路径
|
||||||
|
"""
|
||||||
|
try: # 尝试加载
|
||||||
|
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
|
||||||
|
self.cookie = pickle.load(f) # 反序列化加载Cookie
|
||||||
|
# logger.info(f"已从 {filepath} 加载桔子SEO Cookie")
|
||||||
|
except: # 加载失败
|
||||||
|
self.cookie = requests.cookies.RequestsCookieJar() # 创建空的CookieJar
|
||||||
|
logger.warning(f"加载桔子SEO Cookie失败,创建空Cookie")
|
||||||
|
|
||||||
|
def start_msg_captcha_servlet(self):
|
||||||
|
url = f"{self.base_url}/class/gtcode/msg/StartMsgCaptchaServlet.php"
|
||||||
|
response = self.session.get(url, headers=self.headers, cookies=self.cookie).json()
|
||||||
|
logger.info(response)
|
||||||
|
self.gt = response['gt']
|
||||||
|
self.challenge = response['challenge']
|
||||||
|
return response['success'] == 1
|
||||||
|
|
||||||
|
def get_captcha(self):
|
||||||
|
GETT2 = Geetest2()
|
||||||
|
code = GETT2.get_tp(gt=self.gt, challenge=self.challenge, type_='auto')
|
||||||
|
logger.info(code)
|
||||||
|
if code['result'] == 'success':
|
||||||
|
self.challenge = code['challenge']
|
||||||
|
self.gt = code['gt']
|
||||||
|
self.validate = code['validate']
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
logger.error(f"获取桔子SEO验证码失败: {code.get('msg', '未知错误')}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def login(self, username, password):
|
||||||
|
"""
|
||||||
|
登录桔子SEO
|
||||||
|
|
||||||
|
:param username: 账号
|
||||||
|
:param password: 密码
|
||||||
|
:return: tuple - (是否成功, 消息)
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
for _ in range(3):
|
||||||
|
if self.start_msg_captcha_servlet():
|
||||||
|
if self.get_captcha():
|
||||||
|
break
|
||||||
|
data = {
|
||||||
|
'return_url': '/',
|
||||||
|
'user_name': aes_cbc_encrypt(bytes(username, 'utf-8')),
|
||||||
|
'password': aes_cbc_encrypt(bytes(password, 'utf-8')),
|
||||||
|
'geetest_challenge': self.challenge,
|
||||||
|
'geetest_validate': self.validate,
|
||||||
|
'geetest_seccode': self.validate + '|jordan',
|
||||||
|
'_post_type': 'ajax',
|
||||||
|
}
|
||||||
|
logger.debug(data)
|
||||||
|
# 发送登录请求
|
||||||
|
url = f"{self.base_url}/account/ajax/login_process/"
|
||||||
|
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie)
|
||||||
|
logger.info(response.text)
|
||||||
|
# 解析响应
|
||||||
|
result = response.json()
|
||||||
|
|
||||||
|
if result.get('errno') == 1:
|
||||||
|
# 登录成功,保存Cookie
|
||||||
|
self.cookie = response.cookies
|
||||||
|
self.save_cookies()
|
||||||
|
logger.info("桔子SEO登录成功")
|
||||||
|
return True, "登录成功"
|
||||||
|
else:
|
||||||
|
logger.error(f"桔子SEO登录失败: {result.get('msg', '未知错误')}")
|
||||||
|
return False, result.get('msg', '登录失败')
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"桔子SEO登录异常: {e}")
|
||||||
|
return False, f"登录失败: {str(e)}"
|
||||||
|
|
||||||
|
def check_history(self, domain: str, sensitive_words: list = None):
|
||||||
|
# 检测域名历史是否存在敏感词
|
||||||
|
|
||||||
|
if sensitive_words is None:
|
||||||
|
sensitive_words = []
|
||||||
|
data = {
|
||||||
|
'qrtypeindex': '1',
|
||||||
|
'domains': domain,
|
||||||
|
'_post_type': 'ajax',
|
||||||
|
}
|
||||||
|
url = f"{self.base_url}/snapshot/save/"
|
||||||
|
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json()
|
||||||
|
# logger.info(response)
|
||||||
|
if response['errno'] == 1:
|
||||||
|
url = response['rsm']['url']
|
||||||
|
response_html = self.session.get(url, headers=self.headers, cookies=self.cookie).content.decode('utf-8')
|
||||||
|
title_sensitive_words_match = re.search(r'标题敏感词\D*(\d+)', response_html, re.S)
|
||||||
|
title_suspected_sensitive_words_match = re.search(r'标题有疑似敏感词\D*(\d+)', response_html, re.S)
|
||||||
|
content_sensitive_words_match = re.search(r'内容敏感词\D*(\d+)', response_html, re.S)
|
||||||
|
baidu_sensitive_words_match = re.search(r'百度历史收录敏感\D*(\d+)', response_html, re.S)
|
||||||
|
if bool(title_sensitive_words_match or title_suspected_sensitive_words_match or content_sensitive_words_match or baidu_sensitive_words_match):
|
||||||
|
return False, "存在敏感词"
|
||||||
|
subdomain_match = re.search(r'子域名:\D*(\d+)', response_html, re.S)
|
||||||
|
if subdomain_match:
|
||||||
|
return False, f"存在子域名: {subdomain_match.group(1)}"
|
||||||
|
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
if sensitive_word in response_html:
|
||||||
|
return False, f"存在敏感词: {sensitive_word}"
|
||||||
|
return True, 'success'
|
||||||
|
return False, str(response.get('err', '请求失败'))
|
||||||
|
|
||||||
|
def check_external_link(self, domain: str, sensitive_words: list = None):
|
||||||
|
# 外链查询域名是否存在敏感词
|
||||||
|
if sensitive_words is None:
|
||||||
|
sensitive_words = []
|
||||||
|
data = {
|
||||||
|
'qrtypeindex': '1',
|
||||||
|
'domains': domain,
|
||||||
|
'_post_type': 'ajax',
|
||||||
|
}
|
||||||
|
url = f"{self.base_url}/domain_rank/save_domain/"
|
||||||
|
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json()
|
||||||
|
logger.info(response)
|
||||||
|
if response['errno'] == 1:
|
||||||
|
url = response['rsm']['url']
|
||||||
|
response_html = self.session.get(url, headers=self.headers, cookies=self.cookie).content.decode('utf-8')
|
||||||
|
subdomain_match = re.search(r'子域名:\D*(\d+)', response_html, re.S)
|
||||||
|
if subdomain_match:
|
||||||
|
return False, f"存在子域名: {subdomain_match.group(1)}"
|
||||||
|
for sensitive_word in sensitive_words:
|
||||||
|
if sensitive_word in response_html:
|
||||||
|
return False, f"存在敏感词: {sensitive_word}"
|
||||||
|
return True, 'success'
|
||||||
|
return False, str(response.get('err', '请求失败'))
|
||||||
3851
domainCheck/detect/module/crack_geetest2x.js
Normal file
3851
domainCheck/detect/module/crack_geetest2x.js
Normal file
File diff suppressed because one or more lines are too long
53
domainCheck/detect/module/gap.py
Normal file
53
domainCheck/detect/module/gap.py
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
import base64,re
|
||||||
|
import time, os,datetime, sched
|
||||||
|
import cv2
|
||||||
|
from io import BytesIO
|
||||||
|
from PIL import Image, ImageDraw, ImageFont
|
||||||
|
import requests,re
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
class quekou():
|
||||||
|
def __init__(self):
|
||||||
|
self.t = int(time.time() * 1000)
|
||||||
|
pass
|
||||||
|
def save_png(self,path,con):
|
||||||
|
with open(path, "wb") as fp:
|
||||||
|
fp.write(con)
|
||||||
|
|
||||||
|
def get_distance2(self,bg,tp):
|
||||||
|
|
||||||
|
bg = base64.b64decode(bg) # base64转二进制
|
||||||
|
|
||||||
|
tp = base64.b64decode(tp) # base64转二进制
|
||||||
|
|
||||||
|
res = self.det.slide_match(tp, bg, simple_target=True)
|
||||||
|
res=res['target'][0]
|
||||||
|
|
||||||
|
return res
|
||||||
|
|
||||||
|
def get_distance(self, bg, tp):
|
||||||
|
|
||||||
|
bg = base64.b64decode(bg) # base64转二进制
|
||||||
|
|
||||||
|
tp = base64.b64decode(tp) # base64转二进制
|
||||||
|
''' bg: 背景图片 tp: 缺口图片 out:输出图片 '''
|
||||||
|
# 读取背景图片和缺口图片
|
||||||
|
bg_img = Image.open(BytesIO(bg)) # 背景图片
|
||||||
|
tp_img = Image.open(BytesIO(tp)) # 缺口图片
|
||||||
|
bg_edge = cv2.Canny(np.array(bg_img), 100, 200)
|
||||||
|
tp_edge = cv2.Canny(np.array(tp_img), 100, 200)
|
||||||
|
# 转换图片格式
|
||||||
|
bg_pic = cv2.cvtColor(bg_edge, cv2.COLOR_GRAY2RGB)
|
||||||
|
tp_pic = cv2.cvtColor(tp_edge, cv2.COLOR_GRAY2RGB)
|
||||||
|
# 缺口匹配
|
||||||
|
res = cv2.matchTemplate(bg_pic, tp_pic, cv2.TM_CCOEFF_NORMED)
|
||||||
|
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res) # 寻找最优匹配
|
||||||
|
# 绘制方框
|
||||||
|
tl = max_loc # 左上角点的坐标
|
||||||
|
return tl[0]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
85
domainCheck/detect/module/use_ua.py
Normal file
85
domainCheck/detect/module/use_ua.py
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
# encoding: utf-8
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
import random
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
msg="提示:返回出现'ip overtime','forbidden'等信息说明本机IP被目标网站限制,请使用/更换代理IP或更换电脑即可。"
|
||||||
|
warning="仅用于学习交流,请勿用于非法用途,违者后果自负!"
|
||||||
|
logger.warning(warning)
|
||||||
|
logger.info(msg)
|
||||||
|
UA = [
|
||||||
|
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Macintosh; U; PPC Mac OS X; pl-PL; rv:1.0.1) Gecko/20021111 Chimera/0.6",
|
||||||
|
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.8 (KHTML, like Gecko, Safari) Cheshire/1.0.UNOFFICIAL",
|
||||||
|
"Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.8.1b2) Gecko/20060821 BonEcho/2.0b2 (Debian-1.99+2.0b2+dfsg-1)",
|
||||||
|
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
|
||||||
|
"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
|
||||||
|
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
|
||||||
|
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36",
|
||||||
|
"Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
|
||||||
|
"Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; The World)",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko",
|
||||||
|
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; AcooBrowser; .NET CLR 1.1.4322; .NET CLR 2.0.50727)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Acoo Browser; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; AOL 9.5; AOLBuild 4337.35; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)",
|
||||||
|
"Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)",
|
||||||
|
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; Media Center PC 6.0)",
|
||||||
|
"Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 1.0.3705; .NET CLR 1.1.4322)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 3.0.04506.30)",
|
||||||
|
"Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN) AppleWebKit/523.15 (KHTML, like Gecko, Safari/419.3) Arora/0.3 (Change: 287 c9dfb30)",
|
||||||
|
"Mozilla/5.0 (X11; U; Linux; en-US) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) Arora/0.6",
|
||||||
|
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2pre) Gecko/20070215 K-Ninja/2.1.1",
|
||||||
|
"Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9) Gecko/20080705 Firefox/3.0 Kapiko/3.0",
|
||||||
|
"Mozilla/5.0 (X11; Linux i686; U;) Gecko/20070322 Kazehakase/0.4.5",
|
||||||
|
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko Fedora/1.9.0.8-1.fc10 Kazehakase/0.5.6",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11",
|
||||||
|
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/535.20 (KHTML, like Gecko) Chrome/19.0.1036.7 Safari/535.20",
|
||||||
|
"Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; fr) Presto/2.9.168 Version/11.52",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 TaoBrowser/2.0 Safari/536.11",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.71 Safari/537.1 LBBROWSER",
|
||||||
|
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; LBBROWSER)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E; LBBROWSER)",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.84 Safari/535.11 LBBROWSER",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
|
||||||
|
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; QQBrowser/7.0.3698.400)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SV1; QQDownload 732; .NET4.0C; .NET4.0E; 360SE)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E)",
|
||||||
|
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
|
||||||
|
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1",
|
||||||
|
"Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; zh-cn) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b13pre) Gecko/20110307 Firefox/4.0b13pre",
|
||||||
|
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11",
|
||||||
|
"Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.16 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0.6",
|
||||||
|
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1468.0 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36",
|
||||||
|
"Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36"
|
||||||
|
]
|
||||||
|
|
||||||
|
def randomUA():
|
||||||
|
return {"User-Agent": random.choice(UA)}
|
||||||
|
|
||||||
|
|
||||||
|
# if __name__ == '__main__':
|
||||||
|
#
|
||||||
|
# print(randomUA())
|
||||||
75
domainCheck/detect/register.py
Normal file
75
domainCheck/detect/register.py
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :register.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/3/27 22:55
|
||||||
|
@explain : 域名注册状态检测工具
|
||||||
|
'''
|
||||||
|
|
||||||
|
from datetime import datetime, timezone, timedelta # 日期时间处理,用于时区转换
|
||||||
|
|
||||||
|
import requests # HTTP请求库
|
||||||
|
from loguru import logger # 日志记录
|
||||||
|
|
||||||
|
|
||||||
|
def check_register(domain: str, postfix: str = 'com',proxies: dict = None):
|
||||||
|
'''
|
||||||
|
检测注册状态
|
||||||
|
:param domain: 待检测域名(不包含后缀)
|
||||||
|
:param postfix: 域名后缀com、net(默认com)
|
||||||
|
:param proxies: 代理(默认None)
|
||||||
|
:return: 注册状态(2:可注册,3:已注册,-1:检测失败,过期时间(格式:YYYY-MM-DD HH:MM:SS)为空)
|
||||||
|
'''
|
||||||
|
url = f'https://rdap.verisign.com/{postfix}/v1/domain/{domain}' # 构建RDAP查询URL
|
||||||
|
# url = f'https://www.baidu.com' # 构建RDAP查询URL
|
||||||
|
|
||||||
|
# 添加重试机制
|
||||||
|
max_retries = 3
|
||||||
|
for retry in range(max_retries):
|
||||||
|
try:
|
||||||
|
response = requests.get(url, timeout=20, proxies=proxies) # 发送GET请求,设置超时5秒
|
||||||
|
if response.status_code == 200: # HTTP 200表示域名已注册
|
||||||
|
json_data = response.json() # 解析JSON响应
|
||||||
|
for item in json_data['events']: # 遍历事件列表
|
||||||
|
if item['eventAction'] == 'expiration': # 如果是过期时间事件
|
||||||
|
utc_time = datetime.fromisoformat(item['eventDate'].replace('Z', '+00:00')) # 解析UTC时间
|
||||||
|
beijing_tz = timezone(timedelta(hours=8)) # 创建北京时区(UTC+8)
|
||||||
|
beijing_time = utc_time.astimezone(beijing_tz) # 将UTC时间转换为北京时间
|
||||||
|
return 3, beijing_time.strftime("%Y-%m-%d %H:%M:%S") # 返回已注册状态和过期时间
|
||||||
|
|
||||||
|
elif response.status_code == 404: # HTTP 404表示域名不存在,可注册
|
||||||
|
# 可注册状态
|
||||||
|
return 2, '' # 返回可注册状态,过期时间为空
|
||||||
|
logger.error(response.status_code)
|
||||||
|
except requests.exceptions.RequestException as e:
|
||||||
|
# logger.warning(f"注册状态检测第{retry+1}次失败: {domain}, 错误: {e}")
|
||||||
|
if retry < max_retries - 1:
|
||||||
|
import time
|
||||||
|
time.sleep(2) # 等待2秒后重试
|
||||||
|
proxies=None
|
||||||
|
else:
|
||||||
|
# 达到最大重试次数,返回检测失败
|
||||||
|
logger.error(f"注册状态检测失败: {domain}, 已达到最大重试次数")
|
||||||
|
return -1, '' # -1表示检测失败,过期时间为空
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# def get_proxy():
|
||||||
|
#
|
||||||
|
# url='http://api.ch12361.com/getProxy.php?group=A&count=1'
|
||||||
|
# response = requests.get(url).json()
|
||||||
|
# username=response['username']
|
||||||
|
# password=response['password']
|
||||||
|
# ip=response['ip']
|
||||||
|
# port=response['port']
|
||||||
|
#
|
||||||
|
# proxy_url = f"http://{username}:{password}@{ip}:{port}"
|
||||||
|
# logger.info(proxy_url)
|
||||||
|
# return {'http':proxy_url,'https':proxy_url}
|
||||||
|
#
|
||||||
|
# t=datetime.now()
|
||||||
|
# logger.info(check_register('75az.com',proxies=get_proxy())) # 测试查询nnsk.com域名的注册状态
|
||||||
|
# logger.info(datetime.now()-t)
|
||||||
15108
domainCheck/detect/sdk_leg.js
Normal file
15108
domainCheck/detect/sdk_leg.js
Normal file
File diff suppressed because one or more lines are too long
897
domainCheck/detect/sdk_leg_env.js
Normal file
897
domainCheck/detect/sdk_leg_env.js
Normal file
@@ -0,0 +1,897 @@
|
|||||||
|
process_ = process;
|
||||||
|
require_ = require;
|
||||||
|
delete Buffer;
|
||||||
|
// delete process;
|
||||||
|
delete require;
|
||||||
|
delete global;
|
||||||
|
delete module;
|
||||||
|
delete exports;
|
||||||
|
delete __filename;
|
||||||
|
delete __dirname;
|
||||||
|
delete SharedArrayBuffer;
|
||||||
|
|
||||||
|
AsObj = {
|
||||||
|
// print: console.log,
|
||||||
|
print: function () { },
|
||||||
|
// print_:console.log,
|
||||||
|
}
|
||||||
|
|
||||||
|
no_print = ['Boolean','String','parseFloat','Array','Object','prepareStackTrace_'];
|
||||||
|
function watch(object, WatchName) {
|
||||||
|
const handler = {
|
||||||
|
get(target, property, receiver) {
|
||||||
|
if (
|
||||||
|
property !== 'isNaN' &&
|
||||||
|
property !== 'encodeURI' &&
|
||||||
|
property !== "Uint8Array" &&
|
||||||
|
property !== 'undefined' &&
|
||||||
|
property !== 'JSON' &&
|
||||||
|
property !== 'Number' &&
|
||||||
|
!no_print.includes(property) &&
|
||||||
|
property !== Symbol.for('nodejs.util.inspect.custom') &&
|
||||||
|
typeof property !== 'symbol'
|
||||||
|
) {
|
||||||
|
|
||||||
|
if (property === 'global') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (property === 'Buffer') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (property === 'process') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (WatchName === 'config_data') {
|
||||||
|
debugger
|
||||||
|
}
|
||||||
|
if (WatchName.indexOf('.prototype') != -1 && target[property] != undefined) {
|
||||||
|
return Reflect.get(target, property, receiver);
|
||||||
|
}
|
||||||
|
|
||||||
|
AsObj.print(
|
||||||
|
"方法:", "get",
|
||||||
|
"对象:", WatchName,
|
||||||
|
"属性:", property,
|
||||||
|
"属性类型:", typeof property,
|
||||||
|
"属性值:", typeof target[property] == 'object' ? "object" : target[property],
|
||||||
|
"属性值类型:", typeof target[property]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (WatchName === 'top') {
|
||||||
|
return window;
|
||||||
|
}
|
||||||
|
|
||||||
|
return Reflect.get(target, property, receiver);
|
||||||
|
},
|
||||||
|
|
||||||
|
set(target, property, value, receiver) {
|
||||||
|
if (WatchName.indexOf('.prototype') != -1 && value != undefined) {
|
||||||
|
return Reflect.set(target, property, value, receiver);
|
||||||
|
}
|
||||||
|
AsObj.print(
|
||||||
|
"方法:", "set",
|
||||||
|
"对象:", WatchName,
|
||||||
|
"属性:", property,
|
||||||
|
"属性类型:", typeof property,
|
||||||
|
"属性值:", typeof value == 'object' ? "object" : value,
|
||||||
|
"属性值类型:", typeof target[property]
|
||||||
|
);
|
||||||
|
return Reflect.set(target, property, value, receiver);
|
||||||
|
},
|
||||||
|
// in操作 检测
|
||||||
|
has(target, property) {
|
||||||
|
AsObj.print(
|
||||||
|
"代理对象:", WatchName,
|
||||||
|
"方法:", "has",
|
||||||
|
"检查属性:", property,
|
||||||
|
"结果:", typeof target[property] == 'object' ? "object" : target[property],
|
||||||
|
);
|
||||||
|
return Reflect.has(target, property);
|
||||||
|
},
|
||||||
|
// Object.key 检测
|
||||||
|
ownKeys(target) {
|
||||||
|
AsObj.print(
|
||||||
|
"方法:", "ownKeys",
|
||||||
|
"对象:", target+''
|
||||||
|
);
|
||||||
|
return Reflect.ownKeys(target);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
return new Proxy(object, handler);
|
||||||
|
}
|
||||||
|
// function watch(object, WatchName) {
|
||||||
|
// return object
|
||||||
|
// }
|
||||||
|
|
||||||
|
// 保护函数,toString检测
|
||||||
|
const safeFunction = function safeFunction(func) {
|
||||||
|
//处理安全函数
|
||||||
|
Function.prototype.$call = Function.prototype.call;
|
||||||
|
const $toString = Function.toString;
|
||||||
|
const myFunction_toString_symbol = Symbol('('.concat('', ')'));
|
||||||
|
|
||||||
|
const myToString = function myToString() {
|
||||||
|
return typeof this === 'function' && this[myFunction_toString_symbol] || $toString.$call(this);
|
||||||
|
}
|
||||||
|
|
||||||
|
const set_native = function set_native(func, key, value) {
|
||||||
|
Object.defineProperty(func, key, {
|
||||||
|
"enumerable": false,
|
||||||
|
"configurable": true,
|
||||||
|
"writable": true,
|
||||||
|
"value": value
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
delete Function.prototype['toString'];
|
||||||
|
set_native(Function.prototype, "toString", myToString);
|
||||||
|
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
|
||||||
|
|
||||||
|
const safe_Function = function safe_Function(func) {
|
||||||
|
set_native(func, myFunction_toString_symbol, "function" + (func.name ? " " + func.name : "") + "() { [native code] }");
|
||||||
|
}
|
||||||
|
|
||||||
|
return safe_Function(func)
|
||||||
|
}
|
||||||
|
|
||||||
|
//创建函数,并代理上
|
||||||
|
const makeFunction = function makeFunction(name) {
|
||||||
|
v_log = AsObj.print;
|
||||||
|
// 使用 Function 保留函数名
|
||||||
|
func = new Function("v_log", `
|
||||||
|
return function ${name}() {
|
||||||
|
v_log('函数${name}传参-->', arguments);
|
||||||
|
};
|
||||||
|
`)(v_log); // 传递 v_log 到动态函数
|
||||||
|
|
||||||
|
safeFunction(func);
|
||||||
|
func = watch(func,`${name}`);
|
||||||
|
func.prototype = watch(func.prototype, `${name}.prototype`);
|
||||||
|
return func;
|
||||||
|
}
|
||||||
|
|
||||||
|
!(function () {
|
||||||
|
"use strict";
|
||||||
|
const $toString = Function.toString;
|
||||||
|
const myFunction_toString_symbol = Symbol('('.concat('', ')_', (Math.random() + '').toString(36)));
|
||||||
|
const mytoString = function () {
|
||||||
|
return typeof this == 'function' && this[myFunction_toString_symbol] || $toString.call(this);
|
||||||
|
};
|
||||||
|
|
||||||
|
function set_native(func, key, value) {
|
||||||
|
Object.defineProperty(func, key, {
|
||||||
|
"enumerable": false,
|
||||||
|
"configurable": true,
|
||||||
|
"writable": true,
|
||||||
|
"value": value
|
||||||
|
})
|
||||||
|
};
|
||||||
|
delete Function.prototype['toString'];
|
||||||
|
set_native(Function.prototype, "toString", mytoString);
|
||||||
|
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
|
||||||
|
this.func_set_native = function (func) {
|
||||||
|
set_native(func, myFunction_toString_symbol, `function ${myFunction_toString_symbol, func.name || ''}() { [native code] }`)
|
||||||
|
}
|
||||||
|
}).call(globalThis);
|
||||||
|
|
||||||
|
// 重写全局对象原型链
|
||||||
|
function setTostringAndstringTag(obj) {
|
||||||
|
Object.defineProperties(obj.prototype, {
|
||||||
|
[Symbol.toStringTag]: {
|
||||||
|
configurable: true,
|
||||||
|
value: obj.name
|
||||||
|
}
|
||||||
|
});
|
||||||
|
safeFunction(obj);
|
||||||
|
};
|
||||||
|
|
||||||
|
// 创建标签原型
|
||||||
|
function createTagProto(propObj,portotypeObj) {
|
||||||
|
let res = propObj + ' = ' + 'function ' + propObj + '() { throw new TypeError("Illegal constructor"); };\n';
|
||||||
|
res += 'setTostringAndstringTag(' + propObj + ',null);\n';
|
||||||
|
if (portotypeObj) {
|
||||||
|
for (let key in portotypeObj) {
|
||||||
|
res += propObj + '.prototype.' + portotypeObj[key] + '= function ' + portotypeObj[key] + '() {AsObj.print("'+propObj+'.prototype.' + portotypeObj[key] + '原型方法(需在实例对象上补该方法)::",arguments)};\n';
|
||||||
|
res += 'globalThis.func_set_native(' + propObj + '.prototype.' + portotypeObj[key] + ');\n';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
eval(res);
|
||||||
|
}
|
||||||
|
|
||||||
|
Object.defineProperties(globalThis, {
|
||||||
|
[Symbol.toStringTag]: {
|
||||||
|
configurable: true,
|
||||||
|
value: 'Window'
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
for (let key in globalThis) {
|
||||||
|
if (typeof globalThis[key] === 'function') {
|
||||||
|
safeFunction(globalThis[key])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (let key in console) {
|
||||||
|
if (typeof console[key] === 'function') {
|
||||||
|
safeFunction(console[key])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
createTagProto('EventTarget',['addEventListener']);
|
||||||
|
createTagProto('WindowProperties');
|
||||||
|
createTagProto('Window');
|
||||||
|
|
||||||
|
window = globalThis;
|
||||||
|
window.__proto__ = Window.prototype;
|
||||||
|
window.__proto__.__proto__ = WindowProperties.prototype;
|
||||||
|
window.__proto__.__proto__.__proto__ = EventTarget.prototype;
|
||||||
|
Window.__proto__ = EventTarget;
|
||||||
|
|
||||||
|
Object.defineProperty(window, 'WindowProperties', {
|
||||||
|
get: function () {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
function randoms(min, max) {
|
||||||
|
return Math.floor(Math.random() * (max - min + 1) + min)
|
||||||
|
}
|
||||||
|
|
||||||
|
function getRandomValues(buf) {
|
||||||
|
var min = 0,
|
||||||
|
max = 255;
|
||||||
|
if (buf instanceof Uint16Array) {
|
||||||
|
max = 65535;
|
||||||
|
} else if (buf instanceof Uint32Array) {
|
||||||
|
max = 4294967295;
|
||||||
|
}
|
||||||
|
for (var element in buf) {
|
||||||
|
buf[element] = randoms(min, max);
|
||||||
|
}
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
self = window.self = window;
|
||||||
|
frames = window.frames = window;
|
||||||
|
top = window.top = window;
|
||||||
|
parent = window.parent = window;
|
||||||
|
global = window.global = window;
|
||||||
|
|
||||||
|
Object.defineProperty(window, "global", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function global(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "top", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function top(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "self", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function self(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "parent", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function parent(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
Object.defineProperty(window, "frames", {
|
||||||
|
configurable:false,
|
||||||
|
enumerable: true,
|
||||||
|
set: undefined,
|
||||||
|
get: function frames(){
|
||||||
|
return window
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
innerWidth = 1536
|
||||||
|
innerHeight = 715
|
||||||
|
outerWidth = 1536
|
||||||
|
outerHeight = 824
|
||||||
|
devicePixelRatio = 1.25;
|
||||||
|
screenLeft = 0;
|
||||||
|
screenX = 0;
|
||||||
|
screenTop = 0;
|
||||||
|
screenY = 0;
|
||||||
|
opener = null;
|
||||||
|
isSecureContext = true;
|
||||||
|
crypto = {
|
||||||
|
getRandomValues:getRandomValues
|
||||||
|
};
|
||||||
|
|
||||||
|
createTagProto('DOMStringMap')
|
||||||
|
createTagProto('HTMLHeadElement',['insertBefore','removeChild'])
|
||||||
|
createTagProto('HTMLBodyElement',['addEventListener','appendChild','removeChild'])
|
||||||
|
createTagProto('HTMLHtmlElement',['getAttribute'])
|
||||||
|
createTagProto('HTMLDocument')
|
||||||
|
createTagProto('Document',['browsingTopics','appendChild','querySelector','evaluate','querySelectorAll','removeChild','requestStorageAccess','requestStorageAccessFor','hasStorageAccess','getElementsByTagName','hasPrivateToken','createElement','hasRedemptionRecord','hasFocus'])
|
||||||
|
createTagProto('Node')
|
||||||
|
document = {};
|
||||||
|
document.__proto__ = HTMLDocument.prototype;
|
||||||
|
document.__proto__.__proto__ = Document.prototype;
|
||||||
|
document.__proto__.__proto__.__proto__ = Node.prototype;
|
||||||
|
document.__proto__.__proto__.__proto__.__proto__ = EventTarget.prototype;
|
||||||
|
HTMLDocument.__proto__ = Document;
|
||||||
|
HTMLDocument.__proto__.__proto__ = Node;
|
||||||
|
HTMLDocument.__proto__.__proto__.__proto__ = EventTarget;
|
||||||
|
Document.__proto__ = Node;
|
||||||
|
Document.__proto__.__proto__ = EventTarget;
|
||||||
|
Node.__proto__ = EventTarget;
|
||||||
|
|
||||||
|
createTagProto('Plugin');
|
||||||
|
createTagProto('PluginArray');
|
||||||
|
plugins0 = {
|
||||||
|
name: 'PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type: 'application/pdf',
|
||||||
|
},
|
||||||
|
'1':{
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins0['0'].enabledPlugin = plugins0;
|
||||||
|
plugins0['1'].enabledPlugin = plugins0;
|
||||||
|
plugins1 = {
|
||||||
|
name: 'Chrome PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1': {
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins1['0'].enabledPlugin = plugins1;
|
||||||
|
plugins1['1'].enabledPlugin = plugins1;
|
||||||
|
plugins2 = {
|
||||||
|
name: 'Chromium PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1': {
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins2['0'].enabledPlugin = plugins2;
|
||||||
|
plugins2['1'].enabledPlugin = plugins2;
|
||||||
|
plugins3 = {
|
||||||
|
name: 'Microsoft Edge PDF Viewer',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0':{
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1': {
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins3['0'].enabledPlugin = plugins3;
|
||||||
|
plugins3['1'].enabledPlugin = plugins3;
|
||||||
|
plugins4 = {
|
||||||
|
name: 'WebKit built-in PDF',
|
||||||
|
filename: 'internal-pdf-viewer',
|
||||||
|
description:'Portable Document Format',
|
||||||
|
length: 2,
|
||||||
|
'0': {
|
||||||
|
type:'application/pdf'
|
||||||
|
},
|
||||||
|
'1':{
|
||||||
|
type:'text/pdf'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
plugins4['0'].enabledPlugin = plugins4;
|
||||||
|
plugins4['1'].enabledPlugin = plugins4;
|
||||||
|
plugins = {
|
||||||
|
length: 5,
|
||||||
|
'0': plugins0,
|
||||||
|
'1': plugins1,
|
||||||
|
'2': plugins2,
|
||||||
|
'3': plugins3,
|
||||||
|
'4': plugins4,
|
||||||
|
namedItem : function (name) {
|
||||||
|
AsObj.print('Plugin-namedItem:', name)
|
||||||
|
},
|
||||||
|
item: function (index) {
|
||||||
|
AsObj.print('Plugin-item:', index)
|
||||||
|
return watch(plugins0,'item-'+index);
|
||||||
|
},
|
||||||
|
refresh: function () {
|
||||||
|
AsObj.print('Plugin-refresh:',arguments)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
plugins.__proto__ = PluginArray.prototype;
|
||||||
|
|
||||||
|
MimeTypeArray = function MimeTypeArray() {
|
||||||
|
this.length = 2;
|
||||||
|
this['0'] = {
|
||||||
|
suffixes: 'pdf',
|
||||||
|
type: 'application/pdf',
|
||||||
|
description:"Portable Document Format",
|
||||||
|
enabledPlugin: plugins0
|
||||||
|
};
|
||||||
|
this['1'] = {
|
||||||
|
suffixes: 'pdf',
|
||||||
|
type: 'text/pdf',
|
||||||
|
description:"Portable Document Format",
|
||||||
|
enabledPlugin: plugins0
|
||||||
|
};
|
||||||
|
};
|
||||||
|
MimeTypeArray.prototype.toString = function () { return '[object MimeTypeArray]'; }
|
||||||
|
MimeTypeArray.toString = function () { return 'function MimeTypeArray() { [native code] }'; }
|
||||||
|
Object.defineProperties(MimeTypeArray.prototype, { [Symbol.toStringTag]: { value: 'MimeTypeArray' } })
|
||||||
|
MimeTypeArrayc = new MimeTypeArray();
|
||||||
|
MimeTypeArrayc[Symbol.iterator] = function* () {
|
||||||
|
for (let key in this) {
|
||||||
|
yield this[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 创建电池管理器对象原型
|
||||||
|
const BatteryManager = {
|
||||||
|
level: 1,
|
||||||
|
charging: true,
|
||||||
|
chargingTime: 0,
|
||||||
|
dischargingTime: null,
|
||||||
|
onchargingchange: null,
|
||||||
|
onlevelchange: null,
|
||||||
|
toString: function toString() {
|
||||||
|
return `BatteryManager {
|
||||||
|
charging: ${this.charging},
|
||||||
|
level: ${this.level},
|
||||||
|
chargingTime: ${this.chargingTime},
|
||||||
|
dischargingTime: ${this.dischargingTime}
|
||||||
|
}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
window.BatteryManager = BatteryManager;
|
||||||
|
|
||||||
|
Promise2 = {
|
||||||
|
then: function () {
|
||||||
|
return this;
|
||||||
|
},
|
||||||
|
catch: function (){},
|
||||||
|
};
|
||||||
|
|
||||||
|
createTagProto('Bluetooth');
|
||||||
|
createTagProto('Navigator');
|
||||||
|
Navigator.prototype.hardwareConcurrency = 8;
|
||||||
|
Navigator.prototype.userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36';
|
||||||
|
Navigator.prototype.appVersion = '5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36'
|
||||||
|
Navigator.prototype.appName = 'Netscape';
|
||||||
|
Navigator.prototype.appCodeName = 'Mozilla';
|
||||||
|
Navigator.prototype.vendor = 'Google Inc.';
|
||||||
|
Navigator.prototype.maxTouchPoints = 10;
|
||||||
|
Navigator.prototype.platform = 'Win32';
|
||||||
|
Navigator.prototype.adAuctionComponents = function adAuctionComponents() {
|
||||||
|
AsObj.print('adAuctionComponents:::', arguments)
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.adAuctionComponents)
|
||||||
|
Navigator.prototype.runAdAuction = function runAdAuction() {
|
||||||
|
AsObj.print('runAdAuction:::', arguments)
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.runAdAuction)
|
||||||
|
Navigator.prototype.canLoadAdAuctionFencedFrame = makeFunction('canLoadAdAuctionFencedFrame')
|
||||||
|
Navigator.prototype.deprecatedReplaceInURN = makeFunction('deprecatedReplaceInURN')
|
||||||
|
Navigator.prototype.deprecatedURNToURL = makeFunction('deprecatedURNToURL')
|
||||||
|
Navigator.prototype.joinAdInterestGroup = makeFunction('joinAdInterestGroup')
|
||||||
|
Navigator.prototype.leaveAdInterestGroup = makeFunction('leaveAdInterestGroup')
|
||||||
|
Navigator.prototype.updateAdInterestGroups = makeFunction('updateAdInterestGroups')
|
||||||
|
Navigator.prototype.connection = watch({
|
||||||
|
downlink: 9.1,
|
||||||
|
effectiveType: '4g',
|
||||||
|
rtt: 0,
|
||||||
|
saveData: false,
|
||||||
|
},'connection')
|
||||||
|
Navigator.prototype.language = 'zh-CN';
|
||||||
|
Navigator.prototype.languages = ["zh-CN"];
|
||||||
|
Navigator.prototype.plugins = plugins;
|
||||||
|
Navigator.prototype.webdriver = false;
|
||||||
|
Navigator.prototype.cookieEnabled = true;
|
||||||
|
Navigator.prototype.onLine = true;
|
||||||
|
Navigator.prototype.doNotTrack = null;
|
||||||
|
Navigator.prototype.bluetooth = {};
|
||||||
|
Navigator.prototype.product = 'Gecko'
|
||||||
|
Navigator.prototype.deviceMemory = 8
|
||||||
|
Navigator.prototype.mediaDevices = watch({
|
||||||
|
enumerateDevices: function enumerateDevices() {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = [
|
||||||
|
{deviceId: '', kind: 'audioinput', label: '', groupId: ''},
|
||||||
|
{deviceId: '', kind: 'videoinput', label: '', groupId: ''},
|
||||||
|
{deviceId: '', kind: 'audiooutput', label: '', groupId: ''},
|
||||||
|
]
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
getUserMedia: function getUserMedia() {
|
||||||
|
AsObj.print('getUserMedia:::', arguments)
|
||||||
|
}
|
||||||
|
},'mediaDevices')
|
||||||
|
Navigator.prototype.storage = {
|
||||||
|
estimate: function estimate() {
|
||||||
|
AsObj.print('estimate:::', arguments)
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
usage: 0, // 1GB
|
||||||
|
quota: 2147483648, // 1GB,
|
||||||
|
usageDetails: {caches: 512, indexedDB: 2855}
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Navigator.prototype.webkitPersistentStorage = watch({},'webkitPersistentStorage')
|
||||||
|
Navigator.prototype.webkitTemporaryStorage = watch({
|
||||||
|
queryUsageAndQuota: function queryUsageAndQuota() {
|
||||||
|
AsObj.print('queryUsageAndQuota:::', arguments)
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
usage: 1024 * 1024 * 1024, // 1GB
|
||||||
|
quota: 1024 * 1024 * 1024, // 1GB
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
},'webkitTemporaryStorage')
|
||||||
|
Navigator.prototype.bluetooth.__proto__ = Bluetooth.prototype;
|
||||||
|
Navigator.prototype.javaEnabled = function javaEnabled() {
|
||||||
|
return false
|
||||||
|
};
|
||||||
|
safeFunction(Navigator.prototype.javaEnabled)
|
||||||
|
Navigator.prototype.getBattery = function getBattery() {
|
||||||
|
AsObj.print('getBattery:::', arguments)
|
||||||
|
return Promise.resolve({
|
||||||
|
__proto__: BatteryManager,
|
||||||
|
// 动态参数配置(示例值)
|
||||||
|
level: 1,
|
||||||
|
charging: true,
|
||||||
|
dischargingTime: null // 2小时放电时间
|
||||||
|
})
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.getBattery)
|
||||||
|
Navigator.prototype.registerProtocolHandler = function registerProtocolHandler() {
|
||||||
|
AsObj.print('registerProtocolHandler:::',arguments)
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.registerProtocolHandler)
|
||||||
|
Navigator.prototype.mimeTypes = watch(MimeTypeArrayc,'mimeTypes');
|
||||||
|
Navigator.prototype.geolocation = {
|
||||||
|
getCurrentPosition: function getCurrentPosition() {
|
||||||
|
return Promise2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Navigator.prototype.pdfViewerEnabled = true;
|
||||||
|
Navigator.prototype.doNotTrack = null;
|
||||||
|
Navigator.prototype.keyboard = watch({
|
||||||
|
getLayoutMap: function getLayoutMap() {
|
||||||
|
AsObj.print('Navigator.prototype.keyboard:', arguments)
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
// arguments[0](watch({
|
||||||
|
// size: 48,
|
||||||
|
// values: function () {
|
||||||
|
// return ['k', 'g', '2', '0', 'v', 'a', '`', 'l', '\\', "'", 'w', '8', 'm', 'h', '.', '7', '1', 'p', 'd', 'f', 'o', 'q', 'c', 'n', '[', 'z', 'y', '3', '6', '5', 'x', '/', '\\', ',', '-', '4', 'b', 't', '9', 's', 'i', 'u', '=', 'j', ';', 'r', ']', 'e']
|
||||||
|
// }
|
||||||
|
// }, 'navigator.keyboard.getLayoutMap.then'))
|
||||||
|
return {
|
||||||
|
catch: function () {
|
||||||
|
arguments[0]({
|
||||||
|
message:'getLayoutMap() must be called from a top-level browsing context or allowed by the permission policy.'
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},'navigator.keyboard')
|
||||||
|
Navigator.prototype.permissions = watch({
|
||||||
|
query: function query() {
|
||||||
|
arg_obj = arguments[0];
|
||||||
|
if (arg_obj.name === 'audio_capture') {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
state: 'prompt',
|
||||||
|
onchange: null,
|
||||||
|
name:arg_obj.name
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (arg_obj.name === 'microphone') {
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0](watch({
|
||||||
|
state: 'denied',
|
||||||
|
onchange: null,
|
||||||
|
name: 'audio_capture'
|
||||||
|
},'permissions.query.microphone'));
|
||||||
|
return {catch:function(){}}
|
||||||
|
},
|
||||||
|
catch:function(){}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (arg_obj.name === 'camera') {
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0](watch({
|
||||||
|
state: 'prompt',
|
||||||
|
onchange: null,
|
||||||
|
name: 'video_capture'
|
||||||
|
},'permissions.query.camera'));
|
||||||
|
return {catch:function(){}}
|
||||||
|
},
|
||||||
|
catch:function(){}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
AsObj.print('permissions.query:::', arguments)
|
||||||
|
|
||||||
|
}
|
||||||
|
},'permissions')
|
||||||
|
Navigator.prototype.productSub = '20030107'
|
||||||
|
Navigator.prototype.getGamepads = function getGamepads() {
|
||||||
|
AsObj.print('getGamepads:::', arguments)
|
||||||
|
return [null,null,null,null]
|
||||||
|
}
|
||||||
|
safeFunction(Navigator.prototype.getGamepads)
|
||||||
|
|
||||||
|
Navigator.prototype.sendBeacon = makeFunction('sendBeacon')
|
||||||
|
|
||||||
|
Navigator.prototype.deprecatedRunAdAuctionEnforcesKAnonymity = false
|
||||||
|
Navigator.prototype.gpu = watch({
|
||||||
|
getPreferredCanvasFormat: function getPreferredCanvasFormat() {
|
||||||
|
AsObj.print('gpu.getPreferredCanvasFormat:', arguments)
|
||||||
|
return 'bgra8unorm'
|
||||||
|
},
|
||||||
|
wgslLanguageFeatures: watch({
|
||||||
|
size: 7,
|
||||||
|
values: function values() {
|
||||||
|
debugger
|
||||||
|
AsObj.print('wgslLanguageFeatures.values')
|
||||||
|
return ['packed_4x8_integer_dot_product', 'unrestricted_pointer_parameters', 'subgroup_uniformity', 'subgroup_id', 'pointer_composite_access', 'readonly_and_readwrite_storage_textures', 'uniform_buffer_standard_layout']
|
||||||
|
},
|
||||||
|
}, 'gpu.wgslLanguageFeatures'),
|
||||||
|
requestAdapter: function requestAdapter() {
|
||||||
|
AsObj.print('gpu.requestAdapter:', arguments)
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0](watch({
|
||||||
|
features: watch({
|
||||||
|
size: 19,
|
||||||
|
values: function () {
|
||||||
|
return ['depth32float-stencil8', 'rg11b10ufloat-renderable', 'bgra8unorm-storage', 'texture-formats-tier1', 'texture-compression-bc', 'dual-source-blending', 'core-features-and-limits', 'float32-filterable', 'indirect-first-instance', 'float32-blendable', 'depth-clip-control', 'texture-compression-bc-sliced-3d', 'timestamp-query', 'texture-formats-tier2', 'clip-distances', 'shader-f16', 'primitive-index', 'texture-component-swizzle', 'subgroups']
|
||||||
|
}
|
||||||
|
}, 'gpu.requestAdapter.features'),
|
||||||
|
info: watch({ vendor: 'intel', architecture: 'gen-11', device: '', description: '', subgroupMinSize: 16 }, 'gpu.requestAdapter.info'),
|
||||||
|
limits: watch({
|
||||||
|
maxBufferSize: 2147483648,
|
||||||
|
maxStorageBufferBindingSize:2147483644
|
||||||
|
}, 'gpu.requestAdapter.limits'),
|
||||||
|
catch:function(){}
|
||||||
|
}, 'gpu.requestAdapter'));
|
||||||
|
return {
|
||||||
|
catch: function () {
|
||||||
|
return {
|
||||||
|
then: function () {
|
||||||
|
arguments[0]()
|
||||||
|
return {catch:function(){}}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
},
|
||||||
|
catch: function () {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},'navigator.gpu')
|
||||||
|
Navigator.prototype.userAgentData = watch({
|
||||||
|
brands:[
|
||||||
|
{
|
||||||
|
"brand": "Google Chrome",
|
||||||
|
"version": "143"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Chromium",
|
||||||
|
"version": "143"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Not A(Brand",
|
||||||
|
"version": "24"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
mobile: false,
|
||||||
|
platform: "Windows",
|
||||||
|
getHighEntropyValues: function getHighEntropyValues() {
|
||||||
|
if (arguments[0] + '' === 'architecture,bitness,model,platformVersion,uaFullVersion,wow64') {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const offer = {
|
||||||
|
"architecture": "x86",
|
||||||
|
"bitness": "64",
|
||||||
|
"brands": [
|
||||||
|
{
|
||||||
|
"brand": "Not:A-Brand",
|
||||||
|
"version": "99"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Google Chrome",
|
||||||
|
"version": "145"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"brand": "Chromium",
|
||||||
|
"version": "145"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"mobile": false,
|
||||||
|
"model": "",
|
||||||
|
"platform": "Windows",
|
||||||
|
"platformVersion": "10.0.0",
|
||||||
|
"uaFullVersion": "145.0.7632.117",
|
||||||
|
"wow64": false
|
||||||
|
};
|
||||||
|
resolve(offer);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
AsObj.print('getHighEntropyValues:::', arguments)
|
||||||
|
}
|
||||||
|
},'userAgentData')
|
||||||
|
|
||||||
|
navigator = {};
|
||||||
|
navigator.__proto__ = Navigator.prototype;
|
||||||
|
|
||||||
|
createTagProto('Location');
|
||||||
|
location = {
|
||||||
|
"ancestorOrigins": {},
|
||||||
|
"href": "https://www.neimanmarcus.com/",
|
||||||
|
"origin": "https://www.neimanmarcus.com",
|
||||||
|
"protocol": "https:",
|
||||||
|
"host": "www.neimanmarcus.com",
|
||||||
|
"hostname": "www.neimanmarcus.com",
|
||||||
|
"port": "",
|
||||||
|
"pathname": "/",
|
||||||
|
"search": "",
|
||||||
|
"hash": ""
|
||||||
|
};
|
||||||
|
|
||||||
|
location.__proto__ = Location.prototype;
|
||||||
|
location.toString = function toString() {
|
||||||
|
return this.href;
|
||||||
|
}
|
||||||
|
|
||||||
|
createTagProto('Screen');
|
||||||
|
Screen.prototype = Object.assign(Screen.prototype, {
|
||||||
|
availHeight: 824,
|
||||||
|
availLeft: 0,
|
||||||
|
availTop: 0,
|
||||||
|
availWidth: 1536,
|
||||||
|
colorDepth: 32,
|
||||||
|
height: 864,
|
||||||
|
isExtended: true,
|
||||||
|
onchange: null,
|
||||||
|
pixelDepth: 24,
|
||||||
|
width: 1536,
|
||||||
|
orientation: {
|
||||||
|
angle: 0,
|
||||||
|
type: "landscape-primary",
|
||||||
|
onchange: null
|
||||||
|
}
|
||||||
|
})
|
||||||
|
screen = {};
|
||||||
|
screen.__proto__ = Screen.prototype;
|
||||||
|
|
||||||
|
createTagProto('History',['replaceState']);
|
||||||
|
history = {};
|
||||||
|
history.__proto__ = History.prototype;
|
||||||
|
|
||||||
|
chrome = {
|
||||||
|
loadTimes: function loadTimes() { },
|
||||||
|
csi: function csi() { },
|
||||||
|
app: {
|
||||||
|
InstallState: { DISABLED: 'disabled', INSTALLED: 'installed', NOT_INSTALLED: 'not_installed' },
|
||||||
|
RunningState: { CANNOT_RUN: 'cannot_run', READY_TO_RUN: 'ready_to_run', RUNNING: 'running' },
|
||||||
|
getDetails:function getDetails(){},
|
||||||
|
getIsInstalled:function getIsInstalled(){},
|
||||||
|
installState:function installState(){},
|
||||||
|
isInstalled: false,
|
||||||
|
runningState: function runningState(){}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
createTagProto('Storage');
|
||||||
|
local = {
|
||||||
|
};
|
||||||
|
localStorage = {
|
||||||
|
getItem: function getItem(key) {
|
||||||
|
AsObj.print("localStorage.getItem::", arguments);
|
||||||
|
if (!local[key]) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return local[key];
|
||||||
|
},
|
||||||
|
setItem: function setItem(key, value) {
|
||||||
|
AsObj.print("localStorage.setItem::", arguments);
|
||||||
|
local[key] = value;
|
||||||
|
},
|
||||||
|
clear: function clear() {
|
||||||
|
local = {};
|
||||||
|
},
|
||||||
|
removeItem: function removeItem(key) {
|
||||||
|
AsObj.print("localStorage.removeItem::", arguments);
|
||||||
|
delete local[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
localStorage.__proto__ = Storage.prototype;
|
||||||
|
sessionStorage = {
|
||||||
|
getItem: function getItem(key) {
|
||||||
|
AsObj.print("sessionStorage.getItem::", arguments);
|
||||||
|
if (!local[key]) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return local[key];
|
||||||
|
},
|
||||||
|
setItem: function setItem(key, value) {
|
||||||
|
AsObj.print("sessionStorage.setItem::", arguments);
|
||||||
|
local[key] = value;
|
||||||
|
},
|
||||||
|
clear: function clear() {
|
||||||
|
local = {};
|
||||||
|
},
|
||||||
|
removeItem: function removeItem(key) {
|
||||||
|
AsObj.print("sessionStorage.removeItem::", arguments);
|
||||||
|
delete local[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sessionStorage.__proto__ = Storage.prototype;
|
||||||
|
|
||||||
|
// window = watch(window, 'window');
|
||||||
|
// global = watch(global, 'global');
|
||||||
|
// globalThis = watch(globalThis, 'globalThis');
|
||||||
|
// self = watch(self, 'self');
|
||||||
|
// crypto = watch(crypto, 'crypto');
|
||||||
|
// performance = watch(performance, 'performance');
|
||||||
|
// document = watch(document, 'document');
|
||||||
|
// navigator = watch(navigator, 'navigator');
|
||||||
|
// location = watch(location, 'location');
|
||||||
|
// screen = watch(screen, 'screen');
|
||||||
|
// history = watch(history, 'history');
|
||||||
|
// localStorage = watch(localStorage, 'localStorage');
|
||||||
|
// sessionStorage = watch(sessionStorage, 'sessionStorage');
|
||||||
|
// chrome = watch(chrome, 'chrome');
|
||||||
|
|
||||||
|
require_('./sdk_leg.js');
|
||||||
|
|
||||||
|
let input = '';
|
||||||
|
// 收集数据
|
||||||
|
process.stdin.on('data', chunk => {
|
||||||
|
input += chunk;
|
||||||
|
});
|
||||||
|
process.stdin.on('end', async () => {
|
||||||
|
var config_data = JSON.parse(input);
|
||||||
|
var cryptoManager = await CaptchaSDKCorecc();
|
||||||
|
var encryptData = await buildEncryptedVerifyRequestcc(config_data, cryptoManager);
|
||||||
|
console.log(JSON.stringify(encryptData));
|
||||||
|
process.exit(0);
|
||||||
|
})
|
||||||
20
domainCheck/detect_options.json
Normal file
20
domainCheck/detect_options.json
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
{
|
||||||
|
"detect_juziseo": false,
|
||||||
|
"detect_360_site": true,
|
||||||
|
"detect_aizhan": true,
|
||||||
|
"detect_baidu_site": true,
|
||||||
|
"detect_order": [
|
||||||
|
"detect_register",
|
||||||
|
"detect_baidu_site",
|
||||||
|
"detect_360_site",
|
||||||
|
"detect_chinaz",
|
||||||
|
"detect_aizhan",
|
||||||
|
"detect_wayback",
|
||||||
|
"detect_jucha",
|
||||||
|
"detect_juziseo"
|
||||||
|
],
|
||||||
|
"detect_register": true,
|
||||||
|
"detect_wayback": true,
|
||||||
|
"detect_chinaz": true,
|
||||||
|
"detect_jucha": false
|
||||||
|
}
|
||||||
2181
domainCheck/detect_worker.py
Normal file
2181
domainCheck/detect_worker.py
Normal file
File diff suppressed because it is too large
Load Diff
3
domainCheck/domain_suffixes.json
Normal file
3
domainCheck/domain_suffixes.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"suffixes": ".com,.net"
|
||||||
|
}
|
||||||
194702
domainCheck/domains.txt
Normal file
194702
domainCheck/domains.txt
Normal file
File diff suppressed because it is too large
Load Diff
BIN
domainCheck/favicon.ico
Normal file
BIN
domainCheck/favicon.ico
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4.2 KiB |
BIN
domainCheck/favicon2.ico
Normal file
BIN
domainCheck/favicon2.ico
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4.2 KiB |
270
domainCheck/init_database.py
Normal file
270
domainCheck/init_database.py
Normal file
@@ -0,0 +1,270 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :init_database.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/9 21:20
|
||||||
|
@explain : 数据库初始化脚本
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from loguru import logger
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
|
||||||
|
def init_database():
|
||||||
|
"""
|
||||||
|
初始化数据库表结构
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
logger.info("数据库连接成功")
|
||||||
|
|
||||||
|
# 创建domains表
|
||||||
|
create_domains_table = """
|
||||||
|
CREATE TABLE IF NOT EXISTS domains (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
domain VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
tld VARCHAR(50) NOT NULL,
|
||||||
|
source_type INTEGER NOT NULL,
|
||||||
|
use_status INTEGER DEFAULT 0,
|
||||||
|
detect_status INTEGER DEFAULT 0,
|
||||||
|
register_status INTEGER DEFAULT 0,
|
||||||
|
has_beian BOOLEAN DEFAULT FALSE,
|
||||||
|
company_type VARCHAR(100),
|
||||||
|
website_url VARCHAR(255),
|
||||||
|
beian_year INTEGER,
|
||||||
|
snapshot_years TEXT,
|
||||||
|
expire_date TIMESTAMP,
|
||||||
|
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_domains_table)
|
||||||
|
logger.info("创建domains表成功")
|
||||||
|
|
||||||
|
# 添加新字段
|
||||||
|
try:
|
||||||
|
# 添加review_status字段
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS review_status INTEGER DEFAULT 0")
|
||||||
|
# 添加detect_time字段
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS detect_time TIMESTAMP")
|
||||||
|
# 添加backlink_count字段
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0")
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS jucha_status INTEGER DEFAULT 0")
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS juziseo_status INTEGER DEFAULT 0")
|
||||||
|
# 尝试修改has_beian字段类型
|
||||||
|
try:
|
||||||
|
# 先检查字段类型
|
||||||
|
cur.execute("SELECT data_type FROM information_schema.columns WHERE table_name = 'domains' AND column_name = 'has_beian'")
|
||||||
|
result = cur.fetchone()
|
||||||
|
if result and result[0] != 'integer':
|
||||||
|
# 先删除默认值
|
||||||
|
cur.execute("ALTER TABLE domains ALTER COLUMN has_beian DROP DEFAULT")
|
||||||
|
# 修改字段类型
|
||||||
|
cur.execute("ALTER TABLE domains ALTER COLUMN has_beian TYPE INTEGER USING CASE WHEN has_beian THEN 2 ELSE 3 END")
|
||||||
|
# 设置新的默认值
|
||||||
|
cur.execute("ALTER TABLE domains ALTER COLUMN has_beian SET DEFAULT 1")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"修改has_beian字段类型失败: {e}")
|
||||||
|
logger.info("添加新字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"添加新字段失败: {e}")
|
||||||
|
# 回滚当前事务
|
||||||
|
conn.rollback()
|
||||||
|
# 只添加新字段,不修改has_beian字段类型
|
||||||
|
try:
|
||||||
|
# 添加review_status字段
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS review_status INTEGER DEFAULT 0")
|
||||||
|
# 添加detect_time字段
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS detect_time TIMESTAMP")
|
||||||
|
# 添加backlink_count字段
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0")
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS jucha_status INTEGER DEFAULT 0")
|
||||||
|
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS juziseo_status INTEGER DEFAULT 0")
|
||||||
|
logger.info("添加部分新字段成功")
|
||||||
|
except Exception as e2:
|
||||||
|
logger.warning(f"添加新字段失败: {e2}")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.id IS '主键ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.domain IS '域名'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.tld IS '顶级域名'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.source_type IS '来源类型'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.use_status IS '使用状态'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.detect_status IS '检测状态'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.register_status IS '注册状态'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.review_status IS '人工复核状态'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.has_beian IS '是否有备案历史'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.company_type IS '单位性质'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.website_url IS '网站首页网址'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.beian_year IS '备案年份'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.snapshot_years IS '快照年份'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.expire_date IS '过期时间'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.detect_time IS '检测时间'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.jucha_status IS '聚查检测状态:0未检测,1已检测'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.juziseo_status IS '桔子检测状态:0未检测,1已检测'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.create_time IS '创建时间'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domains.update_time IS '更新时间'")
|
||||||
|
logger.info("添加domains表字段注释成功")
|
||||||
|
|
||||||
|
# 创建detect_tasks表
|
||||||
|
create_tasks_table = """
|
||||||
|
CREATE TABLE IF NOT EXISTS detect_tasks (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
domain_id INTEGER REFERENCES domains(id),
|
||||||
|
task_type INTEGER NOT NULL,
|
||||||
|
status INTEGER DEFAULT 0,
|
||||||
|
priority INTEGER DEFAULT 0,
|
||||||
|
retry_count INTEGER DEFAULT 0,
|
||||||
|
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_tasks_table)
|
||||||
|
logger.info("创建detect_tasks表成功")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.id IS '主键ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.domain_id IS '域名ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.task_type IS '任务类型'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.status IS '任务状态'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.priority IS '优先级'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.retry_count IS '重试次数'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.create_time IS '创建时间'")
|
||||||
|
cur.execute("COMMENT ON COLUMN detect_tasks.update_time IS '更新时间'")
|
||||||
|
logger.info("添加detect_tasks表字段注释成功")
|
||||||
|
|
||||||
|
# 创建domain_blacklist表
|
||||||
|
create_blacklist_table = """
|
||||||
|
CREATE TABLE IF NOT EXISTS domain_blacklist (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
domain VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
reason TEXT,
|
||||||
|
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_blacklist_table)
|
||||||
|
logger.info("创建domain_blacklist表成功")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_blacklist.id IS '主键ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_blacklist.domain IS '域名'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_blacklist.reason IS '黑名单原因'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_blacklist.create_time IS '创建时间'")
|
||||||
|
logger.info("添加domain_blacklist表字段注释成功")
|
||||||
|
|
||||||
|
# 创建sensitive_words表
|
||||||
|
create_sensitive_words_table = """
|
||||||
|
CREATE TABLE IF NOT EXISTS sensitive_words (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
word VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_sensitive_words_table)
|
||||||
|
logger.info("创建sensitive_words表成功")
|
||||||
|
|
||||||
|
# 添加缺失的字段
|
||||||
|
try:
|
||||||
|
# 添加category字段
|
||||||
|
cur.execute("ALTER TABLE sensitive_words ADD COLUMN IF NOT EXISTS category VARCHAR(50) DEFAULT 'default'")
|
||||||
|
# 添加priority字段
|
||||||
|
cur.execute("ALTER TABLE sensitive_words ADD COLUMN IF NOT EXISTS priority INTEGER DEFAULT 1")
|
||||||
|
logger.info("添加sensitive_words表字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"添加sensitive_words表字段失败: {e}")
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
try:
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.id IS '主键ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.word IS '敏感词'")
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.category IS '分类'")
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.priority IS '优先级'")
|
||||||
|
cur.execute("COMMENT ON COLUMN sensitive_words.create_time IS '创建时间'")
|
||||||
|
logger.info("添加sensitive_words表字段注释成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"添加sensitive_words表字段注释失败: {e}")
|
||||||
|
|
||||||
|
# 创建domain_detections表
|
||||||
|
create_detections_table = """
|
||||||
|
CREATE TABLE IF NOT EXISTS domain_detections (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
domain_id INTEGER REFERENCES domains(id),
|
||||||
|
baidu_history JSONB,
|
||||||
|
baidu_site JSONB,
|
||||||
|
qihu360_site JSONB,
|
||||||
|
google_site JSONB,
|
||||||
|
chinaz_info JSONB,
|
||||||
|
aizhan_info JSONB,
|
||||||
|
juziseo_info JSONB,
|
||||||
|
jucha_info JSONB,
|
||||||
|
same_url BOOLEAN,
|
||||||
|
is_chinese_title BOOLEAN,
|
||||||
|
backlink_count_gt_10 BOOLEAN,
|
||||||
|
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
cur.execute(create_detections_table)
|
||||||
|
logger.info("创建domain_detections表成功")
|
||||||
|
cur.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_domain_detections_domain_id ON domain_detections(domain_id)")
|
||||||
|
|
||||||
|
# 为已存在的domain_detections表添加新字段
|
||||||
|
try:
|
||||||
|
# 添加same_url字段
|
||||||
|
cur.execute("ALTER TABLE domain_detections ADD COLUMN IF NOT EXISTS same_url BOOLEAN")
|
||||||
|
# 添加is_chinese_title字段
|
||||||
|
cur.execute("ALTER TABLE domain_detections ADD COLUMN IF NOT EXISTS is_chinese_title BOOLEAN")
|
||||||
|
# 添加backlink_count_gt_10字段
|
||||||
|
cur.execute("ALTER TABLE domain_detections ADD COLUMN IF NOT EXISTS backlink_count_gt_10 BOOLEAN")
|
||||||
|
logger.info("为domain_detections表添加新字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"为domain_detections表添加新字段失败: {e}")
|
||||||
|
# 回滚当前事务
|
||||||
|
conn.rollback()
|
||||||
|
|
||||||
|
# 添加字段注释
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.id IS '主键ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.domain_id IS '域名ID'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.baidu_history IS '百度历史'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.baidu_site IS '百度site'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.qihu360_site IS '360 site'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.google_site IS 'Google site'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.chinaz_info IS '站长之家信息'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.aizhan_info IS '爱站网信息'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.juziseo_info IS '桔子SEO信息'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.jucha_info IS '聚查信息'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.same_url IS '首网址和备案网址是否一样'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.is_chinese_title IS 'title是否简体中文'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.backlink_count_gt_10 IS '快照历史友链数量是否大于10'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.create_time IS '创建时间'")
|
||||||
|
cur.execute("COMMENT ON COLUMN domain_detections.update_time IS '更新时间'")
|
||||||
|
logger.info("添加domain_detections表字段注释成功")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
logger.info("数据库初始化完成")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"数据库初始化失败: {e}")
|
||||||
|
if conn:
|
||||||
|
conn.rollback()
|
||||||
|
finally:
|
||||||
|
if cur:
|
||||||
|
cur.close()
|
||||||
|
if conn:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
init_database()
|
||||||
BIN
domainCheck/jucha_cookies.pkl
Normal file
BIN
domainCheck/jucha_cookies.pkl
Normal file
Binary file not shown.
BIN
domainCheck/juming_cookies.pkl
Normal file
BIN
domainCheck/juming_cookies.pkl
Normal file
Binary file not shown.
BIN
domainCheck/juziseo_cookies.pkl
Normal file
BIN
domainCheck/juziseo_cookies.pkl
Normal file
Binary file not shown.
1
domainCheck/logo.svg
Normal file
1
domainCheck/logo.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 12 KiB |
1
domainCheck/new_logo.svg
Normal file
1
domainCheck/new_logo.svg
Normal file
@@ -0,0 +1 @@
|
|||||||
|
<svg t="1776041645024" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="2611" width="200" height="200"><path d="M462.9 858.3c-96 0-192.1-35.4-266.9-106.6C41.5 604.5 35.5 359 182.7 204.5 254 129.6 350.2 87 453.5 84.5c103.3-2.4 201.5 35.4 276.3 106.7 74.9 71.3 117.5 167.5 120 270.8s-35.4 201.5-106.7 276.3c-75.8 79.8-177.9 120-280.2 120z m0.3-730.7c-2.8 0-5.7 0-8.6 0.1-91.8 2.2-177.3 40.1-240.6 106.6C83.2 371.5 88.5 589.6 225.8 720.4S581.2 845.9 712 708.6c63.4-66.5 97-153.7 94.8-245.5-2.2-91.8-40.1-177.3-106.6-240.6-64.5-61.4-148.3-94.9-237-94.9z" fill="#1195FE" p-id="2612"></path><path d="M862.1 928.3c-17.7 0-35.5-6.5-49.3-19.7l-150.1-143 19.3-15.4c12.6-10 24.3-20.7 34.8-31.7 10.5-11 20.6-23.2 30-36.3l14.5-20.1 150.1 143c13.8 13.1 21.6 30.9 22.1 50s-6.5 37.2-19.6 51c-14.1 14.8-33 22.2-51.8 22.2zM727.8 768l114.8 109.4c11.3 10.8 29.2 10.3 40-1s10.3-29.2-1-40L766.8 727c-6.1 7.4-12.3 14.5-18.7 21.2-6.5 6.8-13.2 13.4-20.3 19.8zM268.8 551.8c-9.8-23.6-15.2-49-15.9-75.3-1.4-56.1 19.2-109.5 58-150.1 1.2-1.3 2.4-2.5 3.7-3.8 38.3-38.3 89.1-60.1 143.5-61.4 30.3-0.7 59.2 5 85.5 16l32.6-32.6c-35.8-17.9-76.3-27.6-119.1-26.6-67.7 1.6-130.7 29.3-177.3 78.6-46.7 49-71.5 113.3-69.9 180.9 0.9 37.8 10 74.1 26.4 106.9l32.5-32.6zM657 390.1c9.9 23.5 15.7 49.1 16.3 76.1 2.8 115.9-89.2 212.5-205.1 215.3-30.1 0.7-59.4-4.9-86.4-16.2l-32.6 32.6c36.7 18.5 77.7 27.8 120 26.8 67.6-1.6 128.5-29.8 172.9-74.2 47.3-47.3 76-113.2 74.3-185.3-0.9-38.7-10.6-75.1-26.9-107.6L657 390.1z" fill="#1195FE" p-id="2613"></path><path d="M287.283 615.504l188.401-62.199 6.05 18.327-188.4 62.199zM430.89 361.097l188.4-62.199 6.051 18.327-188.4 62.199z" fill="#1195FE" p-id="2614"></path><path d="M472.411 564.972l-27.36-196.516 19.117-2.661 27.36 196.515z" fill="#1195FE" p-id="2615"></path><path d="M623.8 314m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2616"></path><path d="M623.8 360.8c-25.8 0-46.8-21-46.8-46.8 0-25.8 21-46.8 46.8-46.8 25.8 0 46.8 21 46.8 46.8 0 25.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7s20.7-9.3 20.7-20.7c0-11.4-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2617"></path><path d="M449.7 363.9m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2618"></path><path d="M449.7 410.7c-25.8 0-46.8-21-46.8-46.8 0-25.8 21-46.8 46.8-46.8 25.8 0 46.8 21 46.8 46.8 0 25.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7 11.4 0 20.7-9.3 20.7-20.7 0-11.5-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2619"></path><path d="M302.4 628.7m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2620"></path><path d="M302.4 675.5c-25.8 0-46.8-21-46.8-46.8 0-25.8 21-46.8 46.8-46.8 25.8 0 46.8 21 46.8 46.8 0 25.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7 11.4 0 20.7-9.3 20.7-20.7 0-11.4-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2621"></path><path d="M474.7 560.5m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2622"></path><path d="M474.7 607.3c-25.8 0-46.8-21-46.8-46.8s21-46.8 46.8-46.8 46.8 21 46.8 46.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7s20.7-9.3 20.7-20.7c0-11.4-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2623"></path></svg>
|
||||||
|
After Width: | Height: | Size: 3.2 KiB |
406
domainCheck/project_design.md
Normal file
406
domainCheck/project_design.md
Normal file
@@ -0,0 +1,406 @@
|
|||||||
|
# 域名库系统设计方案
|
||||||
|
|
||||||
|
## 1. 项目结构设计
|
||||||
|
|
||||||
|
```
|
||||||
|
domainScanDemo/
|
||||||
|
├── app/ # 应用主目录
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── main.py # 主入口
|
||||||
|
│ ├── ui/ # 可视化界面
|
||||||
|
│ │ ├── __init__.py
|
||||||
|
│ │ ├── main_window.py # 主窗口
|
||||||
|
│ │ ├── domain_import.py # 域名导入界面
|
||||||
|
│ │ ├── domain_filter.py # 域名筛选界面
|
||||||
|
│ │ └── sensitive_words.py # 敏感词配置界面
|
||||||
|
│ ├── core/ # 核心功能
|
||||||
|
│ │ ├── __init__.py
|
||||||
|
│ │ ├── domain_collector.py # 域名收集器
|
||||||
|
│ │ ├── domain_processor.py # 域名处理器
|
||||||
|
│ │ ├── detect_engine.py # 检测引擎
|
||||||
|
│ │ ├── task_scheduler.py # 任务调度器
|
||||||
|
│ │ └── export_manager.py # 导出管理器
|
||||||
|
│ ├── detectors/ # 检测插件
|
||||||
|
│ │ ├── __init__.py
|
||||||
|
│ │ ├── base.py # 基础检测类
|
||||||
|
│ │ ├── rdap_detector.py # RDAP检测
|
||||||
|
│ │ ├── wayback_detector.py # 时光机检测
|
||||||
|
│ │ ├── baidu_detector.py # 百度检测
|
||||||
|
│ │ ├── qihu360_detector.py # 360检测
|
||||||
|
│ │ ├── google_detector.py # Google检测
|
||||||
|
│ │ ├── chinaz_detector.py # 站长之家检测
|
||||||
|
│ │ ├── aizhan_detector.py # 爱站网检测
|
||||||
|
│ │ ├── juziseo_detector.py # 桔子SEO检测
|
||||||
|
│ │ └── jucha_detector.py # 聚查检测
|
||||||
|
│ ├── utils/ # 工具类
|
||||||
|
│ │ ├── __init__.py
|
||||||
|
│ │ ├── domain_utils.py # 域名工具
|
||||||
|
│ │ ├── http_utils.py # HTTP工具
|
||||||
|
│ │ ├── database.py # 数据库操作
|
||||||
|
│ │ └── config.py # 配置管理
|
||||||
|
│ └── config/ # 配置文件
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── settings.py # 系统设置
|
||||||
|
│ └── sensitive_words.txt # 敏感词列表
|
||||||
|
├── scripts/ # 脚本工具
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── domain_import.py # 域名导入脚本
|
||||||
|
│ ├── detect_runner.py # 检测运行脚本
|
||||||
|
│ └── stats_report.py # 统计报告脚本
|
||||||
|
├── tests/ # 测试代码
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── test_domain_utils.py # 域名工具测试
|
||||||
|
│ ├── test_detectors.py # 检测器测试
|
||||||
|
│ └── test_database.py # 数据库测试
|
||||||
|
├── requirements.txt # 依赖包
|
||||||
|
├── README.md # 项目说明
|
||||||
|
└── .env # 环境变量
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. 核心功能模块设计
|
||||||
|
|
||||||
|
### 2.1 域名收集模块
|
||||||
|
|
||||||
|
**功能**:从多个来源收集域名数据
|
||||||
|
|
||||||
|
**实现**:
|
||||||
|
- **聚名网爬虫**:抓取一口价和过期删除域名
|
||||||
|
- **搜索引擎采集**:通过关键词搜索收集域名
|
||||||
|
- **企业目录抓取**:从企业黄页等网站抓取域名
|
||||||
|
- **Zone File 解析**:解析域名区域文件
|
||||||
|
- **TXT 导入**:支持从文本文件导入域名
|
||||||
|
- **第三方 API**:集成域名数据 API
|
||||||
|
|
||||||
|
**技术要点**:
|
||||||
|
- 使用 `Scrapy` 框架进行网站爬取
|
||||||
|
- 支持代理池,避免 IP 被封禁
|
||||||
|
- 多线程并行采集,提高效率
|
||||||
|
- 域名标准化处理(小写、去空格、去协议、只留主域)
|
||||||
|
|
||||||
|
### 2.2 域名处理模块
|
||||||
|
|
||||||
|
**功能**:处理收集到的域名,进行标准化和去重
|
||||||
|
|
||||||
|
**实现**:
|
||||||
|
- 域名标准化:转换为小写,去除空格和协议
|
||||||
|
- 后缀过滤:只保留 `.com` 和 `.net`
|
||||||
|
- 去重处理:使用布隆过滤器和数据库唯一索引
|
||||||
|
- 来源标记:记录域名来源和批次信息
|
||||||
|
- 任务创建:为新域名创建检测任务
|
||||||
|
|
||||||
|
**技术要点**:
|
||||||
|
- 使用 Redis 布隆过滤器快速去重
|
||||||
|
- 批量处理,减少数据库操作
|
||||||
|
- 事务处理,确保数据一致性
|
||||||
|
|
||||||
|
### 2.3 检测引擎模块
|
||||||
|
|
||||||
|
**功能**:执行域名检测任务,包括注册状态、风险评估等
|
||||||
|
|
||||||
|
**实现**:
|
||||||
|
- **基础检测**:
|
||||||
|
- 注册状态检测(RDAP优先,其次WHOIS)
|
||||||
|
- 黑名单缓存检查
|
||||||
|
- 时光机快照年份采集
|
||||||
|
- 敏感词匹配(快照内容)
|
||||||
|
|
||||||
|
- **深度检测**:
|
||||||
|
- 百度历史/Site
|
||||||
|
- 360 Site
|
||||||
|
- Google Site
|
||||||
|
- 站长之家/爱站
|
||||||
|
- 桔子SEO(历史记录)
|
||||||
|
- 聚查(WHOIS/备案/拦截)
|
||||||
|
- 桔子SEO(外链查询)
|
||||||
|
|
||||||
|
**技术要点**:
|
||||||
|
- 插件化设计,支持添加新的检测模块
|
||||||
|
- 优先级队列,按成本和重要性排序
|
||||||
|
- 失败重试机制,提高检测成功率
|
||||||
|
- 并发处理,提高检测速度
|
||||||
|
|
||||||
|
### 2.4 任务调度模块
|
||||||
|
|
||||||
|
**功能**:管理检测任务队列,分配任务到检测节点
|
||||||
|
|
||||||
|
**实现**:
|
||||||
|
- 任务队列管理:使用 RabbitMQ 存储任务
|
||||||
|
- 任务分配:根据节点负载分配任务
|
||||||
|
- 状态跟踪:实时监控任务执行状态
|
||||||
|
- 断点续跑:支持系统重启后继续执行任务
|
||||||
|
- 失败重试:自动重试失败的任务
|
||||||
|
|
||||||
|
**技术要点**:
|
||||||
|
- 使用 Celery 作为任务队列
|
||||||
|
- 分布式部署,支持多服务器
|
||||||
|
- 实时监控和告警
|
||||||
|
- 任务优先级管理
|
||||||
|
|
||||||
|
### 2.5 导出管理模块
|
||||||
|
|
||||||
|
**功能**:根据运营筛选条件导出域名
|
||||||
|
|
||||||
|
**实现**:
|
||||||
|
- 多条件组合筛选
|
||||||
|
- 预览功能
|
||||||
|
- TXT 导出
|
||||||
|
- 批量更新使用状态
|
||||||
|
- 批量更新人工审核状态
|
||||||
|
|
||||||
|
**技术要点**:
|
||||||
|
- 高效的数据库查询优化
|
||||||
|
- 支持大文件导出
|
||||||
|
- 导出进度实时显示
|
||||||
|
- 导出历史记录
|
||||||
|
|
||||||
|
### 2.6 可视化界面模块
|
||||||
|
|
||||||
|
**功能**:提供用户友好的操作界面
|
||||||
|
|
||||||
|
**实现**:
|
||||||
|
- **主窗口**:系统概览和功能导航
|
||||||
|
- **域名导入**:支持多种方式导入域名
|
||||||
|
- **域名筛选**:多条件组合筛选
|
||||||
|
- **敏感词配置**:管理敏感词列表
|
||||||
|
- **系统设置**:配置系统参数
|
||||||
|
|
||||||
|
**技术要点**:
|
||||||
|
- 使用 PySide6 构建界面
|
||||||
|
- 响应式设计,支持不同屏幕尺寸
|
||||||
|
- 多线程处理,避免界面卡顿
|
||||||
|
- 实时数据更新
|
||||||
|
|
||||||
|
## 3. 数据库设计
|
||||||
|
|
||||||
|
**数据库**:PostgreSQL 15+
|
||||||
|
|
||||||
|
**核心表结构**:
|
||||||
|
|
||||||
|
| 表名 | 功能 | 关键字段 |
|
||||||
|
|------|------|----------|
|
||||||
|
| `domains` | 域名主表 | domain, tld, use_status, detect_status, register_status, source_type |
|
||||||
|
| `domain_detections` | 检测结果 | domain_id, baidu_history, baidu_site, is_simplified, qihu360_site, google_site |
|
||||||
|
| `domain_sources` | 来源信息 | domain_id, source_type, source_url, batch_id |
|
||||||
|
| `detect_tasks` | 检测任务 | domain_id, task_type, status, priority, retry_count |
|
||||||
|
| `sensitive_words` | 敏感词 | word, category, level |
|
||||||
|
| `domain_blacklist` | 黑名单 | domain, reason, blacklist_time, source |
|
||||||
|
| `system_config` | 系统配置 | config_key, config_value, description |
|
||||||
|
| `detect_nodes` | 检测节点 | node_name, ip_address, status, last_heartbeat |
|
||||||
|
| `task_stats` | 任务统计 | date, total_tasks, success_tasks, failed_tasks |
|
||||||
|
| `domain_stats` | 域名统计 | date, total_domains, available_domains, registered_domains |
|
||||||
|
|
||||||
|
**索引策略**:
|
||||||
|
- 域名字段建立唯一索引
|
||||||
|
- 状态字段建立普通索引
|
||||||
|
- 时间字段建立 BRIN 索引
|
||||||
|
- 复合索引优化多条件查询
|
||||||
|
|
||||||
|
**分区策略**:
|
||||||
|
- `domains` 表按 TLD 分片
|
||||||
|
- `domain_detections` 表按时间分区
|
||||||
|
- `domain_sources` 表按时间分区
|
||||||
|
- `detect_logs` 表按时间分区
|
||||||
|
|
||||||
|
## 4. 技术栈选择
|
||||||
|
|
||||||
|
| 分类 | 技术 | 版本 | 选型理由 |
|
||||||
|
|------|------|------|----------|
|
||||||
|
| **编程语言** | Python | 3.10+ | 丰富的库支持,适合数据处理和爬虫开发 |
|
||||||
|
| **Web 框架** | Scrapy | 2.10+ | 强大的爬虫框架,支持分布式爬取 |
|
||||||
|
| **数据库** | PostgreSQL | 15+ | 支持分区表、JSONB、布隆过滤器,适合亿级数据 |
|
||||||
|
| **缓存** | Redis | 7+ | 高性能缓存,支持布隆过滤器和消息队列 |
|
||||||
|
| **消息队列** | RabbitMQ | 3.10+ | 可靠的任务队列,支持优先级 |
|
||||||
|
| **任务调度** | Celery | 5+ | 分布式任务处理框架 |
|
||||||
|
| **界面库** | PySide6 | 6.5+ | 跨平台 GUI 框架,功能丰富 |
|
||||||
|
| **HTTP 客户端** | Requests | 2.31+ | 简单易用的 HTTP 库 |
|
||||||
|
| **异步处理** | asyncio | 内置 | 高效的异步 I/O 处理 |
|
||||||
|
| **监控** | Prometheus + Grafana | 2.40+ | 强大的监控和可视化 |
|
||||||
|
|
||||||
|
## 5. 系统流程
|
||||||
|
|
||||||
|
### 5.1 域名入库流程
|
||||||
|
|
||||||
|
1. **数据采集**:从多个来源收集域名
|
||||||
|
2. **标准化处理**:转换为小写,去除空格和协议
|
||||||
|
3. **后缀过滤**:只保留 `.com` 和 `.net`
|
||||||
|
4. **去重处理**:使用布隆过滤器和数据库唯一索引
|
||||||
|
5. **来源标记**:记录域名来源和批次信息
|
||||||
|
6. **任务创建**:为新域名创建检测任务
|
||||||
|
|
||||||
|
### 5.2 检测流程
|
||||||
|
|
||||||
|
1. **基础检测**:
|
||||||
|
- 注册状态检测(RDAP优先)
|
||||||
|
- 黑名单缓存检查
|
||||||
|
- 时光机快照年份采集
|
||||||
|
- 敏感词匹配(快照内容)
|
||||||
|
- 命中则直接拉黑并停止后续检测
|
||||||
|
|
||||||
|
2. **深度检测**:
|
||||||
|
- 百度历史/Site
|
||||||
|
- 360 Site
|
||||||
|
- Google Site
|
||||||
|
- 站长之家/爱站
|
||||||
|
- 桔子SEO(历史记录)
|
||||||
|
- 聚查(WHOIS/备案/拦截)
|
||||||
|
- 桔子SEO(外链查询)
|
||||||
|
|
||||||
|
### 5.3 运营筛选流程
|
||||||
|
|
||||||
|
1. **多条件组合筛选**:根据注册状态、使用状态、检测状态等条件
|
||||||
|
2. **预览**:查看筛选结果
|
||||||
|
3. **导出**:导出为 TXT 文件
|
||||||
|
4. **批量更新**:更新使用状态和人工审核状态
|
||||||
|
|
||||||
|
## 6. 关键技术挑战及解决方案
|
||||||
|
|
||||||
|
### 6.1 数据量挑战
|
||||||
|
|
||||||
|
**挑战**:亿级域名数据的存储和查询
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
- 使用 PostgreSQL 分区表
|
||||||
|
- 优化索引策略
|
||||||
|
- 数据分片存储
|
||||||
|
- 缓存热点数据
|
||||||
|
|
||||||
|
### 6.2 检测效率挑战
|
||||||
|
|
||||||
|
**挑战**:大量域名的检测任务处理
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
- 分布式检测架构
|
||||||
|
- 任务优先级队列
|
||||||
|
- 并发处理
|
||||||
|
- 失败重试机制
|
||||||
|
|
||||||
|
### 6.3 反爬机制挑战
|
||||||
|
|
||||||
|
**挑战**:网站反爬措施
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
- 使用代理池
|
||||||
|
- 请求头随机化
|
||||||
|
- 访问频率控制
|
||||||
|
- 模拟浏览器行为
|
||||||
|
|
||||||
|
### 6.4 数据一致性挑战
|
||||||
|
|
||||||
|
**挑战**:分布式环境下的数据一致性
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
- 事务处理
|
||||||
|
- 消息队列确保任务可靠传递
|
||||||
|
- 定期数据同步
|
||||||
|
- 冲突检测和解决
|
||||||
|
|
||||||
|
## 7. 部署方案
|
||||||
|
|
||||||
|
### 7.1 服务器配置
|
||||||
|
|
||||||
|
| 服务器类型 | 配置 | 用途 |
|
||||||
|
|------------|------|------|
|
||||||
|
| **主服务器** | 16核32G,2TB SSD | 数据库、任务调度、Web 界面 |
|
||||||
|
| **检测节点** | 8核16G,500GB SSD | 执行检测任务 |
|
||||||
|
| **存储服务器** | 4核8G,4TB HDD | 数据备份、归档 |
|
||||||
|
|
||||||
|
### 7.2 部署步骤
|
||||||
|
|
||||||
|
1. **环境搭建**:
|
||||||
|
- 安装 PostgreSQL、Redis、RabbitMQ
|
||||||
|
- 配置 Python 环境
|
||||||
|
- 安装依赖包
|
||||||
|
|
||||||
|
2. **数据库初始化**:
|
||||||
|
- 执行 SQL 脚本创建表结构
|
||||||
|
- 配置分区和索引
|
||||||
|
- 导入初始数据
|
||||||
|
|
||||||
|
3. **服务启动**:
|
||||||
|
- 启动数据库服务
|
||||||
|
- 启动消息队列服务
|
||||||
|
- 启动任务调度服务
|
||||||
|
- 启动检测节点
|
||||||
|
|
||||||
|
4. **系统配置**:
|
||||||
|
- 配置敏感词列表
|
||||||
|
- 配置检测参数
|
||||||
|
- 配置监控告警
|
||||||
|
|
||||||
|
### 7.3 监控与维护
|
||||||
|
|
||||||
|
1. **监控系统**:
|
||||||
|
- 数据库性能监控
|
||||||
|
- 任务执行状态监控
|
||||||
|
- 系统负载监控
|
||||||
|
- 网络状态监控
|
||||||
|
|
||||||
|
2. **维护计划**:
|
||||||
|
- 每日:数据备份、任务统计
|
||||||
|
- 每周:索引重建、系统更新
|
||||||
|
- 每月:数据归档、性能优化
|
||||||
|
- 每季度:全量备份、系统评估
|
||||||
|
|
||||||
|
## 8. 预期性能指标
|
||||||
|
|
||||||
|
| 指标 | 预期值 |
|
||||||
|
|------|--------|
|
||||||
|
| 数据导入速度 | > 100万条/分钟 |
|
||||||
|
| 域名去重速度 | > 1000万条/秒 |
|
||||||
|
| 检测任务处理 | > 10000个/分钟 |
|
||||||
|
| 单表查询响应 | < 1秒 |
|
||||||
|
| 系统可用性 | > 99.9% |
|
||||||
|
| 检测准确率 | > 99% |
|
||||||
|
|
||||||
|
## 9. 核心竞争力
|
||||||
|
|
||||||
|
1. **全维度检测**:覆盖注册状态、历史快照、平台风险等多维度
|
||||||
|
2. **智能筛选**:基于规则的自动化筛选,减少人工干预
|
||||||
|
3. **增量建设**:不追求一次性完成,通过持续积累构建完整数据库
|
||||||
|
4. **插件化架构**:检测模块可替换、可扩展
|
||||||
|
5. **数据价值**:为域名投资和企业品牌保护提供数据支持
|
||||||
|
6. **高性能**:分布式架构,支持亿级数据处理
|
||||||
|
7. **可靠性**:完善的错误处理和重试机制
|
||||||
|
8. **用户友好**:直观的可视化界面,操作简单
|
||||||
|
|
||||||
|
## 10. 项目实施计划
|
||||||
|
|
||||||
|
### 第一阶段:基础架构搭建(1个月)
|
||||||
|
- 数据库设计和搭建
|
||||||
|
- 核心模块框架搭建
|
||||||
|
- 数据采集模块实现
|
||||||
|
- 基础检测功能实现
|
||||||
|
|
||||||
|
### 第二阶段:功能完善(2个月)
|
||||||
|
- 深度检测功能实现
|
||||||
|
- 运营管理界面开发
|
||||||
|
- 任务调度系统实现
|
||||||
|
- 多服务器部署方案
|
||||||
|
|
||||||
|
### 第三阶段:优化与测试(1个月)
|
||||||
|
- 性能优化
|
||||||
|
- 稳定性测试
|
||||||
|
- 功能测试
|
||||||
|
- 部署上线
|
||||||
|
|
||||||
|
### 第四阶段:持续迭代
|
||||||
|
- 根据运营反馈优化系统
|
||||||
|
- 增加新的检测模块
|
||||||
|
- 扩展数据源
|
||||||
|
- 提升系统性能
|
||||||
|
|
||||||
|
## 11. 风险评估
|
||||||
|
|
||||||
|
| 风险 | 影响 | 应对措施 |
|
||||||
|
|------|------|----------|
|
||||||
|
| **数据量过大** | 存储和查询性能下降 | 分区表、索引优化、数据分片 |
|
||||||
|
| **API 限制** | 检测速度受限 | 多API源、请求频率控制、缓存 |
|
||||||
|
| **反爬机制** | 爬虫被封禁 | 代理池、请求头随机化、模拟浏览器 |
|
||||||
|
| **系统稳定性** | 服务中断 | 监控告警、自动恢复、冗余部署 |
|
||||||
|
| **成本控制** | 服务器和API成本高 | 优化检测流程、合理使用资源、预算规划 |
|
||||||
|
|
||||||
|
## 12. 结论
|
||||||
|
|
||||||
|
本设计方案基于需求文档,详细说明了系统的架构设计、功能模块和实施计划。系统采用模块化设计,支持增量建设和分布式部署,能够有效应对亿级数据量的挑战。通过多维度的检测和智能筛选,为用户提供高质量的域名资源。
|
||||||
|
|
||||||
|
该方案充分考虑了技术可行性和业务需求,为域名库系统的开发和部署提供了全面的指导。
|
||||||
13
domainCheck/proxy_config.json
Normal file
13
domainCheck/proxy_config.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"proxy_url": "http://211.101.244.154:18008/getProxy_batch.php?group=B&count=50",
|
||||||
|
"proxy_enable": true,
|
||||||
|
"allow_direct": true,
|
||||||
|
"proxy_urls": [
|
||||||
|
"http://211.101.244.154:18008/getProxy_batch.php?group=B&count=50",
|
||||||
|
"http://211.101.244.154:18008/getProxy_batch.php?group=C&count=50",
|
||||||
|
"http://211.101.244.154:18008/getProxy_batch.php?group=D&count=50",
|
||||||
|
"http://211.101.244.154:18008/getProxy_batch.php?group=E&count=50",
|
||||||
|
"http://211.101.244.154:18008/getProxy_batch.php?group=F&count=50",
|
||||||
|
"http://211.101.244.154:18008/getProxy_batch.php?group=G&count=50"
|
||||||
|
]
|
||||||
|
}
|
||||||
50
domainCheck/remove_beian_time_field.py
Normal file
50
domainCheck/remove_beian_time_field.py
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
# -*- coding: UTF-8 -*-
|
||||||
|
'''
|
||||||
|
@Project :domainScanDemo
|
||||||
|
@File :remove_beian_time_field.py
|
||||||
|
@IDE :PyCharm
|
||||||
|
@Author :梦伴
|
||||||
|
@Date :2026/4/11 22:40
|
||||||
|
@explain : 移除domains表中的beian_time字段
|
||||||
|
'''
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
from loguru import logger
|
||||||
|
from app.config import config
|
||||||
|
|
||||||
|
def remove_beian_time_field():
|
||||||
|
"""
|
||||||
|
移除domains表中的beian_time字段
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# 连接数据库
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=config.DB_HOST,
|
||||||
|
port=config.DB_PORT,
|
||||||
|
database=config.DB_DATABASE,
|
||||||
|
user=config.DB_USER,
|
||||||
|
password=config.DB_PASSWORD
|
||||||
|
)
|
||||||
|
cur = conn.cursor()
|
||||||
|
logger.info("数据库连接成功")
|
||||||
|
|
||||||
|
# 移除beian_time字段
|
||||||
|
try:
|
||||||
|
cur.execute("ALTER TABLE domains DROP COLUMN IF EXISTS beian_time")
|
||||||
|
logger.info("移除beian_time字段成功")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"移除beian_time字段失败: {e}")
|
||||||
|
|
||||||
|
# 提交事务
|
||||||
|
conn.commit()
|
||||||
|
logger.info("字段移除完成")
|
||||||
|
|
||||||
|
# 关闭连接
|
||||||
|
cur.close()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"执行失败: {e}")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
remove_beian_time_field()
|
||||||
BIN
domainCheck/requirements.txt
Normal file
BIN
domainCheck/requirements.txt
Normal file
Binary file not shown.
10
domainCheck/start_app.ps1
Normal file
10
domainCheck/start_app.ps1
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$env:PYTHONUTF8 = '1'
|
||||||
|
|
||||||
|
$python = Join-Path $PSScriptRoot '.venv\Scripts\python.exe'
|
||||||
|
if (-not (Test-Path $python)) {
|
||||||
|
throw "未找到虚拟环境解释器: $python"
|
||||||
|
}
|
||||||
|
|
||||||
|
Set-Location $PSScriptRoot
|
||||||
|
& $python 'app\main.py'
|
||||||
10
domainCheck/start_worker.ps1
Normal file
10
domainCheck/start_worker.ps1
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$env:PYTHONUTF8 = '1'
|
||||||
|
|
||||||
|
$python = Join-Path $PSScriptRoot '.venv\Scripts\python.exe'
|
||||||
|
if (-not (Test-Path $python)) {
|
||||||
|
throw "未找到虚拟环境解释器: $python"
|
||||||
|
}
|
||||||
|
|
||||||
|
Set-Location $PSScriptRoot
|
||||||
|
& $python 'detect_worker.py'
|
||||||
3
domainCheck/thread_count.json
Normal file
3
domainCheck/thread_count.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"thread_count": "2"
|
||||||
|
}
|
||||||
1397
domainCheck/tools/node-v20.19.4-win-x64/CHANGELOG.md
Normal file
1397
domainCheck/tools/node-v20.19.4-win-x64/CHANGELOG.md
Normal file
File diff suppressed because it is too large
Load Diff
2170
domainCheck/tools/node-v20.19.4-win-x64/LICENSE
Normal file
2170
domainCheck/tools/node-v20.19.4-win-x64/LICENSE
Normal file
File diff suppressed because it is too large
Load Diff
897
domainCheck/tools/node-v20.19.4-win-x64/README.md
Normal file
897
domainCheck/tools/node-v20.19.4-win-x64/README.md
Normal file
@@ -0,0 +1,897 @@
|
|||||||
|
# Node.js
|
||||||
|
|
||||||
|
Node.js is an open-source, cross-platform JavaScript runtime environment.
|
||||||
|
|
||||||
|
For information on using Node.js, see the [Node.js website][].
|
||||||
|
|
||||||
|
The Node.js project uses an [open governance model](./GOVERNANCE.md). The
|
||||||
|
[OpenJS Foundation][] provides support for the project.
|
||||||
|
|
||||||
|
Contributors are expected to act in a collaborative manner to move
|
||||||
|
the project forward. We encourage the constructive exchange of contrary
|
||||||
|
opinions and compromise. The [TSC](./GOVERNANCE.md#technical-steering-committee)
|
||||||
|
reserves the right to limit or block contributors who repeatedly act in ways
|
||||||
|
that discourage, exhaust, or otherwise negatively affect other participants.
|
||||||
|
|
||||||
|
**This project has a [Code of Conduct][].**
|
||||||
|
|
||||||
|
## Table of contents
|
||||||
|
|
||||||
|
* [Support](#support)
|
||||||
|
* [Release types](#release-types)
|
||||||
|
* [Download](#download)
|
||||||
|
* [Current and LTS releases](#current-and-lts-releases)
|
||||||
|
* [Nightly releases](#nightly-releases)
|
||||||
|
* [API documentation](#api-documentation)
|
||||||
|
* [Verifying binaries](#verifying-binaries)
|
||||||
|
* [Building Node.js](#building-nodejs)
|
||||||
|
* [Security](#security)
|
||||||
|
* [Contributing to Node.js](#contributing-to-nodejs)
|
||||||
|
* [Current project team members](#current-project-team-members)
|
||||||
|
* [TSC (Technical Steering Committee)](#tsc-technical-steering-committee)
|
||||||
|
* [Collaborators](#collaborators)
|
||||||
|
* [Triagers](#triagers)
|
||||||
|
* [Release keys](#release-keys)
|
||||||
|
* [License](#license)
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
Looking for help? Check out the
|
||||||
|
[instructions for getting support](.github/SUPPORT.md).
|
||||||
|
|
||||||
|
## Release types
|
||||||
|
|
||||||
|
* **Current**: Under active development. Code for the Current release is in the
|
||||||
|
branch for its major version number (for example,
|
||||||
|
[v22.x](https://github.com/nodejs/node/tree/v22.x)). Node.js releases a new
|
||||||
|
major version every 6 months, allowing for breaking changes. This happens in
|
||||||
|
April and October every year. Releases appearing each October have a support
|
||||||
|
life of 8 months. Releases appearing each April convert to LTS (see below)
|
||||||
|
each October.
|
||||||
|
* **LTS**: Releases that receive Long Term Support, with a focus on stability
|
||||||
|
and security. Every even-numbered major version will become an LTS release.
|
||||||
|
LTS releases receive 12 months of _Active LTS_ support and a further 18 months
|
||||||
|
of _Maintenance_. LTS release lines have alphabetically-ordered code names,
|
||||||
|
beginning with v4 Argon. There are no breaking changes or feature additions,
|
||||||
|
except in some special circumstances.
|
||||||
|
* **Nightly**: Code from the Current branch built every 24-hours when there are
|
||||||
|
changes. Use with caution.
|
||||||
|
|
||||||
|
Current and LTS releases follow [semantic versioning](https://semver.org). A
|
||||||
|
member of the Release Team [signs](#release-keys) each Current and LTS release.
|
||||||
|
For more information, see the
|
||||||
|
[Release README](https://github.com/nodejs/Release#readme).
|
||||||
|
|
||||||
|
### Download
|
||||||
|
|
||||||
|
Binaries, installers, and source tarballs are available at
|
||||||
|
<https://nodejs.org/en/download/>.
|
||||||
|
|
||||||
|
#### Current and LTS releases
|
||||||
|
|
||||||
|
<https://nodejs.org/download/release/>
|
||||||
|
|
||||||
|
The [latest](https://nodejs.org/download/release/latest/) directory is an
|
||||||
|
alias for the latest Current release. The latest-_codename_ directory is an
|
||||||
|
alias for the latest release from an LTS line. For example, the
|
||||||
|
[latest-hydrogen](https://nodejs.org/download/release/latest-hydrogen/)
|
||||||
|
directory contains the latest Hydrogen (Node.js 18) release.
|
||||||
|
|
||||||
|
#### Nightly releases
|
||||||
|
|
||||||
|
<https://nodejs.org/download/nightly/>
|
||||||
|
|
||||||
|
Each directory and filename includes the version (e.g., `v22.0.0`),
|
||||||
|
followed by the UTC date (e.g., `20240424` for April 24, 2024),
|
||||||
|
and the short commit SHA of the HEAD of the release (e.g., `ddd0a9e494`).
|
||||||
|
For instance, a full directory name might look like `v22.0.0-nightly20240424ddd0a9e494`.
|
||||||
|
|
||||||
|
#### API documentation
|
||||||
|
|
||||||
|
Documentation for the latest Current release is at <https://nodejs.org/api/>.
|
||||||
|
Version-specific documentation is available in each release directory in the
|
||||||
|
_docs_ subdirectory. Version-specific documentation is also at
|
||||||
|
<https://nodejs.org/download/docs/>.
|
||||||
|
|
||||||
|
### Verifying binaries
|
||||||
|
|
||||||
|
Download directories contain a `SHASUMS256.txt` file with SHA checksums for the
|
||||||
|
files.
|
||||||
|
|
||||||
|
To download `SHASUMS256.txt` using `curl`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -O https://nodejs.org/dist/vx.y.z/SHASUMS256.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
To check that downloaded files match the checksum, use `sha256sum`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sha256sum -c SHASUMS256.txt --ignore-missing
|
||||||
|
```
|
||||||
|
|
||||||
|
For Current and LTS, the GPG detached signature of `SHASUMS256.txt` is in
|
||||||
|
`SHASUMS256.txt.sig`. You can use it with `gpg` to verify the integrity of
|
||||||
|
`SHASUMS256.txt`. You will first need to import
|
||||||
|
[the GPG keys of individuals authorized to create releases](#release-keys).
|
||||||
|
|
||||||
|
See [Release keys](#release-keys) for commands to import active release keys.
|
||||||
|
|
||||||
|
Next, download the `SHASUMS256.txt.sig` for the release:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -O https://nodejs.org/dist/vx.y.z/SHASUMS256.txt.sig
|
||||||
|
```
|
||||||
|
|
||||||
|
Then use `gpg --verify SHASUMS256.txt.sig SHASUMS256.txt` to verify
|
||||||
|
the file's signature.
|
||||||
|
|
||||||
|
## Building Node.js
|
||||||
|
|
||||||
|
See [BUILDING.md](BUILDING.md) for instructions on how to build Node.js from
|
||||||
|
source and a list of supported platforms.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
For information on reporting security vulnerabilities in Node.js, see
|
||||||
|
[SECURITY.md](./SECURITY.md).
|
||||||
|
|
||||||
|
## Contributing to Node.js
|
||||||
|
|
||||||
|
* [Contributing to the project][]
|
||||||
|
* [Working Groups][]
|
||||||
|
* [Strategic initiatives][]
|
||||||
|
* [Technical values and prioritization][]
|
||||||
|
|
||||||
|
## Current project team members
|
||||||
|
|
||||||
|
For information about the governance of the Node.js project, see
|
||||||
|
[GOVERNANCE.md](./GOVERNANCE.md).
|
||||||
|
|
||||||
|
<!-- node-core-utils and find-inactive-tsc.mjs depend on the format of the TSC
|
||||||
|
list. If the format changes, those utilities need to be tested and
|
||||||
|
updated. -->
|
||||||
|
|
||||||
|
### TSC (Technical Steering Committee)
|
||||||
|
|
||||||
|
#### TSC voting members
|
||||||
|
|
||||||
|
<!--lint disable prohibited-strings-->
|
||||||
|
|
||||||
|
* [aduh95](https://github.com/aduh95) -
|
||||||
|
**Antoine du Hamel** <<duhamelantoine1995@gmail.com>> (he/him)
|
||||||
|
* [anonrig](https://github.com/anonrig) -
|
||||||
|
**Yagiz Nizipli** <<yagiz@nizipli.com>> (he/him)
|
||||||
|
* [benjamingr](https://github.com/benjamingr) -
|
||||||
|
**Benjamin Gruenbaum** <<benjamingr@gmail.com>>
|
||||||
|
* [BridgeAR](https://github.com/BridgeAR) -
|
||||||
|
**Ruben Bridgewater** <<ruben@bridgewater.de>> (he/him)
|
||||||
|
* [gireeshpunathil](https://github.com/gireeshpunathil) -
|
||||||
|
**Gireesh Punathil** <<gpunathi@in.ibm.com>> (he/him)
|
||||||
|
* [jasnell](https://github.com/jasnell) -
|
||||||
|
**James M Snell** <<jasnell@gmail.com>> (he/him)
|
||||||
|
* [joyeecheung](https://github.com/joyeecheung) -
|
||||||
|
**Joyee Cheung** <<joyeec9h3@gmail.com>> (she/her)
|
||||||
|
* [legendecas](https://github.com/legendecas) -
|
||||||
|
**Chengzhong Wu** <<legendecas@gmail.com>> (he/him)
|
||||||
|
* [marco-ippolito](https://github.com/marco-ippolito) -
|
||||||
|
**Marco Ippolito** <<marcoippolito54@gmail.com>> (he/him)
|
||||||
|
* [mcollina](https://github.com/mcollina) -
|
||||||
|
**Matteo Collina** <<matteo.collina@gmail.com>> (he/him)
|
||||||
|
* [mhdawson](https://github.com/mhdawson) -
|
||||||
|
**Michael Dawson** <<midawson@redhat.com>> (he/him)
|
||||||
|
* [RafaelGSS](https://github.com/RafaelGSS) -
|
||||||
|
**Rafael Gonzaga** <<rafael.nunu@hotmail.com>> (he/him)
|
||||||
|
* [richardlau](https://github.com/richardlau) -
|
||||||
|
**Richard Lau** <<rlau@redhat.com>>
|
||||||
|
* [ronag](https://github.com/ronag) -
|
||||||
|
**Robert Nagy** <<ronagy@icloud.com>>
|
||||||
|
* [ruyadorno](https://github.com/ruyadorno) -
|
||||||
|
**Ruy Adorno** <<ruy@vlt.sh>> (he/him)
|
||||||
|
* [ShogunPanda](https://github.com/ShogunPanda) -
|
||||||
|
**Paolo Insogna** <<paolo@cowtech.it>> (he/him)
|
||||||
|
* [targos](https://github.com/targos) -
|
||||||
|
**Michaël Zasso** <<targos@protonmail.com>> (he/him)
|
||||||
|
* [tniessen](https://github.com/tniessen) -
|
||||||
|
**Tobias Nießen** <<tniessen@tnie.de>> (he/him)
|
||||||
|
|
||||||
|
#### TSC regular members
|
||||||
|
|
||||||
|
* [BethGriggs](https://github.com/BethGriggs) -
|
||||||
|
**Beth Griggs** <<bethanyngriggs@gmail.com>> (she/her)
|
||||||
|
* [bnoordhuis](https://github.com/bnoordhuis) -
|
||||||
|
**Ben Noordhuis** <<info@bnoordhuis.nl>>
|
||||||
|
* [cjihrig](https://github.com/cjihrig) -
|
||||||
|
**Colin Ihrig** <<cjihrig@gmail.com>> (he/him)
|
||||||
|
* [codebytere](https://github.com/codebytere) -
|
||||||
|
**Shelley Vohr** <<shelley.vohr@gmail.com>> (she/her)
|
||||||
|
* [GeoffreyBooth](https://github.com/GeoffreyBooth) -
|
||||||
|
**Geoffrey Booth** <<webadmin@geoffreybooth.com>> (he/him)
|
||||||
|
* [MoLow](https://github.com/MoLow) -
|
||||||
|
**Moshe Atlow** <<moshe@atlow.co.il>> (he/him)
|
||||||
|
* [Trott](https://github.com/Trott) -
|
||||||
|
**Rich Trott** <<rtrott@gmail.com>> (he/him)
|
||||||
|
|
||||||
|
<details>
|
||||||
|
|
||||||
|
<summary>TSC emeriti members</summary>
|
||||||
|
|
||||||
|
#### TSC emeriti members
|
||||||
|
|
||||||
|
* [addaleax](https://github.com/addaleax) -
|
||||||
|
**Anna Henningsen** <<anna@addaleax.net>> (she/her)
|
||||||
|
* [apapirovski](https://github.com/apapirovski) -
|
||||||
|
**Anatoli Papirovski** <<apapirovski@mac.com>> (he/him)
|
||||||
|
* [ChALkeR](https://github.com/ChALkeR) -
|
||||||
|
**Сковорода Никита Андреевич** <<chalkerx@gmail.com>> (he/him)
|
||||||
|
* [chrisdickinson](https://github.com/chrisdickinson) -
|
||||||
|
**Chris Dickinson** <<christopher.s.dickinson@gmail.com>>
|
||||||
|
* [danbev](https://github.com/danbev) -
|
||||||
|
**Daniel Bevenius** <<daniel.bevenius@gmail.com>> (he/him)
|
||||||
|
* [danielleadams](https://github.com/danielleadams) -
|
||||||
|
**Danielle Adams** <<adamzdanielle@gmail.com>> (she/her)
|
||||||
|
* [evanlucas](https://github.com/evanlucas) -
|
||||||
|
**Evan Lucas** <<evanlucas@me.com>> (he/him)
|
||||||
|
* [fhinkel](https://github.com/fhinkel) -
|
||||||
|
**Franziska Hinkelmann** <<franziska.hinkelmann@gmail.com>> (she/her)
|
||||||
|
* [Fishrock123](https://github.com/Fishrock123) -
|
||||||
|
**Jeremiah Senkpiel** <<fishrock123@rocketmail.com>> (he/they)
|
||||||
|
* [gabrielschulhof](https://github.com/gabrielschulhof) -
|
||||||
|
**Gabriel Schulhof** <<gabrielschulhof@gmail.com>>
|
||||||
|
* [gibfahn](https://github.com/gibfahn) -
|
||||||
|
**Gibson Fahnestock** <<gibfahn@gmail.com>> (he/him)
|
||||||
|
* [indutny](https://github.com/indutny) -
|
||||||
|
**Fedor Indutny** <<fedor@indutny.com>>
|
||||||
|
* [isaacs](https://github.com/isaacs) -
|
||||||
|
**Isaac Z. Schlueter** <<i@izs.me>>
|
||||||
|
* [joshgav](https://github.com/joshgav) -
|
||||||
|
**Josh Gavant** <<josh.gavant@outlook.com>>
|
||||||
|
* [mmarchini](https://github.com/mmarchini) -
|
||||||
|
**Mary Marchini** <<oss@mmarchini.me>> (she/her)
|
||||||
|
* [mscdex](https://github.com/mscdex) -
|
||||||
|
**Brian White** <<mscdex@mscdex.net>>
|
||||||
|
* [MylesBorins](https://github.com/MylesBorins) -
|
||||||
|
**Myles Borins** <<myles.borins@gmail.com>> (he/him)
|
||||||
|
* [nebrius](https://github.com/nebrius) -
|
||||||
|
**Bryan Hughes** <<bryan@nebri.us>>
|
||||||
|
* [ofrobots](https://github.com/ofrobots) -
|
||||||
|
**Ali Ijaz Sheikh** <<ofrobots@google.com>> (he/him)
|
||||||
|
* [orangemocha](https://github.com/orangemocha) -
|
||||||
|
**Alexis Campailla** <<orangemocha@nodejs.org>>
|
||||||
|
* [piscisaureus](https://github.com/piscisaureus) -
|
||||||
|
**Bert Belder** <<bertbelder@gmail.com>>
|
||||||
|
* [RaisinTen](https://github.com/RaisinTen) -
|
||||||
|
**Darshan Sen** <<raisinten@gmail.com>> (he/him)
|
||||||
|
* [rvagg](https://github.com/rvagg) -
|
||||||
|
**Rod Vagg** <<r@va.gg>>
|
||||||
|
* [sam-github](https://github.com/sam-github) -
|
||||||
|
**Sam Roberts** <<vieuxtech@gmail.com>>
|
||||||
|
* [shigeki](https://github.com/shigeki) -
|
||||||
|
**Shigeki Ohtsu** <<ohtsu@ohtsu.org>> (he/him)
|
||||||
|
* [thefourtheye](https://github.com/thefourtheye) -
|
||||||
|
**Sakthipriyan Vairamani** <<thechargingvolcano@gmail.com>> (he/him)
|
||||||
|
* [TimothyGu](https://github.com/TimothyGu) -
|
||||||
|
**Tiancheng "Timothy" Gu** <<timothygu99@gmail.com>> (he/him)
|
||||||
|
* [trevnorris](https://github.com/trevnorris) -
|
||||||
|
**Trevor Norris** <<trev.norris@gmail.com>>
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<!-- node-core-utils and find-inactive-collaborators.mjs depend on the format
|
||||||
|
of the collaborator list. If the format changes, those utilities need to be
|
||||||
|
tested and updated. -->
|
||||||
|
|
||||||
|
### Collaborators
|
||||||
|
|
||||||
|
* [abmusse](https://github.com/abmusse) -
|
||||||
|
**Abdirahim Musse** <<abdirahim.musse@ibm.com>>
|
||||||
|
* [addaleax](https://github.com/addaleax) -
|
||||||
|
**Anna Henningsen** <<anna@addaleax.net>> (she/her)
|
||||||
|
* [aduh95](https://github.com/aduh95) -
|
||||||
|
**Antoine du Hamel** <<duhamelantoine1995@gmail.com>> (he/him) - [Support me](https://github.com/sponsors/aduh95)
|
||||||
|
* [anonrig](https://github.com/anonrig) -
|
||||||
|
**Yagiz Nizipli** <<yagiz@nizipli.com>> (he/him) - [Support me](https://github.com/sponsors/anonrig)
|
||||||
|
* [atlowChemi](https://github.com/atlowChemi) -
|
||||||
|
**Chemi Atlow** <<chemi@atlow.co.il>> (he/him)
|
||||||
|
* [Ayase-252](https://github.com/Ayase-252) -
|
||||||
|
**Qingyu Deng** <<i@ayase-lab.com>>
|
||||||
|
* [bengl](https://github.com/bengl) -
|
||||||
|
**Bryan English** <<bryan@bryanenglish.com>> (he/him)
|
||||||
|
* [benjamingr](https://github.com/benjamingr) -
|
||||||
|
**Benjamin Gruenbaum** <<benjamingr@gmail.com>>
|
||||||
|
* [BethGriggs](https://github.com/BethGriggs) -
|
||||||
|
**Beth Griggs** <<bethanyngriggs@gmail.com>> (she/her)
|
||||||
|
* [bnb](https://github.com/bnb) -
|
||||||
|
**Tierney Cyren** <<hello@bnb.im>> (they/them)
|
||||||
|
* [bnoordhuis](https://github.com/bnoordhuis) -
|
||||||
|
**Ben Noordhuis** <<info@bnoordhuis.nl>>
|
||||||
|
* [BridgeAR](https://github.com/BridgeAR) -
|
||||||
|
**Ruben Bridgewater** <<ruben@bridgewater.de>> (he/him)
|
||||||
|
* [cclauss](https://github.com/cclauss) -
|
||||||
|
**Christian Clauss** <<cclauss@me.com>> (he/him)
|
||||||
|
* [cjihrig](https://github.com/cjihrig) -
|
||||||
|
**Colin Ihrig** <<cjihrig@gmail.com>> (he/him)
|
||||||
|
* [codebytere](https://github.com/codebytere) -
|
||||||
|
**Shelley Vohr** <<shelley.vohr@gmail.com>> (she/her)
|
||||||
|
* [cola119](https://github.com/cola119) -
|
||||||
|
**Kohei Ueno** <<kohei.ueno119@gmail.com>> (he/him)
|
||||||
|
* [daeyeon](https://github.com/daeyeon) -
|
||||||
|
**Daeyeon Jeong** <<daeyeon.dev@gmail.com>> (he/him)
|
||||||
|
* [dario-piotrowicz](https://github.com/dario-piotrowicz) -
|
||||||
|
**Dario Piotrowicz** <<dario.piotrowicz@gmail.com>> (he/him)
|
||||||
|
* [debadree25](https://github.com/debadree25) -
|
||||||
|
**Debadree Chatterjee** <<debadree333@gmail.com>> (he/him)
|
||||||
|
* [deokjinkim](https://github.com/deokjinkim) -
|
||||||
|
**Deokjin Kim** <<deokjin81.kim@gmail.com>> (he/him)
|
||||||
|
* [edsadr](https://github.com/edsadr) -
|
||||||
|
**Adrian Estrada** <<edsadr@gmail.com>> (he/him)
|
||||||
|
* [ErickWendel](https://github.com/ErickWendel) -
|
||||||
|
**Erick Wendel** <<erick.workspace@gmail.com>> (he/him)
|
||||||
|
* [Ethan-Arrowood](https://github.com/Ethan-Arrowood) -
|
||||||
|
**Ethan Arrowood** <<ethan@arrowood.dev>> (he/him)
|
||||||
|
* [F3n67u](https://github.com/F3n67u) -
|
||||||
|
**Feng Yu** <<F3n67u@outlook.com>> (he/him)
|
||||||
|
* [fhinkel](https://github.com/fhinkel) -
|
||||||
|
**Franziska Hinkelmann** <<franziska.hinkelmann@gmail.com>> (she/her)
|
||||||
|
* [Flarna](https://github.com/Flarna) -
|
||||||
|
**Gerhard Stöbich** <<deb2001-github@yahoo.de>> (he/they)
|
||||||
|
* [gabrielschulhof](https://github.com/gabrielschulhof) -
|
||||||
|
**Gabriel Schulhof** <<gabrielschulhof@gmail.com>>
|
||||||
|
* [gengjiawen](https://github.com/gengjiawen) -
|
||||||
|
**Jiawen Geng** <<technicalcute@gmail.com>>
|
||||||
|
* [GeoffreyBooth](https://github.com/GeoffreyBooth) -
|
||||||
|
**Geoffrey Booth** <<webadmin@geoffreybooth.com>> (he/him)
|
||||||
|
* [gireeshpunathil](https://github.com/gireeshpunathil) -
|
||||||
|
**Gireesh Punathil** <<gpunathi@in.ibm.com>> (he/him)
|
||||||
|
* [guybedford](https://github.com/guybedford) -
|
||||||
|
**Guy Bedford** <<guybedford@gmail.com>> (he/him)
|
||||||
|
* [H4ad](https://github.com/H4ad) -
|
||||||
|
**Vinícius Lourenço Claro Cardoso** <<contact@viniciusl.com.br>> (he/him)
|
||||||
|
* [HarshithaKP](https://github.com/HarshithaKP) -
|
||||||
|
**Harshitha K P** <<harshitha014@gmail.com>> (she/her)
|
||||||
|
* [himself65](https://github.com/himself65) -
|
||||||
|
**Zeyu "Alex" Yang** <<himself65@outlook.com>> (he/him)
|
||||||
|
* [jakecastelli](https://github.com/jakecastelli) -
|
||||||
|
**Jake Yuesong Li** <<jake.yuesong@gmail.com>> (he/him)
|
||||||
|
* [JakobJingleheimer](https://github.com/JakobJingleheimer) -
|
||||||
|
**Jacob Smith** <<jacob@frende.me>> (he/him)
|
||||||
|
* [jasnell](https://github.com/jasnell) -
|
||||||
|
**James M Snell** <<jasnell@gmail.com>> (he/him)
|
||||||
|
* [jazelly](https://github.com/jazelly) -
|
||||||
|
**Jason Zhang** <<xzha4350@gmail.com>> (he/him)
|
||||||
|
* [jkrems](https://github.com/jkrems) -
|
||||||
|
**Jan Krems** <<jan.krems@gmail.com>> (he/him)
|
||||||
|
* [joyeecheung](https://github.com/joyeecheung) -
|
||||||
|
**Joyee Cheung** <<joyeec9h3@gmail.com>> (she/her)
|
||||||
|
* [juanarbol](https://github.com/juanarbol) -
|
||||||
|
**Juan José Arboleda** <<soyjuanarbol@gmail.com>> (he/him)
|
||||||
|
* [JungMinu](https://github.com/JungMinu) -
|
||||||
|
**Minwoo Jung** <<nodecorelab@gmail.com>> (he/him)
|
||||||
|
* [KhafraDev](https://github.com/KhafraDev) -
|
||||||
|
**Matthew Aitken** <<maitken033380023@gmail.com>> (he/him)
|
||||||
|
* [kvakil](https://github.com/kvakil) -
|
||||||
|
**Keyhan Vakil** <<kvakil@sylph.kvakil.me>>
|
||||||
|
* [legendecas](https://github.com/legendecas) -
|
||||||
|
**Chengzhong Wu** <<legendecas@gmail.com>> (he/him)
|
||||||
|
* [lemire](https://github.com/lemire) -
|
||||||
|
**Daniel Lemire** <<daniel@lemire.me>>
|
||||||
|
* [Linkgoron](https://github.com/Linkgoron) -
|
||||||
|
**Nitzan Uziely** <<linkgoron@gmail.com>>
|
||||||
|
* [LiviaMedeiros](https://github.com/LiviaMedeiros) -
|
||||||
|
**LiviaMedeiros** <<livia@cirno.name>>
|
||||||
|
* [ljharb](https://github.com/ljharb) -
|
||||||
|
**Jordan Harband** <<ljharb@gmail.com>>
|
||||||
|
* [lpinca](https://github.com/lpinca) -
|
||||||
|
**Luigi Pinca** <<luigipinca@gmail.com>> (he/him)
|
||||||
|
* [lukekarrys](https://github.com/lukekarrys) -
|
||||||
|
**Luke Karrys** <<luke@lukekarrys.com>> (he/him)
|
||||||
|
* [Lxxyx](https://github.com/Lxxyx) -
|
||||||
|
**Zijian Liu** <<lxxyxzj@gmail.com>> (he/him)
|
||||||
|
* [marco-ippolito](https://github.com/marco-ippolito) -
|
||||||
|
**Marco Ippolito** <<marcoippolito54@gmail.com>> (he/him) - [Support me](https://github.com/sponsors/marco-ippolito)
|
||||||
|
* [marsonya](https://github.com/marsonya) -
|
||||||
|
**Akhil Marsonya** <<akhil.marsonya27@gmail.com>> (he/him)
|
||||||
|
* [MattiasBuelens](https://github.com/MattiasBuelens) -
|
||||||
|
**Mattias Buelens** <<mattias@buelens.com>> (he/him)
|
||||||
|
* [mcollina](https://github.com/mcollina) -
|
||||||
|
**Matteo Collina** <<matteo.collina@gmail.com>> (he/him) - [Support me](https://github.com/sponsors/mcollina)
|
||||||
|
* [meixg](https://github.com/meixg) -
|
||||||
|
**Xuguang Mei** <<meixuguang@gmail.com>> (he/him)
|
||||||
|
* [mhdawson](https://github.com/mhdawson) -
|
||||||
|
**Michael Dawson** <<midawson@redhat.com>> (he/him)
|
||||||
|
* [mildsunrise](https://github.com/mildsunrise) -
|
||||||
|
**Alba Mendez** <<me@alba.sh>> (she/her)
|
||||||
|
* [MoLow](https://github.com/MoLow) -
|
||||||
|
**Moshe Atlow** <<moshe@atlow.co.il>> (he/him)
|
||||||
|
* [MrJithil](https://github.com/MrJithil) -
|
||||||
|
**Jithil P Ponnan** <<jithil@outlook.com>> (he/him)
|
||||||
|
* [ovflowd](https://github.com/ovflowd) -
|
||||||
|
**Claudio Wunder** <<cwunder@gnome.org>> (he/they)
|
||||||
|
* [panva](https://github.com/panva) -
|
||||||
|
**Filip Skokan** <<panva.ip@gmail.com>> (he/him)
|
||||||
|
* [pimterry](https://github.com/pimterry) -
|
||||||
|
**Tim Perry** <<pimterry@gmail.com>> (he/him)
|
||||||
|
* [pmarchini](https://github.com/pmarchini)
|
||||||
|
**Pietro Marchini** <<pietro.marchini94@gmail.com>> (he/him)
|
||||||
|
* [Qard](https://github.com/Qard) -
|
||||||
|
**Stephen Belanger** <<admin@stephenbelanger.com>> (he/him)
|
||||||
|
* [RafaelGSS](https://github.com/RafaelGSS) -
|
||||||
|
**Rafael Gonzaga** <<rafael.nunu@hotmail.com>> (he/him)
|
||||||
|
* [richardlau](https://github.com/richardlau) -
|
||||||
|
**Richard Lau** <<rlau@redhat.com>>
|
||||||
|
* [rluvaton](https://github.com/rluvaton) -
|
||||||
|
**Raz Luvaton** <<rluvaton@gmail.com>> (he/him)
|
||||||
|
* [ronag](https://github.com/ronag) -
|
||||||
|
**Robert Nagy** <<ronagy@icloud.com>>
|
||||||
|
* [ruyadorno](https://github.com/ruyadorno) -
|
||||||
|
**Ruy Adorno** <<ruy@vlt.sh>> (he/him)
|
||||||
|
* [santigimeno](https://github.com/santigimeno) -
|
||||||
|
**Santiago Gimeno** <<santiago.gimeno@gmail.com>>
|
||||||
|
* [ShogunPanda](https://github.com/ShogunPanda) -
|
||||||
|
**Paolo Insogna** <<paolo@cowtech.it>> (he/him)
|
||||||
|
* [srl295](https://github.com/srl295) -
|
||||||
|
**Steven R Loomis** <<srl295@gmail.com>>
|
||||||
|
* [StefanStojanovic](https://github.com/StefanStojanovic) -
|
||||||
|
**Stefan Stojanovic** <<stefan.stojanovic@janeasystems.com>> (he/him)
|
||||||
|
* [sxa](https://github.com/sxa) -
|
||||||
|
**Stewart X Addison** <<sxa@redhat.com>> (he/him)
|
||||||
|
* [targos](https://github.com/targos) -
|
||||||
|
**Michaël Zasso** <<targos@protonmail.com>> (he/him)
|
||||||
|
* [theanarkh](https://github.com/theanarkh) -
|
||||||
|
**theanarkh** <<theratliter@gmail.com>> (he/him)
|
||||||
|
* [tniessen](https://github.com/tniessen) -
|
||||||
|
**Tobias Nießen** <<tniessen@tnie.de>> (he/him)
|
||||||
|
* [trivikr](https://github.com/trivikr) -
|
||||||
|
**Trivikram Kamat** <<trivikr.dev@gmail.com>>
|
||||||
|
* [Trott](https://github.com/Trott) -
|
||||||
|
**Rich Trott** <<rtrott@gmail.com>> (he/him)
|
||||||
|
* [UlisesGascon](https://github.com/UlisesGascon) -
|
||||||
|
**Ulises Gascón** <<ulisesgascongonzalez@gmail.com>> (he/him)
|
||||||
|
* [vmoroz](https://github.com/vmoroz) -
|
||||||
|
**Vladimir Morozov** <<vmorozov@microsoft.com>> (he/him)
|
||||||
|
* [VoltrexKeyva](https://github.com/VoltrexKeyva) -
|
||||||
|
**Mohammed Keyvanzadeh** <<mohammadkeyvanzade94@gmail.com>> (he/him)
|
||||||
|
* [zcbenz](https://github.com/zcbenz) -
|
||||||
|
**Cheng Zhao** <<zcbenz@gmail.com>> (he/him)
|
||||||
|
* [ZYSzys](https://github.com/ZYSzys) -
|
||||||
|
**Yongsheng Zhang** <<zyszys98@gmail.com>> (he/him)
|
||||||
|
|
||||||
|
<details>
|
||||||
|
|
||||||
|
<summary>Emeriti</summary>
|
||||||
|
|
||||||
|
<!-- find-inactive-collaborators.mjs depends on the format of the emeriti list.
|
||||||
|
If the format changes, those utilities need to be tested and updated. -->
|
||||||
|
|
||||||
|
### Collaborator emeriti
|
||||||
|
|
||||||
|
* [ak239](https://github.com/ak239) -
|
||||||
|
**Aleksei Koziatinskii** <<ak239spb@gmail.com>>
|
||||||
|
* [andrasq](https://github.com/andrasq) -
|
||||||
|
**Andras** <<andras@kinvey.com>>
|
||||||
|
* [AndreasMadsen](https://github.com/AndreasMadsen) -
|
||||||
|
**Andreas Madsen** <<amwebdk@gmail.com>> (he/him)
|
||||||
|
* [AnnaMag](https://github.com/AnnaMag) -
|
||||||
|
**Anna M. Kedzierska** <<anna.m.kedzierska@gmail.com>>
|
||||||
|
* [antsmartian](https://github.com/antsmartian) -
|
||||||
|
**Anto Aravinth** <<anto.aravinth.cse@gmail.com>> (he/him)
|
||||||
|
* [apapirovski](https://github.com/apapirovski) -
|
||||||
|
**Anatoli Papirovski** <<apapirovski@mac.com>> (he/him)
|
||||||
|
* [aqrln](https://github.com/aqrln) -
|
||||||
|
**Alexey Orlenko** <<eaglexrlnk@gmail.com>> (he/him)
|
||||||
|
* [AshCripps](https://github.com/AshCripps) -
|
||||||
|
**Ash Cripps** <<email@ashleycripps.co.uk>>
|
||||||
|
* [bcoe](https://github.com/bcoe) -
|
||||||
|
**Ben Coe** <<bencoe@gmail.com>> (he/him)
|
||||||
|
* [bmeck](https://github.com/bmeck) -
|
||||||
|
**Bradley Farias** <<bradley.meck@gmail.com>>
|
||||||
|
* [bmeurer](https://github.com/bmeurer) -
|
||||||
|
**Benedikt Meurer** <<benedikt.meurer@gmail.com>>
|
||||||
|
* [boneskull](https://github.com/boneskull) -
|
||||||
|
**Christopher Hiller** <<boneskull@boneskull.com>> (he/him)
|
||||||
|
* [brendanashworth](https://github.com/brendanashworth) -
|
||||||
|
**Brendan Ashworth** <<brendan.ashworth@me.com>>
|
||||||
|
* [bzoz](https://github.com/bzoz) -
|
||||||
|
**Bartosz Sosnowski** <<bartosz@janeasystems.com>>
|
||||||
|
* [calvinmetcalf](https://github.com/calvinmetcalf) -
|
||||||
|
**Calvin Metcalf** <<calvin.metcalf@gmail.com>>
|
||||||
|
* [ChALkeR](https://github.com/ChALkeR) -
|
||||||
|
**Сковорода Никита Андреевич** <<chalkerx@gmail.com>> (he/him)
|
||||||
|
* [chrisdickinson](https://github.com/chrisdickinson) -
|
||||||
|
**Chris Dickinson** <<christopher.s.dickinson@gmail.com>>
|
||||||
|
* [claudiorodriguez](https://github.com/claudiorodriguez) -
|
||||||
|
**Claudio Rodriguez** <<cjrodr@yahoo.com>>
|
||||||
|
* [danbev](https://github.com/danbev) -
|
||||||
|
**Daniel Bevenius** <<daniel.bevenius@gmail.com>> (he/him)
|
||||||
|
* [danielleadams](https://github.com/danielleadams) -
|
||||||
|
**Danielle Adams** <<adamzdanielle@gmail.com>> (she/her)
|
||||||
|
* [DavidCai1993](https://github.com/DavidCai1993) -
|
||||||
|
**David Cai** <<davidcai1993@yahoo.com>> (he/him)
|
||||||
|
* [davisjam](https://github.com/davisjam) -
|
||||||
|
**Jamie Davis** <<davisjam@vt.edu>> (he/him)
|
||||||
|
* [devnexen](https://github.com/devnexen) -
|
||||||
|
**David Carlier** <<devnexen@gmail.com>>
|
||||||
|
* [devsnek](https://github.com/devsnek) -
|
||||||
|
**Gus Caplan** <<me@gus.host>> (they/them)
|
||||||
|
* [digitalinfinity](https://github.com/digitalinfinity) -
|
||||||
|
**Hitesh Kanwathirtha** <<digitalinfinity@gmail.com>> (he/him)
|
||||||
|
* [dmabupt](https://github.com/dmabupt) -
|
||||||
|
**Xu Meng** <<dmabupt@gmail.com>> (he/him)
|
||||||
|
* [dnlup](https://github.com/dnlup)
|
||||||
|
**dnlup** <<dnlup.dev@gmail.com>>
|
||||||
|
* [eljefedelrodeodeljefe](https://github.com/eljefedelrodeodeljefe) -
|
||||||
|
**Robert Jefe Lindstaedt** <<robert.lindstaedt@gmail.com>>
|
||||||
|
* [estliberitas](https://github.com/estliberitas) -
|
||||||
|
**Alexander Makarenko** <<estliberitas@gmail.com>>
|
||||||
|
* [eugeneo](https://github.com/eugeneo) -
|
||||||
|
**Eugene Ostroukhov** <<eostroukhov@google.com>>
|
||||||
|
* [evanlucas](https://github.com/evanlucas) -
|
||||||
|
**Evan Lucas** <<evanlucas@me.com>> (he/him)
|
||||||
|
* [firedfox](https://github.com/firedfox) -
|
||||||
|
**Daniel Wang** <<wangyang0123@gmail.com>>
|
||||||
|
* [Fishrock123](https://github.com/Fishrock123) -
|
||||||
|
**Jeremiah Senkpiel** <<fishrock123@rocketmail.com>> (he/they)
|
||||||
|
* [gdams](https://github.com/gdams) -
|
||||||
|
**George Adams** <<gadams@microsoft.com>> (he/him)
|
||||||
|
* [geek](https://github.com/geek) -
|
||||||
|
**Wyatt Preul** <<wpreul@gmail.com>>
|
||||||
|
* [gibfahn](https://github.com/gibfahn) -
|
||||||
|
**Gibson Fahnestock** <<gibfahn@gmail.com>> (he/him)
|
||||||
|
* [glentiki](https://github.com/glentiki) -
|
||||||
|
**Glen Keane** <<glenkeane.94@gmail.com>> (he/him)
|
||||||
|
* [hashseed](https://github.com/hashseed) -
|
||||||
|
**Yang Guo** <<yangguo@chromium.org>> (he/him)
|
||||||
|
* [hiroppy](https://github.com/hiroppy) -
|
||||||
|
**Yuta Hiroto** <<hello@hiroppy.me>> (he/him)
|
||||||
|
* [iansu](https://github.com/iansu) -
|
||||||
|
**Ian Sutherland** <<ian@iansutherland.ca>>
|
||||||
|
* [iarna](https://github.com/iarna) -
|
||||||
|
**Rebecca Turner** <<me@re-becca.org>>
|
||||||
|
* [imran-iq](https://github.com/imran-iq) -
|
||||||
|
**Imran Iqbal** <<imran@imraniqbal.org>>
|
||||||
|
* [imyller](https://github.com/imyller) -
|
||||||
|
**Ilkka Myller** <<ilkka.myller@nodefield.com>>
|
||||||
|
* [indutny](https://github.com/indutny) -
|
||||||
|
**Fedor Indutny** <<fedor@indutny.com>>
|
||||||
|
* [isaacs](https://github.com/isaacs) -
|
||||||
|
**Isaac Z. Schlueter** <<i@izs.me>>
|
||||||
|
* [italoacasas](https://github.com/italoacasas) -
|
||||||
|
**Italo A. Casas** <<me@italoacasas.com>> (he/him)
|
||||||
|
* [JacksonTian](https://github.com/JacksonTian) -
|
||||||
|
**Jackson Tian** <<shyvo1987@gmail.com>>
|
||||||
|
* [jasongin](https://github.com/jasongin) -
|
||||||
|
**Jason Ginchereau** <<jasongin@microsoft.com>>
|
||||||
|
* [jbergstroem](https://github.com/jbergstroem) -
|
||||||
|
**Johan Bergström** <<bugs@bergstroem.nu>>
|
||||||
|
* [jdalton](https://github.com/jdalton) -
|
||||||
|
**John-David Dalton** <<john.david.dalton@gmail.com>>
|
||||||
|
* [jhamhader](https://github.com/jhamhader) -
|
||||||
|
**Yuval Brik** <<yuval@brik.org.il>>
|
||||||
|
* [joaocgreis](https://github.com/joaocgreis) -
|
||||||
|
**João Reis** <<reis@janeasystems.com>>
|
||||||
|
* [joesepi](https://github.com/joesepi) -
|
||||||
|
**Joe Sepi** <<sepi@joesepi.com>> (he/him)
|
||||||
|
* [joshgav](https://github.com/joshgav) -
|
||||||
|
**Josh Gavant** <<josh.gavant@outlook.com>>
|
||||||
|
* [julianduque](https://github.com/julianduque) -
|
||||||
|
**Julian Duque** <<julianduquej@gmail.com>> (he/him)
|
||||||
|
* [kfarnung](https://github.com/kfarnung) -
|
||||||
|
**Kyle Farnung** <<kfarnung@microsoft.com>> (he/him)
|
||||||
|
* [kunalspathak](https://github.com/kunalspathak) -
|
||||||
|
**Kunal Pathak** <<kunal.pathak@microsoft.com>>
|
||||||
|
* [kuriyosh](https://github.com/kuriyosh) -
|
||||||
|
**Yoshiki Kurihara** <<yosyos0306@gmail.com>> (he/him)
|
||||||
|
* [lance](https://github.com/lance) -
|
||||||
|
**Lance Ball** <<lball@redhat.com>> (he/him)
|
||||||
|
* [Leko](https://github.com/Leko) -
|
||||||
|
**Shingo Inoue** <<leko.noor@gmail.com>> (he/him)
|
||||||
|
* [lucamaraschi](https://github.com/lucamaraschi) -
|
||||||
|
**Luca Maraschi** <<luca.maraschi@gmail.com>> (he/him)
|
||||||
|
* [lundibundi](https://github.com/lundibundi) -
|
||||||
|
**Denys Otrishko** <<shishugi@gmail.com>> (he/him)
|
||||||
|
* [lxe](https://github.com/lxe) -
|
||||||
|
**Aleksey Smolenchuk** <<lxe@lxe.co>>
|
||||||
|
* [maclover7](https://github.com/maclover7) -
|
||||||
|
**Jon Moss** <<me@jonathanmoss.me>> (he/him)
|
||||||
|
* [mafintosh](https://github.com/mafintosh) -
|
||||||
|
**Mathias Buus** <<mathiasbuus@gmail.com>> (he/him)
|
||||||
|
* [matthewloring](https://github.com/matthewloring) -
|
||||||
|
**Matthew Loring** <<mattloring@google.com>>
|
||||||
|
* [Mesteery](https://github.com/Mesteery) -
|
||||||
|
**Mestery** <<mestery@protonmail.com>> (he/him)
|
||||||
|
* [micnic](https://github.com/micnic) -
|
||||||
|
**Nicu Micleușanu** <<micnic90@gmail.com>> (he/him)
|
||||||
|
* [mikeal](https://github.com/mikeal) -
|
||||||
|
**Mikeal Rogers** <<mikeal.rogers@gmail.com>>
|
||||||
|
* [miladfarca](https://github.com/miladfarca) -
|
||||||
|
**Milad Fa** <<mfarazma@redhat.com>> (he/him)
|
||||||
|
* [misterdjules](https://github.com/misterdjules) -
|
||||||
|
**Julien Gilli** <<jgilli@netflix.com>>
|
||||||
|
* [mmarchini](https://github.com/mmarchini) -
|
||||||
|
**Mary Marchini** <<oss@mmarchini.me>> (she/her)
|
||||||
|
* [monsanto](https://github.com/monsanto) -
|
||||||
|
**Christopher Monsanto** <<chris@monsan.to>>
|
||||||
|
* [MoonBall](https://github.com/MoonBall) -
|
||||||
|
**Chen Gang** <<gangc.cxy@foxmail.com>>
|
||||||
|
* [mscdex](https://github.com/mscdex) -
|
||||||
|
**Brian White** <<mscdex@mscdex.net>>
|
||||||
|
* [MylesBorins](https://github.com/MylesBorins) -
|
||||||
|
**Myles Borins** <<myles.borins@gmail.com>> (he/him)
|
||||||
|
* [not-an-aardvark](https://github.com/not-an-aardvark) -
|
||||||
|
**Teddy Katz** <<teddy.katz@gmail.com>> (he/him)
|
||||||
|
* [ofrobots](https://github.com/ofrobots) -
|
||||||
|
**Ali Ijaz Sheikh** <<ofrobots@google.com>> (he/him)
|
||||||
|
* [Olegas](https://github.com/Olegas) -
|
||||||
|
**Oleg Elifantiev** <<oleg@elifantiev.ru>>
|
||||||
|
* [orangemocha](https://github.com/orangemocha) -
|
||||||
|
**Alexis Campailla** <<orangemocha@nodejs.org>>
|
||||||
|
* [othiym23](https://github.com/othiym23) -
|
||||||
|
**Forrest L Norvell** <<ogd@aoaioxxysz.net>> (they/them/themself)
|
||||||
|
* [oyyd](https://github.com/oyyd) -
|
||||||
|
**Ouyang Yadong** <<oyydoibh@gmail.com>> (he/him)
|
||||||
|
* [petkaantonov](https://github.com/petkaantonov) -
|
||||||
|
**Petka Antonov** <<petka_antonov@hotmail.com>>
|
||||||
|
* [phillipj](https://github.com/phillipj) -
|
||||||
|
**Phillip Johnsen** <<johphi@gmail.com>>
|
||||||
|
* [piscisaureus](https://github.com/piscisaureus) -
|
||||||
|
**Bert Belder** <<bertbelder@gmail.com>>
|
||||||
|
* [pmq20](https://github.com/pmq20) -
|
||||||
|
**Minqi Pan** <<pmq2001@gmail.com>>
|
||||||
|
* [PoojaDurgad](https://github.com/PoojaDurgad) -
|
||||||
|
**Pooja D P** <<Pooja.D.P@ibm.com>> (she/her)
|
||||||
|
* [princejwesley](https://github.com/princejwesley) -
|
||||||
|
**Prince John Wesley** <<princejohnwesley@gmail.com>>
|
||||||
|
* [psmarshall](https://github.com/psmarshall) -
|
||||||
|
**Peter Marshall** <<petermarshall@chromium.org>> (he/him)
|
||||||
|
* [puzpuzpuz](https://github.com/puzpuzpuz) -
|
||||||
|
**Andrey Pechkurov** <<apechkurov@gmail.com>> (he/him)
|
||||||
|
* [RaisinTen](https://github.com/RaisinTen) -
|
||||||
|
**Darshan Sen** <<raisinten@gmail.com>> (he/him)
|
||||||
|
* [refack](https://github.com/refack) -
|
||||||
|
**Refael Ackermann (רפאל פלחי)** <<refack@gmail.com>> (he/him/הוא/אתה)
|
||||||
|
* [rexagod](https://github.com/rexagod) -
|
||||||
|
**Pranshu Srivastava** <<rexagod@gmail.com>> (he/him)
|
||||||
|
* [rickyes](https://github.com/rickyes) -
|
||||||
|
**Ricky Zhou** <<0x19951125@gmail.com>> (he/him)
|
||||||
|
* [rlidwka](https://github.com/rlidwka) -
|
||||||
|
**Alex Kocharin** <<alex@kocharin.ru>>
|
||||||
|
* [rmg](https://github.com/rmg) -
|
||||||
|
**Ryan Graham** <<r.m.graham@gmail.com>>
|
||||||
|
* [robertkowalski](https://github.com/robertkowalski) -
|
||||||
|
**Robert Kowalski** <<rok@kowalski.gd>>
|
||||||
|
* [romankl](https://github.com/romankl) -
|
||||||
|
**Roman Klauke** <<romaaan.git@gmail.com>>
|
||||||
|
* [ronkorving](https://github.com/ronkorving) -
|
||||||
|
**Ron Korving** <<ron@ronkorving.nl>>
|
||||||
|
* [RReverser](https://github.com/RReverser) -
|
||||||
|
**Ingvar Stepanyan** <<me@rreverser.com>>
|
||||||
|
* [rubys](https://github.com/rubys) -
|
||||||
|
**Sam Ruby** <<rubys@intertwingly.net>>
|
||||||
|
* [rvagg](https://github.com/rvagg) -
|
||||||
|
**Rod Vagg** <<rod@vagg.org>>
|
||||||
|
* [ryzokuken](https://github.com/ryzokuken) -
|
||||||
|
**Ujjwal Sharma** <<ryzokuken@disroot.org>> (he/him)
|
||||||
|
* [saghul](https://github.com/saghul) -
|
||||||
|
**Saúl Ibarra Corretgé** <<s@saghul.net>>
|
||||||
|
* [sam-github](https://github.com/sam-github) -
|
||||||
|
**Sam Roberts** <<vieuxtech@gmail.com>>
|
||||||
|
* [sebdeckers](https://github.com/sebdeckers) -
|
||||||
|
**Sebastiaan Deckers** <<sebdeckers83@gmail.com>>
|
||||||
|
* [seishun](https://github.com/seishun) -
|
||||||
|
**Nikolai Vavilov** <<vvnicholas@gmail.com>>
|
||||||
|
* [shigeki](https://github.com/shigeki) -
|
||||||
|
**Shigeki Ohtsu** <<ohtsu@ohtsu.org>> (he/him)
|
||||||
|
* [shisama](https://github.com/shisama) -
|
||||||
|
**Masashi Hirano** <<shisama07@gmail.com>> (he/him)
|
||||||
|
* [silverwind](https://github.com/silverwind) -
|
||||||
|
**Roman Reiss** <<me@silverwind.io>>
|
||||||
|
* [starkwang](https://github.com/starkwang) -
|
||||||
|
**Weijia Wang** <<starkwang@126.com>>
|
||||||
|
* [stefanmb](https://github.com/stefanmb) -
|
||||||
|
**Stefan Budeanu** <<stefan@budeanu.com>>
|
||||||
|
* [tellnes](https://github.com/tellnes) -
|
||||||
|
**Christian Tellnes** <<christian@tellnes.no>>
|
||||||
|
* [thefourtheye](https://github.com/thefourtheye) -
|
||||||
|
**Sakthipriyan Vairamani** <<thechargingvolcano@gmail.com>> (he/him)
|
||||||
|
* [thlorenz](https://github.com/thlorenz) -
|
||||||
|
**Thorsten Lorenz** <<thlorenz@gmx.de>>
|
||||||
|
* [TimothyGu](https://github.com/TimothyGu) -
|
||||||
|
**Tiancheng "Timothy" Gu** <<timothygu99@gmail.com>> (he/him)
|
||||||
|
* [trevnorris](https://github.com/trevnorris) -
|
||||||
|
**Trevor Norris** <<trev.norris@gmail.com>>
|
||||||
|
* [tunniclm](https://github.com/tunniclm) -
|
||||||
|
**Mike Tunnicliffe** <<m.j.tunnicliffe@gmail.com>>
|
||||||
|
* [vdeturckheim](https://github.com/vdeturckheim) -
|
||||||
|
**Vladimir de Turckheim** <<vlad2t@hotmail.com>> (he/him)
|
||||||
|
* [vkurchatkin](https://github.com/vkurchatkin) -
|
||||||
|
**Vladimir Kurchatkin** <<vladimir.kurchatkin@gmail.com>>
|
||||||
|
* [vsemozhetbyt](https://github.com/vsemozhetbyt) -
|
||||||
|
**Vse Mozhet Byt** <<vsemozhetbyt@gmail.com>> (he/him)
|
||||||
|
* [watilde](https://github.com/watilde) -
|
||||||
|
**Daijiro Wachi** <<daijiro.wachi@gmail.com>> (he/him)
|
||||||
|
* [watson](https://github.com/watson) -
|
||||||
|
**Thomas Watson** <<w@tson.dk>>
|
||||||
|
* [whitlockjc](https://github.com/whitlockjc) -
|
||||||
|
**Jeremy Whitlock** <<jwhitlock@apache.org>>
|
||||||
|
* [XadillaX](https://github.com/XadillaX) -
|
||||||
|
**Khaidi Chu** <<i@2333.moe>> (he/him)
|
||||||
|
* [yashLadha](https://github.com/yashLadha) -
|
||||||
|
**Yash Ladha** <<yash@yashladha.in>> (he/him)
|
||||||
|
* [yhwang](https://github.com/yhwang) -
|
||||||
|
**Yihong Wang** <<yh.wang@ibm.com>>
|
||||||
|
* [yorkie](https://github.com/yorkie) -
|
||||||
|
**Yorkie Liu** <<yorkiefixer@gmail.com>>
|
||||||
|
* [yosuke-furukawa](https://github.com/yosuke-furukawa) -
|
||||||
|
**Yosuke Furukawa** <<yosuke.furukawa@gmail.com>>
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<!--lint enable prohibited-strings-->
|
||||||
|
|
||||||
|
Collaborators follow the [Collaborator Guide](./doc/contributing/collaborator-guide.md) in
|
||||||
|
maintaining the Node.js project.
|
||||||
|
|
||||||
|
### Triagers
|
||||||
|
|
||||||
|
* [atlowChemi](https://github.com/atlowChemi) -
|
||||||
|
**Chemi Atlow** <<chemi@atlow.co.il>> (he/him)
|
||||||
|
* [Ayase-252](https://github.com/Ayase-252) -
|
||||||
|
**Qingyu Deng** <<i@ayase-lab.com>>
|
||||||
|
* [bmuenzenmeyer](https://github.com/bmuenzenmeyer) -
|
||||||
|
**Brian Muenzenmeyer** <<brian.muenzenmeyer@gmail.com>> (he/him)
|
||||||
|
* [CanadaHonk](https://github.com/CanadaHonk) -
|
||||||
|
**Oliver Medhurst** <<honk@goose.icu>> (they/them)
|
||||||
|
* [daeyeon](https://github.com/daeyeon) -
|
||||||
|
**Daeyeon Jeong** <<daeyeon.dev@gmail.com>> (he/him)
|
||||||
|
* [F3n67u](https://github.com/F3n67u) -
|
||||||
|
**Feng Yu** <<F3n67u@outlook.com>> (he/him)
|
||||||
|
* [gireeshpunathil](https://github.com/gireeshpunathil) -
|
||||||
|
**Gireesh Punathil** <<gpunathi@in.ibm.com>> (he/him)
|
||||||
|
* [gurgunday](https://github.com/gurgunday) -
|
||||||
|
**Gürgün Dayıoğlu** <<hey@gurgun.day>>
|
||||||
|
* [iam-frankqiu](https://github.com/iam-frankqiu) -
|
||||||
|
**Frank Qiu** <<iam.frankqiu@gmail.com>> (he/him)
|
||||||
|
* [KevinEady](https://github.com/KevinEady) -
|
||||||
|
**Kevin Eady** <<kevin.c.eady@gmail.com>> (he/him)
|
||||||
|
* [kvakil](https://github.com/kvakil) -
|
||||||
|
**Keyhan Vakil** <<kvakil@sylph.kvakil.me>>
|
||||||
|
* [marsonya](https://github.com/marsonya) -
|
||||||
|
**Akhil Marsonya** <<akhil.marsonya27@gmail.com>> (he/him)
|
||||||
|
* [meixg](https://github.com/meixg) -
|
||||||
|
**Xuguang Mei** <<meixuguang@gmail.com>> (he/him)
|
||||||
|
* [mertcanaltin](https://github.com/mertcanaltin) -
|
||||||
|
**Mert Can Altin** <<mertgold60@gmail.com>>
|
||||||
|
* [preveen-stack](https://github.com/preveen-stack) -
|
||||||
|
**Preveen Padmanabhan** <<wide4head@gmail.com>> (he/him)
|
||||||
|
* [RedYetiDev](https://github.com/redyetidev) -
|
||||||
|
**Aviv Keller** <<redyetidev@gmail.com>> (they/them)
|
||||||
|
* [VoltrexKeyva](https://github.com/VoltrexKeyva) -
|
||||||
|
**Mohammed Keyvanzadeh** <<mohammadkeyvanzade94@gmail.com>> (he/him)
|
||||||
|
|
||||||
|
Triagers follow the [Triage Guide](./doc/contributing/issues.md#triaging-a-bug-report) when
|
||||||
|
responding to new issues.
|
||||||
|
|
||||||
|
### Release keys
|
||||||
|
|
||||||
|
Primary GPG keys for Node.js Releasers (some Releasers sign with subkeys):
|
||||||
|
|
||||||
|
* **Antoine du Hamel** <<duhamelantoine1995@gmail.com>>
|
||||||
|
`C0D6248439F1D5604AAFFB4021D900FFDB233756`
|
||||||
|
* **Juan José Arboleda** <<soyjuanarbol@gmail.com>>
|
||||||
|
`DD792F5973C6DE52C432CBDAC77ABFA00DDBF2B7`
|
||||||
|
* **Marco Ippolito** <<marcoippolito54@gmail.com>>
|
||||||
|
`CC68F5A3106FF448322E48ED27F5E38D5B0A215F`
|
||||||
|
* **Michaël Zasso** <<targos@protonmail.com>>
|
||||||
|
`8FCCA13FEF1D0C2E91008E09770F7A9A5AE15600`
|
||||||
|
* **Rafael Gonzaga** <<rafael.nunu@hotmail.com>>
|
||||||
|
`890C08DB8579162FEE0DF9DB8BEAB4DFCF555EF4`
|
||||||
|
* **Richard Lau** <<rlau@redhat.com>>
|
||||||
|
`C82FA3AE1CBEDC6BE46B9360C43CEC45C17AB93C`
|
||||||
|
* **Ruy Adorno** <<ruyadorno@hotmail.com>>
|
||||||
|
`108F52B48DB57BB0CC439B2997B01419BD92F80A`
|
||||||
|
* **Ulises Gascón** <<ulisesgascongonzalez@gmail.com>>
|
||||||
|
`A363A499291CBBC940DD62E41F10027AF002F8B0`
|
||||||
|
|
||||||
|
To import the full set of trusted release keys (including subkeys possibly used
|
||||||
|
to sign releases):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys C0D6248439F1D5604AAFFB4021D900FFDB233756 # Antoine du Hamel
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys DD792F5973C6DE52C432CBDAC77ABFA00DDBF2B7 # Juan José Arboleda
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys CC68F5A3106FF448322E48ED27F5E38D5B0A215F # Marco Ippolito
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys 8FCCA13FEF1D0C2E91008E09770F7A9A5AE15600 # Michaël Zasso
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys 890C08DB8579162FEE0DF9DB8BEAB4DFCF555EF4 # Rafael Gonzaga
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys C82FA3AE1CBEDC6BE46B9360C43CEC45C17AB93C # Richard Lau
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys 108F52B48DB57BB0CC439B2997B01419BD92F80A # Ruy Adorno
|
||||||
|
gpg --keyserver hkps://keys.openpgp.org --recv-keys A363A499291CBBC940DD62E41F10027AF002F8B0 # Ulises Gascón
|
||||||
|
```
|
||||||
|
|
||||||
|
See [Verifying binaries](#verifying-binaries) for how to use these keys to
|
||||||
|
verify a downloaded file.
|
||||||
|
|
||||||
|
<details>
|
||||||
|
|
||||||
|
<summary>Other keys used to sign some previous releases</summary>
|
||||||
|
|
||||||
|
* **Beth Griggs** <<bethanyngriggs@gmail.com>>
|
||||||
|
`4ED778F539E3634C779C87C6D7062848A1AB005C`
|
||||||
|
* **Bryan English** <<bryan@bryanenglish.com>>
|
||||||
|
`141F07595B7B3FFE74309A937405533BE57C7D57`
|
||||||
|
* **Chris Dickinson** <<christopher.s.dickinson@gmail.com>>
|
||||||
|
`9554F04D7259F04124DE6B476D5A82AC7E37093B`
|
||||||
|
* **Colin Ihrig** <<cjihrig@gmail.com>>
|
||||||
|
`94AE36675C464D64BAFA68DD7434390BDBE9B9C5`
|
||||||
|
* **Danielle Adams** <<adamzdanielle@gmail.com>>
|
||||||
|
`1C050899334244A8AF75E53792EF661D867B9DFA`
|
||||||
|
`74F12602B6F1C4E913FAA37AD3A89613643B6201`
|
||||||
|
* **Evan Lucas** <<evanlucas@me.com>>
|
||||||
|
`B9AE9905FFD7803F25714661B63B535A4C206CA9`
|
||||||
|
* **Gibson Fahnestock** <<gibfahn@gmail.com>>
|
||||||
|
`77984A986EBC2AA786BC0F66B01FBB92821C587A`
|
||||||
|
* **Isaac Z. Schlueter** <<i@izs.me>>
|
||||||
|
`93C7E9E91B49E432C2F75674B0A78B0A6C481CF6`
|
||||||
|
* **Italo A. Casas** <<me@italoacasas.com>>
|
||||||
|
`56730D5401028683275BD23C23EFEFE93C4CFFFE`
|
||||||
|
* **James M Snell** <<jasnell@keybase.io>>
|
||||||
|
`71DCFD284A79C3B38668286BC97EC7A07EDE3FC1`
|
||||||
|
* **Jeremiah Senkpiel** <<fishrock@keybase.io>>
|
||||||
|
`FD3A5288F042B6850C66B31F09FE44734EB7990E`
|
||||||
|
* **Juan José Arboleda** <<soyjuanarbol@gmail.com>>
|
||||||
|
`61FC681DFB92A079F1685E77973F295594EC4689`
|
||||||
|
* **Julien Gilli** <<jgilli@fastmail.fm>>
|
||||||
|
`114F43EE0176B71C7BC219DD50A3051F888C628D`
|
||||||
|
* **Myles Borins** <<myles.borins@gmail.com>>
|
||||||
|
`C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8`
|
||||||
|
* **Rod Vagg** <<rod@vagg.org>>
|
||||||
|
`DD8F2338BAE7501E3DD5AC78C273792F7D83545D`
|
||||||
|
* **Ruben Bridgewater** <<ruben@bridgewater.de>>
|
||||||
|
`A48C2BEE680E841632CD4E44F07496B3EB3C1762`
|
||||||
|
* **Shelley Vohr** <<shelley.vohr@gmail.com>>
|
||||||
|
`B9E2F5981AA6E0CD28160D9FF13993A75599653C`
|
||||||
|
* **Timothy J Fontaine** <<tjfontaine@gmail.com>>
|
||||||
|
`7937DFD2AB06298B2293C3187D33FF9D0246406D`
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
### Security release stewards
|
||||||
|
|
||||||
|
When possible, the commitment to take slots in the
|
||||||
|
security release steward rotation is made by companies in order
|
||||||
|
to ensure individuals who act as security stewards have the
|
||||||
|
support and recognition from their employer to be able to
|
||||||
|
prioritize security releases. Security release stewards manage security
|
||||||
|
releases on a rotation basis as outlined in the
|
||||||
|
[security release process](./doc/contributing/security-release-process.md).
|
||||||
|
|
||||||
|
* [Datadog](https://www.datadoghq.com/)
|
||||||
|
* [bengl](https://github.com/bengl) -
|
||||||
|
**Bryan English** <<bryan@bryanenglish.com>> (he/him)
|
||||||
|
* [NodeSource](https://nodesource.com/)
|
||||||
|
* [juanarbol](https://github.com/juanarbol) -
|
||||||
|
**Juan José Arboleda** <<soyjuanarbol@gmail.com>> (he/him)
|
||||||
|
* [RafaelGSS](https://github.com/RafaelGSS) -
|
||||||
|
**Rafael Gonzaga** <<rafael.nunu@hotmail.com>> (he/him)
|
||||||
|
* [Platformatic](https://platformatic.dev/)
|
||||||
|
* [mcollina](https://github.com/mcollina) -
|
||||||
|
**Matteo Collina** <<matteo.collina@gmail.com>> (he/him)
|
||||||
|
* [Red Hat](https://redhat.com) / [IBM](https://ibm.com)
|
||||||
|
* [joesepi](https://github.com/joesepi) -
|
||||||
|
**Joe Sepi** <<joesepi@ibm.com>> (he/him)
|
||||||
|
* [mhdawson](https://github.com/mhdawson) -
|
||||||
|
**Michael Dawson** <<midawson@redhat.com>> (he/him)
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Node.js is available under the
|
||||||
|
[MIT License](https://opensource.org/licenses/MIT). Node.js also includes
|
||||||
|
external libraries that are available under a variety of licenses. See
|
||||||
|
[LICENSE](https://github.com/nodejs/node/blob/HEAD/LICENSE) for the full
|
||||||
|
license text.
|
||||||
|
|
||||||
|
[Code of Conduct]: https://github.com/nodejs/admin/blob/HEAD/CODE_OF_CONDUCT.md
|
||||||
|
[Contributing to the project]: CONTRIBUTING.md
|
||||||
|
[Node.js website]: https://nodejs.org/
|
||||||
|
[OpenJS Foundation]: https://openjsf.org/
|
||||||
|
[Strategic initiatives]: doc/contributing/strategic-initiatives.md
|
||||||
|
[Technical values and prioritization]: doc/contributing/technical-values.md
|
||||||
|
[Working Groups]: https://github.com/nodejs/TSC/blob/HEAD/WORKING_GROUPS.md
|
||||||
12
domainCheck/tools/node-v20.19.4-win-x64/corepack
Normal file
12
domainCheck/tools/node-v20.19.4-win-x64/corepack
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
|
||||||
|
|
||||||
|
case `uname` in
|
||||||
|
*CYGWIN*) basedir=`cygpath -w "$basedir"`;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
if [ -x "$basedir/node" ]; then
|
||||||
|
exec "$basedir/node" "$basedir/node_modules/corepack/dist/corepack.js" "$@"
|
||||||
|
else
|
||||||
|
exec node "$basedir/node_modules/corepack/dist/corepack.js" "$@"
|
||||||
|
fi
|
||||||
7
domainCheck/tools/node-v20.19.4-win-x64/corepack.cmd
Normal file
7
domainCheck/tools/node-v20.19.4-win-x64/corepack.cmd
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
@SETLOCAL
|
||||||
|
@IF EXIST "%~dp0\node.exe" (
|
||||||
|
"%~dp0\node.exe" "%~dp0\node_modules\corepack\dist\corepack.js" %*
|
||||||
|
) ELSE (
|
||||||
|
@SET PATHEXT=%PATHEXT:;.JS;=;%
|
||||||
|
node "%~dp0\node_modules\corepack\dist\corepack.js" %*
|
||||||
|
)
|
||||||
66
domainCheck/tools/node-v20.19.4-win-x64/install_tools.bat
Normal file
66
domainCheck/tools/node-v20.19.4-win-x64/install_tools.bat
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
@echo off
|
||||||
|
|
||||||
|
setlocal
|
||||||
|
title Install Additional Tools for Node.js
|
||||||
|
|
||||||
|
cls
|
||||||
|
|
||||||
|
echo ====================================================
|
||||||
|
echo Tools for Node.js Native Modules Installation Script
|
||||||
|
echo ====================================================
|
||||||
|
echo.
|
||||||
|
echo This script will install Python and the Visual Studio Build Tools, necessary
|
||||||
|
echo to compile Node.js native modules. Note that Chocolatey and required Windows
|
||||||
|
echo updates will also be installed.
|
||||||
|
echo.
|
||||||
|
echo This will require about 3 GiB of free disk space, plus any space necessary to
|
||||||
|
echo install Windows updates. This will take a while to run.
|
||||||
|
echo.
|
||||||
|
echo Please close all open programs for the duration of the installation. If the
|
||||||
|
echo installation fails, please ensure Windows is fully updated, reboot your
|
||||||
|
echo computer and try to run this again. This script can be found in the
|
||||||
|
echo Start menu under Node.js.
|
||||||
|
echo.
|
||||||
|
echo You can close this window to stop now. Detailed instructions to install these
|
||||||
|
echo tools manually are available at https://github.com/nodejs/node-gyp#on-windows
|
||||||
|
echo.
|
||||||
|
pause
|
||||||
|
|
||||||
|
cls
|
||||||
|
|
||||||
|
REM Adapted from https://github.com/Microsoft/windows-dev-box-setup-scripts/blob/79bbe5bdc4867088b3e074f9610932f8e4e192c2/README.md#legal
|
||||||
|
echo Using this script downloads third party software
|
||||||
|
echo ------------------------------------------------
|
||||||
|
echo This script will direct to Chocolatey to install packages. By using
|
||||||
|
echo Chocolatey to install a package, you are accepting the license for the
|
||||||
|
echo application, executable(s), or other artifacts delivered to your machine as a
|
||||||
|
echo result of a Chocolatey install. This acceptance occurs whether you know the
|
||||||
|
echo license terms or not. Read and understand the license terms of the packages
|
||||||
|
echo being installed and their dependencies prior to installation:
|
||||||
|
echo - https://chocolatey.org/packages/chocolatey
|
||||||
|
echo - https://chocolatey.org/packages/python
|
||||||
|
echo - https://chocolatey.org/packages/visualstudio2019-workload-vctools
|
||||||
|
echo.
|
||||||
|
echo This script is provided AS-IS without any warranties of any kind
|
||||||
|
echo ----------------------------------------------------------------
|
||||||
|
echo Chocolatey has implemented security safeguards in their process to help
|
||||||
|
echo protect the community from malicious or pirated software, but any use of this
|
||||||
|
echo script is at your own risk. Please read the Chocolatey's legal terms of use
|
||||||
|
echo as well as how the community repository for Chocolatey.org is maintained.
|
||||||
|
echo.
|
||||||
|
pause
|
||||||
|
|
||||||
|
cls
|
||||||
|
|
||||||
|
"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" ^
|
||||||
|
-NoProfile ^
|
||||||
|
-InputFormat None ^
|
||||||
|
-ExecutionPolicy Bypass ^
|
||||||
|
-Command Start-Process ^
|
||||||
|
'%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe' ^
|
||||||
|
-ArgumentList '-NoProfile -InputFormat None -ExecutionPolicy Bypass -Command ^
|
||||||
|
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; ^
|
||||||
|
iex ((New-Object System.Net.WebClient).DownloadString(''https://chocolatey.org/install.ps1'')); ^
|
||||||
|
choco upgrade -y python visualstudio2019-workload-vctools; ^
|
||||||
|
Read-Host ''Type ENTER to exit'' ' ^
|
||||||
|
-Verb RunAs
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user