convert domainCheck to regular directory

This commit is contained in:
Your Name
2026-04-16 13:33:06 +08:00
parent e406d73334
commit d37c444929
107 changed files with 254185 additions and 1 deletions

Submodule domainCheck deleted from 83ab7a79e8

12
domainCheck/.env.example Normal file
View File

@@ -0,0 +1,12 @@
# 数据库配置
DB_HOST=localhost
DB_PORT=5432
DB_DATABASE=domain_scan_db
DB_USER=postgres
DB_PASSWORD=postgres
# Redis配置
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
REDIS_DB=0

10
domainCheck/.gitignore vendored Normal file
View File

@@ -0,0 +1,10 @@
# 忽略整个目录
/node_modules/
/build/
/dist/
# 忽略所有压缩文件
*.spec
.env
.venv
.idea

View File

@@ -0,0 +1,221 @@
# 域名工具项目部署与使用文档
## 1. 项目概述
域名工具是一个用于域名筛选、检测和管理的综合性工具,主要功能包括:
- 域名批量导入
- 域名筛选与查询
- 域名状态检测(注册状态、使用状态、检测状态等)
- 域名详细信息检测(备案历史、快照年份、百度收录等)
- 域名数据导出Excel、CSV
- 批量更新域名状态
## 2. 环境要求
### 2.1 系统要求
- Windows 10/11 64位操作系统
- 至少4GB内存
- 至少500MB磁盘空间
### 2.2 软件依赖
- Python 3.9+
- PostgreSQL 12+
- Redis用于缓存+布隆过滤器(用于域名去重)
## 3. 安装步骤
### 3.1 克隆项目
```bash
git clone <项目仓库地址>
cd domainScanDemo
```
### 3.2 创建虚拟环境
```bash
python -m venv .venv
```
### 3.3 激活虚拟环境
```bash
# Windows
.venv\Scripts\activate
# Linux/Mac
source .venv/bin/activate
```
### 3.4 安装依赖
```bash
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### 3.5 安装额外依赖用于Excel导出
```bash
pip install openpyxl
```
## 4. 配置说明
### 4.1 环境变量配置
复制 `.env.example` 文件为 `.env`,并根据实际情况修改配置:
```env
# 数据库连接信息
DATABASE_URL=postgresql://username:password@localhost:5432/domain_db
# Redis连接信息可选
REDIS_URL=redis://localhost:6379/0
# 日志配置
LOG_LEVEL=INFO
LOG_FILE=logs/app.log
# 线程配置
MAX_WORKERS=10
# 检测配置
DETECT_TIMEOUT=30
```
### 4.2 其他配置文件
- `detect_options.json` - 检测选项配置
- `proxy_config.json` - 代理配置
- `domain_suffixes.json` - 域名后缀配置
- `thread_count.json` - 线程数量配置
## 5. 数据库初始化
### 5.1 创建数据库
在PostgreSQL中创建数据库
```sql
CREATE DATABASE domain_db;
```
### 5.2 初始化表结构
运行数据库初始化脚本:
```bash
python init_database.py
```
此脚本会创建以下表:
- `domains` - 域名基本信息表
- `domain_detections` - 域名检测详情表
- `sensitive_words` - 敏感词表
### 5.3 添加敏感词(可选)
```bash
python add_sensitive_words_table.py
```
## 6. 运行项目
### 6.1 启动主应用
```bash
python app/main.py
```
### 6.2 启动检测端(可选,用于分布式检测)
```bash
python detect_worker.py
```
## 7. 打包指南
### 7.1 打包主应用
```bash
pyinstaller --onefile --windowed --add-data="app/sdk_leg.js;detect" --add-data="app/sdk_leg_env.js;detect" --add-data="app/chinaz.js;detect" --add-data="detect/module/crack_geetest2x.js;detect/module" --add-data="favicon.ico;." --add-data="new_logo.svg;." --add-data=".env;." --icon="favicon.ico" --name="域名工具" app/main.py
```
### 7.2 打包检测端
```bash
pyinstaller --onefile --windowed --add-data="detect/sdk_leg.js;detect" --add-data="detect/sdk_leg_env.js;detect" --add-data="detect/chinaz.js;detect" --add-data="detect/module/crack_geetest2x.js;detect/module" --add-data="favicon2.ico;." --add-data="new_logo.svg;." --add-data=".env;." --icon="favicon2.ico" --name="域名检测端" detect_worker.py
```
### 7.3 打包结果
打包完成后,可执行文件会生成在 `dist` 目录中:
- `域名工具.exe` - 主应用
- `域名检测端.exe` - 检测端
## 8. 使用说明
### 8.1 域名导入
1. 点击"域名导入"标签页
2. 选择导入方式(文件导入或聚名网爬取)
3. 选择文件或输入聚名网URL
4. 点击"开始导入"按钮
5. 等待导入完成
### 8.2 域名筛选
1. 点击"域名筛选"标签页
2. 设置筛选条件(注册状态、使用状态、检测状态等)
3. 点击"查询"按钮
4. 查看查询结果
5. 可选择域名进行批量更新或导出
### 8.3 域名检测
1. 确保检测端已启动
2. 在主应用中选择需要检测的域名
3. 点击"开始检测"按钮
4. 等待检测完成
5. 查看检测结果
### 8.4 数据导出
1. 在域名筛选界面选择需要导出的域名
2. 点击"导出"按钮
3. 选择导出格式Excel或CSV
4. 选择保存路径
5. 等待导出完成
## 9. 常见问题
### 9.1 数据库连接失败
- 检查 `.env` 文件中的数据库连接信息是否正确
- 确保 PostgreSQL 服务已启动
- 确保数据库用户有足够的权限
### 9.2 Redis 连接失败
- 检查 `.env` 文件中的 Redis 连接信息是否正确
- 确保 Redis 服务已启动
- 如果不需要 Redis 缓存,可以忽略此警告
### 9.3 检测失败
- 检查网络连接是否正常
- 检查代理配置是否正确
- 检查检测端是否已启动
### 9.4 导出失败
- 确保已安装 `openpyxl`用于Excel导出
- 确保导出路径有写入权限
- 检查导出文件格式是否正确
## 10. 技术支持
如果遇到问题,请查看日志文件 `logs/app.log` 获取详细错误信息,或联系技术支持。
## 11. 版本信息
- 项目版本1.0.0
- 最后更新2026-04-14
---
**注意**:本文档仅供参考,具体配置和使用方法可能因环境不同而有所差异。

View File

@@ -0,0 +1,192 @@
# acw_sc__v2 生成器
从混淆的HTML代码中提取的 `acw_sc__v2` cookie 生成函数。
## 文件说明
### JavaScript版本
- **acw_sc__v2_generator.js** - JavaScript版本的生成函数可直接在浏览器控制台中使用
### Python版本
- **acw_sc__v2_fixed.py** - 简化版Python生成器适合快速使用
- **acw_sc__v2_complete.py** - 完整版Python生成器包含更多功能和错误处理
- **acw_sc__v2_simple.py** - 最简化版本,直接使用硬编码密钥
## 使用方法
### JavaScript版本
在浏览器控制台中执行:
```javascript
// 加载acw_sc__v2_generator.js文件后
var arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0';
var acwScV2 = generateAcwScV2(arg1);
console.log('acw_sc__v2的值:', acwScV2);
// 设置cookie可选
setAcwScV2Cookie(acwScV2);
```
### Python版本
#### 简化版acw_sc__v2_fixed.py
```python
from acw_sc__v2_fixed import generate_acw_sc_v2
arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0'
acw_sc_v2 = generate_acw_sc_v2(arg1)
print(f'acw_sc__v2的值: {acw_sc_v2}')
```
#### 完整版acw_sc__v2_complete.py
```python
from acw_sc__v2_complete import AcwScV2Generator, generate_acw_sc_v2
# 方法1使用便捷函数
arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0'
result = generate_acw_sc_v2(arg1)
print(f'acw_sc__v2的值: {result}')
# 方法2使用生成器类
generator = AcwScV2Generator()
result = generator.generate(arg1)
print(f'acw_sc__v2的值: {result}')
# 方法3批量生成
test_args = [
'1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0',
'A13D8B4174A10C57667C1CD92E54D01F449CA76E',
'F0286855D50CD9E96C1CDF3D4A913DB01EEB3321'
]
batch_results = generator.generate_batch(test_args)
for arg, res in batch_results.items():
print(f'{arg} -> {res}')
# 方法4使用自定义密钥
custom_key = 'YXBwbHk=' # Base64编码的密钥
generator.set_key(custom_key)
result = generator.generate(arg1)
print(f'使用自定义密钥的acw_sc__v2值: {result}')
```
## 核心算法
### 1. 字符重排
根据索引映射数组 `m` 重新排列输入字符串 `arg1`
```python
m = [0xf, 0x23, 0x1d, 0x18, 0x21, 0x10, 0x1, 0x26, 0xa, 0x9, 0x13, 0x1f, 0x28, 0x1b, 0x16, 0x17, 0x19, 0xd, 0x6, 0xb, 0x27, 0x12, 0x14, 0x8, 0xe, 0x15, 0x20, 0x1a, 0x2, 0x1e, 0x7, 0x4, 0x11, 0x5, 0x3, 0x1c, 0x22, 0x25, 0xc, 0x24]
```
### 2. 密钥解码
将Base64编码的密钥解码为十六进制字符串
```python
key_encoded = 'Aw5PDg9JAw4='
key_hex = base64.b64decode(key_encoded).hex()
# 结果: '030e4f0e0f49030e'
```
### 3. 异或运算
将重排后的字符串与解码后的密钥进行按位异或:
```python
for x in range(0, min(len(u), len(p)), 2):
u_sub = u[x:x+2]
p_sub = p[x:x+2]
a = format(int(u_sub, 16) ^ int(p_sub, 16), '02x')
v += a
```
## 输入输出示例
### 输入
```python
arg1 = '1EEB3321F0286855D50CD9E96C1CDF3D4A913DB0'
```
### 输出
```python
acw_sc__v2 = '5ad70a13004a0290'
```
## 注意事项
1. **输入长度**`arg1` 必须是40位的十六进制字符串
2. **输入格式**`arg1` 必须是有效的十六进制字符串
3. **密钥**:默认使用内置密钥,也可以自定义密钥
4. **输出格式**:输出为十六进制字符串
## 错误处理
完整版生成器包含以下错误处理:
- 输入长度验证
- 输入格式验证
- Base64解码错误处理
- 异常捕获和错误信息输出
## 应用场景
- 网站爬虫绕过阿里云CDN的反爬虫保护
- 安全测试:测试网站的反爬虫机制
- 学习研究了解混淆JavaScript代码的逆向工程
## 技术细节
### 混淆技术
原始HTML代码使用了多种混淆技术
1. **字符串编码**Base64编码 + URL编码
2. **变量名混淆**:使用无意义的变量名
3. **控制流混淆**:使用闭包和条件函数
4. **反调试**:通过数学计算和正则表达式检测调试器
### 逆向工程
通过分析原始代码,提取出核心算法:
1. 移除反调试逻辑
2. 简化字符串解码
3. 提取核心计算逻辑
4. 重构为可读代码
## 性能优化
- 使用缓存机制避免重复解码
- 批量生成功能提高效率
- 简化算法减少计算量
## 安全提示
本工具仅供学习和研究使用,请勿用于非法用途。使用本工具绕过网站的反爬虫保护可能违反网站的使用条款。
## 许可证
本代码仅供学习和研究使用。
## 联系方式
如有问题或建议,请通过以下方式联系:
- 提交Issue
- 发送Pull Request
- 联系作者
## 更新日志
### v1.0.0 (2024-01-01)
- 初始版本发布
- 支持JavaScript和Python版本
- 包含完整的使用示例
## 致谢
感谢所有为逆向工程和反爬虫技术研究做出贡献的开发者。

View File

@@ -0,0 +1,76 @@
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :add_backlink_count_column.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/10 03:50
@explain : 为domains表添加backlink_count字段
'''
import psycopg2
from app.config import config
def add_backlink_count_column():
"""
为domains表添加backlink_count字段
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
print(f"成功连接到数据库: {config.DB_HOST}:{config.DB_PORT}/{config.DB_DATABASE}")
# 创建游标
cur = conn.cursor()
# 添加backlink_count字段
add_column_sql = """
ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0
"""
cur.execute(add_column_sql)
print("添加backlink_count字段成功")
# 添加字段注释
add_comment_sql = """
COMMENT ON COLUMN domains.backlink_count IS '友情链接数量'
"""
cur.execute(add_comment_sql)
print("添加字段注释成功")
# 创建索引
create_index_sql = """
CREATE INDEX IF NOT EXISTS idx_domains_backlink_count ON domains(backlink_count)
"""
cur.execute(create_index_sql)
print("创建索引成功")
# 提交事务
conn.commit()
print("事务提交成功")
# 关闭游标和连接
cur.close()
conn.close()
print("连接关闭成功")
return True
except Exception as e:
print(f"添加backlink_count字段失败: {e}")
return False
if __name__ == "__main__":
print("开始为domains表添加backlink_count字段...")
success = add_backlink_count_column()
if success:
print("backlink_count字段添加成功")
else:
print("backlink_count字段添加失败")

View File

@@ -0,0 +1,8 @@
-- 为domains表添加backlink_count字段
ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0;
-- 添加字段注释
COMMENT ON COLUMN domains.backlink_count IS '友情链接数量';
-- 创建索引
CREATE INDEX IF NOT EXISTS idx_domains_backlink_count ON domains(backlink_count);

View File

@@ -0,0 +1,51 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :add_expire_date_field.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/11 22:40
@explain : 为domains表添加expire_date字段
'''
import psycopg2
from loguru import logger
from app.config import config
def add_expire_date_field():
"""
为domains表添加expire_date字段
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
logger.info("数据库连接成功")
# 添加expire_date字段
try:
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS expire_date DATE")
cur.execute("COMMENT ON COLUMN domains.expire_date IS '过期时间'")
logger.info("添加expire_date字段成功")
except Exception as e:
logger.error(f"添加expire_date字段失败: {e}")
# 提交事务
conn.commit()
logger.info("字段添加完成")
# 关闭连接
cur.close()
conn.close()
except Exception as e:
logger.error(f"执行失败: {e}")
if __name__ == "__main__":
add_expire_date_field()

View File

@@ -0,0 +1,59 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :add_fields_to_domains.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/11 22:40
@explain : 为domains表添加缺失的字段
'''
import psycopg2
from loguru import logger
from app.config import config
def add_fields_to_domains():
"""
为domains表添加缺失的字段
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
logger.info("数据库连接成功")
# 添加company_type字段
try:
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS company_type VARCHAR(100)")
cur.execute("COMMENT ON COLUMN domains.company_type IS '单位性质'")
logger.info("添加company_type字段成功")
except Exception as e:
logger.error(f"添加company_type字段失败: {e}")
# 添加website_url字段
try:
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS website_url VARCHAR(255)")
cur.execute("COMMENT ON COLUMN domains.website_url IS '网站首页网址'")
logger.info("添加website_url字段成功")
except Exception as e:
logger.error(f"添加website_url字段失败: {e}")
# 提交事务
conn.commit()
logger.info("字段添加完成")
# 关闭连接
cur.close()
conn.close()
except Exception as e:
logger.error(f"执行失败: {e}")
if __name__ == "__main__":
add_fields_to_domains()

View File

@@ -0,0 +1,58 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :add_sensitive_words_table.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/12 00:03
@explain : 为数据库添加sensitive_words表
'''
import psycopg2
from loguru import logger
from app.config import config
def add_sensitive_words_table():
"""
为数据库添加sensitive_words表
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
logger.info("数据库连接成功")
# 创建sensitive_words表
create_sensitive_words_table = """
CREATE TABLE IF NOT EXISTS sensitive_words (
id SERIAL PRIMARY KEY,
word VARCHAR(255) UNIQUE NOT NULL
)
"""
cur.execute(create_sensitive_words_table)
logger.info("创建sensitive_words表成功")
# 添加字段注释
cur.execute("COMMENT ON COLUMN sensitive_words.id IS '主键ID'")
cur.execute("COMMENT ON COLUMN sensitive_words.word IS '敏感词'")
logger.info("添加sensitive_words表字段注释成功")
# 提交事务
conn.commit()
logger.info("sensitive_words表添加完成")
# 关闭连接
cur.close()
conn.close()
except Exception as e:
logger.error(f"执行失败: {e}")
if __name__ == "__main__":
add_sensitive_words_table()

View File

@@ -0,0 +1,6 @@
# -*- coding: UTF-8 -*-
'''
域名库系统
'''
__version__ = "1.0.0"

1257
domainCheck/app/chinaz.js Normal file

File diff suppressed because one or more lines are too long

265
domainCheck/app/config.py Normal file
View File

@@ -0,0 +1,265 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :config.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:08
@explain : 系统配置
'''
import os
import sys
from dotenv import load_dotenv
def _safe_echo(message):
"""
在不同 Windows 控制台编码下安全输出文本,避免导入阶段因中文打印失败。
"""
try:
print(message)
except UnicodeEncodeError:
try:
encoding = sys.stdout.encoding or 'utf-8'
sys.stdout.buffer.write((message + '\n').encode(encoding, errors='replace'))
except Exception:
pass
# 确定基础目录
if hasattr(sys, '_MEIPASS'):
# PyInstaller 打包后的临时目录
BASE_DIR = sys._MEIPASS
else:
# 开发环境目录
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# 加载环境变量
env_path = os.path.join(BASE_DIR, '.env')
if os.path.exists(env_path):
load_dotenv(env_path)
_safe_echo(f"成功加载环境变量文件: {env_path}")
else:
load_dotenv()
_safe_echo(f"环境变量文件不存在: {env_path},使用默认环境变量")
class Config:
"""
系统配置
"""
# 数据库配置
DB_HOST = os.getenv('DB_HOST', 'localhost')
DB_PORT = int(os.getenv('DB_PORT', 5432))
DB_DATABASE = os.getenv('DB_DATABASE', 'domain_scan_db')
DB_USER = os.getenv('DB_USER', 'postgres')
DB_PASSWORD = os.getenv('DB_PASSWORD', 'postgres')
DB_POOL_SIZE = int(os.getenv('DB_POOL_SIZE', 5))
# 消息队列配置
RABBITMQ_HOST = os.getenv('RABBITMQ_HOST', 'localhost')
RABBITMQ_PORT = int(os.getenv('RABBITMQ_PORT', 5672))
RABBITMQ_USER = os.getenv('RABBITMQ_USER', 'guest')
RABBITMQ_PASSWORD = os.getenv('RABBITMQ_PASSWORD', 'guest')
RABBITMQ_VHOST = os.getenv('RABBITMQ_VHOST', '/')
# Redis配置
REDIS_HOST = os.getenv('REDIS_HOST', 'localhost')
REDIS_PORT = int(os.getenv('REDIS_PORT', 6379))
REDIS_PASSWORD = os.getenv('REDIS_PASSWORD', '')
REDIS_DB = int(os.getenv('REDIS_DB', 0))
# 聚名网配置
JUMING_COOKIE = os.getenv('JUMING_COOKIE', '')
JUMING_REFERER = os.getenv('JUMING_REFERER', 'https://www.juming.com/')
JUMING_USER_AGENT = os.getenv('JUMING_USER_AGENT', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36')
# 代理配置
PROXY_ENABLED = os.getenv('PROXY_ENABLED', 'false').lower() == 'true'
PROXY_URL = os.getenv('PROXY_URL', '')
# 检测配置
DETECT_TIMEOUT = int(os.getenv('DETECT_TIMEOUT', 30))
DETECT_RETRY_COUNT = int(os.getenv('DETECT_RETRY_COUNT', 3))
DETECT_CONCURRENCY = int(os.getenv('DETECT_CONCURRENCY', 10))
WAYBACK_CDX_TIMEOUT = int(os.getenv('WAYBACK_CDX_TIMEOUT', 15))
WAYBACK_SNAPSHOT_TIMEOUT = int(os.getenv('WAYBACK_SNAPSHOT_TIMEOUT', 12))
WAYBACK_RETRY_COUNT = int(os.getenv('WAYBACK_RETRY_COUNT', 2))
WAYBACK_REQUEST_DELAY = float(os.getenv('WAYBACK_REQUEST_DELAY', 0))
WAYBACK_PROGRESS_INTERVAL = int(os.getenv('WAYBACK_PROGRESS_INTERVAL', 500))
WAYBACK_DOMAIN_CONCURRENCY = int(os.getenv('WAYBACK_DOMAIN_CONCURRENCY', 3))
WAYBACK_TITLE_MAX_BYTES = int(os.getenv('WAYBACK_TITLE_MAX_BYTES', 65536))
WAYBACK_TIMESTAMP_CACHE_TTL = int(os.getenv('WAYBACK_TIMESTAMP_CACHE_TTL', 86400))
WAYBACK_TITLE_CACHE_TTL = int(os.getenv('WAYBACK_TITLE_CACHE_TTL', 2592000))
WAYBACK_USER_AGENT = os.getenv(
'WAYBACK_USER_AGENT',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
)
# 域名配置
DOMAIN_TLDS = ['com', 'net']
DOMAIN_BATCH_SIZE = int(os.getenv('DOMAIN_BATCH_SIZE', 1000))
# 日志配置
LOG_LEVEL = os.getenv('LOG_LEVEL', 'INFO')
LOG_FILE = os.getenv('LOG_FILE', 'app.log')
# 任务配置
TASK_PRIORITY = {
'rdap': 10,
'wayback': 8,
'baidu': 6,
'qihu360': 5,
'google': 5,
'chinaz': 4,
'aizhan': 4,
'juziseo': 3,
'jucha': 3
}
# 敏感词配置 - 现在从数据库加载
SENSITIVE_WORDS = []
# 检测项配置
DETECT_ITEMS = {
'rdap': True,
'wayback': True,
'baidu': True,
'qihu360': True,
'google': True,
'chinaz': True,
'aizhan': True,
'juziseo': True,
'jucha': True
}
# 目录配置
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA_DIR = os.path.join(BASE_DIR, 'data')
LOG_DIR = os.path.join(BASE_DIR, 'logs')
# 确保目录存在
os.makedirs(DATA_DIR, exist_ok=True)
os.makedirs(LOG_DIR, exist_ok=True)
# 从数据库加载敏感词
def load_sensitive_words():
"""
从数据库加载敏感词
"""
try:
from app.utils.database import Database
db = Database()
sensitive_words = db.get_sensitive_words()
words = [word['word'] for word in sensitive_words]
db.close()
return words
except Exception as e:
_safe_echo(f"加载敏感词失败: {e}")
return []
# 从文件加载检测选项
def load_detect_options():
"""
从文件加载检测选项
"""
try:
default_order = [
'detect_register',
'detect_baidu_site',
'detect_360_site',
'detect_chinaz',
'detect_aizhan',
'detect_wayback',
'detect_jucha',
'detect_juziseo',
]
import json
import os
defaults = {
'detect_register': True,
'detect_wayback': True,
'detect_chinaz': True,
'detect_aizhan': True,
'detect_baidu_site': True,
'detect_360_site': True,
'detect_jucha': False,
'detect_juziseo': False,
'detect_order': default_order,
}
if os.path.exists('detect_options.json'):
with open('detect_options.json', 'r', encoding='utf-8') as f:
detect_options = json.load(f)
defaults.update(detect_options)
if defaults.get('detect_whois') or defaults.get('detect_beian') or defaults.get('detect_intercept'):
defaults['detect_jucha'] = True
if defaults.get('detect_juziseo_outlink'):
defaults['detect_juziseo'] = True
order = defaults.get('detect_order') or []
normalized_order = [key for key in order if key in default_order]
for key in default_order:
if key not in normalized_order:
normalized_order.append(key)
defaults['detect_order'] = normalized_order
return defaults
except Exception as e:
_safe_echo(f"加载检测选项失败: {e}")
return {
'detect_register': True,
'detect_wayback': True,
'detect_chinaz': True,
'detect_aizhan': True,
'detect_baidu_site': True,
'detect_360_site': True,
'detect_jucha': False,
'detect_juziseo': False,
}
# 导出配置
config = Config()
# 加载检测选项
config.DETECT_OPTIONS = load_detect_options()
# 延迟加载敏感词,避免循环导入
def load_sensitive_words_lazy():
"""
延迟加载敏感词
"""
return load_sensitive_words()
# 设置敏感词属性为延迟加载
config.load_sensitive_words = load_sensitive_words_lazy
# 检查检测类型是否应该执行
def should_detect(detect_type):
"""
检查检测类型是否应该执行
:param detect_type: 检测类型
:return: bool - 是否应该执行
"""
# 检测类型映射
type_mapping = {
'register': 'detect_register',
'wayback': 'detect_wayback',
'chinaz': 'detect_chinaz',
'aizhan': 'detect_aizhan',
'baidu_site': 'detect_baidu_site',
'360_site': 'detect_360_site',
'whois': 'detect_jucha',
'beian': 'detect_jucha',
'intercept': 'detect_jucha',
'jucha': 'detect_jucha',
'juziseo': 'detect_juziseo',
'juziseo_outlink': 'detect_juziseo'
}
# 获取对应的配置键
config_key = type_mapping.get(detect_type)
if not config_key:
return True # 默认执行
# 检查是否在配置中默认为True
return config.DETECT_OPTIONS.get(config_key, True)

View File

@@ -0,0 +1,4 @@
# -*- coding: UTF-8 -*-
'''
核心功能模块
'''

View File

@@ -0,0 +1,286 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :detect_engine.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:52
@explain : 检测引擎
'''
import time
from loguru import logger
from app.utils.database import Database
from app.detectors.rdap_detector import RDAPDetector
from app.detectors.wayback_detector import WaybackDetector
from app.detectors.baidu_detector import BaiduDetector
from app.detectors.qihu360_detector import Qihu360Detector
from app.detectors.google_detector import GoogleDetector
from app.detectors.chinaz_detector import ChinazDetector
from app.detectors.aizhan_detector import AizhanDetector
from app.detectors.juziseo_detector import JuziseoDetector
from app.detectors.jucha_detector import JuchaDetector
from app.utils.status_codes import (
DETECT_STATUS_BLACKLISTED,
DETECT_STATUS_COMPLETED,
DETECT_STATUS_FAILED,
DETECT_STATUS_RUNNING,
)
class DetectEngine:
"""
检测引擎
"""
def __init__(self):
"""
初始化检测引擎
"""
self.db = Database()
self.rdap_detector = RDAPDetector()
self.wayback_detector = WaybackDetector()
self.baidu_detector = BaiduDetector()
self.qihu360_detector = Qihu360Detector()
self.google_detector = GoogleDetector()
self.chinaz_detector = ChinazDetector()
self.aizhan_detector = AizhanDetector()
self.juziseo_detector = JuziseoDetector()
self.jucha_detector = JuchaDetector()
def detect_domain(self, domain_id):
"""
检测域名
:param domain_id: 域名ID
:return: bool - 是否检测成功
"""
try:
# 获取域名信息
domain_info = self.db.get_domain_by_id(domain_id)
if not domain_info:
logger.error(f"域名不存在: {domain_id}")
return False
domain = domain_info['domain']
logger.info(f"开始检测域名: {domain}")
# 更新检测状态为检测中
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_RUNNING)
# 1. 基础检测
if not self._basic_detect(domain_id, domain):
logger.info(f"基础检测失败,停止后续检测: {domain}")
return False
# 2. 深度检测
if not self._deep_detect(domain_id, domain):
logger.info(f"深度检测失败: {domain}")
return False
# 更新检测状态为正常
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_COMPLETED)
logger.info(f"域名检测完成: {domain}")
return True
except Exception as e:
logger.error(f"检测域名出错: {e}")
# 更新检测状态为检测失败
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_FAILED)
return False
def _basic_detect(self, domain_id, domain):
"""
基础检测
:param domain_id: 域名ID
:param domain: 域名
:return: bool - 是否检测通过
"""
# 1. 检查是否为一口价域名
is_ykj = self.db.is_ykj_domain(domain_id)
# 2. 注册状态检测(一口价域名跳过)
if not is_ykj:
register_status = self.rdap_detector.check_register_status(domain)
self.db.update_domain_register_status(domain_id, register_status)
# 3. 黑名单缓存检查
if self.db.is_blacklisted(domain):
logger.info(f"域名在黑名单中: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
return False
# 4. 时光机快照年份采集
snapshot_years = self.wayback_detector.get_snapshot_years(domain)
if snapshot_years:
self.db.update_domain_snapshot_years(domain_id, ','.join(map(str, snapshot_years)))
# 5. 时光机正文抽样与敏感词匹配
if self.wayback_detector.has_sensitive_content(domain):
logger.info(f"域名包含敏感词: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
self.db.add_to_blacklist(domain, "快照包含敏感词")
return False
return True
def _deep_detect(self, domain_id, domain):
"""
深度检测
:param domain_id: 域名ID
:param domain: 域名
:return: bool - 是否检测通过
"""
# 1. 百度历史/Site
baidu_history = self.baidu_detector.check_history(domain)
baidu_site = self.baidu_detector.check_site(domain)
# 2. 360 Site
qihu360_site = self.qihu360_detector.check_site(domain)
# 3. Google Site
google_site = self.google_detector.check_site(domain)
# 4. 站长之家
chinaz_info = self.chinaz_detector.check_domain(domain)
# 5. 爱站网
aizhan_info = self.aizhan_detector.check_domain(domain)
# 6. 桔子SEO
juziseo_info = self.juziseo_detector.check_domain(domain)
# 7. 聚查
jucha_info = self.jucha_detector.check_domain(domain)
# 检查是否有风险
if self._check_risk(domain_id, domain, baidu_history, baidu_site, qihu360_site, google_site, chinaz_info, aizhan_info, juziseo_info, jucha_info):
return False
# 保存检测结果
self.db.add_detection_result(domain_id, baidu_history, baidu_site, qihu360_site, google_site, chinaz_info, aizhan_info, juziseo_info, jucha_info)
return True
def _check_risk(self, domain_id, domain, baidu_history, baidu_site, qihu360_site, google_site, chinaz_info, aizhan_info, juziseo_info, jucha_info):
"""
检查风险
:param domain_id: 域名ID
:param domain: 域名
:param baidu_history: 百度历史
:param baidu_site: 百度site
:param qihu360_site: 360 site
:param google_site: Google site
:param chinaz_info: 站长之家信息
:param aizhan_info: 爱站网信息
:param juziseo_info: 桔子SEO信息
:param jucha_info: 聚查信息
:return: bool - 是否有风险
"""
# 检查百度历史过灰
if baidu_history and '' in str(baidu_history):
logger.info(f"百度历史过灰: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
self.db.add_to_blacklist(domain, "百度历史过灰")
return True
# 检查标题敏感词
if chinaz_info and 'title' in chinaz_info:
if self._contains_sensitive_words(chinaz_info['title']):
logger.info(f"标题包含敏感词: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
self.db.add_to_blacklist(domain, "标题包含敏感词")
return True
# 检查子域名
if baidu_site and 'subdomains' in baidu_site:
subdomains = baidu_site['subdomains']
# 排除 www, @, m
valid_subdomains = [sub for sub in subdomains if sub not in ['www', '@', 'm']]
if valid_subdomains:
logger.info(f"存在子域名: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
self.db.add_to_blacklist(domain, "存在子域名")
return True
# 检查风险提示
if aizhan_info and 'risk' in aizhan_info:
if aizhan_info['risk'] in ['低风险', '疑似色情博彩风险', '严重影响权重']:
logger.info(f"风险提示: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
self.db.add_to_blacklist(domain, f"风险提示: {aizhan_info['risk']}")
return True
# 检查WHOIS状态
if jucha_info and 'whois' in jucha_info:
if jucha_info['whois'].get('status') in ['clientHold', 'serverHold']:
logger.info(f"WHOIS状态异常: {domain}")
self.db.update_domain_detect_status(domain_id, DETECT_STATUS_BLACKLISTED)
self.db.add_to_blacklist(domain, "WHOIS状态异常")
return True
# 检查拦截检测
if jucha_info and 'intercept' in jucha_info:
if not jucha_info['intercept'].get('normal', True):
logger.info(f"拦截检测异常: {domain}")
self.db.update_domain_detect_status(domain_id, 4) # 4 表示黑名单
self.db.add_to_blacklist(domain, "拦截检测异常")
return True
return False
def _contains_sensitive_words(self, text):
"""
检查文本是否包含敏感词
:param text: 文本
:return: bool - 是否包含敏感词
"""
sensitive_words = self.db.get_all_sensitive_words()
for word in sensitive_words:
if word in text:
return True
return False
def process_task(self, task_id):
"""
处理检测任务
:param task_id: 任务ID
:return: bool - 是否处理成功
"""
try:
# 获取任务信息
task = self.db.get_task_by_id(task_id)
if not task:
logger.error(f"任务不存在: {task_id}")
return False
domain_id = task['domain_id']
# 更新任务状态为执行中
self.db.update_task_status(task_id, 1) # 1 表示执行中
# 执行检测
success = self.detect_domain(domain_id)
# 更新任务状态
if success:
self.db.update_task_status(task_id, 2) # 2 表示完成
else:
# 增加重试次数
retry_count = task.get('retry_count', 0) + 1
if retry_count < 3:
self.db.update_task_retry_count(task_id, retry_count)
self.db.update_task_status(task_id, 0) # 0 表示待执行
else:
self.db.update_task_status(task_id, 3) # 3 表示失败
return success
except Exception as e:
logger.error(f"处理任务出错: {e}")
# 更新任务状态为失败
self.db.update_task_status(task_id, 3) # 3 表示失败
return False

View File

@@ -0,0 +1,261 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :domain_collector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:50
@explain : 域名收集器
'''
import re
import tldextract
from loguru import logger
from app.utils.database import Database
from app.utils.domain_utils import normalize_domain
from app.config import config
class DomainCollector:
"""
域名收集器
"""
def __init__(self):
"""
初始化域名收集器
"""
self.db = Database(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
def add_domain(self, domain, source_type):
"""
添加域名
:param domain: 域名
:param source_type: 来源类型
:return: bool - 是否添加成功
"""
try:
# 标准化域名
normalized_domain = normalize_domain(domain)
if not normalized_domain:
logger.warning(f"无效域名: {domain}")
return False
# 提取顶级域名
ext = tldextract.extract(normalized_domain)
tld = ext.suffix
# 只保留 .com 和 .net
if tld not in ['com', 'net']:
logger.warning(f"不支持的顶级域名: {tld}")
return False
# 检查是否已存在
if self.db.domain_exists(normalized_domain):
logger.info(f"域名已存在: {normalized_domain}")
return False
# 添加域名
domain_id = self.db.add_domain(normalized_domain, tld, source_type)
if domain_id:
# 创建检测任务
self.db.create_detect_task(domain_id, 1) # 1 表示基础检测
logger.info(f"成功添加域名: {normalized_domain}")
return True
else:
logger.error(f"添加域名失败: {normalized_domain}")
return False
except Exception as e:
logger.error(f"添加域名出错: {e}")
return False
def add_domains_batch(self, domains, source_type, batch_size=1000, dry_run=False):
"""
批量添加域名
:param domains: 域名列表
:param source_type: 来源类型
:param batch_size: 批量大小
:param dry_run: 是否仅进行干运行(不实际添加域名)
:return: dict - 统计信息
"""
try:
# 统计信息
stats = {
'total': len(domains),
'valid': 0,
'added': 0,
'exists': 0,
'invalid': 0,
'failed': 0
}
# 处理大规模数据时,分批进行标准化和过滤
normalized_domains = []
batch_domains = []
for i, domain in enumerate(domains):
normalized = normalize_domain(domain)
if not normalized:
stats['invalid'] += 1
continue
# 提取顶级域名
ext = tldextract.extract(normalized)
tld = ext.suffix
# 只保留 .com 和 .net
if tld not in ['com', 'net']:
stats['invalid'] += 1
continue
normalized_domains.append((normalized, tld))
batch_domains.append(normalized)
stats['valid'] += 1
# 每1000个域名检查一次避免内存占用过高
if (i + 1) % 1000 == 0:
logger.info(f"已处理 {i + 1}/{len(domains)} 个域名")
logger.info(f"域名标准化完成,有效域名: {stats['valid']}")
# 提取所有域名
all_domains = [domain for domain, tld in normalized_domains]
# 批量检查域名是否存在
existing_domains = self.db.check_domains_exist(all_domains)
existing_set = set(existing_domains)
# 准备批量添加数据
batch_data = []
for domain, tld in normalized_domains:
if domain not in existing_set:
batch_data.append((domain, tld, source_type))
stats['exists'] = len(existing_domains)
stats['valid'] = len(normalized_domains)
# 干运行模式下直接返回统计信息
if dry_run:
stats['added'] = len(batch_data)
logger.info(f"干运行模式:准备添加 {len(batch_data)} 个新域名")
return stats
logger.info(f"准备添加 {len(batch_data)} 个新域名")
# 分批次添加
for i in range(0, len(batch_data), batch_size):
batch = batch_data[i:i+batch_size]
added_count = self.db.add_domains_batch(batch)
stats['added'] += added_count
stats['failed'] += len(batch) - added_count
# 每处理一批,记录一次进度
if (i + len(batch)) % (batch_size * 10) == 0:
logger.info(f"已添加 {i + len(batch)}/{len(batch_data)} 个域名")
logger.info(f"批量添加域名完成: {stats}")
return stats
except Exception as e:
logger.error(f"批量添加域名出错: {e}")
return {
'total': len(domains),
'valid': 0,
'added': 0,
'exists': 0,
'invalid': 0,
'failed': len(domains)
}
def import_from_file(self, file_path, source_type, batch_size=1000):
"""
从文件导入域名
:param file_path: 文件路径
:param source_type: 来源类型
:param batch_size: 批量大小
:return: dict - 导入统计信息
"""
try:
with open(file_path, 'r', encoding='utf-8') as f:
domains = f.readlines()
# 提取域名
domain_list = []
for domain in domains:
domain = domain.strip()
if domain:
domain_list.append(domain)
# 批量添加域名
stats = self.add_domains_batch(domain_list, source_type, batch_size)
logger.info(f"从文件导入完成: {stats}")
return stats
except Exception as e:
logger.error(f"从文件导入出错: {e}")
return {
'total': 0,
'valid': 0,
'added': 0,
'exists': 0,
'invalid': 0,
'failed': 0
}
def collect_from_juming(self, type_='一口价'):
"""
从聚名网收集域名
:param type_: 类型,一口价或过期删除
:return: int - 收集到的域名数量
"""
# 这里可以添加从聚名网收集域名的逻辑
logger.info(f"从聚名网收集 {type_} 域名")
# 模拟收集结果
return 0
def collect_from_search_engine(self, keyword, limit=100):
"""
从搜索引擎收集域名
:param keyword: 关键词
:param limit: 限制数量
:return: int - 收集到的域名数量
"""
# 这里可以添加从搜索引擎收集域名的逻辑
logger.info(f"从搜索引擎收集域名,关键词: {keyword}, 限制: {limit}")
# 模拟收集结果
return 0
def collect_from_enterprise_directory(self, url, limit=100):
"""
从企业目录收集域名
:param url: 企业目录URL
:param limit: 限制数量
:return: int - 收集到的域名数量
"""
# 这里可以添加从企业目录收集域名的逻辑
logger.info(f"从企业目录收集域名URL: {url}, 限制: {limit}")
# 模拟收集结果
return 0
def collect_from_zone_file(self, file_path):
"""
从Zone File收集域名
:param file_path: Zone File路径
:return: int - 收集到的域名数量
"""
# 这里可以添加从Zone File收集域名的逻辑
logger.info(f"从Zone File收集域名文件: {file_path}")
# 模拟收集结果
return 0

View File

@@ -0,0 +1,150 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :domain_processor.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:51
@explain : 域名处理器
'''
import re
import tldextract
from loguru import logger
from app.utils.database import Database
from app.utils.domain_utils import normalize_domain
class DomainProcessor:
"""
域名处理器
"""
def __init__(self):
"""
初始化域名处理器
"""
self.db = Database()
def process_domain(self, domain):
"""
处理域名
:param domain: 域名
:return: dict - 处理结果
"""
result = {
'original': domain,
'normalized': None,
'tld': None,
'valid': False,
'reason': ''
}
try:
# 标准化域名
normalized = normalize_domain(domain)
if not normalized:
result['reason'] = '无效域名格式'
return result
# 提取顶级域名
ext = tldextract.extract(normalized)
tld = ext.suffix
# 检查顶级域名
if tld not in ['com', 'net']:
result['reason'] = '不支持的顶级域名'
return result
result['normalized'] = normalized
result['tld'] = tld
result['valid'] = True
return result
except Exception as e:
logger.error(f"处理域名出错: {e}")
result['reason'] = f"处理出错: {str(e)}"
return result
def batch_process(self, domains):
"""
批量处理域名
:param domains: 域名列表
:return: list - 处理结果列表
"""
results = []
for domain in domains:
result = self.process_domain(domain)
results.append(result)
return results
def filter_valid_domains(self, domains):
"""
过滤有效的域名
:param domains: 域名列表
:return: list - 有效的域名列表
"""
valid_domains = []
for domain in domains:
result = self.process_domain(domain)
if result['valid']:
valid_domains.append(result['normalized'])
return valid_domains
def update_domain_status(self, domain_id, status_type, status_value):
"""
更新域名状态
:param domain_id: 域名ID
:param status_type: 状态类型
:param status_value: 状态值
:return: bool - 是否更新成功
"""
try:
if status_type == 'use_status':
return self.db.update_domain_use_status(domain_id, status_value)
elif status_type == 'detect_status':
return self.db.update_domain_detect_status(domain_id, status_value)
elif status_type == 'register_status':
return self.db.update_domain_register_status(domain_id, status_value)
else:
logger.error(f"未知的状态类型: {status_type}")
return False
except Exception as e:
logger.error(f"更新域名状态出错: {e}")
return False
def blacklist_domain(self, domain_id, reason):
"""
将域名加入黑名单
:param domain_id: 域名ID
:param reason: 黑名单原因
:return: bool - 是否操作成功
"""
try:
# 更新检测状态为黑名单
if self.db.update_domain_detect_status(domain_id, 4): # 4 表示黑名单
# 添加到黑名单表
domain = self.db.get_domain_by_id(domain_id)
if domain:
return self.db.add_to_blacklist(domain['domain'], reason)
return False
except Exception as e:
logger.error(f"将域名加入黑名单出错: {e}")
return False
def get_domain_statistics(self):
"""
获取域名统计信息
:return: dict - 统计信息
"""
try:
return self.db.get_domain_statistics()
except Exception as e:
logger.error(f"获取域名统计信息出错: {e}")
return {}

View File

@@ -0,0 +1,289 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :export_manager.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:54
@explain : 导出管理器
'''
import os
from loguru import logger
from app.utils.database import Database
class ExportManager:
"""
导出管理器
"""
def __init__(self):
"""
初始化导出管理器
"""
self.db = Database()
def export_domains(self, filter_conditions, output_file):
"""
导出域名
:param filter_conditions: 筛选条件
:param output_file: 输出文件路径
:return: int - 导出的域名数量
"""
try:
# 查询符合条件的域名
domains = self.db.get_domains_by_conditions(filter_conditions)
if not domains:
logger.warning("没有符合条件的域名")
return 0
# 导出到文件
with open(output_file, 'w', encoding='utf-8') as f:
for domain in domains:
f.write(domain['domain'] + '\n')
logger.info(f"成功导出 {len(domains)} 个域名到 {output_file}")
return len(domains)
except Exception as e:
logger.error(f"导出域名出错: {e}")
return 0
def export_with_details(self, filter_conditions, output_file):
"""
导出域名及其详细信息
:param filter_conditions: 筛选条件
:param output_file: 输出文件路径
:return: int - 导出的域名数量
"""
try:
# 查询符合条件的域名及其详细信息
domains = self.db.get_domains_with_details(filter_conditions)
if not domains:
logger.warning("没有符合条件的域名")
return 0
# 导出到文件
with open(output_file, 'w', encoding='utf-8') as f:
# 写入表头
f.write('域名,注册状态,使用状态,检测状态,备案历史,备案年份,快照年份,友情链接数量\n')
# 写入数据
for domain in domains:
line = f"{domain['domain']},{domain['register_status']},{domain['use_status']},{domain['detect_status']},{domain['has_beian']},{domain['beian_year']},{domain['snapshot_years']},{domain['backlink_count']}\n"
f.write(line)
logger.info(f"成功导出 {len(domains)} 个域名及其详细信息到 {output_file}")
return len(domains)
except Exception as e:
logger.error(f"导出域名详细信息出错: {e}")
return 0
def batch_update_status(self, domain_ids, status_type, status_value):
"""
批量更新域名状态
:param domain_ids: 域名ID列表
:param status_type: 状态类型
:param status_value: 状态值
:return: int - 更新成功的域名数量
"""
try:
success_count = 0
for domain_id in domain_ids:
if self.db.update_domain_status(domain_id, status_type, status_value):
success_count += 1
logger.info(f"成功更新 {success_count} 个域名的状态")
return success_count
except Exception as e:
logger.error(f"批量更新域名状态出错: {e}")
return 0
def get_filtered_domains(self, filter_conditions, limit=1000):
"""
获取符合条件的域名
:param filter_conditions: 筛选条件
:param limit: 限制数量
:return: list - 域名列表
"""
try:
domains = self.db.get_domains_by_conditions(filter_conditions, limit)
logger.info(f"获取到 {len(domains)} 个符合条件的域名")
return domains
except Exception as e:
logger.error(f"获取符合条件的域名出错: {e}")
return []
def generate_report(self, output_file):
"""
生成统计报告
:param output_file: 输出文件路径
:return: bool - 是否生成成功
"""
try:
# 获取域名统计信息
domain_stats = self.db.get_domain_statistics()
# 获取任务统计信息
task_stats = self.db.get_task_statistics()
# 生成报告
with open(output_file, 'w', encoding='utf-8') as f:
f.write('域名库系统统计报告\n')
f.write('=' * 50 + '\n')
f.write('\n域名统计:\n')
f.write(f'总域名数: {domain_stats.get("total", 0)}\n')
f.write(f'可注册域名: {domain_stats.get("available", 0)}\n')
f.write(f'已注册域名: {domain_stats.get("registered", 0)}\n')
f.write(f'黑名单域名: {domain_stats.get("blacklisted", 0)}\n')
f.write('\n任务统计:\n')
f.write(f'总任务数: {task_stats.get("total", 0)}\n')
f.write(f'待执行任务: {task_stats.get("pending", 0)}\n')
f.write(f'执行中任务: {task_stats.get("running", 0)}\n')
f.write(f'已完成任务: {task_stats.get("completed", 0)}\n')
f.write(f'失败任务: {task_stats.get("failed", 0)}\n')
logger.info(f"成功生成统计报告到 {output_file}")
return True
except Exception as e:
logger.error(f"生成统计报告出错: {e}")
return False
def export_to_excel(self, domains, output_file):
"""
导出域名到Excel文件
:param domains: 域名列表
:param output_file: 输出文件路径
:return: int - 导出的域名数量
"""
try:
# 尝试导入openpyxl
try:
from openpyxl import Workbook
except ImportError:
# 如果没有安装openpyxl使用CSV格式作为替代
logger.warning("openpyxl库未安装将使用CSV格式导出")
return self.export_to_csv(domains, output_file.replace('.xlsx', '.csv'))
# 创建工作簿
wb = Workbook()
ws = wb.active
# 写入表头
headers = ['域名', '注册状态', '使用状态', '检测状态', '人工复核状态', '过期时间', '单位性质', '网站首页网址', '检测时间', '备案历史', '备案年份', '快照年份', '百度历史', '百度Site', '是否中文标题', '360 Site', 'Google Site', '友情链接数量']
ws.append(headers)
# 写入数据
for domain in domains:
row = [
domain.get('domain', ''),
domain.get('register_status', ''),
domain.get('use_status', ''),
domain.get('detect_status', ''),
domain.get('review_status', ''),
domain.get('expire_date', ''),
domain.get('company_type', ''),
domain.get('website_url', ''),
domain.get('detect_time', ''),
domain.get('has_beian', ''),
domain.get('beian_year', ''),
domain.get('snapshot_years', ''),
domain.get('baidu_history', ''),
domain.get('baidu_site', ''),
domain.get('is_chinese_title', ''),
domain.get('qihu360_site', ''),
domain.get('google_site', ''),
domain.get('backlink_count', '')
]
ws.append(row)
# 保存文件
wb.save(output_file)
logger.info(f"成功导出 {len(domains)} 个域名到Excel文件: {output_file}")
return len(domains)
except Exception as e:
logger.error(f"导出Excel文件出错: {e}")
# 尝试使用CSV格式作为替代
try:
csv_file = output_file.replace('.xlsx', '.csv')
logger.info(f"尝试使用CSV格式导出到: {csv_file}")
return self.export_to_csv(domains, csv_file)
except Exception as e2:
logger.error(f"导出CSV文件也失败: {e2}")
raise
def export_to_csv(self, domains, output_file):
"""
导出域名到CSV文件
:param domains: 域名列表
:param output_file: 输出文件路径
:return: int - 导出的域名数量
"""
try:
import csv
# 写入文件
with open(output_file, 'w', encoding='utf-8', newline='') as f:
writer = csv.writer(f)
# 写入表头
headers = ['域名', '注册状态', '使用状态', '检测状态', '人工复核状态', '过期时间', '单位性质', '网站首页网址', '检测时间', '备案历史', '备案年份', '快照年份', '百度历史', '百度Site', '是否中文标题', '360 Site', 'Google Site', '友情链接数量']
writer.writerow(headers)
# 写入数据
for domain in domains:
row = [
domain.get('domain', ''),
domain.get('register_status', ''),
domain.get('use_status', ''),
domain.get('detect_status', ''),
domain.get('review_status', ''),
domain.get('expire_date', ''),
domain.get('company_type', ''),
domain.get('website_url', ''),
domain.get('detect_time', ''),
domain.get('has_beian', ''),
domain.get('beian_year', ''),
domain.get('snapshot_years', ''),
domain.get('baidu_history', ''),
domain.get('baidu_site', ''),
domain.get('is_chinese_title', ''),
domain.get('qihu360_site', ''),
domain.get('google_site', ''),
domain.get('backlink_count', '')
]
writer.writerow(row)
logger.info(f"成功导出 {len(domains)} 个域名到CSV文件: {output_file}")
return len(domains)
except Exception as e:
logger.error(f"导出CSV文件出错: {e}")
raise
def export_to_txt(self, domains, output_file):
"""
导出域名到TXT文件一行一个域名。
"""
try:
with open(output_file, 'w', encoding='utf-8') as f:
for domain in domains:
value = domain.get('domain', '').strip()
if value:
f.write(value + '\n')
logger.info(f"成功导出 {len(domains)} 个域名到TXT文件: {output_file}")
return len(domains)
except Exception as e:
logger.error(f"导出TXT文件出错: {e}")
raise

View File

@@ -0,0 +1,162 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :task_scheduler.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:53
@explain : 任务调度器
'''
import time
import threading
from loguru import logger
from app.utils.database import Database
from app.core.detect_engine import DetectEngine
class TaskScheduler:
"""
任务调度器
"""
def __init__(self):
"""
初始化任务调度器
"""
self.db = Database()
self.detect_engine = DetectEngine()
self.running = False
self.threads = []
self.max_threads = 10
def start(self):
"""
启动任务调度器
"""
if self.running:
logger.info("任务调度器已经在运行中")
return
self.running = True
logger.info("启动任务调度器")
# 启动多个线程处理任务
for i in range(self.max_threads):
thread = threading.Thread(target=self._process_tasks, daemon=True)
thread.start()
self.threads.append(thread)
logger.info(f"启动任务处理线程 {i+1}")
def stop(self):
"""
停止任务调度器
"""
self.running = False
logger.info("停止任务调度器")
# 等待线程结束
for thread in self.threads:
thread.join(timeout=5)
self.threads.clear()
logger.info("任务调度器已停止")
def _process_tasks(self):
"""
处理任务
"""
while self.running:
try:
# 获取待执行的任务
task = self.db.get_pending_task()
if not task:
# 没有任务,休眠一段时间
time.sleep(1)
continue
task_id = task['id']
domain_id = task['domain_id']
logger.info(f"处理任务: {task_id}, 域名ID: {domain_id}")
# 执行任务
success = self.detect_engine.process_task(task_id)
if success:
logger.info(f"任务处理成功: {task_id}")
else:
logger.warning(f"任务处理失败: {task_id}")
# 短暂休眠,避免过于频繁的数据库操作
time.sleep(0.1)
except Exception as e:
logger.error(f"处理任务出错: {e}")
# 休眠一段时间,避免出错后无限循环
time.sleep(5)
def add_task(self, domain_id, task_type=1, priority=0):
"""
添加任务
:param domain_id: 域名ID
:param task_type: 任务类型1-基础检测2-深度检测
:param priority: 优先级0-低1-中2-高
:return: int - 任务ID
"""
try:
task_id = self.db.create_detect_task(domain_id, task_type, priority)
logger.info(f"添加任务成功: {task_id}, 域名ID: {domain_id}")
return task_id
except Exception as e:
logger.error(f"添加任务失败: {e}")
return None
def get_task_stats(self):
"""
获取任务统计信息
:return: dict - 任务统计信息
"""
try:
return self.db.get_task_statistics()
except Exception as e:
logger.error(f"获取任务统计信息出错: {e}")
return {}
def retry_failed_tasks(self):
"""
重试失败的任务
:return: int - 重试的任务数量
"""
try:
tasks = self.db.get_failed_tasks()
retry_count = 0
for task in tasks:
task_id = task['id']
self.db.update_task_status(task_id, 0) # 0 表示待执行
self.db.update_task_retry_count(task_id, 0) # 重置重试次数
retry_count += 1
logger.info(f"重试 {retry_count} 个失败的任务")
return retry_count
except Exception as e:
logger.error(f"重试失败任务出错: {e}")
return 0
def clear_completed_tasks(self, days=7):
"""
清理已完成的任务
:param days: 保留天数
:return: int - 清理的任务数量
"""
try:
count = self.db.clear_completed_tasks(days)
logger.info(f"清理 {count} 个已完成的任务")
return count
except Exception as e:
logger.error(f"清理已完成任务出错: {e}")
return 0

View File

@@ -0,0 +1,10 @@
{
"juming": {
"email": "chaofanai1998@gmail.com",
"password": "llzz123,./"
},
"juziseo": {
"email": "mamian",
"password": "Abc123456"
}
}

View File

@@ -0,0 +1,20 @@
{
"detect_register": true,
"detect_wayback": true,
"detect_chinaz": true,
"detect_aizhan": true,
"detect_baidu_site": true,
"detect_360_site": true,
"detect_jucha": false,
"detect_juziseo": false,
"detect_order": [
"detect_register",
"detect_baidu_site",
"detect_360_site",
"detect_chinaz",
"detect_aizhan",
"detect_wayback",
"detect_jucha",
"detect_juziseo"
]
}

View File

@@ -0,0 +1,4 @@
# -*- coding: UTF-8 -*-
'''
检测插件模块
'''

View File

@@ -0,0 +1,130 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :aizhan_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:02
@explain : 爱站网检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class AizhanDetector(BaseDetector):
"""
爱站网检测器
"""
def __init__(self):
"""
初始化爱站网检测器
"""
super().__init__()
self.url = 'https://www.aizhan.com'
self.query_url = 'https://www.aizhan.com/cha/{domain}'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.query_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取标题
title = self._extract_title(content)
# 提取风险信息
risk = self._extract_risk(content)
# 检查是否包含敏感词
has_sensitive = self._check_sensitive(title, risk)
return {
'title': title,
'risk': risk,
'has_sensitive': has_sensitive
}
else:
self._log_warning(f"爱站网查询失败: {response.status_code}")
return {'title': '', 'risk': '', 'has_sensitive': False}
except Exception as e:
return self._handle_exception(e, domain)
def _extract_title(self, content):
"""
提取标题
:param content: 页面内容
:return: str - 标题
"""
try:
pattern = r'<title>(.*?)</title>'
match = re.search(pattern, content)
if match:
return match.group(1).strip()
return ''
except Exception as e:
self._handle_exception(e, 'extract_title')
return ''
def _extract_risk(self, content):
"""
提取风险信息
:param content: 页面内容
:return: str - 风险信息
"""
try:
# 这里需要根据实际页面结构调整正则表达式
pattern = r'百度网址检测:<span[^>]+>(.*?)</span>'
match = re.search(pattern, content)
if match:
return match.group(1).strip()
return ''
except Exception as e:
self._handle_exception(e, 'extract_risk')
return ''
def _check_sensitive(self, title, risk):
"""
检查是否包含敏感词
:param title: 标题
:param risk: 风险信息
:return: bool - 是否包含敏感词
"""
# 风险类型
sensitive_risks = ['低风险', '疑似色情博彩风险', '严重影响权重']
# 敏感词
sensitive_words = ['色情', '赌博', '博彩', '毒品', '暴力', '诈骗']
# 检查风险
for r in sensitive_risks:
if r in risk:
return True
# 检查标题
for word in sensitive_words:
if word in title:
return True
return False

View File

@@ -0,0 +1,151 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :baidu_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:58
@explain : 百度检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class BaiduDetector(BaseDetector):
"""
百度检测器
"""
def __init__(self):
"""
初始化百度检测器
"""
super().__init__()
self.site_url = 'https://www.baidu.com/s'
self.history_url = 'https://www.baidu.com/s'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 检查百度site
site_result = self.check_site(domain)
# 检查百度历史
history_result = self.check_history(domain)
return {
'site': site_result,
'history': history_result
}
except Exception as e:
return self._handle_exception(e, domain)
def check_site(self, domain):
"""
检查百度site收录
:param domain: 域名
:return: dict - 检测结果
"""
try:
params = {
'wd': f'site:{domain}',
'rn': '50'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(self.site_url, params=params, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取子域名
subdomains = self._extract_subdomains(content, domain)
# 检查是否有收录
has_收录 = '没有找到相关结果' not in content
return {
'has_收录': has_收录,
'subdomains': subdomains
}
else:
self._log_warning(f"百度site查询失败: {response.status_code}")
return {'has_收录': False, 'subdomains': []}
except Exception as e:
self._handle_exception(e, domain)
return {'has_收录': False, 'subdomains': []}
def check_history(self, domain):
"""
检查百度历史收录
:param domain: 域名
:return: dict - 检测结果
"""
try:
params = {
'wd': f'cache:{domain}',
'rn': '50'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(self.history_url, params=params, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 检查是否有历史收录
has_history = '百度快照' in content
# 检查是否有灰色内容
has_gray = '风险提示' in content or '安全警告' in content
return {
'has_history': has_history,
'has_gray': has_gray
}
else:
self._log_warning(f"百度历史查询失败: {response.status_code}")
return {'has_history': False, 'has_gray': False}
except Exception as e:
self._handle_exception(e, domain)
return {'has_history': False, 'has_gray': False}
def _extract_subdomains(self, content, domain):
"""
提取子域名
:param content: 搜索结果内容
:param domain: 主域名
:return: list - 子域名列表
"""
try:
# 提取所有包含域名的链接
pattern = r'https?://([a-zA-Z0-9-]+)\.' + re.escape(domain)
matches = re.findall(pattern, content)
# 去重并过滤空值
subdomains = list(set(matches))
subdomains = [sub for sub in subdomains if sub]
return subdomains
except Exception as e:
self._handle_exception(e, domain)
return []

View File

@@ -0,0 +1,70 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :base.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:55
@explain : 基础检测类
'''
from abc import ABC, abstractmethod
from loguru import logger
class BaseDetector(ABC):
"""
基础检测类
"""
def __init__(self):
"""
初始化检测类
"""
self.name = self.__class__.__name__
logger.info(f"初始化检测器: {self.name}")
@abstractmethod
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
pass
def _log_info(self, message):
"""
记录信息日志
:param message: 消息
"""
logger.info(f"[{self.name}] {message}")
def _log_warning(self, message):
"""
记录警告日志
:param message: 消息
"""
logger.warning(f"[{self.name}] {message}")
def _log_error(self, message):
"""
记录错误日志
:param message: 消息
"""
logger.error(f"[{self.name}] {message}")
def _handle_exception(self, e, domain):
"""
处理异常
:param e: 异常
:param domain: 域名
:return: dict - 错误结果
"""
self._log_error(f"检测域名 {domain} 出错: {e}")
return {'error': str(e)}

View File

@@ -0,0 +1,130 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :chinaz_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:01
@explain : 站长之家检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class ChinazDetector(BaseDetector):
"""
站长之家检测器
"""
def __init__(self):
"""
初始化站长之家检测器
"""
super().__init__()
self.url = 'https://seo.chinaz.com'
self.query_url = 'https://seo.chinaz.com/{domain}'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.query_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取标题
title = self._extract_title(content)
# 提取网站分类
category = self._extract_category(content)
# 检查是否包含敏感词
has_sensitive = self._check_sensitive(title, category)
return {
'title': title,
'category': category,
'has_sensitive': has_sensitive
}
else:
self._log_warning(f"站长之家查询失败: {response.status_code}")
return {'title': '', 'category': '', 'has_sensitive': False}
except Exception as e:
return self._handle_exception(e, domain)
def _extract_title(self, content):
"""
提取标题
:param content: 页面内容
:return: str - 标题
"""
try:
pattern = r'<title>(.*?)</title>'
match = re.search(pattern, content)
if match:
return match.group(1).strip()
return ''
except Exception as e:
self._handle_exception(e, 'extract_title')
return ''
def _extract_category(self, content):
"""
提取网站分类
:param content: 页面内容
:return: str - 分类
"""
try:
# 这里需要根据实际页面结构调整正则表达式
pattern = r'网站分类:<a[^>]+>(.*?)</a>'
match = re.search(pattern, content)
if match:
return match.group(1).strip()
return ''
except Exception as e:
self._handle_exception(e, 'extract_category')
return ''
def _check_sensitive(self, title, category):
"""
检查是否包含敏感词
:param title: 标题
:param category: 分类
:return: bool - 是否包含敏感词
"""
# 敏感分类
sensitive_categories = ['视频电影', '体育运动', '常用查询']
# 敏感词
sensitive_words = ['色情', '赌博', '博彩', '毒品', '暴力', '诈骗']
# 检查分类
for cat in sensitive_categories:
if cat in category:
return True
# 检查标题
for word in sensitive_words:
if word in title:
return True
return False

View File

@@ -0,0 +1,80 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :google_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:00
@explain : Google检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class GoogleDetector(BaseDetector):
"""
Google检测器
"""
def __init__(self):
"""
初始化Google检测器
"""
super().__init__()
self.site_url = 'https://www.google.com/search'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 检查Google site
site_result = self.check_site(domain)
return {
'site': site_result
}
except Exception as e:
return self._handle_exception(e, domain)
def check_site(self, domain):
"""
检查Google site收录
:param domain: 域名
:return: dict - 检测结果
"""
try:
params = {
'q': f'site:{domain}',
'num': '50'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(self.site_url, params=params, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 检查是否有收录
has_收录 = 'No results found for' not in content
return {
'has_收录': has_收录
}
else:
self._log_warning(f"Google site查询失败: {response.status_code}")
return {'has_收录': False}
except Exception as e:
self._handle_exception(e, domain)
return {'has_收录': False}

View File

@@ -0,0 +1,228 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :jucha_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:04
@explain : 聚查检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class JuchaDetector(BaseDetector):
"""
聚查检测器
"""
def __init__(self):
"""
初始化聚查检测器
"""
super().__init__()
self.url = 'https://www.jucha.com'
self.whois_url = 'https://www.jucha.com/whois/{domain}'
self.beian_url = 'https://www.jucha.com/beian/{domain}'
self.intercept_url = 'https://www.jucha.com/intercept/{domain}'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 检查WHOIS
whois_result = self.check_whois(domain)
# 检查备案
beian_result = self.check_beian(domain)
# 检查拦截
intercept_result = self.check_intercept(domain)
return {
'whois': whois_result,
'beian': beian_result,
'intercept': intercept_result
}
except Exception as e:
return self._handle_exception(e, domain)
def check_whois(self, domain):
"""
检查WHOIS
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.whois_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取WHOIS信息
whois_info = self._extract_whois_info(content)
return whois_info
else:
self._log_warning(f"聚查WHOIS查询失败: {response.status_code}")
return {'status': ''}
except Exception as e:
self._handle_exception(e, domain)
return {'status': ''}
def check_beian(self, domain):
"""
检查备案
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.beian_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取备案信息
beian_info = self._extract_beian_info(content)
return beian_info
else:
self._log_warning(f"聚查备案查询失败: {response.status_code}")
return {'has_beian': False, 'beian_year': '', 'is_enterprise': False, 'beian_match': False}
except Exception as e:
self._handle_exception(e, domain)
return {'has_beian': False, 'beian_year': '', 'is_enterprise': False, 'beian_match': False}
def check_intercept(self, domain):
"""
检查拦截
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.intercept_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 检查是否被拦截
is_normal = self._check_intercept_status(content)
return {
'normal': is_normal
}
else:
self._log_warning(f"聚查拦截查询失败: {response.status_code}")
return {'normal': False}
except Exception as e:
self._handle_exception(e, domain)
return {'normal': False}
def _extract_whois_info(self, content):
"""
提取WHOIS信息
:param content: 页面内容
:return: dict - WHOIS信息
"""
try:
# 提取状态信息
pattern = r'域名状态:<span[^>]+>(.*?)</span>'
match = re.search(pattern, content)
status = match.group(1).strip() if match else ''
return {
'status': status
}
except Exception as e:
self._handle_exception(e, 'extract_whois_info')
return {'status': ''}
def _extract_beian_info(self, content):
"""
提取备案信息
:param content: 页面内容
:return: dict - 备案信息
"""
try:
# 检查是否有备案
has_beian = '备案信息' in content
# 提取备案年份
beian_year = ''
pattern = r'审核时间:(\d{4})-\d{2}-\d{2}'
match = re.search(pattern, content)
if match:
beian_year = match.group(1)
# 提取单位性质
is_enterprise = '企业' in content
# 检查首网址和备案网址是否一致
beian_match = '网站首页网址' in content
return {
'has_beian': has_beian,
'beian_year': beian_year,
'is_enterprise': is_enterprise,
'beian_match': beian_match
}
except Exception as e:
self._handle_exception(e, 'extract_beian_info')
return {'has_beian': False, 'beian_year': '', 'is_enterprise': False, 'beian_match': False}
def _check_intercept_status(self, content):
"""
检查拦截状态
:param content: 页面内容
:return: bool - 是否正常
"""
try:
# 检查是否包含正常标识
if '正常' in content:
return True
# 检查是否包含拦截标识
if '拦截' in content:
return False
return False
except Exception as e:
self._handle_exception(e, 'check_intercept_status')
return False

View File

@@ -0,0 +1,214 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :juziseo_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:03
@explain : 桔子SEO检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class JuziseoDetector(BaseDetector):
"""
桔子SEO检测器
"""
def __init__(self):
"""
初始化桔子SEO检测器
"""
super().__init__()
self.url = 'https://seo.juziseo.com'
self.history_url = 'https://seo.juziseo.com/history/{domain}'
self.backlink_url = 'https://seo.juziseo.com/backlink/{domain}'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 检查历史信息
history_result = self.check_history(domain)
# 检查外链
backlink_result = self.check_backlink(domain)
return {
'history': history_result,
'backlink': backlink_result
}
except Exception as e:
return self._handle_exception(e, domain)
def check_history(self, domain):
"""
检查历史信息
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.history_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取历史信息
history_info = self._extract_history_info(content)
# 检查是否包含敏感词
has_sensitive = self._check_sensitive(history_info)
# 检查是否有百度历史收录
has_baidu_history = '百度历史收录' in content
# 检查是否有子域名
has_subdomains = '子域名' in content
# 检查是否为简体中文
is_simplified = self._check_simplified(content)
return {
'has_sensitive': has_sensitive,
'has_baidu_history': has_baidu_history,
'has_subdomains': has_subdomains,
'is_simplified': is_simplified
}
else:
self._log_warning(f"桔子SEO历史查询失败: {response.status_code}")
return {'has_sensitive': False, 'has_baidu_history': False, 'has_subdomains': False, 'is_simplified': True}
except Exception as e:
self._handle_exception(e, domain)
return {'has_sensitive': False, 'has_baidu_history': False, 'has_subdomains': False, 'is_simplified': True}
def check_backlink(self, domain):
"""
检查外链
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 构建查询URL
query_url = self.backlink_url.format(domain=domain)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(query_url, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 检查是否包含敏感词
has_sensitive = self._check_backlink_sensitive(content)
# 检查是否有子域名
has_subdomains = '子域名' in content
return {
'has_sensitive': has_sensitive,
'has_subdomains': has_subdomains
}
else:
self._log_warning(f"桔子SEO外链查询失败: {response.status_code}")
return {'has_sensitive': False, 'has_subdomains': False}
except Exception as e:
self._handle_exception(e, domain)
return {'has_sensitive': False, 'has_subdomains': False}
def _extract_history_info(self, content):
"""
提取历史信息
:param content: 页面内容
:return: str - 历史信息
"""
try:
# 这里需要根据实际页面结构调整正则表达式
pattern = r'<div class="history-info">(.*?)</div>'
match = re.search(pattern, content, re.DOTALL)
if match:
return match.group(1).strip()
return ''
except Exception as e:
self._handle_exception(e, 'extract_history_info')
return ''
def _check_sensitive(self, history_info):
"""
检查是否包含敏感词
:param history_info: 历史信息
:return: bool - 是否包含敏感词
"""
# 敏感词
sensitive_words = [
'色情', '赌博', '博彩', '毒品', '暴力', '诈骗',
'足球', '直播', '证券', '配资', '软件',
'体育', '商行', '下载', '影视', '网络',
'计算', 'app', 'HTML SiteMap', '模拟器', '传媒',
'二次元', '成人', '米乐', '小说', '凯发',
'人才', '华体', '娱乐', '开户'
]
for word in sensitive_words:
if word in history_info:
return True
return False
def _check_simplified(self, content):
"""
检查是否为简体中文
:param content: 页面内容
:return: bool - 是否为简体中文
"""
# 检查是否包含简体中文标识
if '简体中文' in content:
return True
# 检查是否包含繁体中文标识
if '繁体中文' in content:
return False
return True
def _check_backlink_sensitive(self, content):
"""
检查外链是否包含敏感词
:param content: 页面内容
:return: bool - 是否包含敏感词
"""
# 敏感词
sensitive_words = [
'内幕', '猛料', '精料', '高手', '绝杀',
'权威', '澳门', '色情', '赌博', '博彩'
]
for word in sensitive_words:
if word in content:
return True
return False

View File

@@ -0,0 +1,107 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :qihu360_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:59
@explain : 360检测器
'''
import requests
from curl_cffi import requests as curl_requests
import re
from app.detectors.base import BaseDetector
class Qihu360Detector(BaseDetector):
"""
360检测器
"""
def __init__(self):
"""
初始化360检测器
"""
super().__init__()
self.site_url = 'https://www.so.com/s'
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 检查360 site
site_result = self.check_site(domain)
return {
'site': site_result
}
except Exception as e:
return self._handle_exception(e, domain)
def check_site(self, domain):
"""
检查360 site收录
:param domain: 域名
:return: dict - 检测结果
"""
try:
params = {
'q': f'site:{domain}',
'pn': '1',
'rn': '50'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36'
}
# 使用curl_cffi模拟浏览器
response = curl_requests.get(self.site_url, params=params, headers=headers, impersonate='chrome', timeout=10)
if response.status_code == 200:
content = response.text
# 提取子域名
subdomains = self._extract_subdomains(content, domain)
# 检查是否有收录
has_收录 = '没有找到相关结果' not in content
return {
'has_收录': has_收录,
'subdomains': subdomains
}
else:
self._log_warning(f"360 site查询失败: {response.status_code}")
return {'has_收录': False, 'subdomains': []}
except Exception as e:
self._handle_exception(e, domain)
return {'has_收录': False, 'subdomains': []}
def _extract_subdomains(self, content, domain):
"""
提取子域名
:param content: 搜索结果内容
:param domain: 主域名
:return: list - 子域名列表
"""
try:
# 提取所有包含域名的链接
pattern = r'https?://([a-zA-Z0-9-]+)\.' + re.escape(domain)
matches = re.findall(pattern, content)
# 去重并过滤空值
subdomains = list(set(matches))
subdomains = [sub for sub in subdomains if sub]
return subdomains
except Exception as e:
self._handle_exception(e, domain)
return []

View File

@@ -0,0 +1,128 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :rdap_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:56
@explain : RDAP检测器
'''
import requests
from app.detectors.base import BaseDetector
class RDAPDetector(BaseDetector):
"""
RDAP检测器
"""
def __init__(self):
"""
初始化RDAP检测器
"""
super().__init__()
self.rdap_urls = {
'com': 'https://rdap.verisign.com/com/v1/domain/',
'net': 'https://rdap.verisign.com/net/v1/domain/'
}
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
# 提取顶级域名
tld = domain.split('.')[-1]
if tld not in self.rdap_urls:
return {'error': '不支持的顶级域名'}
# 构建RDAP查询URL
url = self.rdap_urls[tld] + domain
# 发送请求
response = requests.get(url, timeout=10)
if response.status_code == 200:
data = response.json()
return self._parse_rdap_response(data)
elif response.status_code == 404:
return {'status': 'available'}
else:
return {'error': f'RDAP查询失败: {response.status_code}'}
except Exception as e:
return self._handle_exception(e, domain)
def check_register_status(self, domain):
"""
检查注册状态
:param domain: 域名
:return: int - 注册状态码
"""
try:
result = self.check_domain(domain)
if 'error' in result:
return 9 # 状态未知
if result.get('status') == 'available':
return 2 # 可注册
# 检查域名状态
statuses = result.get('status', [])
if 'clientHold' in statuses:
return 7 # clientHold
elif 'serverHold' in statuses:
return 8 # serverHold
elif 'autoRenewPeriod' in statuses:
return 4 # 宽限期
elif 'redemptionPeriod' in statuses:
return 5 # 赎回期
elif 'pendingDelete' in statuses:
return 6 # 删除期
else:
return 3 # 已注册
except Exception as e:
self._handle_exception(e, domain)
return 10 # 检测失败
def _parse_rdap_response(self, data):
"""
解析RDAP响应
:param data: RDAP响应数据
:return: dict - 解析结果
"""
result = {
'status': 'registered',
'domain': data.get('ldhName'),
'statuses': data.get('status', []),
'registrar': None,
'creation_date': None,
'expiration_date': None,
'last_update': None
}
# 解析注册商信息
for entity in data.get('entities', []):
if 'registrar' in entity.get('roles', []):
result['registrar'] = entity.get('vcardArray', [[], []])[1][1][3]
break
# 解析时间信息
for event in data.get('events', []):
event_action = event.get('eventAction')
event_date = event.get('eventDate')
if event_action == 'registration':
result['creation_date'] = event_date
elif event_action == 'expiration':
result['expiration_date'] = event_date
elif event_action == 'last update':
result['last_update'] = event_date
return result

View File

@@ -0,0 +1,562 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :wayback_detector.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:57
@explain : Wayback检测器
'''
import html
import json
import re
import threading
import time
import zlib
from base64 import b64decode, b64encode
from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
import redis
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from app.config import config
from app.detectors.base import BaseDetector
class WaybackDetector(BaseDetector):
"""
Wayback检测器
"""
TITLE_PATTERN = re.compile(r'<title[^>]*>(.*?)</title>', re.IGNORECASE | re.DOTALL)
def __init__(self):
"""
初始化Wayback检测器
"""
super().__init__()
self.cdx_api_url = 'https://web.archive.org/cdx/search/cdx'
# 使用 id_ 直接取快照内容,避免回放页面额外壳层干扰正文匹配。
self.snapshot_url = 'https://web.archive.org/web/{timestamp}id_/{domain}'
self._timestamp_cache = {}
self._title_cache = {}
self._cache_lock = threading.Lock()
self.session = self._build_session()
self.redis_client = self._build_redis_client()
def _build_session(self):
session = requests.Session()
retry = Retry(
total=max(0, config.WAYBACK_RETRY_COUNT),
backoff_factor=0.5,
status_forcelist=(429, 500, 502, 503, 504),
allowed_methods=frozenset(["GET"]),
raise_on_status=False,
)
adapter = HTTPAdapter(max_retries=retry, pool_connections=10, pool_maxsize=10)
session.mount('http://', adapter)
session.mount('https://', adapter)
session.headers.update({
'User-Agent': config.WAYBACK_USER_AGENT,
'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
})
return session
def _build_redis_client(self):
try:
client = redis.Redis(
host=config.REDIS_HOST,
port=config.REDIS_PORT,
password=config.REDIS_PASSWORD,
db=config.REDIS_DB,
decode_responses=True,
socket_connect_timeout=5,
socket_timeout=5,
)
client.ping()
return client
except Exception:
return None
def _title_cache_key(self, domain, timestamp):
return f"domain_tool:wayback_title:{domain}:{timestamp}"
def _timestamp_cache_key(self, domain):
return f"domain_tool:wayback_timestamps:{domain}"
def _record_cache_key(self, domain):
return f"domain_tool:wayback_records:{domain}"
def _normalize_title(self, title):
normalized = html.unescape(title or '')
normalized = re.sub(r'\s+', ' ', normalized, flags=re.DOTALL).strip().lower()
return normalized
def _extract_title(self, content):
match = self.TITLE_PATTERN.search(content or '')
if not match:
return ''
return html.unescape(match.group(1)).strip()
def _load_cached_title(self, domain, timestamp):
cache_key = self._title_cache_key(domain, timestamp)
with self._cache_lock:
if cache_key in self._title_cache:
return self._title_cache[cache_key]
if self.redis_client:
try:
raw_value = self.redis_client.get(cache_key)
if raw_value:
data = json.loads(raw_value)
with self._cache_lock:
self._title_cache[cache_key] = data
return data
except Exception:
pass
return None
def _save_cached_title(self, domain, timestamp, data):
cache_key = self._title_cache_key(domain, timestamp)
with self._cache_lock:
self._title_cache[cache_key] = data
if self.redis_client:
try:
self.redis_client.set(
cache_key,
json.dumps(data, ensure_ascii=False),
ex=max(0, config.WAYBACK_TITLE_CACHE_TTL) or None,
)
except Exception:
pass
def _load_cached_timestamps(self, domain):
if domain in self._timestamp_cache:
return list(self._timestamp_cache[domain])
if not self.redis_client:
return None
try:
raw_value = self.redis_client.get(self._timestamp_cache_key(domain))
if not raw_value:
return None
compressed = b64decode(raw_value.encode('ascii'))
timestamps = json.loads(zlib.decompress(compressed).decode('utf-8'))
if isinstance(timestamps, list):
self._timestamp_cache[domain] = tuple(timestamps)
return list(timestamps)
except Exception:
return None
return None
def _save_cached_timestamps(self, domain, timestamps):
self._timestamp_cache[domain] = tuple(timestamps)
if not self.redis_client:
return
try:
payload = json.dumps(timestamps, separators=(',', ':')).encode('utf-8')
compressed = zlib.compress(payload, level=6)
self.redis_client.set(
self._timestamp_cache_key(domain),
b64encode(compressed).decode('ascii'),
ex=max(0, config.WAYBACK_TIMESTAMP_CACHE_TTL) or None,
)
except Exception:
pass
def _load_cached_records(self, domain):
cache_key = self._record_cache_key(domain)
with self._cache_lock:
if cache_key in self._timestamp_cache:
return [
{'timestamp': item[0], 'digest': item[1]}
for item in self._timestamp_cache[cache_key]
]
if not self.redis_client:
return None
try:
raw_value = self.redis_client.get(cache_key)
if not raw_value:
return None
compressed = b64decode(raw_value.encode('ascii'))
records = json.loads(zlib.decompress(compressed).decode('utf-8'))
if isinstance(records, list):
normalized = tuple(
(item.get('timestamp', ''), item.get('digest', ''))
for item in records if isinstance(item, dict)
)
with self._cache_lock:
self._timestamp_cache[cache_key] = normalized
return [
{'timestamp': item[0], 'digest': item[1]}
for item in normalized if item[0]
]
except Exception:
return None
return None
def _save_cached_records(self, domain, records):
cache_key = self._record_cache_key(domain)
normalized = tuple(
(item.get('timestamp', ''), item.get('digest', ''))
for item in (records or [])
if item and item.get('timestamp')
)
with self._cache_lock:
self._timestamp_cache[cache_key] = normalized
if not self.redis_client:
return
try:
payload = json.dumps(
[{'timestamp': item[0], 'digest': item[1]} for item in normalized],
separators=(',', ':')
).encode('utf-8')
compressed = zlib.compress(payload, level=6)
self.redis_client.set(
cache_key,
b64encode(compressed).decode('ascii'),
ex=max(0, config.WAYBACK_TIMESTAMP_CACHE_TTL) or None,
)
except Exception:
pass
def _fetch_cdx_records(self, domain, limit=None, fast_latest=False):
response = None
try:
params = {
'url': domain,
'output': 'txt',
'fl': 'timestamp,digest',
'filter': ['statuscode:200', 'mimetype:text/html'],
}
if limit is not None:
params['limit'] = str(limit)
if fast_latest:
params['fastLatest'] = 'true'
response = self.session.get(
self.cdx_api_url,
params=params,
timeout=config.WAYBACK_CDX_TIMEOUT,
stream=True,
)
if response.status_code != 200:
self._log_warning(f"获取快照记录失败: {response.status_code}")
return []
records = []
seen = set()
for raw_line in response.iter_lines(decode_unicode=True):
line = (raw_line or '').strip()
if not line:
continue
parts = line.split()
timestamp = parts[0].strip() if parts else ''
digest = parts[1].strip() if len(parts) > 1 else ''
if not timestamp or timestamp in seen:
continue
seen.add(timestamp)
records.append({'timestamp': timestamp, 'digest': digest})
return records
except Exception as e:
self._handle_exception(e, domain)
return []
finally:
try:
if response is not None:
response.close()
except Exception:
pass
def get_latest_snapshot_record(self, domain):
records = self._fetch_cdx_records(domain, limit=-1, fast_latest=True)
return records[0] if records else None
def get_snapshot_records(self, domain):
cached_records = self._load_cached_records(domain)
if cached_records is not None:
return cached_records
records = self._fetch_cdx_records(domain)
if records:
self._save_cached_records(domain, records)
self._save_cached_timestamps(domain, [item['timestamp'] for item in records])
return records
def check_domain(self, domain):
"""
检测域名
:param domain: 域名
:return: dict - 检测结果
"""
try:
return self.scan_snapshots(domain)
except Exception as e:
return self._handle_exception(e, domain)
def get_snapshot_timestamps(self, domain):
cached_records = self._load_cached_records(domain)
if cached_records is not None:
return [item['timestamp'] for item in cached_records if item.get('timestamp')]
cached_timestamps = self._load_cached_timestamps(domain)
if cached_timestamps is not None:
return cached_timestamps
records = self.get_snapshot_records(domain)
return [item['timestamp'] for item in records if item.get('timestamp')]
def _fetch_snapshot_title(self, domain, timestamp):
cached = self._load_cached_title(domain, timestamp)
if cached is not None:
return cached
snapshot_url = self.snapshot_url.format(timestamp=timestamp, domain=domain)
response = None
try:
response = self.session.get(
snapshot_url,
timeout=config.WAYBACK_SNAPSHOT_TIMEOUT,
stream=True,
)
if response.status_code != 200:
data = {'timestamp': timestamp, 'title': '', 'ok': False}
self._save_cached_title(domain, timestamp, data)
return data
content_type = (response.headers.get('Content-Type') or '').lower()
if content_type and 'text/html' not in content_type and 'application/xhtml+xml' not in content_type:
data = {'timestamp': timestamp, 'title': '', 'ok': False}
self._save_cached_title(domain, timestamp, data)
return data
chunks = []
total_bytes = 0
found_title = False
for chunk in response.iter_content(chunk_size=4096, decode_unicode=True):
if not chunk:
continue
chunks.append(chunk)
total_bytes += len(chunk.encode('utf-8', errors='ignore'))
current_text = ''.join(chunks)
if '</title>' in current_text.lower():
found_title = True
break
if total_bytes >= config.WAYBACK_TITLE_MAX_BYTES:
break
content = ''.join(chunks)
title = self._extract_title(content) if found_title or content else ''
data = {'timestamp': timestamp, 'title': title, 'ok': True}
self._save_cached_title(domain, timestamp, data)
return data
except Exception:
data = {'timestamp': timestamp, 'title': '', 'ok': False}
self._save_cached_title(domain, timestamp, data)
return data
finally:
try:
if response is not None:
response.close()
except Exception:
pass
def get_snapshot_years(self, domain):
"""
获取快照年份
:param domain: 域名
:return: list - 快照年份列表
"""
try:
years = set()
for timestamp in self.get_snapshot_timestamps(domain):
if len(timestamp) >= 4:
years.add(int(timestamp[:4]))
return sorted(years)
except Exception as e:
self._handle_exception(e, domain)
return []
def has_sensitive_content(self, domain):
"""
检查是否包含敏感内容
:param domain: 域名
:return: bool - 是否包含敏感内容
"""
try:
result = self.scan_snapshots(domain)
return result.get('has_sensitive_content', False)
except Exception as e:
self._handle_exception(e, domain)
return False
def scan_snapshots(self, domain, sensitive_words=None, stop_on_first_hit=True):
sensitive_words = sensitive_words or config.load_sensitive_words()
latest_record = self.get_latest_snapshot_record(domain)
latest_timestamp = (latest_record or {}).get('timestamp')
latest_digest = (latest_record or {}).get('digest', '')
matched_word = None
matched_timestamp = None
matched_title = None
fetched_snapshot_count = 0
failed_snapshot_count = 0
unique_title_count = 0
duplicate_title_skipped = 0
digest_duplicate_skipped = 0
started_at = time.time()
progress_interval = max(1, config.WAYBACK_PROGRESS_INTERVAL)
title_seen = set()
digest_seen = set()
checked_snapshot_count = 0
domain_concurrency = max(1, config.WAYBACK_DOMAIN_CONCURRENCY)
if latest_timestamp:
latest_result = self._fetch_snapshot_title(domain, latest_timestamp)
checked_snapshot_count = 1
if latest_result and latest_result.get('ok'):
fetched_snapshot_count = 1
title = latest_result.get('title', '')
normalized_title = self._normalize_title(title)
if latest_digest:
digest_seen.add(latest_digest)
if normalized_title:
title_seen.add(normalized_title)
unique_title_count = 1
matched_word = self._find_sensitive_word(title, sensitive_words)
if matched_word and stop_on_first_hit:
matched_timestamp = latest_timestamp
matched_title = title
return {
'snapshot_years': [int(latest_timestamp[:4])] if len(latest_timestamp) >= 4 else [],
'has_sensitive_content': True,
'matched_word': matched_word,
'matched_timestamp': matched_timestamp,
'matched_title': matched_title,
'backlink_count': 0,
'backlink_count_gt_10': False,
'checked_snapshot_count': checked_snapshot_count,
'fetched_snapshot_count': fetched_snapshot_count,
'failed_snapshot_count': failed_snapshot_count,
'unique_title_count': unique_title_count,
'duplicate_title_skipped': duplicate_title_skipped,
'digest_duplicate_skipped': digest_duplicate_skipped,
'elapsed_seconds': round(time.time() - started_at, 2),
}
else:
failed_snapshot_count += 1
records = sorted(self.get_snapshot_records(domain), key=lambda item: item.get('timestamp', ''), reverse=True)
years = sorted({int(item['timestamp'][:4]) for item in records if len(item.get('timestamp', '')) >= 4})
checked_snapshot_count = len(records)
pending_records = []
for item in records:
timestamp = item.get('timestamp', '')
digest = item.get('digest', '')
if not timestamp:
continue
if latest_timestamp and timestamp == latest_timestamp:
continue
if digest and digest in digest_seen:
digest_duplicate_skipped += 1
continue
if digest:
digest_seen.add(digest)
pending_records.append(item)
with ThreadPoolExecutor(max_workers=domain_concurrency) as executor:
pending = {}
index = 0
finished_count = 1 if latest_timestamp else 0
stop_requested = False
while (index < len(pending_records) or pending) and not stop_requested:
while index < len(pending_records) and len(pending) < domain_concurrency and not stop_requested:
timestamp = pending_records[index]['timestamp']
future = executor.submit(self._fetch_snapshot_title, domain, timestamp)
pending[future] = timestamp
index += 1
if not pending:
break
done, _ = wait(list(pending.keys()), return_when=FIRST_COMPLETED)
for future in done:
timestamp = pending.pop(future, None)
finished_count += 1
try:
result = future.result()
except Exception as e:
failed_snapshot_count += 1
self._handle_exception(e, domain)
continue
if not result or not result.get('ok'):
failed_snapshot_count += 1
continue
fetched_snapshot_count += 1
title = result.get('title', '')
normalized_title = self._normalize_title(title)
if normalized_title:
if normalized_title in title_seen:
duplicate_title_skipped += 1
else:
title_seen.add(normalized_title)
unique_title_count += 1
matched_word = self._find_sensitive_word(title, sensitive_words)
if matched_word:
matched_timestamp = timestamp
matched_title = title
if stop_on_first_hit:
stop_requested = True
if finished_count % progress_interval == 0:
elapsed = round(time.time() - started_at, 2)
self._log_info(
f"{domain} 时光机进度: {finished_count}/{checked_snapshot_count},成功 {fetched_snapshot_count},失败 {failed_snapshot_count},唯一标题 {unique_title_count},标题重复跳过 {duplicate_title_skipped}digest 重复跳过 {digest_duplicate_skipped},耗时 {elapsed}s"
)
if config.WAYBACK_REQUEST_DELAY > 0:
time.sleep(config.WAYBACK_REQUEST_DELAY)
if stop_requested:
for future in pending:
future.cancel()
return {
'snapshot_years': years,
'has_sensitive_content': matched_word is not None,
'matched_word': matched_word,
'matched_timestamp': matched_timestamp,
'matched_title': matched_title,
'backlink_count': 0,
'backlink_count_gt_10': False,
'checked_snapshot_count': checked_snapshot_count,
'fetched_snapshot_count': fetched_snapshot_count,
'failed_snapshot_count': failed_snapshot_count,
'unique_title_count': unique_title_count,
'duplicate_title_skipped': duplicate_title_skipped,
'digest_duplicate_skipped': digest_duplicate_skipped,
'elapsed_seconds': round(time.time() - started_at, 2),
}
def _check_sensitive_words(self, content):
"""
检查敏感词
:param content: 内容
:return: bool - 是否包含敏感词
"""
return self._find_sensitive_word(content, config.load_sensitive_words()) is not None
def _find_sensitive_word(self, content, sensitive_words):
for word in sensitive_words or []:
if word and word in (content or ''):
return word
return None
def get_backlink_count(self, domain):
"""
当前策略仅扫描标题,不再抓取正文,友链数量默认返回 0。
"""
return 0
def _count_backlinks(self, content):
return 0

View File

@@ -0,0 +1,3 @@
{
"suffixes": ".com,.net"
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

68
domainCheck/app/main.py Normal file
View File

@@ -0,0 +1,68 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :main.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:09
@explain : 系统主入口
'''
import sys
import os
# 添加项目根目录到 sys.path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from loguru import logger
from PySide6.QtWidgets import QApplication
from app.ui.main_window import MainWindow
from app.utils.database import Database
from app.config import config
# 配置日志
logger.add(
os.path.join(config.LOG_DIR, config.LOG_FILE),
level=config.LOG_LEVEL,
rotation="10 MB",
compression="zip"
)
def main():
"""
主函数
"""
try:
# 初始化数据库连接
db = Database(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
# 测试数据库连接
db.execute("SELECT 1")
logger.info("数据库连接成功")
# 初始化应用程序
app = QApplication(sys.argv)
window = MainWindow()
window.show()
# 运行应用程序
sys.exit(app.exec())
except Exception as e:
logger.error(f"启动应用程序失败: {e}")
sys.exit(1)
finally:
# 关闭数据库连接
if 'db' in locals():
db.close()
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,4 @@
{
"proxy_enable": false,
"proxy_url": ""
}

15108
domainCheck/app/sdk_leg.js Normal file

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,897 @@
process_ = process;
require_ = require;
delete Buffer;
// delete process;
delete require;
delete global;
delete module;
delete exports;
delete __filename;
delete __dirname;
delete SharedArrayBuffer;
AsObj = {
// print: console.log,
print: function () { },
// print_:console.log,
}
no_print = ['Boolean','String','parseFloat','Array','Object','prepareStackTrace_'];
function watch(object, WatchName) {
const handler = {
get(target, property, receiver) {
if (
property !== 'isNaN' &&
property !== 'encodeURI' &&
property !== "Uint8Array" &&
property !== 'undefined' &&
property !== 'JSON' &&
property !== 'Number' &&
!no_print.includes(property) &&
property !== Symbol.for('nodejs.util.inspect.custom') &&
typeof property !== 'symbol'
) {
if (property === 'global') {
return undefined;
}
if (property === 'Buffer') {
return undefined;
}
if (property === 'process') {
return undefined;
}
if (WatchName === 'config_data') {
debugger
}
if (WatchName.indexOf('.prototype') != -1 && target[property] != undefined) {
return Reflect.get(target, property, receiver);
}
AsObj.print(
"方法:", "get",
"对象:", WatchName,
"属性:", property,
"属性类型:", typeof property,
"属性值:", typeof target[property] == 'object' ? "object" : target[property],
"属性值类型:", typeof target[property]
);
}
if (WatchName === 'top') {
return window;
}
return Reflect.get(target, property, receiver);
},
set(target, property, value, receiver) {
if (WatchName.indexOf('.prototype') != -1 && value != undefined) {
return Reflect.set(target, property, value, receiver);
}
AsObj.print(
"方法:", "set",
"对象:", WatchName,
"属性:", property,
"属性类型:", typeof property,
"属性值:", typeof value == 'object' ? "object" : value,
"属性值类型:", typeof target[property]
);
return Reflect.set(target, property, value, receiver);
},
// in操作 检测
has(target, property) {
AsObj.print(
"代理对象:", WatchName,
"方法:", "has",
"检查属性:", property,
"结果:", typeof target[property] == 'object' ? "object" : target[property],
);
return Reflect.has(target, property);
},
// Object.key 检测
ownKeys(target) {
AsObj.print(
"方法:", "ownKeys",
"对象:", target+''
);
return Reflect.ownKeys(target);
}
};
return new Proxy(object, handler);
}
// function watch(object, WatchName) {
// return object
// }
// 保护函数toString检测
const safeFunction = function safeFunction(func) {
//处理安全函数
Function.prototype.$call = Function.prototype.call;
const $toString = Function.toString;
const myFunction_toString_symbol = Symbol('('.concat('', ')'));
const myToString = function myToString() {
return typeof this === 'function' && this[myFunction_toString_symbol] || $toString.$call(this);
}
const set_native = function set_native(func, key, value) {
Object.defineProperty(func, key, {
"enumerable": false,
"configurable": true,
"writable": true,
"value": value
});
}
delete Function.prototype['toString'];
set_native(Function.prototype, "toString", myToString);
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
const safe_Function = function safe_Function(func) {
set_native(func, myFunction_toString_symbol, "function" + (func.name ? " " + func.name : "") + "() { [native code] }");
}
return safe_Function(func)
}
//创建函数,并代理上
const makeFunction = function makeFunction(name) {
v_log = AsObj.print;
// 使用 Function 保留函数名
func = new Function("v_log", `
return function ${name}() {
v_log('函数${name}传参-->', arguments);
};
`)(v_log); // 传递 v_log 到动态函数
safeFunction(func);
func = watch(func,`${name}`);
func.prototype = watch(func.prototype, `${name}.prototype`);
return func;
}
!(function () {
"use strict";
const $toString = Function.toString;
const myFunction_toString_symbol = Symbol('('.concat('', ')_', (Math.random() + '').toString(36)));
const mytoString = function () {
return typeof this == 'function' && this[myFunction_toString_symbol] || $toString.call(this);
};
function set_native(func, key, value) {
Object.defineProperty(func, key, {
"enumerable": false,
"configurable": true,
"writable": true,
"value": value
})
};
delete Function.prototype['toString'];
set_native(Function.prototype, "toString", mytoString);
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
this.func_set_native = function (func) {
set_native(func, myFunction_toString_symbol, `function ${myFunction_toString_symbol, func.name || ''}() { [native code] }`)
}
}).call(globalThis);
// 重写全局对象原型链
function setTostringAndstringTag(obj) {
Object.defineProperties(obj.prototype, {
[Symbol.toStringTag]: {
configurable: true,
value: obj.name
}
});
safeFunction(obj);
};
// 创建标签原型
function createTagProto(propObj,portotypeObj) {
let res = propObj + ' = ' + 'function ' + propObj + '() { throw new TypeError("Illegal constructor"); };\n';
res += 'setTostringAndstringTag(' + propObj + ',null);\n';
if (portotypeObj) {
for (let key in portotypeObj) {
res += propObj + '.prototype.' + portotypeObj[key] + '= function ' + portotypeObj[key] + '() {AsObj.print("'+propObj+'.prototype.' + portotypeObj[key] + '原型方法(需在实例对象上补该方法)::",arguments)};\n';
res += 'globalThis.func_set_native(' + propObj + '.prototype.' + portotypeObj[key] + ');\n';
}
}
eval(res);
}
Object.defineProperties(globalThis, {
[Symbol.toStringTag]: {
configurable: true,
value: 'Window'
}
});
for (let key in globalThis) {
if (typeof globalThis[key] === 'function') {
safeFunction(globalThis[key])
}
}
for (let key in console) {
if (typeof console[key] === 'function') {
safeFunction(console[key])
}
}
createTagProto('EventTarget',['addEventListener']);
createTagProto('WindowProperties');
createTagProto('Window');
window = globalThis;
window.__proto__ = Window.prototype;
window.__proto__.__proto__ = WindowProperties.prototype;
window.__proto__.__proto__.__proto__ = EventTarget.prototype;
Window.__proto__ = EventTarget;
Object.defineProperty(window, 'WindowProperties', {
get: function () {
return undefined;
}
})
function randoms(min, max) {
return Math.floor(Math.random() * (max - min + 1) + min)
}
function getRandomValues(buf) {
var min = 0,
max = 255;
if (buf instanceof Uint16Array) {
max = 65535;
} else if (buf instanceof Uint32Array) {
max = 4294967295;
}
for (var element in buf) {
buf[element] = randoms(min, max);
}
return buf;
}
self = window.self = window;
frames = window.frames = window;
top = window.top = window;
parent = window.parent = window;
global = window.global = window;
Object.defineProperty(window, "global", {
configurable:false,
enumerable: true,
set: undefined,
get: function global(){
return window
}
})
Object.defineProperty(window, "top", {
configurable:false,
enumerable: true,
set: undefined,
get: function top(){
return window
}
})
Object.defineProperty(window, "self", {
configurable:false,
enumerable: true,
set: undefined,
get: function self(){
return window
}
})
Object.defineProperty(window, "parent", {
configurable:false,
enumerable: true,
set: undefined,
get: function parent(){
return window
}
})
Object.defineProperty(window, "frames", {
configurable:false,
enumerable: true,
set: undefined,
get: function frames(){
return window
}
})
innerWidth = 1536
innerHeight = 715
outerWidth = 1536
outerHeight = 824
devicePixelRatio = 1.25;
screenLeft = 0;
screenX = 0;
screenTop = 0;
screenY = 0;
opener = null;
isSecureContext = true;
crypto = {
getRandomValues:getRandomValues
};
createTagProto('DOMStringMap')
createTagProto('HTMLHeadElement',['insertBefore','removeChild'])
createTagProto('HTMLBodyElement',['addEventListener','appendChild','removeChild'])
createTagProto('HTMLHtmlElement',['getAttribute'])
createTagProto('HTMLDocument')
createTagProto('Document',['browsingTopics','appendChild','querySelector','evaluate','querySelectorAll','removeChild','requestStorageAccess','requestStorageAccessFor','hasStorageAccess','getElementsByTagName','hasPrivateToken','createElement','hasRedemptionRecord','hasFocus'])
createTagProto('Node')
document = {};
document.__proto__ = HTMLDocument.prototype;
document.__proto__.__proto__ = Document.prototype;
document.__proto__.__proto__.__proto__ = Node.prototype;
document.__proto__.__proto__.__proto__.__proto__ = EventTarget.prototype;
HTMLDocument.__proto__ = Document;
HTMLDocument.__proto__.__proto__ = Node;
HTMLDocument.__proto__.__proto__.__proto__ = EventTarget;
Document.__proto__ = Node;
Document.__proto__.__proto__ = EventTarget;
Node.__proto__ = EventTarget;
createTagProto('Plugin');
createTagProto('PluginArray');
plugins0 = {
name: 'PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type: 'application/pdf',
},
'1':{
type:'text/pdf'
}
}
plugins0['0'].enabledPlugin = plugins0;
plugins0['1'].enabledPlugin = plugins0;
plugins1 = {
name: 'Chrome PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type:'application/pdf'
},
'1': {
type:'text/pdf'
}
}
plugins1['0'].enabledPlugin = plugins1;
plugins1['1'].enabledPlugin = plugins1;
plugins2 = {
name: 'Chromium PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type:'application/pdf'
},
'1': {
type:'text/pdf'
}
}
plugins2['0'].enabledPlugin = plugins2;
plugins2['1'].enabledPlugin = plugins2;
plugins3 = {
name: 'Microsoft Edge PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0':{
type:'application/pdf'
},
'1': {
type:'text/pdf'
}
}
plugins3['0'].enabledPlugin = plugins3;
plugins3['1'].enabledPlugin = plugins3;
plugins4 = {
name: 'WebKit built-in PDF',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type:'application/pdf'
},
'1':{
type:'text/pdf'
}
}
plugins4['0'].enabledPlugin = plugins4;
plugins4['1'].enabledPlugin = plugins4;
plugins = {
length: 5,
'0': plugins0,
'1': plugins1,
'2': plugins2,
'3': plugins3,
'4': plugins4,
namedItem : function (name) {
AsObj.print('Plugin-namedItem:', name)
},
item: function (index) {
AsObj.print('Plugin-item:', index)
return watch(plugins0,'item-'+index);
},
refresh: function () {
AsObj.print('Plugin-refresh:',arguments)
},
}
plugins.__proto__ = PluginArray.prototype;
MimeTypeArray = function MimeTypeArray() {
this.length = 2;
this['0'] = {
suffixes: 'pdf',
type: 'application/pdf',
description:"Portable Document Format",
enabledPlugin: plugins0
};
this['1'] = {
suffixes: 'pdf',
type: 'text/pdf',
description:"Portable Document Format",
enabledPlugin: plugins0
};
};
MimeTypeArray.prototype.toString = function () { return '[object MimeTypeArray]'; }
MimeTypeArray.toString = function () { return 'function MimeTypeArray() { [native code] }'; }
Object.defineProperties(MimeTypeArray.prototype, { [Symbol.toStringTag]: { value: 'MimeTypeArray' } })
MimeTypeArrayc = new MimeTypeArray();
MimeTypeArrayc[Symbol.iterator] = function* () {
for (let key in this) {
yield this[key];
}
}
// 创建电池管理器对象原型
const BatteryManager = {
level: 1,
charging: true,
chargingTime: 0,
dischargingTime: null,
onchargingchange: null,
onlevelchange: null,
toString: function toString() {
return `BatteryManager {
charging: ${this.charging},
level: ${this.level},
chargingTime: ${this.chargingTime},
dischargingTime: ${this.dischargingTime}
}`
}
}
window.BatteryManager = BatteryManager;
Promise2 = {
then: function () {
return this;
},
catch: function (){},
};
createTagProto('Bluetooth');
createTagProto('Navigator');
Navigator.prototype.hardwareConcurrency = 8;
Navigator.prototype.userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36';
Navigator.prototype.appVersion = '5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36'
Navigator.prototype.appName = 'Netscape';
Navigator.prototype.appCodeName = 'Mozilla';
Navigator.prototype.vendor = 'Google Inc.';
Navigator.prototype.maxTouchPoints = 10;
Navigator.prototype.platform = 'Win32';
Navigator.prototype.adAuctionComponents = function adAuctionComponents() {
AsObj.print('adAuctionComponents:::', arguments)
}
safeFunction(Navigator.prototype.adAuctionComponents)
Navigator.prototype.runAdAuction = function runAdAuction() {
AsObj.print('runAdAuction:::', arguments)
}
safeFunction(Navigator.prototype.runAdAuction)
Navigator.prototype.canLoadAdAuctionFencedFrame = makeFunction('canLoadAdAuctionFencedFrame')
Navigator.prototype.deprecatedReplaceInURN = makeFunction('deprecatedReplaceInURN')
Navigator.prototype.deprecatedURNToURL = makeFunction('deprecatedURNToURL')
Navigator.prototype.joinAdInterestGroup = makeFunction('joinAdInterestGroup')
Navigator.prototype.leaveAdInterestGroup = makeFunction('leaveAdInterestGroup')
Navigator.prototype.updateAdInterestGroups = makeFunction('updateAdInterestGroups')
Navigator.prototype.connection = watch({
downlink: 9.1,
effectiveType: '4g',
rtt: 0,
saveData: false,
},'connection')
Navigator.prototype.language = 'zh-CN';
Navigator.prototype.languages = ["zh-CN"];
Navigator.prototype.plugins = plugins;
Navigator.prototype.webdriver = false;
Navigator.prototype.cookieEnabled = true;
Navigator.prototype.onLine = true;
Navigator.prototype.doNotTrack = null;
Navigator.prototype.bluetooth = {};
Navigator.prototype.product = 'Gecko'
Navigator.prototype.deviceMemory = 8
Navigator.prototype.mediaDevices = watch({
enumerateDevices: function enumerateDevices() {
return new Promise((resolve, reject) => {
const offer = [
{deviceId: '', kind: 'audioinput', label: '', groupId: ''},
{deviceId: '', kind: 'videoinput', label: '', groupId: ''},
{deviceId: '', kind: 'audiooutput', label: '', groupId: ''},
]
resolve(offer);
});
},
getUserMedia: function getUserMedia() {
AsObj.print('getUserMedia:::', arguments)
}
},'mediaDevices')
Navigator.prototype.storage = {
estimate: function estimate() {
AsObj.print('estimate:::', arguments)
return new Promise((resolve, reject) => {
const offer = {
usage: 0, // 1GB
quota: 2147483648, // 1GB,
usageDetails: {caches: 512, indexedDB: 2855}
};
resolve(offer);
});
}
}
Navigator.prototype.webkitPersistentStorage = watch({},'webkitPersistentStorage')
Navigator.prototype.webkitTemporaryStorage = watch({
queryUsageAndQuota: function queryUsageAndQuota() {
AsObj.print('queryUsageAndQuota:::', arguments)
return new Promise((resolve, reject) => {
const offer = {
usage: 1024 * 1024 * 1024, // 1GB
quota: 1024 * 1024 * 1024, // 1GB
};
resolve(offer);
});
}
},'webkitTemporaryStorage')
Navigator.prototype.bluetooth.__proto__ = Bluetooth.prototype;
Navigator.prototype.javaEnabled = function javaEnabled() {
return false
};
safeFunction(Navigator.prototype.javaEnabled)
Navigator.prototype.getBattery = function getBattery() {
AsObj.print('getBattery:::', arguments)
return Promise.resolve({
__proto__: BatteryManager,
// 动态参数配置(示例值)
level: 1,
charging: true,
dischargingTime: null // 2小时放电时间
})
}
safeFunction(Navigator.prototype.getBattery)
Navigator.prototype.registerProtocolHandler = function registerProtocolHandler() {
AsObj.print('registerProtocolHandler:::',arguments)
}
safeFunction(Navigator.prototype.registerProtocolHandler)
Navigator.prototype.mimeTypes = watch(MimeTypeArrayc,'mimeTypes');
Navigator.prototype.geolocation = {
getCurrentPosition: function getCurrentPosition() {
return Promise2;
}
}
Navigator.prototype.pdfViewerEnabled = true;
Navigator.prototype.doNotTrack = null;
Navigator.prototype.keyboard = watch({
getLayoutMap: function getLayoutMap() {
AsObj.print('Navigator.prototype.keyboard:', arguments)
return {
then: function () {
// arguments[0](watch({
// size: 48,
// values: function () {
// return ['k', 'g', '2', '0', 'v', 'a', '`', 'l', '\\', "'", 'w', '8', 'm', 'h', '.', '7', '1', 'p', 'd', 'f', 'o', 'q', 'c', 'n', '[', 'z', 'y', '3', '6', '5', 'x', '/', '\\', ',', '-', '4', 'b', 't', '9', 's', 'i', 'u', '=', 'j', ';', 'r', ']', 'e']
// }
// }, 'navigator.keyboard.getLayoutMap.then'))
return {
catch: function () {
arguments[0]({
message:'getLayoutMap() must be called from a top-level browsing context or allowed by the permission policy.'
})
}
}
}
}
}
},'navigator.keyboard')
Navigator.prototype.permissions = watch({
query: function query() {
arg_obj = arguments[0];
if (arg_obj.name === 'audio_capture') {
return new Promise((resolve, reject) => {
const offer = {
state: 'prompt',
onchange: null,
name:arg_obj.name
};
resolve(offer);
});
}
if (arg_obj.name === 'microphone') {
return {
then: function () {
arguments[0](watch({
state: 'denied',
onchange: null,
name: 'audio_capture'
},'permissions.query.microphone'));
return {catch:function(){}}
},
catch:function(){}
}
}
if (arg_obj.name === 'camera') {
return {
then: function () {
arguments[0](watch({
state: 'prompt',
onchange: null,
name: 'video_capture'
},'permissions.query.camera'));
return {catch:function(){}}
},
catch:function(){}
}
}
AsObj.print('permissions.query:::', arguments)
}
},'permissions')
Navigator.prototype.productSub = '20030107'
Navigator.prototype.getGamepads = function getGamepads() {
AsObj.print('getGamepads:::', arguments)
return [null,null,null,null]
}
safeFunction(Navigator.prototype.getGamepads)
Navigator.prototype.sendBeacon = makeFunction('sendBeacon')
Navigator.prototype.deprecatedRunAdAuctionEnforcesKAnonymity = false
Navigator.prototype.gpu = watch({
getPreferredCanvasFormat: function getPreferredCanvasFormat() {
AsObj.print('gpu.getPreferredCanvasFormat:', arguments)
return 'bgra8unorm'
},
wgslLanguageFeatures: watch({
size: 7,
values: function values() {
debugger
AsObj.print('wgslLanguageFeatures.values')
return ['packed_4x8_integer_dot_product', 'unrestricted_pointer_parameters', 'subgroup_uniformity', 'subgroup_id', 'pointer_composite_access', 'readonly_and_readwrite_storage_textures', 'uniform_buffer_standard_layout']
},
}, 'gpu.wgslLanguageFeatures'),
requestAdapter: function requestAdapter() {
AsObj.print('gpu.requestAdapter:', arguments)
return {
then: function () {
arguments[0](watch({
features: watch({
size: 19,
values: function () {
return ['depth32float-stencil8', 'rg11b10ufloat-renderable', 'bgra8unorm-storage', 'texture-formats-tier1', 'texture-compression-bc', 'dual-source-blending', 'core-features-and-limits', 'float32-filterable', 'indirect-first-instance', 'float32-blendable', 'depth-clip-control', 'texture-compression-bc-sliced-3d', 'timestamp-query', 'texture-formats-tier2', 'clip-distances', 'shader-f16', 'primitive-index', 'texture-component-swizzle', 'subgroups']
}
}, 'gpu.requestAdapter.features'),
info: watch({ vendor: 'intel', architecture: 'gen-11', device: '', description: '', subgroupMinSize: 16 }, 'gpu.requestAdapter.info'),
limits: watch({
maxBufferSize: 2147483648,
maxStorageBufferBindingSize:2147483644
}, 'gpu.requestAdapter.limits'),
catch:function(){}
}, 'gpu.requestAdapter'));
return {
catch: function () {
return {
then: function () {
arguments[0]()
return {catch:function(){}}
}
}
}
};
},
catch: function () {
}
}
}
},'navigator.gpu')
Navigator.prototype.userAgentData = watch({
brands:[
{
"brand": "Google Chrome",
"version": "143"
},
{
"brand": "Chromium",
"version": "143"
},
{
"brand": "Not A(Brand",
"version": "24"
}
],
mobile: false,
platform: "Windows",
getHighEntropyValues: function getHighEntropyValues() {
if (arguments[0] + '' === 'architecture,bitness,model,platformVersion,uaFullVersion,wow64') {
return new Promise((resolve, reject) => {
const offer = {
"architecture": "x86",
"bitness": "64",
"brands": [
{
"brand": "Not:A-Brand",
"version": "99"
},
{
"brand": "Google Chrome",
"version": "145"
},
{
"brand": "Chromium",
"version": "145"
}
],
"mobile": false,
"model": "",
"platform": "Windows",
"platformVersion": "10.0.0",
"uaFullVersion": "145.0.7632.117",
"wow64": false
};
resolve(offer);
});
}
AsObj.print('getHighEntropyValues:::', arguments)
}
},'userAgentData')
navigator = {};
navigator.__proto__ = Navigator.prototype;
createTagProto('Location');
location = {
"ancestorOrigins": {},
"href": "https://www.neimanmarcus.com/",
"origin": "https://www.neimanmarcus.com",
"protocol": "https:",
"host": "www.neimanmarcus.com",
"hostname": "www.neimanmarcus.com",
"port": "",
"pathname": "/",
"search": "",
"hash": ""
};
location.__proto__ = Location.prototype;
location.toString = function toString() {
return this.href;
}
createTagProto('Screen');
Screen.prototype = Object.assign(Screen.prototype, {
availHeight: 824,
availLeft: 0,
availTop: 0,
availWidth: 1536,
colorDepth: 32,
height: 864,
isExtended: true,
onchange: null,
pixelDepth: 24,
width: 1536,
orientation: {
angle: 0,
type: "landscape-primary",
onchange: null
}
})
screen = {};
screen.__proto__ = Screen.prototype;
createTagProto('History',['replaceState']);
history = {};
history.__proto__ = History.prototype;
chrome = {
loadTimes: function loadTimes() { },
csi: function csi() { },
app: {
InstallState: { DISABLED: 'disabled', INSTALLED: 'installed', NOT_INSTALLED: 'not_installed' },
RunningState: { CANNOT_RUN: 'cannot_run', READY_TO_RUN: 'ready_to_run', RUNNING: 'running' },
getDetails:function getDetails(){},
getIsInstalled:function getIsInstalled(){},
installState:function installState(){},
isInstalled: false,
runningState: function runningState(){}
},
}
createTagProto('Storage');
local = {
};
localStorage = {
getItem: function getItem(key) {
AsObj.print("localStorage.getItem::", arguments);
if (!local[key]) {
return null;
}
return local[key];
},
setItem: function setItem(key, value) {
AsObj.print("localStorage.setItem::", arguments);
local[key] = value;
},
clear: function clear() {
local = {};
},
removeItem: function removeItem(key) {
AsObj.print("localStorage.removeItem::", arguments);
delete local[key];
}
}
localStorage.__proto__ = Storage.prototype;
sessionStorage = {
getItem: function getItem(key) {
AsObj.print("sessionStorage.getItem::", arguments);
if (!local[key]) {
return null;
}
return local[key];
},
setItem: function setItem(key, value) {
AsObj.print("sessionStorage.setItem::", arguments);
local[key] = value;
},
clear: function clear() {
local = {};
},
removeItem: function removeItem(key) {
AsObj.print("sessionStorage.removeItem::", arguments);
delete local[key];
}
}
sessionStorage.__proto__ = Storage.prototype;
// window = watch(window, 'window');
// global = watch(global, 'global');
// globalThis = watch(globalThis, 'globalThis');
// self = watch(self, 'self');
// crypto = watch(crypto, 'crypto');
// performance = watch(performance, 'performance');
// document = watch(document, 'document');
// navigator = watch(navigator, 'navigator');
// location = watch(location, 'location');
// screen = watch(screen, 'screen');
// history = watch(history, 'history');
// localStorage = watch(localStorage, 'localStorage');
// sessionStorage = watch(sessionStorage, 'sessionStorage');
// chrome = watch(chrome, 'chrome');
require_('./sdk_leg.js');
let input = '';
// 收集数据
process.stdin.on('data', chunk => {
input += chunk;
});
process.stdin.on('end', async () => {
var config_data = JSON.parse(input);
var cryptoManager = await CaptchaSDKCorecc();
var encryptData = await buildEncryptedVerifyRequestcc(config_data, cryptoManager);
console.log(JSON.stringify(encryptData));
process.exit(0);
})

View File

@@ -0,0 +1,3 @@
{
"thread_count": "1"
}

View File

@@ -0,0 +1,4 @@
# -*- coding: UTF-8 -*-
'''
UI 模块
'''

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,510 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :domain_import.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:47
@explain : 域名导入界面
'''
from PySide6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QTextEdit, QFileDialog, QProgressBar
from PySide6.QtCore import Qt, QThread, Signal
from loguru import logger
from app.core.domain_collector import DomainCollector
class ImportThread(QThread):
"""
导入线程
"""
progress_updated = Signal(int)
finished = Signal(bool, str)
def __init__(self, domain_list, source_type):
"""
初始化导入线程
:param domain_list: 域名列表
:param source_type: 来源类型
"""
super().__init__()
self.domain_list = domain_list
self.source_type = source_type
def run(self):
"""
运行导入线程
"""
try:
collector = DomainCollector()
total = len(self.domain_list)
# 实时更新进度:开始
self.progress_updated.emit(0)
# 标准化域名和检查是否存在占30%进度)
normalized_domains = []
for i, domain in enumerate(self.domain_list):
from app.utils.domain_utils import normalize_domain
import tldextract
normalized = normalize_domain(domain)
if normalized:
# 提取顶级域名
ext = tldextract.extract(normalized)
tld = ext.suffix
normalized_domains.append((normalized, tld))
# 更新进度
progress = int((i + 1) / total * 30)
self.progress_updated.emit(progress)
# 批量检查域名是否存在
batch_data = []
existing_domains = []
if normalized_domains:
all_domains = [domain for domain, tld in normalized_domains]
existing_domains = collector.db.check_domains_exist(all_domains)
existing_set = set(existing_domains)
# 准备批量添加数据
for domain, tld in normalized_domains:
if domain not in existing_set:
batch_data.append((domain, tld, self.source_type))
# 分批次添加域名占70%进度)
batch_size = 1000
total_batches = len(batch_data)
for i in range(0, len(batch_data), batch_size):
batch = batch_data[i:i+batch_size]
collector.db.add_domains_batch(batch)
# 更新进度
processed = min(i + len(batch), total_batches)
progress = 30 + int(processed / total_batches * 70)
self.progress_updated.emit(progress)
# 完成导入
self.progress_updated.emit(100)
# 计算统计信息
stats = {
'total': total,
'valid': len(normalized_domains),
'added': len(batch_data),
'exists': len(existing_domains),
'invalid': total - len(normalized_domains),
'failed': 0
}
# 根据统计信息生成消息
if stats['added'] > 0:
message = f"导入完成: 总域名数 {stats['total']}, 有效域名数 {stats['valid']}, 新增域名数 {stats['added']}, 已存在域名数 {stats['exists']}, 无效域名数 {stats['invalid']}"
else:
message = f"导入完成: 所有域名已存在,未添加新域名"
self.finished.emit(True, message)
except Exception as e:
logger.error(f"导入失败: {e}")
self.finished.emit(False, f"导入失败: {str(e)}")
class ImportFileThread(QThread):
"""
文件导入线程,用于处理大文件
"""
progress_updated = Signal(int)
finished = Signal(bool, str)
def __init__(self, file_path, source_type):
"""
初始化文件导入线程
:param file_path: 文件路径
:param source_type: 来源类型
"""
super().__init__()
self.file_path = file_path
self.source_type = source_type
def run(self):
"""
运行文件导入线程
"""
try:
collector = DomainCollector()
# 首先计算文件中的域名数量
total = 0
encodings = ['utf-8', 'utf-8-sig', 'gbk', 'gb2312', 'cp936', 'latin-1', 'ascii']
encoding = 'utf-8' # 默认编码
# 尝试不同的编码格式计算域名数量
for enc in encodings:
try:
with open(self.file_path, 'r', encoding=enc) as f:
total = sum(1 for line in f if line.strip())
encoding = enc
break
except UnicodeDecodeError:
continue
if total == 0:
# 尝试使用二进制模式读取
try:
import chardet
with open(self.file_path, 'rb') as f:
raw_data = f.read()
result = chardet.detect(raw_data)
encoding = result['encoding']
if encoding:
total = sum(1 for line in raw_data.decode(encoding).split('\n') if line.strip())
else:
# 最后尝试使用 replace 模式读取
with open(self.file_path, 'r', encoding='utf-8', errors='replace') as f:
total = sum(1 for line in f if line.strip())
encoding = 'utf-8'
except Exception:
# 最后尝试使用 replace 模式读取
with open(self.file_path, 'r', encoding='utf-8', errors='replace') as f:
total = sum(1 for line in f if line.strip())
encoding = 'utf-8'
# 实时更新进度:开始
self.progress_updated.emit(0)
# 逐行读取文件并处理域名
normalized_domains = []
processed = 0
with open(self.file_path, 'r', encoding=encoding, errors='replace') as f:
for line in f:
domain = line.strip()
if domain:
from app.utils.domain_utils import normalize_domain
import tldextract
normalized = normalize_domain(domain)
if normalized:
# 提取顶级域名
ext = tldextract.extract(normalized)
tld = ext.suffix
normalized_domains.append((normalized, tld))
processed += 1
# 更新进度占30%
progress = int(processed / total * 30)
self.progress_updated.emit(progress)
# 批量检查域名是否存在
batch_data = []
existing_domains = []
if normalized_domains:
all_domains = [domain for domain, tld in normalized_domains]
existing_domains = collector.db.check_domains_exist(all_domains)
existing_set = set(existing_domains)
# 准备批量添加数据
for domain, tld in normalized_domains:
if domain not in existing_set:
batch_data.append((domain, tld, self.source_type))
# 分批次添加域名占70%进度)
batch_size = 1000
total_batches = len(batch_data)
for i in range(0, len(batch_data), batch_size):
batch = batch_data[i:i+batch_size]
collector.db.add_domains_batch(batch)
# 更新进度
processed_batches = min(i + len(batch), total_batches)
progress = 30 + int(processed_batches / total_batches * 70)
self.progress_updated.emit(progress)
# 完成导入
self.progress_updated.emit(100)
# 计算统计信息
stats = {
'total': total,
'valid': len(normalized_domains),
'added': len(batch_data),
'exists': len(existing_domains),
'invalid': total - len(normalized_domains),
'failed': 0
}
# 根据统计信息生成消息
if stats['added'] > 0:
message = f"导入完成: 总域名数 {stats['total']}, 有效域名数 {stats['valid']}, 新增域名数 {stats['added']}, 已存在域名数 {stats['exists']}, 无效域名数 {stats['invalid']}"
else:
message = f"导入完成: 所有域名已存在,未添加新域名"
self.finished.emit(True, message)
except Exception as e:
logger.error(f"导入失败: {e}")
self.finished.emit(False, f"导入失败: {str(e)}")
class DomainImportWidget(QWidget):
"""
域名导入界面
"""
def __init__(self):
"""
初始化域名导入界面
"""
super().__init__()
# 创建布局
layout = QVBoxLayout(self)
# 创建文本编辑框
self.text_edit = QTextEdit()
self.text_edit.setPlaceholderText("请输入域名,一行一个")
self.text_edit.setStyleSheet("""
QTextEdit {
font-size: 14px;
padding: 10px;
border: 1px solid #ddd;
border-radius: 4px;
background-color: #f9f9f9;
min-height: 300px;
}
""")
layout.addWidget(self.text_edit)
# 创建按钮布局
button_layout = QHBoxLayout()
# 导入文件按钮
self.import_file_btn = QPushButton("导入文件")
self.import_file_btn.clicked.connect(self.import_file)
self.import_file_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #2196F3;
color: white;
border: none;
border-radius: 4px;
}
QPushButton:hover {
background-color: #0b7dda;
}
QPushButton:disabled {
background-color: #cccccc;
}
""")
button_layout.addWidget(self.import_file_btn)
# 开始导入按钮
self.start_import_btn = QPushButton("开始导入")
self.start_import_btn.clicked.connect(self.start_import)
self.start_import_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #4CAF50;
color: white;
border: none;
border-radius: 4px;
margin-left: 10px;
}
QPushButton:hover {
background-color: #45a049;
}
QPushButton:disabled {
background-color: #cccccc;
}
""")
button_layout.addWidget(self.start_import_btn)
button_layout.setContentsMargins(0, 15, 0, 15)
layout.addLayout(button_layout)
# 创建进度条
self.progress_bar = QProgressBar()
self.progress_bar.setVisible(False)
self.progress_bar.setStyleSheet("""
QProgressBar {
height: 20px;
border: 1px solid #ddd;
border-radius: 10px;
background-color: #f0f0f0;
margin-bottom: 10px;
}
QProgressBar::chunk {
background-color: #4CAF50;
border-radius: 10px;
}
""")
layout.addWidget(self.progress_bar)
# 创建状态标签
self.status_label = QLabel("")
self.status_label.setAlignment(Qt.AlignCenter)
self.status_label.setStyleSheet("font-size: 14px; color: #333; padding: 10px; background-color: #f0f8ff; border-radius: 4px;")
layout.addWidget(self.status_label)
logger.info("域名导入界面创建完成")
def import_file(self):
"""
导入文件
"""
file_path, _ = QFileDialog.getOpenFileName(self, "选择文件", "", "文本文件 (*.txt)")
if file_path:
try:
# 尝试不同的编码格式
encodings = ['utf-8', 'utf-8-sig', 'gbk', 'gb2312', 'cp936', 'latin-1', 'ascii']
domain_count = 0
# 尝试使用不同编码读取并计数
for encoding in encodings:
try:
with open(file_path, 'r', encoding=encoding) as f:
domain_count = sum(1 for line in f if line.strip())
logger.info(f"使用编码 {encoding} 成功读取文件")
break
except UnicodeDecodeError:
continue
# 如果仍然失败,尝试使用二进制模式读取并猜测编码
if domain_count == 0:
try:
import chardet
with open(file_path, 'rb') as f:
raw_data = f.read()
result = chardet.detect(raw_data)
encoding = result['encoding']
if encoding:
domain_count = sum(1 for line in raw_data.decode(encoding).split('\n') if line.strip())
logger.info(f"使用 chardet 检测到编码 {encoding} 并成功读取文件")
else:
raise Exception("无法识别文件编码")
except Exception as e:
logger.warning(f"chardet 检测失败: {e}")
# 最后尝试使用 replace 模式读取
with open(file_path, 'r', encoding='utf-8', errors='replace') as f:
domain_count = sum(1 for line in f if line.strip())
logger.info("使用 utf-8 replace 模式读取文件")
# 对于大文件,不显示所有域名,只显示文件路径和域名数量
if domain_count > 1000:
self.text_edit.setText(f"文件路径: {file_path}\n域名数量: {domain_count}\n\n提示: 由于文件较大,仅显示文件信息,不显示具体域名。")
# 保存文件路径,用于后续导入
self.imported_file_path = file_path
else:
# 对于小文件,显示所有域名
domains = []
for encoding in encodings:
try:
with open(file_path, 'r', encoding=encoding) as f:
domains = f.readlines()
break
except UnicodeDecodeError:
continue
if not domains:
# 尝试使用二进制模式读取
try:
import chardet
with open(file_path, 'rb') as f:
raw_data = f.read()
result = chardet.detect(raw_data)
encoding = result['encoding']
if encoding:
domains = raw_data.decode(encoding).split('\n')
else:
with open(file_path, 'r', encoding='utf-8', errors='replace') as f:
domains = f.readlines()
except Exception:
with open(file_path, 'r', encoding='utf-8', errors='replace') as f:
domains = f.readlines()
domains = [domain.strip() for domain in domains if domain.strip()]
self.text_edit.setText('\n'.join(domains))
# 清除文件路径,使用文本框中的域名
self.imported_file_path = None
self.status_label.setText(f"成功读取 {domain_count} 个域名")
logger.info(f"成功读取文件: {file_path}, 共 {domain_count} 个域名")
except Exception as e:
self.status_label.setText(f"读取文件失败: {str(e)}")
logger.error(f"读取文件失败: {e}")
def start_import(self):
"""
开始导入
"""
# 检查是否有导入的文件路径
if hasattr(self, 'imported_file_path') and self.imported_file_path:
# 大文件导入,使用文件路径
file_path = self.imported_file_path
# 显示进度条
self.progress_bar.setVisible(True)
self.progress_bar.setValue(0)
self.status_label.setText("正在导入...")
# 禁用按钮
self.import_file_btn.setEnabled(False)
self.start_import_btn.setEnabled(False)
# 创建并启动导入线程
self.import_thread = ImportFileThread(file_path, 7) # 7 表示 TXT 导入
self.import_thread.progress_updated.connect(self.update_progress)
self.import_thread.finished.connect(self.import_finished)
self.import_thread.start()
logger.info(f"开始从文件导入: {file_path}")
else:
# 小文件或手动输入的域名
domains = self.text_edit.toPlainText().split('\n')
domains = [domain.strip() for domain in domains if domain.strip()]
if not domains:
self.status_label.setText("请输入域名")
return
# 显示进度条
self.progress_bar.setVisible(True)
self.progress_bar.setValue(0)
self.status_label.setText("正在导入...")
# 禁用按钮
self.import_file_btn.setEnabled(False)
self.start_import_btn.setEnabled(False)
# 创建并启动导入线程
self.import_thread = ImportThread(domains, 7) # 7 表示 TXT 导入
self.import_thread.progress_updated.connect(self.update_progress)
self.import_thread.finished.connect(self.import_finished)
self.import_thread.start()
logger.info(f"开始导入 {len(domains)} 个域名")
def update_progress(self, progress):
"""
更新进度
:param progress: 进度值
"""
self.progress_bar.setValue(progress)
def import_finished(self, success, message):
"""
导入完成
:param success: 是否成功
:param message: 消息
"""
self.status_label.setText(message)
self.progress_bar.setVisible(False)
# 启用按钮
self.import_file_btn.setEnabled(True)
self.start_import_btn.setEnabled(True)
logger.info(f"导入完成: {message}")

View File

@@ -0,0 +1,577 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :juming_crawler.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 15:00
@explain : 聚名网爬取页面
'''
from PySide6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QTextEdit, QProgressBar, QLineEdit, QComboBox, QDateEdit, QCheckBox
from PySide6.QtCore import Qt, QThread, Signal, QDate
from PySide6.QtGui import QIntValidator
from loguru import logger
import re
import datetime
import time
from app.core.domain_collector import DomainCollector
from detect.juming import JM
class JumingCrawlThread(QThread):
"""
聚名爬取线程
"""
progress_updated = Signal(int)
status_updated = Signal(str)
finished = Signal(bool, str)
def __init__(self, crawl_type, page_start=1, page_size=50, crawl_date=None, auto_date=True):
"""
初始化聚名爬取线程
:param crawl_type: 爬取类型 (1: 一口价, 2: 删除列表)
:param page_start: 起始页码
:param page_size: 每页数量
:param crawl_date: 爬取日期(删除列表用)
:param auto_date: 是否自动新增日期
"""
super().__init__()
self.crawl_type = crawl_type
self.page_start = page_start
self.page_size = page_size
self.crawl_date = crawl_date
self.auto_date = auto_date
self.is_paused = False
self.is_stopped = False
self.current_page = 0
self.total_count = 0
def run(self):
"""
运行聚名爬取线程
"""
try:
# 初始化聚名客户端
jm = JM()
# 加载 Cookie
jm.load_cookies()
logger.info("已加载 Cookie")
self.status_updated.emit("已加载 Cookie")
# 直接开始爬取,不需要登录,因为 Cookie 已经在系统设置页面加载了
self.progress_updated.emit(10)
if self.crawl_type == 1: # 一口价
self.progress_updated.emit(30)
logger.info("开始获取一口价域名")
self.status_updated.emit("开始获取一口价域名")
# 自动爬取多页
page = self.page_start
while not self.is_stopped:
if self.is_paused:
time.sleep(0.1)
continue
self.current_page = page
self.status_updated.emit(f"正在爬取第 {page}")
logger.info(f"正在爬取第 {page}")
# 获取当前页
success, html = jm.ykj_get_list(page=page, page_size=self.page_size)
if success:
pattern_ym = r"<a class='yda1 ydz' ym='([^']*)'"
results = re.findall(pattern_ym, html)
domains = [domain.strip() for domain in results if domain.strip()]
domain_count = len(domains)
self.total_count += domain_count
self.status_updated.emit(f"{page} 页找到 {domain_count} 个域名,累计 {self.total_count}")
logger.info(f"{page} 页找到 {domain_count} 个域名,累计 {self.total_count}")
# 自动入库
if domains:
collector = DomainCollector()
stats = collector.add_domains_batch(domains, 1) # 1 表示一口价
logger.info(f"自动入库完成: {stats}")
self.status_updated.emit(f"自动入库完成: 成功添加 {stats['added']} 个域名")
# 如果返回的数量小于指定的数量,停止爬取
if domain_count < self.page_size:
logger.info(f"返回数量小于指定数量,停止爬取")
self.status_updated.emit("返回数量小于指定数量,停止爬取")
break
# 增加页码
page += 1
# 模拟网络延迟
time.sleep(1)
else:
logger.error(f"获取一口价域名失败: {html}")
self.status_updated.emit(f"获取一口价域名失败: {html}")
break
elif self.crawl_type == 2: # 删除列表
self.progress_updated.emit(30)
logger.info("开始获取删除域名列表")
self.status_updated.emit("开始获取删除域名列表")
# 使用传入的日期或默认今天
start_date_str = self.crawl_date if self.crawl_date else datetime.date.today().strftime("%Y-%m-%d")
start_date = datetime.datetime.strptime(start_date_str, "%Y-%m-%d").date()
# 计算结束日期:今天 + 4天
end_date = datetime.date.today() + datetime.timedelta(days=4)
if self.auto_date:
# 自动新增日期,从起始日期到今天+4天
current_date = start_date
while current_date <= end_date and not self.is_stopped:
if self.is_paused:
time.sleep(0.1)
continue
crawl_date = current_date.strftime("%Y-%m-%d")
self.status_updated.emit(f"正在爬取 {crawl_date} 的删除域名")
logger.info(f"正在爬取 {crawl_date} 的删除域名")
deleted_domains = jm.new_cha_del(crawl_date)
domains = [domain.strip() for domain in deleted_domains if domain.strip()]
domain_count = len(domains)
self.total_count += domain_count
self.status_updated.emit(f"{crawl_date} 找到 {domain_count} 个删除域名,累计 {self.total_count}")
logger.info(f"{crawl_date} 找到 {domain_count} 个删除域名,累计 {self.total_count}")
# 自动入库
if domains:
collector = DomainCollector()
stats = collector.add_domains_batch(domains, 2) # 2 表示删除列表
logger.info(f"{crawl_date} 自动入库完成: {stats}")
self.status_updated.emit(f"{crawl_date} 自动入库完成: 成功添加 {stats['added']} 个域名")
# 增加日期
current_date = current_date + datetime.timedelta(days=1)
# 模拟网络延迟
time.sleep(1)
else:
# 只爬取指定日期
crawl_date = start_date_str
self.status_updated.emit(f"正在爬取 {crawl_date} 的删除域名")
logger.info(f"正在爬取 {crawl_date} 的删除域名")
deleted_domains = jm.new_cha_del(crawl_date)
domains = [domain.strip() for domain in deleted_domains if domain.strip()]
domain_count = len(domains)
self.total_count = domain_count
self.status_updated.emit(f"找到 {domain_count} 个删除域名")
logger.info(f"找到 {domain_count} 个删除域名")
# 自动入库
if domains:
collector = DomainCollector()
stats = collector.add_domains_batch(domains, 2) # 2 表示删除列表
logger.info(f"自动入库完成: {stats}")
self.status_updated.emit(f"自动入库完成: 成功添加 {stats['added']} 个域名")
self.progress_updated.emit(90)
self.progress_updated.emit(100)
self.finished.emit(True, f"成功获取 {self.total_count} 个域名")
except Exception as e:
logger.error(f"从聚名网爬取失败: {e}")
self.status_updated.emit(f"爬取失败: {str(e)}")
self.finished.emit(False, f"爬取失败: {str(e)}")
def pause(self):
"""
暂停爬取
"""
self.is_paused = True
logger.info("爬取已暂停")
self.status_updated.emit("爬取已暂停")
def resume(self):
"""
恢复爬取
"""
self.is_paused = False
logger.info("爬取已恢复")
self.status_updated.emit("爬取已恢复")
def stop(self):
"""
停止爬取
"""
self.is_stopped = True
logger.info("爬取已停止")
self.status_updated.emit("爬取已停止")
class JumingCrawlerWidget(QWidget):
"""
聚名网爬取页面
"""
def __init__(self):
"""
初始化聚名网爬取页面
"""
super().__init__()
# 创建布局
layout = QVBoxLayout(self)
# 爬取类型选择
type_layout = QHBoxLayout()
type_label = QLabel("爬取类型:")
type_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px;")
self.type_combo = QComboBox()
self.type_combo.addItem("一口价域名", 1)
self.type_combo.addItem("删除列表域名", 2)
self.type_combo.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px;")
# 监听类型变化
self.type_combo.currentIndexChanged.connect(self.on_type_changed)
type_layout.addWidget(type_label)
type_layout.addWidget(self.type_combo)
type_layout.setContentsMargins(0, 0, 0, 15)
layout.addLayout(type_layout)
# 页码和每页数量设置
page_layout = QHBoxLayout()
# 起始页码
page_start_label = QLabel("起始页码:")
page_start_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px;")
self.page_start_edit = QLineEdit("1")
# 移除所有限制,允许输入任意正整数
self.page_start_edit.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px; width: 120px;")
page_layout.addWidget(page_start_label)
page_layout.addWidget(self.page_start_edit)
# 每页数量
page_size_label = QLabel("每页数量:")
page_size_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px; margin-left: 20px;")
self.page_size_edit = QLineEdit("500")
self.page_size_edit.setValidator(QIntValidator(1, 1000))
self.page_size_edit.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px; width: 80px;")
page_layout.addWidget(page_size_label)
page_layout.addWidget(self.page_size_edit)
page_layout.setContentsMargins(0, 0, 0, 20)
layout.addLayout(page_layout)
# 日期设置(删除列表用)
date_container = QWidget()
date_layout = QHBoxLayout(date_container)
# 起始日期
date_label = QLabel("起始日期:")
date_label.setStyleSheet("font-size: 14px; color: #666; min-width: 80px;")
self.date_edit = QDateEdit()
self.date_edit.setDate(QDate.currentDate())
self.date_edit.setCalendarPopup(True)
self.date_edit.setStyleSheet("font-size: 14px; padding: 5px; border: 1px solid #ddd; border-radius: 4px; width: 150px;")
# 设置最大日期为今天+4天最小日期为今天的前4天
max_date = QDate.currentDate().addDays(4)
min_date = QDate.currentDate().addDays(-4)
self.date_edit.setMinimumDate(min_date)
self.date_edit.setMaximumDate(max_date)
date_layout.addWidget(date_label)
date_layout.addWidget(self.date_edit)
# 自动新增日期选项
auto_date_checkbox = QCheckBox("自动新增日期")
auto_date_checkbox.setChecked(True)
auto_date_checkbox.setStyleSheet("font-size: 14px; margin-left: 20px;")
self.auto_date_checkbox = auto_date_checkbox
date_layout.addWidget(auto_date_checkbox)
date_layout.setContentsMargins(0, 0, 0, 20)
self.date_container = date_container
layout.addWidget(date_container)
# 默认隐藏日期输入框
self.date_container.setVisible(False)
# 按钮布局
button_layout = QHBoxLayout()
# 创建开始按钮
self.start_btn = QPushButton("开始爬取")
self.start_btn.clicked.connect(self.start_crawl)
self.start_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #4CAF50;
color: white;
border: none;
border-radius: 4px;
}
QPushButton:hover {
background-color: #45a049;
}
QPushButton:disabled {
background-color: #cccccc;
}
""")
button_layout.addWidget(self.start_btn)
# 创建暂停按钮
self.pause_btn = QPushButton("暂停爬取")
self.pause_btn.clicked.connect(self.pause_crawl)
self.pause_btn.setEnabled(False)
self.pause_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #ff9800;
color: white;
border: none;
border-radius: 4px;
margin-left: 10px;
}
QPushButton:hover {
background-color: #f57c00;
}
QPushButton:disabled {
background-color: #cccccc;
}
""")
button_layout.addWidget(self.pause_btn)
# 创建停止按钮
self.stop_btn = QPushButton("停止爬取")
self.stop_btn.clicked.connect(self.stop_crawl)
self.stop_btn.setEnabled(False)
self.stop_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #f44336;
color: white;
border: none;
border-radius: 4px;
margin-left: 10px;
}
QPushButton:hover {
background-color: #d32f2f;
}
QPushButton:disabled {
background-color: #cccccc;
}
""")
button_layout.addWidget(self.stop_btn)
button_layout.setContentsMargins(0, 0, 0, 20)
layout.addLayout(button_layout)
# 创建日志显示区域
self.log_edit = QTextEdit()
self.log_edit.setPlaceholderText("爬取日志将显示在这里")
self.log_edit.setReadOnly(True)
self.log_edit.setStyleSheet("""
QTextEdit {
font-size: 13px;
font-family: Consolas, Monaco, monospace;
padding: 10px;
border: 1px solid #ddd;
border-radius: 4px;
background-color: #f9f9f9;
}
""")
layout.addWidget(self.log_edit)
# 创建进度条
self.progress_bar = QProgressBar()
self.progress_bar.setVisible(False)
self.progress_bar.setStyleSheet("""
QProgressBar {
height: 20px;
border: 1px solid #ddd;
border-radius: 10px;
background-color: #f0f0f0;
margin-top: 10px;
margin-bottom: 10px;
}
QProgressBar::chunk {
background-color: #4CAF50;
border-radius: 10px;
}
""")
layout.addWidget(self.progress_bar)
# 创建状态标签
self.status_label = QLabel("")
self.status_label.setAlignment(Qt.AlignCenter)
self.status_label.setStyleSheet("font-size: 14px; color: #333; margin-top: 10px; padding: 8px; background-color: #f0f8ff; border-radius: 4px;")
layout.addWidget(self.status_label)
# 存储爬取线程
self.crawl_thread = None
logger.info("聚名网爬取页面创建完成")
def start_crawl(self):
"""
开始爬取
"""
crawl_type = self.type_combo.currentData()
# 获取起始页码和每页数量
try:
page_start = int(self.page_start_edit.text())
page_size = int(self.page_size_edit.text())
if page_start < 1:
self.status_label.setText("起始页码必须大于0")
return
if page_size < 1:
self.status_label.setText("每页数量必须大于0")
return
except ValueError:
self.status_label.setText("请输入有效的页码和每页数量")
return
# 显示进度条
self.progress_bar.setVisible(True)
self.progress_bar.setValue(0)
self.status_label.setText("正在爬取域名...")
# 清空日志
self.log_edit.clear()
# 启用/禁用按钮
self.start_btn.setEnabled(False)
self.pause_btn.setEnabled(True)
self.stop_btn.setEnabled(True)
# 获取爬取日期和自动新增日期选项(删除列表用)
crawl_date = None
auto_date = False
if crawl_type == 2: # 删除列表
crawl_date = self.date_edit.date().toString("yyyy-MM-dd")
auto_date = self.auto_date_checkbox.isChecked()
# 创建并启动爬取线程
self.crawl_thread = JumingCrawlThread(crawl_type, page_start, page_size, crawl_date, auto_date)
self.crawl_thread.progress_updated.connect(self.update_progress)
self.crawl_thread.status_updated.connect(self.update_status)
self.crawl_thread.finished.connect(self.crawl_finished)
self.crawl_thread.start()
if crawl_type == 2: # 删除列表
auto_date_str = "" if auto_date else ""
logger.info(f"开始爬取聚名网域名, 类型: {crawl_type}, 爬取日期: {crawl_date}, 自动新增日期: {auto_date_str}")
self.log_edit.append(f"开始爬取聚名网域名, 类型: {crawl_type}, 爬取日期: {crawl_date}, 自动新增日期: {auto_date_str}")
else: # 一口价
logger.info(f"开始爬取聚名网域名, 类型: {crawl_type}, 起始页码: {page_start}, 每页数量: {page_size}")
self.log_edit.append(f"开始爬取聚名网域名, 类型: {crawl_type}, 起始页码: {page_start}, 每页数量: {page_size}")
def pause_crawl(self):
"""
暂停爬取
"""
if self.crawl_thread:
if self.crawl_thread.is_paused:
self.crawl_thread.resume()
self.pause_btn.setText("暂停爬取")
else:
self.crawl_thread.pause()
self.pause_btn.setText("恢复爬取")
def stop_crawl(self):
"""
停止爬取
"""
if self.crawl_thread:
self.crawl_thread.stop()
def update_progress(self, progress):
"""
更新进度
:param progress: 进度值
"""
self.progress_bar.setValue(progress)
def update_status(self, status):
"""
更新状态
:param status: 状态消息
"""
self.status_label.setText(status)
self.log_edit.append(status)
def crawl_finished(self, success, message):
"""
爬取完成
:param success: 是否成功
:param message: 消息
"""
self.status_label.setText(message)
self.log_edit.append(message)
# 启用/禁用按钮
self.start_btn.setEnabled(True)
self.pause_btn.setEnabled(False)
self.pause_btn.setText("暂停爬取")
self.stop_btn.setEnabled(False)
self.progress_bar.setVisible(False)
logger.info(f"聚名网爬取完成: {message}")
def on_type_changed(self, index):
"""
爬取类型变化时的处理
:param index: 选择的索引
"""
crawl_type = self.type_combo.currentData()
if crawl_type == 2: # 删除列表
self.date_container.setVisible(True)
else: # 一口价
self.date_container.setVisible(False)
class ImportThread(QThread):
"""
导入线程
"""
progress_updated = Signal(int)
finished = Signal(bool, str)
def __init__(self, domain_list, source_type):
"""
初始化导入线程
:param domain_list: 域名列表
:param source_type: 来源类型
"""
super().__init__()
self.domain_list = domain_list
self.source_type = source_type
def run(self):
"""
运行导入线程
"""
try:
collector = DomainCollector()
total = len(self.domain_list)
for i, domain in enumerate(self.domain_list):
collector.add_domain(domain, self.source_type)
progress = int((i + 1) / total * 100)
self.progress_updated.emit(progress)
self.finished.emit(True, f"成功导入 {total} 个域名")
except Exception as e:
logger.error(f"导入失败: {e}")
self.finished.emit(False, f"导入失败: {str(e)}")

View File

@@ -0,0 +1,137 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :main_window.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:46
@explain : 主窗口
'''
from PySide6.QtWidgets import QMainWindow, QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QTabWidget, QLabel, QScrollArea, QFrame
from PySide6.QtCore import Qt
from PySide6.QtGui import QIcon
import os
from loguru import logger
from app.ui.domain_import import DomainImportWidget
from app.ui.domain_filter import DomainFilterWidget
from app.ui.sensitive_words import SensitiveWordsWidget
from app.ui.juming_crawler import JumingCrawlerWidget
from app.ui.system_settings import SystemSettingsWidget
class MainWindow(QMainWindow):
"""
主窗口
"""
def __init__(self):
"""
初始化主窗口
"""
super().__init__()
# 设置窗口标题和大小
self.setWindowTitle("域名工具")
self.setGeometry(100, 100, 2000, 800)
self.setMinimumSize(1200, 720)
# 设置窗口图标
icon_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "favicon.ico")
if os.path.exists(icon_path):
self.setWindowIcon(QIcon(icon_path))
logger.info(f"设置窗口图标成功: {icon_path}")
else:
logger.warning(f"窗口图标文件不存在: {icon_path}")
# 创建中央部件
central_widget = QWidget()
self.setCentralWidget(central_widget)
# 创建主布局
main_layout = QVBoxLayout(central_widget)
# 创建标签页
self.tab_widget = QTabWidget()
self.tab_widget.setStyleSheet("""
QTabWidget {
font-size: 14px;
}
QTabBar::tab {
padding: 10px 20px;
background-color: #f0f0f0;
border: 1px solid #ddd;
border-bottom: none;
border-top-left-radius: 4px;
border-top-right-radius: 4px;
margin-right: 2px;
}
QTabBar::tab:hover {
background-color: #e0e0e0;
}
QTabBar::tab:selected {
background-color: white;
color: #4CAF50;
font-weight: bold;
border-color: #4CAF50;
}
QTabWidget::pane {
border: 1px solid #ddd;
border-top: none;
border-radius: 0 0 4px 4px;
padding: 10px;
}
""")
main_layout.addWidget(self.tab_widget)
# 创建标签页内容
self.create_tabs()
# 记录日志
logger.info("主窗口创建完成")
def create_tabs(self):
"""
创建标签页
"""
# 聚名爬取标签页
juming_widget = JumingCrawlerWidget()
self.tab_widget.addTab(self.wrap_scrollable_tab(juming_widget), "聚名爬取")
# 域名筛选标签页
filter_widget = DomainFilterWidget()
self.tab_widget.addTab(self.wrap_scrollable_tab(filter_widget), "域名筛选")
# 域名导入标签页
import_widget = DomainImportWidget()
self.tab_widget.addTab(self.wrap_scrollable_tab(import_widget), "域名导入")
# 敏感词配置标签页
sensitive_widget = SensitiveWordsWidget()
self.tab_widget.addTab(self.wrap_scrollable_tab(sensitive_widget), "敏感词配置")
# 系统设置标签页
settings_widget = SystemSettingsWidget()
self.tab_widget.addTab(self.wrap_scrollable_tab(settings_widget), "系统设置")
logger.info("标签页创建完成")
def wrap_scrollable_tab(self, widget):
"""
给标签页统一包一层滚动区域,保证右侧滚动条始终可用
"""
container = QWidget()
container_layout = QVBoxLayout(container)
container_layout.setContentsMargins(0, 0, 0, 0)
container_layout.setSpacing(0)
scroll_area = QScrollArea()
scroll_area.setWidgetResizable(True)
scroll_area.setFrameShape(QFrame.NoFrame)
scroll_area.setVerticalScrollBarPolicy(Qt.ScrollBarAsNeeded)
scroll_area.setHorizontalScrollBarPolicy(Qt.ScrollBarAsNeeded)
scroll_area.setWidget(widget)
container_layout.addWidget(scroll_area)
return container

View File

@@ -0,0 +1,284 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :sensitive_words.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/8 23:49
@explain : 敏感词配置界面
'''
from PySide6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QTextEdit, QFileDialog
from PySide6.QtCore import Qt, QThread, Signal
from loguru import logger
from app.utils.database import Database
class SaveWordsThread(QThread):
"""
保存敏感词线程
"""
finished = Signal(bool, str, int)
def __init__(self, words):
super().__init__()
self.words = words
def run(self):
try:
db = Database()
# 先清空现有敏感词
db.execute("DELETE FROM sensitive_words")
# 批量添加敏感词
word_tuples = [(word, 'default', 1) for word in self.words]
if word_tuples:
db.batch_add_sensitive_words(word_tuples)
db.close()
self.finished.emit(True, "成功保存敏感词", len(self.words))
except Exception as e:
self.finished.emit(False, str(e), 0)
class LoadWordsThread(QThread):
"""
加载敏感词线程
"""
finished = Signal(bool, list, str)
def run(self):
try:
db = Database()
sensitive_words = db.get_sensitive_words()
words = [word['word'] for word in sensitive_words]
db.close()
self.finished.emit(True, words, f"成功加载 {len(words)} 个敏感词")
except Exception as e:
self.finished.emit(False, [], str(e))
class SensitiveWordsWidget(QWidget):
"""
敏感词配置界面
"""
def __init__(self):
"""
初始化敏感词配置界面
"""
super().__init__()
# 创建布局
layout = QVBoxLayout(self)
# 创建文本编辑框
self.text_edit = QTextEdit()
self.text_edit.setPlaceholderText("请输入敏感词,一行一个")
self.text_edit.setStyleSheet("""
QTextEdit {
font-size: 14px;
padding: 10px;
border: 1px solid #ddd;
border-radius: 4px;
background-color: #f9f9f9;
min-height: 300px;
}
""")
layout.addWidget(self.text_edit)
# 创建按钮布局
button_layout = QHBoxLayout()
# 导入按钮
self.import_btn = QPushButton("导入")
self.import_btn.clicked.connect(self.import_words)
self.import_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #2196F3;
color: white;
border: none;
border-radius: 4px;
}
QPushButton:hover {
background-color: #0b7dda;
}
""")
button_layout.addWidget(self.import_btn)
# 导出按钮
self.export_btn = QPushButton("导出")
self.export_btn.clicked.connect(self.export_words)
self.export_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #ff9800;
color: white;
border: none;
border-radius: 4px;
margin-left: 10px;
}
QPushButton:hover {
background-color: #f57c00;
}
""")
button_layout.addWidget(self.export_btn)
# 保存按钮
self.save_btn = QPushButton("保存")
self.save_btn.clicked.connect(self.save_words)
self.save_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #4CAF50;
color: white;
border: none;
border-radius: 4px;
margin-left: 10px;
}
QPushButton:hover {
background-color: #45a049;
}
""")
button_layout.addWidget(self.save_btn)
# 加载按钮
self.load_btn = QPushButton("加载")
self.load_btn.clicked.connect(self.load_words)
self.load_btn.setStyleSheet("""
QPushButton {
font-size: 14px;
padding: 8px 16px;
background-color: #9c27b0;
color: white;
border: none;
border-radius: 4px;
margin-left: 10px;
}
QPushButton:hover {
background-color: #7b1fa2;
}
""")
button_layout.addWidget(self.load_btn)
button_layout.setContentsMargins(0, 15, 0, 15)
layout.addLayout(button_layout)
# 创建状态标签
self.status_label = QLabel("")
self.status_label.setAlignment(Qt.AlignCenter)
self.status_label.setStyleSheet("font-size: 14px; color: #333; padding: 10px; background-color: #f0f8ff; border-radius: 4px;")
layout.addWidget(self.status_label)
# 初始化线程
self.save_thread = None
self.load_thread = None
# 加载敏感词
self.load_words()
logger.info("敏感词配置界面创建完成")
def import_words(self):
"""
导入敏感词
"""
file_path, _ = QFileDialog.getOpenFileName(self, "选择文件", "", "文本文件 (*.txt)")
if file_path:
try:
with open(file_path, 'r', encoding='utf-8') as f:
words = f.readlines()
words = [word.strip() for word in words if word.strip()]
self.text_edit.setText('\n'.join(words))
self.status_label.setText(f"成功导入 {len(words)} 个敏感词")
logger.info(f"成功导入敏感词文件: {file_path}, 共 {len(words)} 个敏感词")
except Exception as e:
self.status_label.setText(f"导入失败: {str(e)}")
logger.error(f"导入敏感词失败: {e}")
def export_words(self):
"""
导出敏感词
"""
words = self.text_edit.toPlainText().split('\n')
words = [word.strip() for word in words if word.strip()]
if not words:
self.status_label.setText("没有敏感词可导出")
return
file_path, _ = QFileDialog.getSaveFileName(self, "保存文件", "", "文本文件 (*.txt)")
if file_path:
try:
with open(file_path, 'w', encoding='utf-8') as f:
for word in words:
f.write(word + '\n')
self.status_label.setText(f"成功导出 {len(words)} 个敏感词")
logger.info(f"成功导出 {len(words)} 个敏感词到 {file_path}")
except Exception as e:
self.status_label.setText(f"导出失败: {str(e)}")
logger.error(f"导出敏感词失败: {e}")
def save_words(self):
"""
保存敏感词
"""
words = self.text_edit.toPlainText().split('\n')
words = [word.strip() for word in words if word.strip()]
# 禁用按钮,防止重复点击
self.save_btn.setEnabled(False)
self.status_label.setText("正在保存敏感词...")
# 创建并启动保存线程
self.save_thread = SaveWordsThread(words)
self.save_thread.finished.connect(self.on_save_finished)
self.save_thread.start()
def on_save_finished(self, success, message, count):
"""
保存完成的回调函数
"""
if success:
self.status_label.setText(f"成功保存 {count} 个敏感词")
logger.info(f"成功保存 {count} 个敏感词到数据库")
else:
self.status_label.setText(f"保存失败: {message}")
logger.error(f"保存敏感词失败: {message}")
# 重新启用按钮
self.save_btn.setEnabled(True)
def load_words(self):
"""
加载敏感词
"""
# 禁用按钮,防止重复点击
self.load_btn.setEnabled(False)
self.status_label.setText("正在加载敏感词...")
# 创建并启动加载线程
self.load_thread = LoadWordsThread()
self.load_thread.finished.connect(self.on_load_finished)
self.load_thread.start()
def on_load_finished(self, success, words, message):
"""
加载完成的回调函数
"""
if success:
self.text_edit.setText('\n'.join(words))
self.status_label.setText(message)
logger.info(message)
else:
self.status_label.setText(f"加载失败: {message}")
logger.error(f"加载敏感词失败: {message}")
# 重新启用按钮
self.load_btn.setEnabled(True)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,4 @@
# -*- coding: UTF-8 -*-
'''
工具类模块
'''

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,195 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :domain_utils.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:05
@explain : 域名工具类
'''
import re
import tldextract
from loguru import logger
from app.utils.status_codes import DETECT_STATUS_LABELS, REGISTER_STATUS_LABELS, USE_STATUS_LABELS
def normalize_domain(domain):
"""
标准化域名
:param domain: 域名
:return: str - 标准化后的域名
"""
try:
# 转换为小写
domain = domain.lower()
# 去除空格
domain = domain.strip()
# 去除协议
domain = re.sub(r'^https?://', '', domain)
# 去除路径和查询参数
domain = domain.split('/')[0]
domain = domain.split('?')[0]
# 去除端口
domain = domain.split(':')[0]
# 只保留主域
ext = tldextract.extract(domain)
if ext.domain and ext.suffix:
domain = f"{ext.domain}.{ext.suffix}"
# 验证域名格式
if not is_valid_domain(domain):
return None
return domain
except Exception as e:
logger.error(f"标准化域名出错: {e}")
return None
def is_valid_domain(domain):
"""
验证域名格式
:param domain: 域名
:return: bool - 是否有效
"""
try:
# 域名格式正则
pattern = r'^[a-zA-Z0-9][a-zA-Z0-9-]{0,61}[a-zA-Z0-9]\.[a-zA-Z]{2,}$'
return bool(re.match(pattern, domain))
except Exception as e:
logger.error(f"验证域名格式出错: {e}")
return False
def extract_tld(domain):
"""
提取顶级域名
:param domain: 域名
:return: str - 顶级域名
"""
try:
ext = tldextract.extract(domain)
return ext.suffix
except Exception as e:
logger.error(f"提取顶级域名出错: {e}")
return ''
def extract_domain(domain):
"""
提取主域名
:param domain: 域名
:return: str - 主域名
"""
try:
ext = tldextract.extract(domain)
if ext.domain and ext.suffix:
return f"{ext.domain}.{ext.suffix}"
return domain
except Exception as e:
logger.error(f"提取主域名出错: {e}")
return domain
def is_com_or_net(domain):
"""
检查是否为 .com 或 .net 域名
:param domain: 域名
:return: bool - 是否为 .com 或 .net 域名
"""
try:
tld = extract_tld(domain)
return tld in ['com', 'net']
except Exception as e:
logger.error(f"检查域名后缀出错: {e}")
return False
def generate_domain_variants(domain):
"""
生成域名变体
:param domain: 域名
:return: list - 域名变体列表
"""
try:
variants = []
# 原始域名
variants.append(domain)
# 添加 www
if not domain.startswith('www.'):
variants.append(f"www.{domain}")
# 移除 www
if domain.startswith('www.'):
variants.append(domain[4:])
return variants
except Exception as e:
logger.error(f"生成域名变体出错: {e}")
return [domain]
def parse_domain_status(status_code):
"""
解析域名状态码
:param status_code: 状态码
:return: str - 状态描述
"""
return REGISTER_STATUS_LABELS.get(status_code, '未知')
def parse_use_status(status_code):
"""
解析使用状态码
:param status_code: 状态码
:return: str - 状态描述
"""
return USE_STATUS_LABELS.get(status_code, '未知')
def parse_detect_status(status_code):
"""
解析检测状态码
:param status_code: 状态码
:return: str - 状态描述
"""
return DETECT_STATUS_LABELS.get(status_code, '未知')
def parse_source_type(source_type):
"""
解析来源类型
:param source_type: 来源类型
:return: str - 来源描述
"""
source_map = {
1: '聚名一口价',
2: '聚名过期删除',
3: 'zone file',
4: '搜索引擎采集',
5: '企业目录采集',
6: '手工录入',
7: 'TXT 导入',
8: '第三方接口',
9: '其它'
}
return source_map.get(source_type, '未知')

View File

@@ -0,0 +1,157 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :http_utils.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 0:06
@explain : HTTP工具类
'''
import requests
from curl_cffi import requests as curl_requests
from loguru import logger
class HTTPUtils:
"""
HTTP工具类
"""
@staticmethod
def get(url, headers=None, params=None, timeout=10, proxies=None, use_curl=False):
"""
发送GET请求
:param url: 请求URL
:param headers: 请求头
:param params: 查询参数
:param timeout: 超时时间
:param proxies: 代理
:param use_curl: 是否使用curl_cffi
:return: requests.Response - 响应对象
"""
try:
if use_curl:
# 使用curl_cffi模拟浏览器
response = curl_requests.get(url, headers=headers, params=params, timeout=timeout, proxies=proxies, impersonate='chrome')
else:
# 使用requests
response = requests.get(url, headers=headers, params=params, timeout=timeout, proxies=proxies)
response.raise_for_status() # 检查状态码
return response
except Exception as e:
logger.error(f"GET请求失败: {url}, 错误: {e}")
return None
@staticmethod
def post(url, headers=None, data=None, json=None, timeout=10, proxies=None, use_curl=False):
"""
发送POST请求
:param url: 请求URL
:param headers: 请求头
:param data: 表单数据
:param json: JSON数据
:param timeout: 超时时间
:param proxies: 代理
:param use_curl: 是否使用curl_cffi
:return: requests.Response - 响应对象
"""
try:
if use_curl:
# 使用curl_cffi模拟浏览器
response = curl_requests.post(url, headers=headers, data=data, json=json, timeout=timeout, proxies=proxies, impersonate='chrome')
else:
# 使用requests
response = requests.post(url, headers=headers, data=data, json=json, timeout=timeout, proxies=proxies)
response.raise_for_status() # 检查状态码
return response
except Exception as e:
logger.error(f"POST请求失败: {url}, 错误: {e}")
return None
@staticmethod
def get_random_user_agent():
"""
获取随机用户代理
:return: str - 用户代理
"""
user_agents = [
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Firefox/138.0',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/146.0.0.0',
'Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.6 Safari/605.1.15'
]
import random
return random.choice(user_agents)
@staticmethod
def get_default_headers():
"""
获取默认请求头
:return: dict - 请求头
"""
return {
'User-Agent': HTTPUtils.get_random_user_agent(),
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
'Accept-Language': 'zh-CN,zh;q=0.9',
'Accept-Encoding': 'gzip, deflate, br',
'Connection': 'keep-alive',
'Upgrade-Insecure-Requests': '1',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1'
}
@staticmethod
def retry_request(func, max_retries=3, delay=1):
"""
重试请求
:param func: 请求函数
:param max_retries: 最大重试次数
:param delay: 重试延迟
:return: 函数返回值
"""
import time
for i in range(max_retries):
try:
result = func()
if result:
return result
except Exception as e:
logger.warning(f"请求失败,第 {i+1} 次重试: {e}")
if i < max_retries - 1:
time.sleep(delay)
return None
@staticmethod
def check_proxy(proxy):
"""
检查代理是否可用
:param proxy: 代理URL
:return: bool - 是否可用
"""
try:
proxies = {
'http': proxy,
'https': proxy
}
response = requests.get('https://www.baidu.com', proxies=proxies, timeout=5)
return response.status_code == 200
except Exception as e:
logger.error(f"代理检查失败: {proxy}, 错误: {e}")
return False

View File

@@ -0,0 +1,66 @@
"""Centralized status codes and labels used across detection, database and UI."""
REGISTER_STATUS_PENDING = 0
REGISTER_STATUS_AVAILABLE = 2
REGISTER_STATUS_REGISTERED = 3
REGISTER_STATUS_GRACE = 4
REGISTER_STATUS_REDEMPTION = 5
REGISTER_STATUS_PENDING_DELETE = 6
REGISTER_STATUS_CLIENT_HOLD = 7
REGISTER_STATUS_SERVER_HOLD = 8
REGISTER_STATUS_UNKNOWN = 9
REGISTER_STATUS_FAILED = 10
DETECT_STATUS_PENDING = 0
DETECT_STATUS_COMPLETED = 1
DETECT_STATUS_RUNNING = 2
DETECT_STATUS_BLACKLISTED = 3
DETECT_STATUS_FAILED = 4
USE_STATUS_UNUSED = 0
USE_STATUS_USED = 1
USE_STATUS_SOLD = 2
USE_STATUS_RESERVED = 3
REVIEW_STATUS_NONE = 0
REVIEW_STATUS_PENDING = 1
REVIEW_STATUS_APPROVED = 2
REVIEW_STATUS_REJECTED = 3
THIRD_PARTY_STATUS_PENDING = 0
THIRD_PARTY_STATUS_DONE = 1
REGISTER_STATUS_LABELS = {
REGISTER_STATUS_PENDING: '待检测',
REGISTER_STATUS_AVAILABLE: '可注册',
REGISTER_STATUS_REGISTERED: '已注册',
REGISTER_STATUS_GRACE: '宽限期',
REGISTER_STATUS_REDEMPTION: '赎回期',
REGISTER_STATUS_PENDING_DELETE: '删除期',
REGISTER_STATUS_CLIENT_HOLD: 'clientHold',
REGISTER_STATUS_SERVER_HOLD: 'serverHold',
REGISTER_STATUS_UNKNOWN: '状态未知',
REGISTER_STATUS_FAILED: '检测失败',
}
DETECT_STATUS_LABELS = {
DETECT_STATUS_PENDING: '待检测',
DETECT_STATUS_COMPLETED: '检测完成',
DETECT_STATUS_RUNNING: '检测中',
DETECT_STATUS_BLACKLISTED: '黑名单',
DETECT_STATUS_FAILED: '检测失败',
}
USE_STATUS_LABELS = {
USE_STATUS_UNUSED: '未使用',
USE_STATUS_USED: '已经使用',
USE_STATUS_SOLD: '已经卖出',
USE_STATUS_RESERVED: '已经预定',
}
REVIEW_STATUS_LABELS = {
REVIEW_STATUS_NONE: '无需复核',
REVIEW_STATUS_PENDING: '待人工复核',
REVIEW_STATUS_APPROVED: '人工通过',
REVIEW_STATUS_REJECTED: '人工拒绝',
}

View File

@@ -0,0 +1,74 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :check_database.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 21:40
@explain : 检查数据库表结构和注释
'''
import psycopg2
from app.config import config
def check_database():
"""
检查数据库表结构和注释
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
print("数据库连接成功")
# 检查所有表
print("\n=== 所有表 ===")
cur.execute("SELECT table_name FROM information_schema.tables WHERE table_schema = 'public'")
tables = cur.fetchall()
for table in tables:
print(f"表名: {table[0]}")
# 检查domains表结构
print("\n=== domains表结构 ===")
cur.execute("\d+ domains.txt")
result = cur.fetchall()
for row in result:
print(row)
# 检查其他表结构
print("\n=== detect_tasks表结构 ===")
cur.execute("\d+ detect_tasks")
result = cur.fetchall()
for row in result:
print(row)
print("\n=== domain_blacklist表结构 ===")
cur.execute("\d+ domain_blacklist")
result = cur.fetchall()
for row in result:
print(row)
print("\n=== domain_detections表结构 ===")
cur.execute("\d+ domain_detections")
result = cur.fetchall()
for row in result:
print(row)
except Exception as e:
print(f"检查数据库失败: {e}")
finally:
if cur:
cur.close()
if conn:
conn.close()
if __name__ == "__main__":
check_database()

View File

@@ -0,0 +1,69 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :check_db_structure.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 21:50
@explain : 检查数据库表结构和索引
'''
import psycopg2
from app.config import config
def check_db_structure():
"""
检查数据库表结构和索引
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
print("数据库连接成功")
# 检查domains表结构
print("\n=== domains表结构 ===")
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'domains.txt'")
for row in cur.fetchall():
print(row)
# 检查domains表索引
print("\n=== domains表索引 ===")
cur.execute("SELECT indexname, indexdef FROM pg_indexes WHERE tablename = 'domains.txt'")
for row in cur.fetchall():
print(row)
# 检查其他表结构
print("\n=== detect_tasks表结构 ===")
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'detect_tasks'")
for row in cur.fetchall():
print(row)
print("\n=== domain_blacklist表结构 ===")
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'domain_blacklist'")
for row in cur.fetchall():
print(row)
print("\n=== domain_detections表结构 ===")
cur.execute("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'domain_detections'")
for row in cur.fetchall():
print(row)
except Exception as e:
print(f"检查数据库失败: {e}")
finally:
if cur:
cur.close()
if conn:
conn.close()
if __name__ == "__main__":
check_db_structure()

View File

@@ -0,0 +1,128 @@
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :create_sensitive_words_table.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/10 03:30
@explain : 创建敏感词表
'''
import psycopg2
from app.config import config
def create_sensitive_words_table():
"""
创建敏感词表
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
print(f"成功连接到数据库: {config.DB_HOST}:{config.DB_PORT}/{config.DB_DATABASE}")
# 创建游标
cur = conn.cursor()
# 创建敏感词表
create_table_sql = """
CREATE TABLE IF NOT EXISTS sensitive_words (
id SERIAL PRIMARY KEY,
word VARCHAR(255) UNIQUE NOT NULL,
category VARCHAR(50) DEFAULT 'default',
priority INTEGER DEFAULT 1,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cur.execute(create_table_sql)
print("创建敏感词表成功")
# 添加表注释
add_table_comment_sql = """
COMMENT ON TABLE sensitive_words IS '敏感词表,存储需要过滤的敏感词'
"""
cur.execute(add_table_comment_sql)
print("添加表注释成功")
# 添加字段注释
add_column_comments_sql = """
COMMENT ON COLUMN sensitive_words.id IS '主键ID';
COMMENT ON COLUMN sensitive_words.word IS '敏感词';
COMMENT ON COLUMN sensitive_words.category IS '敏感词分类';
COMMENT ON COLUMN sensitive_words.priority IS '优先级,数字越大优先级越高';
COMMENT ON COLUMN sensitive_words.created_at IS '创建时间';
COMMENT ON COLUMN sensitive_words.updated_at IS '更新时间';
"""
cur.execute(add_column_comments_sql)
print("添加字段注释成功")
# 创建索引
create_index_sql = """
CREATE INDEX IF NOT EXISTS idx_sensitive_words_word ON sensitive_words(word);
CREATE INDEX IF NOT EXISTS idx_sensitive_words_category ON sensitive_words(category);
"""
cur.execute(create_index_sql)
print("创建索引成功")
# 创建更新触发器
create_function_sql = """
CREATE OR REPLACE FUNCTION update_updated_at_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = CURRENT_TIMESTAMP;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
"""
cur.execute(create_function_sql)
print("创建更新函数成功")
# 检查触发器是否存在
check_trigger_sql = """
SELECT COUNT(*) FROM pg_trigger WHERE tgname = 'update_sensitive_words_updated_at'
"""
cur.execute(check_trigger_sql)
trigger_exists = cur.fetchone()[0] > 0
if not trigger_exists:
create_trigger_sql = """
CREATE TRIGGER update_sensitive_words_updated_at
BEFORE UPDATE ON sensitive_words
FOR EACH ROW
EXECUTE FUNCTION update_updated_at_column();
"""
cur.execute(create_trigger_sql)
print("创建触发器成功")
else:
print("触发器已存在,跳过创建")
# 提交事务
conn.commit()
print("事务提交成功")
# 关闭游标和连接
cur.close()
conn.close()
print("连接关闭成功")
return True
except Exception as e:
print(f"创建敏感词表失败: {e}")
return False
if __name__ == "__main__":
print("开始创建敏感词表...")
success = create_sensitive_words_table()
if success:
print("敏感词表创建成功!")
else:
print("敏感词表创建失败!")

View File

@@ -0,0 +1,27 @@
-- 创建敏感词表
CREATE TABLE IF NOT EXISTS sensitive_words (
id SERIAL PRIMARY KEY,
word VARCHAR(255) UNIQUE NOT NULL,
category VARCHAR(50) DEFAULT 'default',
priority INTEGER DEFAULT 1,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- 创建索引
CREATE INDEX IF NOT EXISTS idx_sensitive_words_word ON sensitive_words(word);
CREATE INDEX IF NOT EXISTS idx_sensitive_words_category ON sensitive_words(category);
-- 更新updated_at触发器
CREATE OR REPLACE FUNCTION update_updated_at_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = CURRENT_TIMESTAMP;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_sensitive_words_updated_at
BEFORE UPDATE ON sensitive_words
FOR EACH ROW
EXECUTE FUNCTION update_updated_at_column();

View File

@@ -0,0 +1,10 @@
{
"juming": {
"email": "chaofanai1998@gmail.com",
"password": "dpapi:AQAAANCMnd8BFdERjHoAwE/Cl+sBAAAA75Mg9f3V6US9VHIqKDeFygAAAAACAAAAAAAQZgAAAAEAACAAAABWF/Vfy5UuhJJCDfZysE6bKwCAYElSlyA9DNYer/f33gAAAAAOgAAAAAIAACAAAAC5SFpZMi3xCmsyafDYXG3Y6xgBQYBp4cbLVjOw/WzVyhAAAABvn1gHcy2q1P7N3EQwllBOQAAAANIVDlnB3xQo4T74pBRoI/15OZB6StRQMGT8OLjXIXzEcQBbsjOETUBP0mJF3Cx5eR5G9kZdAEc7GzW0QlVa5uA="
},
"juziseo": {
"email": "",
"password": ""
}
}

View File

@@ -0,0 +1,155 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :aizhan.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/3/29 20:59
@explain : 爱站网域名查询工具 - 获取域名的网站标题信息
'''
import re # 正则表达式模块用于从HTML中提取网站标题
import requests # HTTP请求库用于发送网络请求
from loguru import logger # 日志记录模块,用于记录程序运行日志
from typing import List, Optional # 类型提示
def check_aizhan(domain: str, sensitive_words: Optional[List[str]] = None, proxies: dict = None):
'''
查询域名的网站标题信息是否存在敏感词
通过爱站网aizhan.com查询域名的网站标题
:param domain: 待查询的域名www.baidu.com不带协议头
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
:return: str - 网站标题字符串,如果查询失败则返回空字符串
'''
# 初始化敏感词列表
if sensitive_words is None:
sensitive_words = []
# 构建爱站网查询URL
# 格式https://www.aizhan.com/cha/{域名}/
url = f'https://www.aizhan.com/cha/{domain}/'
# 构建HTTP请求头模拟浏览器访问
headers = {
'Accept': '*/*', # 接受所有类型的内容
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0',
# 用户代理(浏览器标识)
'Referer': url, # 来源页面(防盗链)
'host': 'www.aizhan.com', # 目标主机地址
}
# 添加重试机制
max_retries = 3
for retry in range(max_retries):
try:
# 发送GET请求获取页面内容
# url: 请求URL
# headers: HTTP请求头
# timeout=10: 设置超时时间10秒
# proxies: 代理配置(如需使用代理)
response = requests.get(url, headers=headers, timeout=10, proxies=proxies)
# 判断请求是否成功HTTP 200表示成功
if response.status_code == 200:
# 使用正则表达式提取网站标题
# 正则解释:匹配 id="webpage_title"> 开头,</div> 结尾,中间的内容
# 匹配模式id="webpage_title">标题内容</div>
# ([^<]+) 表示匹配一个或多个非<字符,作为捕获组
match = re.search(r'id="webpage_title">([^<]+)</div>', response.text)
# 如果找到匹配则返回标题文本,否则返回空字符串
title = match.group(1) if match else ''
if title:
for sensitive_word in sensitive_words:
# 使用正则表达式检查是否包含敏感词
if re.search(sensitive_word, title, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}'
return True, title
# 请求失败,继续重试
logger.warning(f"爱站网检测第{retry+1}次失败: {domain}, 状态码: {response.status_code}")
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
else:
# 达到最大重试次数,尝试不使用代理
if proxies:
logger.info('爱站网检测失败,尝试不使用代理')
try:
response = requests.get(url, headers=headers, timeout=10, proxies=None)
if response.status_code == 200:
match = re.search(r'id="webpage_title">([^<]+)</div>', response.text)
title = match.group(1) if match else ''
if title:
for sensitive_word in sensitive_words:
if re.search(sensitive_word, title, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}'
return True, title
return True, ''
except Exception as e2:
logger.error(f"不使用代理的爱站网检测也失败: {domain}, 错误: {e2}")
return False, str(e2)
return True, ''
except Exception as e: # 捕获所有异常(网络错误、超时等)
logger.warning(f"爱站网检测第{retry+1}次失败: {domain}, 错误: {e}")
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
else:
# 达到最大重试次数,尝试不使用代理
if proxies:
logger.info('爱站网检测失败,尝试不使用代理')
try:
response = requests.get(url, headers=headers, timeout=10, proxies=None)
if response.status_code == 200:
match = re.search(r'id="webpage_title">([^<]+)</div>', response.text)
title = match.group(1) if match else ''
if title:
for sensitive_word in sensitive_words:
if re.search(sensitive_word, title, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}'
return True, title
return True, ''
except Exception as e2:
logger.error(f"不使用代理的爱站网检测也失败: {domain}, 错误: {e2}")
return False, str(e2)
return False, str(e)
# # ========================= 【测试代码】 =========================
# if __name__ == '__main__':
# # 测试用例1查询常见域名
# logger.info("=== 测试爱站网查询 ===")
# test_domain_1 = 'baidu.com'
# logger.info(f"查询域名: {test_domain_1}")
#
# result_1 = check_aizhan(test_domain_1)
# logger.info(f"网站标题: {result_1}")
# logger.info("")
#
# # 测试用例2查询另一个域名
# logger.info("=== 测试查询QQ域名 ===")
# test_domain_2 = 'qq.com'
# logger.info(f"查询域名: {test_domain_2}")
#
# result_2 = check_aizhan(test_domain_2)
# logger.info(f"网站标题: {result_2}")
# logger.info("")
#
#
#
# # 测试用例4查询可能不存在的域名
# logger.info("=== 测试不存在域名 ===")
# test_domain_4 = 'nonexistentdomain123456.com'
# logger.info(f"查询域名: {test_domain_4}")
#
# result_4 = check_aizhan(test_domain_4)
# logger.info(f"网站标题: {result_4 if result_4 else '(无结果)'}")

263
domainCheck/detect/baidu.py Normal file
View File

@@ -0,0 +1,263 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :baidu.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/3/29 21:54
@explain : 百度域名检测工具 - 包含百度搜索结果查询和百度安全API检测功能
'''
import random # 随机数生成模块,用于随机选择浏览器类型
import re # 正则表达式模块用于从HTML中提取搜索结果
import time
from loguru import logger # 日志记录模块,用于记录程序运行日志
from curl_cffi import requests # curl_cffi库支持浏览器模拟的HTTP请求
from typing import List, Optional, Union, Any # 类型提示
def check_site(domain: str, sensitive_words: Optional[List[str]] = None, proxies: dict = None) -> Union[
None, tuple[bool, str], list[Any]]:
'''
通过百度搜索查询域名的相关信息
使用curl_cffi模拟浏览器访问百度搜索获取site:domain的搜索结果标题
:param domain: 待查询的域名www.baidu.com
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
:return: list - 搜索结果标题列表,如果查询失败则返回空列表
'''
# 初始化敏感词列表
if sensitive_words is None:
sensitive_words = []
# 添加重试机制
max_retries = 3
for retry in range(max_retries):
try: # try-except块用于捕获网络请求中的异常
# 构建搜索查询参数
# wd: 搜索关键词格式为site:域名
params = {
'wd': f'site:{domain}' # 搜索site:domain限制搜索结果为指定域名
}
cookies = {
'BIDUPSID': '053DBE4D820C6EFB729DC7B13B1F82B2',
'PSTM': '1775657119',
'H_PS_PSSID': '63148_67862_67986_68002_68142_68148_68152_68141_68165_68189_68226_68267_68296_68336_68369_68453_68438_68464_68541_68546_68558_68520_68589_68621_68615_68606_68601_68682_68671_68735_68544_68733_68766_68807_68901_68918_68836_68921_68955_68976_68997_69007_69010_69018_69024_69014',
'BAIDUID': '053DBE4D820C6EFB729DC7B13B1F82B2:FG=1',
'BDORZ': 'B490B5EBF6F3CD402E515D22BCDA1598',
'BAIDUID_BFESS': '053DBE4D820C6EFB729DC7B13B1F82B2:FG=1',
'BA_HECTOR': 'ah0l24210la4050l0g0k0g2g0h20071ktcocj27',
'ZFY': 'Qxyr75Xm7o8zUxYzuFnYoW7cnS:AnVp:BpnrNVkr3usno:C',
'delPer': '0',
'BAIDUID_REF': '053DBE4D820C6EFB729DC7B13B1F82B2:FG=1',
'H_WISE_SIDS': '110085_661771_667681_673676_683089_682564_685373_660925_687556_686285_690306_690303_690478_690576_690334_691545_685595_690883_692754_692777_692118_692910_693221_693364_693403_693656_693941_694124_693392_694171_694236_693996_694316_694324_693886_694409_694379_694696_694780_694783_692006_694884_694918_694996_694991_695032_695118_694840_695140_695193_695110_695262_688610_694933_695278_694865_695214_695374_690651_694577_692378_695452_695457_695480_695603_695642_695631_695668_695824_695843_695866_695888_695892_695897_695975_694177_695900_695939_696145_696150_696153_696069_696078_696066_694359_696288_696308_694985_696297_695715_696128_696313_696457_696431_696473_696492_696590_693385_696610_696682_696679_696662_696651_696655_696658_696673_696643_696110_696729_696733_696772_8000116_8000133_8000138_8000159_8000163_8000167_8000176_8000186_8000190_8000204',
'H_WISE_SIDS_BFESS': '110085_661771_667681_673676_683089_682564_685373_660925_687556_686285_690306_690303_690478_690576_690334_691545_685595_690883_692754_692777_692118_692910_693221_693364_693403_693656_693941_694124_693392_694171_694236_693996_694316_694324_693886_694409_694379_694696_694780_694783_692006_694884_694918_694996_694991_695032_695118_694840_695140_695193_695110_695262_688610_694933_695278_694865_695214_695374_690651_694577_692378_695452_695457_695480_695603_695642_695631_695668_695824_695843_695866_695888_695892_695897_695975_694177_695900_695939_696145_696150_696153_696069_696078_696066_694359_696288_696308_694985_696297_695715_696128_696313_696457_696431_696473_696492_696590_693385_696610_696682_696679_696662_696651_696655_696658_696673_696643_696110_696729_696733_696772_8000116_8000133_8000138_8000159_8000163_8000167_8000176_8000186_8000190_8000204',
'MSA_PBT': '147',
'MSA_ZOOM': '1000',
'wpr': '0',
'COOKIE_SESSION': '0_0_0_0_0_0_0_0_0_0_0_0_0_1775657396%7C1%230_0_0_0_0_0_0_0_1775657396%7C1',
'MSA_PHY_WH': '1440_3440',
'POLYFILL': '0',
'MSA_WH': '1254_940',
'kleck': '7ce2abac229d2e8ba435a8bcc6256f57f53e9d08954443bf',
'PSCBD': '16%3A1%3A3',
'SE_LAUNCH': '5%3A1775657396_16%3A29594323%3A3',
'BDSVRTM': '3',
'PSINO': '6',
'__bsi': '17976785662214065461_00_7_R_R_6_0303_c02f_Y',
}
headers = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
'Accept-Language': 'zh-CN,zh;q=0.9',
'Connection': 'keep-alive',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0',
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Microsoft Edge";v="146"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Windows"',
}
# 百度搜索URL
url = 'https://m.baidu.com/s'
# 发送GET请求获取搜索结果
# url: 请求URL
# params: 查询参数
# headers: HTTP请求头
# timeout=15: 超时时间15秒
# proxies: 代理配置
# impersonate: 模拟的浏览器类型
# .content: 获取响应内容(字节流)
# .decode('utf-8'): 解码为UTF-8字符串
response_text = requests.get(url, params=params, headers=headers, cookies=cookies, timeout=15,
proxies=proxies).content.decode('utf-8')
# 使用正则表达式提取搜索结果标题
# re.findall()返回所有匹配的结果列表
# 正则解释:匹配 <!--s-text--> 和 <!--/s-text--> 之间的内容
match = re.findall(r'<!--s-text-->([^<]+)<!--/s-text-->', response_text,re.S)
# 返回匹配结果列表,如果没有匹配则返回空列表
search_results= match if match else []
# 如果没有搜索结果,直接返回通过
if not search_results:
return True, ''
# 检查每个搜索结果是否包含敏感词
for result in search_results:
for sensitive_word in sensitive_words:
# 使用正则表达式检查是否包含敏感词
if re.search(sensitive_word, result, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
# 所有搜索结果都不包含敏感词,返回通过
return True, ''
except Exception as e: # 捕获所有异常
# logger.warning(f"百度site检测第{retry+1}次失败: {domain}, 错误: {e}")
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
proxies=None
else:
# 达到最大重试次数,返回检测失败
logger.error(f"百度site检测失败: {domain}, 已达到最大重试次数")
return False, str(e) # 返回错误信息
def baidu(domain: str, proxies: dict = None) -> str:
'''
通过百度安全API检测域名安全状态
调用百度移动安全API检测域名是否为风险网站
:param domain: 待检测的域名www.baidu.com
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
:return: str - 安全状态描述("风险网站提示""高危网站提示""正常"),如果检测失败则返回错误信息
'''
# 构建HTTP请求头
headers = {
'Cache-Control': 'no-cache', # 不使用缓存
'Content-Type': 'application/json; charset=utf-8', # 内容类型为JSON
'Host': 'mobsec-sec.baidu.com', # 目标主机
'User-Agent': 'okhttp/3.12.12', # 模拟OKHttp客户端
}
# 构建请求参数
params = {
'auth_ver': '2', # 认证版本
'appkey': '4665fd2c6b0922a551e8ae74', # API应用密钥
'nonce': '1751309255340', # 随机数(注意:实际使用时应该动态生成)
'lc': '77qHTv4VtmRiXYtd', # 本地配置参数
'pkg': 'com.baidu.searchbox', # 应用包名
'vc': '-1', # 版本码
'cuid': 'CF0E6FCCD824D146605C16AC1C8BAC95%7CVFWAO56TC', # 设备ID注意实际使用时应该动态生成
'tk': '', # 令牌(空)
'type': '3', # 检测类型
'vn': '2.6.0', # 版本号
's': 'fdfef2ce96c1c7193f9b80425020f63e', # 签名(注意:实际使用时应该动态生成)
}
# 构建JSON请求数据
json_data = {
'url': f'https://{domain}/', # 待检测的URL添加https协议头
}
# 提取URL作为后续获取结果的键
key = json_data['url']
# 添加重试机制
max_retries = 3
for retry in range(max_retries):
try: # try-except块用于捕获API调用中的异常
# 发送POST请求到百度安全API
# url: API接口地址
# params: 查询参数
# headers: HTTP请求头
# json: JSON请求数据
# proxies: 代理配置
# .json(): 解析JSON响应
response = requests.post('https://mobsec-sec.baidu.com/3.1/scanurl', params=params, headers=headers,
json=json_data, proxies=proxies, timeout=15).json()
# # 记录API响应调试用
# logger.info(response)
# 安全状态码映射表
msgs = {
2193: '风险网站提示', # 风险网站
2243: '高危网站提示', # 高危网站
'_': '正常' # 默认正常
}
# 从响应中获取安全等级并映射为状态描述
# response['response']['datas'][key].get('grand_level', 'level'): 获取安全等级,默认值为'level'
# msgs.get(): 根据安全等级获取对应的状态描述,默认值为'正常'
return msgs.get(response['response']['datas'][key].get('grand_level', 'level'), '正常')
except Exception as e: # 捕获所有异常
logger.warning(f"百度网页安全中心检测第{retry+1}次失败: {domain}, 错误: {e}")
if retry < max_retries - 1:
time.sleep(2) # 等待2秒后重试
else:
# 达到最大重试次数,尝试不使用代理
if proxies:
logger.info('百度网页安全中心检测失败,尝试不使用代理')
try:
response = requests.post('https://mobsec-sec.baidu.com/3.1/scanurl', params=params, headers=headers,
json=json_data, proxies=None, timeout=15).json()
msgs = {
2193: '风险网站提示', # 风险网站
2243: '高危网站提示', # 高危网站
'_': '正常' # 默认正常
}
return msgs.get(response['response']['datas'][key].get('grand_level', 'level'), '正常')
except Exception as e2:
logger.error(f"不使用代理的百度网页安全中心检测也失败: {domain}, 错误: {e2}")
return str(e2) # 返回错误信息
return str(e) # 返回错误信息
# # ========================= 【测试代码】 =========================
# if __name__ == '__main__': # 程序入口,当直接运行此文件时执行以下代码
# # proxies = get_proxies()
# proxies = None
# # 测试用例1查询域名搜索结果
# logger.info("=== 测试百度搜索结果查询 ===")
# test_domain_1 = '920zl.com'
# logger.info(f"查询域名: {test_domain_1}")
#
# result_1 = check_site(test_domain_1, proxies=proxies)
# logger.info(result_1)
#
# # 测试用例3检测域名安全状态
# logger.info("=== 测试域名安全检测 ===")
# test_domain_3 = 'baidu.com'
# logger.info(f"检测域名: {test_domain_3}")
#
# result_3 = baidu(test_domain_3, proxies=proxies)
# logger.info(f"安全状态: {result_3}")
# logger.debug(proxies)
# domains = ['Lqyingye.com', 'tjjinLikeji.com', 'guokangLxs.com', 'zhuoyijz.com', 'jiandanrongyi.com', 'fromhhc.com',
# 'djxow.com', '725game.com', 'gcLpchd.cn', '6rdb.com.cn', 'sunmantech.com', 'hroxa.com', 'guLhu65.cn',
# 'xxjdch.com', '7e7fm1.cn', 'kaoxueb.com', 'jabas.cn', 'shzhongyi.com', 'figuangze.com', 'eztw9k.cn',
# 'foxok.com', 'oLLbmr.com', 'vwpwwkp.cn', 'cbmrs.com', 'zaozhuang56.com', 'hengxiangracing.com',
# 'jdyfbpq.cn', 'czca.cn', 'shibingxiongdi.com', 'ubxbm.cn', 'ydbaoshi.com', '2046hd.cn', 'mffmcp.com',
# 'hgLdhzz.com', 'okswmi.cn', 'caidaozg.com', 'dkdywx.cn', 'qkjps.com', 'Lwxgk.cn', 'drtechnoLogy.cn',
# 'akcie.cn', '588uu.com', 'kuaiyijian.net', 'c5b.cn', 'faka866.cn', 'wiraf03.cn', 'bobjibar.com',
# '166233.cn', 'adLfjcjq.cn', 'whrsom.com']
# for domain in domains:
# result_1 = check_site(domain)
# logger.info(f"搜索结果标题数: {len(result_1)}")
# for i, title in enumerate(result_1, 1):
# logger.info(f" 结果{i}: {title}")
# time.sleep(1)

191
domainCheck/detect/c360.py Normal file
View File

@@ -0,0 +1,191 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :c360.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/2 15:57
@explain : 360搜索敏感词检测工具 - 检测域名是否包含敏感词
'''
import re # 正则表达式模块
import requests # HTTP请求库
from loguru import logger # 日志记录
from typing import List, Tuple, Optional, Dict # 类型提示
# 常量定义
SO_SEARCH_URL = 'https://www.so.com/s' # 360搜索URL
SO_REFERER_TEMPLATE = 'https://www.so.com/s?ie=utf-8&q=site%3A{domain}' # Referer模板
SEARCH_PATTERN = r'target="_blank">([^<]+)</a></h3>' # 搜索结果匹配模式
BLOCKED_CODE = 3 # 拦截状态码
def check_domain(
domain: str,
sensitive_words: Optional[List[str]] = None,
proxies: Optional[Dict] = None
) -> Tuple[bool, str]:
"""
检查域名是否包含敏感词
通过360搜索获取域名的搜索结果然后检查搜索结果中是否包含敏感词
Args:
domain: 要检测的域名
sensitive_words: 敏感词列表如果为None则初始化为空列表
proxies: 代理配置
Returns:
Tuple[bool, str]: (是否通过, 错误信息)
- True, '': 通过检测,不包含敏感词
- False, '敏感词xxx': 包含敏感词,返回具体的敏感词
"""
# 初始化敏感词列表
if sensitive_words is None:
sensitive_words = []
# 构建请求头
headers = {
'Accept': '*/*', # 接受所有类型
'Accept-Language': 'zh-cn', # 中文语言
'Referer': SO_REFERER_TEMPLATE.format(domain=domain), # 来源页面
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.87 Safari/537.36', # 用户代理
'Host': 'www.so.com', # 主机名
}
# 构建查询参数
params = {
'ie': 'utf-8', # 编码格式
'q': f'site:{domain}', # 搜索查询
}
# 添加重试机制
max_retries = 3
for retry in range(max_retries):
try:
# 发送GET请求获取搜索结果
response = requests.get(
SO_SEARCH_URL,
params=params,
headers=headers,
proxies=proxies,
timeout=15 # 增加超时时间到15秒
)
# 解析响应内容
response_html = response.content.decode('utf-8', errors='ignore')
# 使用正则表达式提取搜索结果
search_results = re.findall(SEARCH_PATTERN, response_html)
# 记录搜索结果数量
logger.info(f'360搜索结果数量: {len(search_results)}')
# 如果没有搜索结果,直接返回通过
if not search_results:
return True, ''
# 检查每个搜索结果是否包含敏感词
for result in search_results:
for sensitive_word in sensitive_words:
# 使用正则表达式检查是否包含敏感词
if re.search(sensitive_word, result, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
# 所有搜索结果都不包含敏感词,返回通过
return True, ''
except requests.exceptions.Timeout:
logger.warning(f'360搜索请求第{retry+1}次超时')
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
else:
# 达到最大重试次数,尝试不使用代理
if proxies:
logger.info('360搜索请求超时尝试不使用代理')
try:
response = requests.get(
SO_SEARCH_URL,
params=params,
headers=headers,
proxies=None,
timeout=15
)
response_html = response.content.decode('utf-8', errors='ignore')
search_results = re.findall(SEARCH_PATTERN, response_html)
logger.info(f'不使用代理的360搜索结果数量: {len(search_results)}')
if not search_results:
return True, ''
for result in search_results:
for sensitive_word in sensitive_words:
if re.search(sensitive_word, result, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
return True, ''
except Exception as e:
logger.error(f'不使用代理的360搜索请求也失败: {str(e)}')
return False, f'360搜索请求超时'
return False, '360搜索请求超时'
except requests.exceptions.RequestException as e:
error_str = str(e)
logger.warning(f'360搜索请求第{retry+1}次失败: {error_str}')
# 检查是否是"Too many open files"错误
if 'Too many open files' in error_str:
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
else:
# 达到最大重试次数返回failure标记不拉黑域名
logger.error(f'360搜索请求失败已达到最大重试次数: {error_str}')
return True, 'failure'
else:
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
else:
# 达到最大重试次数,尝试不使用代理
if proxies:
logger.info('360搜索请求失败尝试不使用代理')
try:
response = requests.get(
SO_SEARCH_URL,
params=params,
headers=headers,
proxies=None,
timeout=15
)
response_html = response.content.decode('utf-8', errors='ignore')
search_results = re.findall(SEARCH_PATTERN, response_html)
logger.info(f'不使用代理的360搜索结果数量: {len(search_results)}')
if not search_results:
return True, ''
for result in search_results:
for sensitive_word in sensitive_words:
if re.search(sensitive_word, result, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {result}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {result}'
return True, ''
except Exception as e2:
logger.error(f'不使用代理的360搜索请求也失败: {str(e2)}')
return False, f'360搜索请求失败: {str(e)}'
return False, f'360搜索请求失败: {str(e)}'
except Exception as e:
logger.warning(f'360搜索第{retry+1}次未知错误: {str(e)}')
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
else:
return False, f'未知错误: {str(e)}'
# if __name__ == '__main__':
# # 测试用例
# test_domain = 'niuniushouka.com'
# test_sensitive_words = ['赌博', '色情', '暴力','收卡']
#
# logger.info(f'开始检测域名: {test_domain}')
# result = check_domain(test_domain, test_sensitive_words)
# logger.info(f'检测结果: {result}')

1257
domainCheck/detect/chinaz.js Normal file

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,242 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :chinaz.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/3/27 23:48
@explain : 站长工具SEO查询工具 - 获取域名网站标题、SEO排名等信息
'''
import json # JSON处理模块用于解析API响应数据
import os # 操作系统接口模块,用于获取脚本所在目录路径
import re # 正则表达式模块,用于提取网页中的标题和加密密钥
import time # 时间处理模块,用于生成时间戳
from typing import List, Optional # 类型提示
import quickjs # JavaScript执行引擎用于执行JS加密代码生成签名
import requests # HTTP请求库用于发送网络请求
from loguru import logger # 日志记录模块,用于记录程序运行日志
def check_title(domain: str,sensitive_words: Optional[List[str]] = None, proxies: dict = None):
'''
检测域名标题并获取SEO数据
:param domain: 待检测域名www.baidu.com不带协议头
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
:return: Tuple[bool, str, Tuple[bool, Any]] - (是否成功, 消息, (SEO查询是否成功, SEO数据或错误信息))
'''
# 初始化敏感词列表
if sensitive_words is None:
sensitive_words = []
# 添加重试机制
max_retries = 3
for retry in range(max_retries):
try:
# 构建SEO查询URL站长工具网站
url = f'https://seo.chinaz.com/{domain}'
# 发送GET请求获取页面内容设置超时10秒
response = requests.get(url, timeout=10, proxies=proxies)
# 判断请求是否成功HTTP 200表示成功
if response.status_code == 200:
# 使用正则表达式提取网站标题从id="site_title"的div标签中提取文本内容
# 正则解释:匹配 id="site_title"> 开头,</div> 结尾,中间的内容
match = re.search(r'id="site_title">([^<]+)</div>', response.text)
# 使用正则表达式提取加密密钥enkey用于后续API请求的身份验证
# 正则解释:匹配 var enkey = '...' 中的单引号内容
pattern = r"var enkey = '([^']+)''"
match_enkey = re.search(pattern, response.text)
# 如果上面的正则匹配失败,尝试匹配正确的单引号版本
if not match_enkey:
pattern = r"var enkey = '([^']+)" # 匹配 var enkey = '...' 格式
match_enkey = re.search(pattern, response.text)
# 如果找到enkey则提取并去除首尾空格否则为空字符串
enkey = match_enkey.group(1).strip() if match_enkey else ''
# 提取网站标题,如果找到则返回,否则返回空字符串
title = match.group(1).strip() if match else ''
if title:
for sensitive_word in sensitive_words:
# 使用正则表达式检查是否包含敏感词
if re.search(sensitive_word, title, re.IGNORECASE):
logger.warning(f'检测到敏感词: {sensitive_word} 在结果: {title}')
return False, f'检测到敏感词: {sensitive_word} 在结果: {title}', ()
# 调用get_site_data获取详细SEO数据
success, result = get_site_data(domain, enkey, proxies=proxies)
# 检查网站分类
if success and result:
# 定义需要拉黑的分类
blacklist_categories = ['视频电影', '体育运动', '常用查询', '游戏网站', '游戏', '视频', '体育']
# 检查分类是否在黑名单中
if isinstance(result, dict):
# 检查可能的分类字段
category_fields = ['Category', 'category', '分类', '网站分类']
for field in category_fields:
if field in result:
category = result[field]
if isinstance(category, str):
for blacklist_category in blacklist_categories:
if blacklist_category in category:
logger.warning(f'检测到黑名单分类: {category} 在域名: {domain}')
return False, f'网站分类: {category} 属于黑名单分类', ()
# 检查Result是否为列表
elif isinstance(result, list):
for item in result:
if isinstance(item, dict):
category_fields = ['Category', 'category', '分类', '网站分类']
for field in category_fields:
if field in item:
category = item[field]
if isinstance(category, str):
for blacklist_category in blacklist_categories:
if blacklist_category in category:
logger.warning(f'检测到黑名单分类: {category} 在域名: {domain}')
return False, f'网站分类: {category} 属于黑名单分类', ()
return True, 'success', (success, result)
# 请求失败返回空标题和空元组
return True, '', ()
except Exception as e: # 捕获所有异常
error_str = str(e)
# 检查是否是需要重试的错误类型
if any(error_type in error_str for error_type in ['Too many open files', 'EOF occurred in violation of protocol', 'Max retries exceeded', 'Read timed out']):
logger.warning(f"站长工具检测第{retry+1}次失败: {domain}, 错误: {e}")
if retry < max_retries - 1:
time.sleep(2) # 等待2秒后重试
continue
else:
# 达到最大重试次数,返回失败但不拉黑
logger.error(f"站长工具检测失败,已达到最大重试次数: {domain}, 错误: {e}")
return True, 'failure', (False, str(e))
else:
# 其他错误直接返回
logger.error(f"站长工具检测失败: {domain}, 错误: {e}")
return False, str(e), ()
# ========================= 【最终请求】 =========================
def get_site_data(domain, enkey: str, proxies: dict = None):
'''
获取站点的详细SEO数据
通过站长工具API获取域名的SEO排名、权重等信息
:param domain: 待检测域名www.baidu.com
:param enkey: 加密密钥从check_title函数获取
:param proxies: 代理配置(如:{'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}
:return: Tuple[bool, Any] - (是否成功, 结果数据或错误信息)
- (True, Result数据): 查询成功
- (False, 错误信息): 查询失败
'''
try:
# 获取当前脚本所在目录用于构建JS文件的完整路径
current_dir = os.path.dirname(__file__)
# 构建JS加密代码文件的完整路径
js_path = os.path.join(current_dir, "chinaz.js")
# 读取JS加密代码文件包含生成签名的函数
with open(js_path, "r", encoding="utf-8") as f:
js_code = f.read()
# 创建JavaScript执行上下文QuickJS引擎
ctx = quickjs.Context()
# 【关键步骤】加载JS代码到上下文使其可以被调用
# 这一步必须先执行,否则后续的函数调用会失败
ctx.eval(js_code)
# 调用JS函数生成host_key主机密钥是生成签名的基础
# 格式generateHostKey("domain")
host_key = ctx.eval(f'generateHostKey("{domain}")')
# 获取当前时间戳(毫秒),用于防止缓存和重放攻击
ts = str(int(time.time() * 1000))
# 构建请求参数字典包含API所需的全部参数
params = {
# jQuery回调函数名格式jQuery + 随机数字 + 时间戳)
"callback": f"jQuery11130943805094955342_{int(time.time() * 1000) - 1000}",
"action": "GetCategory", # 动作类型:获取分类/排名数据
"host": domain, # 主机域名
"secretkey": enkey, # 加密密钥(从页面获取)
# 随机数字调用JS函数生成与host_key相关
"rd": ctx.eval(f'getRandomNum("{host_key}")'),
"ts": ts, # 时间戳(毫秒)
# MD5令牌调用JS函数生成用于请求签名验证
"token": ctx.eval(f'generateMD5Token("{host_key}","{ts}")'),
"_": int(time.time() * 1000) # 额外时间戳参数(用于缓存破坏)
}
# 构建HTTP请求头模拟浏览器访问
headers = {
'Accept': '*/*', # 接受所有类型的内容
'Accept-Language': 'zh-cn', # 接受的语言:简体中文
'Referer': 'https://othertool.chinaz.com/GetTopRanked.ashx', # 来源页面(防盗链)
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.87 Safari/537.36', # 用户代理(浏览器标识)
'Host': 'othertool.chinaz.com', # 目标主机地址
}
# API请求URL站长工具SEO数据查询接口
url = 'https://othertool.chinaz.com/GetTopRanked.ashx'
# 发送GET请求获取SEO数据
resp = requests.get(
url, # 请求URL
params=params, # 查询参数自动拼接为URL参数
headers=headers, # 请求头
timeout=10, # 超时时间10秒
proxies=proxies # 代理配置
)
# 判断请求是否成功
if resp.status_code == 200:
# 使用正则表达式提取JSON数据API返回格式callback(json_data)
# 正则解释匹配括号内的JSON内容
match = re.search(r'\((.*?)\)', resp.text)
# 提取JSON字符串
json_str = match.group(1)
# 解析JSON数据为Python字典
data = json.loads(json_str)
# 返回状态码和结果StateCode为1表示成功
return data['StateCode'] == 1, data['Result']
except Exception as e: # 捕获所有异常
logger.error(e) # 记录错误日志
return False, str(e) # 返回失败状态和错误信息
# # ========================= 【测试代码】 =========================
# if __name__ == '__main__':
#
#
# # 测试用例3使用代理查询
# logger.info("=== 测试使用代理 ===")
# test_domain_3 = 'tt.com'
#
# logger.info(f"查询域名: {test_domain_3}")
#
#
# title_3, seo_data_3 = check_title(test_domain_3)
# logger.info(f"域名标题: {title_3}")
# logger.info(f"SEO数据: {seo_data_3}")

View File

@@ -0,0 +1,253 @@
# -*- coding:utf-8 -*-
import requests, re, json, base64
import io, os, random
import time, cv2, json
from PIL import Image
from functools import partial
from io import BytesIO
from urllib3.util import proxy
from detect.module.use_ua import randomUA
from detect.module.gap import quekou
import subprocess
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8')
import execjs
class slide():
def __init__(self):
self.headers = randomUA()
self.t = round(time.time() * 1000)
def __ease_out_expo(self, sep):
if sep == 1:
return 1
else:
return 1 - pow(2, -10 * sep)
def get_slide_track(self, vdict):
distance = self.huak(vdict)
slide_track = [
[random.randint(-50, -10), random.randint(-50, -10), 0],
[0, 0, 0],
]
count = 30 + int(distance / 2)
t = random.randint(50, 100)
_x = 0
_y = 0
for i in range(count):
x = round(self.__ease_out_expo(i / count) * distance)
t += random.randint(10, 20)
if x == _x:
continue
slide_track.append([x, _y, t])
_x = x
slide_track.append(slide_track[-1])
return slide_track
def tp_huanyuan(self, content):
_img = Image.open(BytesIO(content))
_Ge = [{"x": -157, "y": -58}, {"x": -145, "y": -58}, {"x": -265, "y": -58}, {"x": -277, "y": -58},
{"x": -181, "y": -58}, {"x": -169, "y": -58}, {"x": -241, "y": -58}, {"x": -253, "y": -58},
{"x": -109, "y": -58}, {"x": -97, "y": -58}, {"x": -289, "y": -58}, {"x": -301, "y": -58},
{"x": -85, "y": -58}, {"x": -73, "y": -58}, {"x": -25, "y": -58}, {"x": -37, "y": -58},
{"x": -13, "y": -58}, {"x": -1, "y": -58}, {"x": -121, "y": -58}, {"x": -133, "y": -58},
{"x": -61, "y": -58}, {"x": -49, "y": -58}, {"x": -217, "y": -58}, {"x": -229, "y": -58},
{"x": -205, "y": -58}, {"x": -193, "y": -58}, {"x": -145, "y": 0}, {"x": -157, "y": 0},
{"x": -277, "y": 0}, {"x": -265, "y": 0}, {"x": -169, "y": 0}, {"x": -181, "y": 0},
{"x": -253, "y": 0}, {"x": -241, "y": 0}, {"x": -97, "y": 0}, {"x": -109, "y": 0},
{"x": -301, "y": 0}, {"x": -289, "y": 0}, {"x": -73, "y": 0}, {"x": -85, "y": 0},
{"x": -37, "y": 0}, {"x": -25, "y": 0}, {"x": -1, "y": 0}, {"x": -13, "y": 0},
{"x": -133, "y": 0}, {"x": -121, "y": 0}, {"x": -49, "y": 0}, {"x": -61, "y": 0},
{"x": -229, "y": 0}, {"x": -217, "y": 0}, {"x": -193, "y": 0}, {"x": -205, "y": 0}]
w_sep, h_sep = 10, 58
new_img = Image.new('RGB', (260, 116))
for idx in range(len(_Ge)):
x = abs(_Ge[idx]['x'])
y = 58 if _Ge[idx]['y'] == -58 else 0
img_cut = _img.crop((x, y, x + w_sep, y + h_sep))
new_x = idx % 26 * 10
new_y = 0 if idx < 26 else 58
new_img.paste(img_cut, (new_x, new_y))
img_byte = BytesIO()
new_img.save(img_byte, 'png')
return img_byte.getvalue()
def huak(self, vdict):
count = 1
bgbase64 = ""
tpbase64 = ""
for idv, p_url in vdict.items():
p_url = 'http://static.geetest.com/' + p_url
# print(p_url)
vcode = requests.get(p_url, headers=self.headers)
text = vcode.content
if idv == 'bg':
text = self.tp_huanyuan(text)
bgbase64 = base64.encodebytes(text).decode()
else:
tpbase64 = base64.encodebytes(text).decode()
count += 1
if bgbase64 and tpbase64:
dis = quekou().get_distance(bgbase64, tpbase64)
else:
dis = 0
return dis
class Geetest2():
def __init__(self):
self.header = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36",
}
# 获取当前文件的目录
current_dir = os.path.dirname(__file__)
with open(os.path.join(current_dir, 'module', 'crack_geetest2x.js'), "r", encoding="gb2312",
errors='ignore') as f:
js_encrypt = f.read()
self.js = execjs.compile(js_encrypt)
self.t = round(time.time() * 1000)
self.gct_vdi = {}
self.gct_volue = ['', ''] # 初始值
self.token_vdi = {}
self.timeout = (7, 9.05)
self.geetest_type = 'slide'
self.geetest_path = "/static/js/geetest.6.0.9.js"
def new_requests(self):
self.session = requests.session()
self.session.headers = self.header
def get_vsion(self, http, gt, proxies, pparmas):
url = "https://api.geetest.com/gettype.php"
params = {
"gt": gt,
"callback": f"geetest_{self.t}"
}
try:
respe = http.get(url=url, params=params, proxies=proxies).text
except Exception as e:
if proxy:
if pparmas:
vdict = {'result': '错误', 'message': f'代理IP超时!,请重新更换代理IP,{e}', 'proxy': proxies,
'pparmas': pparmas}
else:
vdict = {'result': '错误', 'message': f'代理IP超时!,请重新更换代理IP,{e}', 'proxy': proxies}
else:
vdict = {'result': '错误', 'message': f'请求错误,{e}'}
return vdict
try:
data = json.loads(respe[22:-1])["data"]
except:
vdict = {'result': '错误', 'type': 'auto', 'message': respe}
return vdict
self.geetest_type = data["type"]
self.geetest_path = data['path']
self.token_vdi[gt] = 1
def get_tp(self, gt, challenge, type_, proxies=None, pparmas=None):
self.new_requests()
self.t = round(time.time() * 1000)
if pparmas:
h_list = re.findall('([\w-]+):(.+)', pparmas)
for idx in h_list:
print('新增协议头' + str(idx))
self.session.headers[idx[0]] = idx[1].replace('\r', '').replace('\n', '').replace('\r\n', '')
print(self.session.headers)
if not self.token_vdi.get('gt', ''):
self.get_vsion(self.session, gt, proxies, pparmas)
url = f"https://api.geetest.com/get.php"
params = {
"gt": gt,
"challenge": challenge,
"product": "popup",
"offline": "false",
"protocol": "https://",
"type": self.geetest_type,
"path": self.geetest_path,
"callback": f"geetest_{self.t}"
}
response = self.session.get(url, params=params, timeout=self.timeout).text
geetest_type = self.geetest_type
if geetest_type == 'slide':
data = json.loads(response[22:-1])
else:
try:
data = json.loads(response[22:-1])["data"]
except:
vdict = {'result': '错误', 'type': type_, 'message': response}
return vdict
##print(data)
nc, ns = data['c'], data['s']
if geetest_type == 'slide':
data = json.loads(response[22:-1])
else:
try:
data = json.loads(response[22:-1])["data"]
except:
vdict = {'result': '错误', 'type': type_, 'message': json.loads(response)}
return vdict
##print(data)
nc, ns = data['c'], data['s']
gct_url = 'http://static.geetest.com' + data['gct_path']
##print('gct地址:', gct_url)
if geetest_type == 'click':
return {'result': '请求类型不是滑块', 'type': type_, 'gt': gt, 'challenge': challenge,
'当前类型': '点选'}
if geetest_type == 'slide': # slide
challenge = data['challenge'] # 这里challenge改变了
bg = data['bg'] # 背景图片
slice = data['slice'] # 缺口图片
vdict = {
"bg": bg,
"slice": slice,
}
slide_track = slide().get_slide_track(vdict)
imgload = 37
w = self.js.call('get_slide_w', nc, ns, gt, challenge, slide_track, imgload, self.gct_volue)
params = (
('gt', gt),
('challenge', challenge),
('w', w),
('callback', f"geetest_{self.t}"),
)
response = self.session.get('https://api.geetest.com/ajax.php', params=params, proxies=proxies,
timeout=self.timeout).text
data = json.loads(response[22:-1])
validate = data.get('validate')
vdict = {'result': data.get('message'), 'type': geetest_type, 'gt': gt, 'challenge': challenge,
'validate': validate}
return vdict
if __name__ == '__main__':
a = Geetest2()
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36",
}
url = "https://seo.juziseo.com/class/gtcode/msg/StartMsgCaptchaServlet.php"
url = "https://gsxt.hlj.gov.cn/registerValidate.jspx"
for i in range(1):
index = requests.get(url, headers=headers)
# print(index.text)
html = index.json()
challenge = html["challenge"]
gt = html["gt"]
print('gt', gt, 'challenge', challenge)
r = a.get_tp(gt, challenge, 'auto', '')
print(r)

557
domainCheck/detect/jucha.py Normal file
View File

@@ -0,0 +1,557 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :jucha.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/2 22:49
@explain : 聚查网API封装类 - 优化版,减少冗余代码,添加详细注释
'''
# 导入标准库
import base64 # Base64编码解码
import hashlib # 哈希加密
import json # JSON数据处理
import os # 操作系统接口
import pickle # 序列化反序列化
import random # 随机数生成
import subprocess # 子进程管理
import time # 时间处理
from functools import partial
# 导入第三方库
import requests # HTTP请求库
from loguru import logger # 日志记录
from requests.cookies import RequestsCookieJar # Cookie管理
# # 移除自定义子进程类的替换,避免影响其他模块
# class MySubprocessPopen(subprocess.Popen): # 自定义子进程类
# def __init__(self, *args, **kwargs): # 初始化方法
# kwargs['encoding'] = "UTF-8" # 设置默认编码为UTF-8
# super().__init__(*args, **kwargs) # 调用父类初始化方法
#
#
# subprocess.Popen = MySubprocessPopen # 替换subprocess.Popen为自定义类
os.environ["EXECJS_RUNTIME"] = "Node" # 设置JavaScript运行时环境为Node.js
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8', startupinfo=startupinfo)
def resolve_node_executable() -> str:
current_dir = os.path.dirname(__file__)
project_dir = os.path.dirname(current_dir)
bundled_node = os.path.join(project_dir, "tools", "node-v20.19.4-win-x64", "node.exe")
if os.path.exists(bundled_node):
return bundled_node
return "node"
def calculate_seed(t: int) -> str: # 计算验证码种子值
now = int(time.time()) # 获取当前时间戳
c = now - t # 计算时间差
seed = base64.b64encode(str(c).encode()).decode() # Base64编码
return seed # 返回种子字符串
def random_fingerprint() -> str: # 生成随机设备指纹
data = str(random.random()) + str(time.time()) # 组合随机数和时间戳
return hashlib.sha256(data.encode()).hexdigest() # 返回SHA256哈希值
class JC(object): # 聚查网API封装类
token: str # 验证码token
session_id: str # 会话ID
fingerprint: str # 设备指纹
captchaId: str # 验证码ID
encryptionPublicKey: str # 加密公钥
cookie: RequestsCookieJar = {} # 聚查网Cookie
juming_cookie: RequestsCookieJar = {} # 聚名网Cookie
# 通用请求头配置
headers = {
'accept': 'application/json, text/javascript, */*; q=0.01', # 接受的内容类型
'accept-language': 'zh-CN,zh;q=0.9', # 接受的语言
'content-type': 'application/x-www-form-urlencoded', # 内容类型
'origin': 'https://www.jucha.com', # 请求源
'priority': 'u=1, i', # 请求优先级
'referer': 'https://www.jucha.com/login', # 来源页面
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"', # 浏览器标识
'sec-ch-ua-mobile': '?0', # 是否移动端
'sec-ch-ua-platform': '"Windows"', # 操作系统平台
'sec-fetch-dest': 'empty', # 请求目标
'sec-fetch-mode': 'cors', # 请求模式
'sec-fetch-site': 'same-origin', # 请求站点
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36',
# 用户代理
'x-requested-with': 'XMLHttpRequest', # AJAX请求标识
}
def __init__(self, proxies: dict = None): # 初始化JC类
self.session = requests.Session() # 创建会话对象
self.session.timeout = 10 # 设置超时时间
self.session.proxies = proxies # 设置代理
self.base_url = "https://www.jucha.com" # 设置基础URL
self.cookie = requests.cookies.RequestsCookieJar() # 初始化CookieJar
self.juming_cookie = requests.cookies.RequestsCookieJar() # 初始化聚名网CookieJar
def _get_headers(self, referer: str = None) -> dict: # 获取请求头可自定义referer
headers = self.headers.copy() # 复制默认请求头
if referer: # 如果指定了referer
headers['referer'] = referer # 更新referer
return headers # 返回请求头
def _get_route_headers(self, route: str) -> dict: # 根据路由获取对应的请求头
referer_map = { # 路由到referer的映射
'whois': f'{self.base_url}/whois/', # WHOIS查询的referer
'beian': f'{self.base_url}/baian/', # 备案查询的referer
'safe': f'{self.base_url}/safe/', # 安全检测的referer
}
return self._get_headers(referer_map.get(route)) # 返回对应的请求头
def _handle_captcha(self, retry_times: int = 5) -> tuple: # 处理滑块验证码
for _ in range(retry_times): # 循环重试
init_result = self.captcha_init() # 初始化验证码
if init_result[0]: # 初始化成功
verify_result = self.captcha_verify() # 验证验证码
if verify_result[0]: # 验证成功
return True, '验证码验证成功' # 返回成功
return False, f'滑块验证码失败,{retry_times}次内未成功' # 返回失败
def _check_request(self, url: str, data: dict, headers: dict, cookies=None) -> dict: # 检查请求并处理验证码
res = self.session.post(url=url, data=data, headers=headers, cookies=cookies) # 发送POST请求
try:
response = res.json() # 尝试解析JSON
except Exception as e: # JSON解析失败
logger.error(f"JSON解析失败: {str(e)}") # 记录错误
logger.error(f"原始响应内容前500字符: {res.text[:500]}") # 记录原始响应
return {'code': -1, 'msg': f'JSON解析失败: {str(e)}'} # 返回错误
return response # 返回响应
def _check_and_handle_captcha(self, response: dict, domain: str, route: str, xm_codes,
data: dict = None) -> tuple: # 检查并处理验证码
# 检查response是否为字典
if not isinstance(response, dict):
return False, None, response # 返回失败
if response.get('code') == 1001: # 需要验证码
captcha_result = self._handle_captcha() # 处理验证码
if captcha_result[0]: # 验证码验证成功
new_data = data.copy() if data else {} # 复制数据
new_data.update({ # 更新验证码参数
'_csrf': '', # CSRF令牌
'ymlb': domain, # 域名
'xm_codes[]': xm_codes, # 检测代码
'type': '2' if route in ['beian', 'safe'] else '1', # 类型备案和安全检测为2WHOIS为1
'route': route, # 路由
'captcha_verify_param': self.token, # 验证码token
'sessionId': self.session_id, # 会话ID
})
return True, new_data, None # 返回需要重试
return False, None, captcha_result # 返回失败
return False, None, response # 不需要验证码
def captcha_init(self): # 初始化滑块验证码
self.fingerprint = random_fingerprint() # 生成随机指纹
data = { # 构建请求数据
"request_id": self.fingerprint, # 请求ID
"scene": "default", # 场景
"seed": calculate_seed(286) # 种子值
}
url = f"{self.base_url}/captcha/init" # 初始化URL
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json() # 发送请求
if response['code'] == 1: # 初始化成功
self.captchaId = response['data']['captchaId'] # 保存验证码ID
self.encryptionPublicKey = response['data']['encryptionPublicKey'] # 保存加密公钥
return response['code'] == 1, response['msg'] # 返回结果
def captcha_verify(self): # 验证滑块验证码
data = { # 构建验证数据
"offset": 290, # 滑块偏移量
"duration": 611, # 滑动持续时间
"trail": [ # 滑动轨迹
{"x": 0, "y": 0, "time": 0}, {"x": 0, "y": 0, "time": 17}, {"x": 1, "y": 0, "time": 89},
{"x": 5, "y": 0, "time": 106}, {"x": 14, "y": 0, "time": 123}, {"x": 28, "y": 0, "time": 139},
{"x": 47, "y": 0, "time": 156}, {"x": 74, "y": 0, "time": 173}, {"x": 102, "y": 0, "time": 189},
{"x": 131, "y": 0, "time": 206}, {"x": 159, "y": 0, "time": 223}, {"x": 184, "y": 0, "time": 239},
{"x": 205, "y": 0, "time": 256}, {"x": 227, "y": 1, "time": 273}, {"x": 248, "y": 0, "time": 289},
{"x": 263, "y": 0, "time": 306}, {"x": 276, "y": 0, "time": 323}, {"x": 286, "y": 0, "time": 339},
{"x": 290, "y": 0, "time": 356}, {"x": 290, "y": 0, "time": 373}, {"x": 290, "y": 0, "time": 389},
{"x": 290, "y": 0, "time": 406}, {"x": 290, "y": 0, "time": 423},
{"x": 290, "y": 0, "time": 478}
],
"fingerprint": self.fingerprint, # 设备指纹
"captchaId": self.captchaId, # 验证码ID
"serverPublicKey": self.encryptionPublicKey # 服务器公钥
}
data_str = json.dumps(data, separators=(",", ":")) # 转换为JSON字符串
current_dir = os.path.dirname(__file__)
jsFile_path = current_dir + "/sdk_leg_env.js" # JavaScript脚本路径
if not os.path.exists(jsFile_path): # 检查脚本文件是否存在
logger.error(f"Node.js脚本文件不存在: {jsFile_path}") # 记录错误
return False, f"缺少签名脚本: {jsFile_path}" # 返回失败
try: # 尝试执行Node.js脚本
with subprocess.Popen( # 创建子进程
[resolve_node_executable(), jsFile_path], # 执行node命令
stdin=subprocess.PIPE, # 标准输入管道
stdout=subprocess.PIPE, # 标准输出管道
stderr=subprocess.PIPE, # 标准错误管道
text=True, # 文本模式
encoding="utf-8", # UTF-8编码
errors="ignore" # 忽略编码错误
) as proc: # 进程上下文
stdout, stderr = proc.communicate(data_str, timeout=10) # 传入参数并获取输出
verify_data = stdout.strip() # 去除空白字符
if stderr: # 如果有错误输出
logger.error(f'genData_stderr->[{stderr}]') # 记录错误
if proc.returncode != 0: # 检查退出码
return False, f"Node.js脚本执行失败退出码: {proc.returncode}, 错误信息: {stderr}" # 返回失败
url = f"{self.base_url}/captcha/verify" # 验证URL
verify_headers = self.headers.copy() # 复制请求头
verify_headers['Content-Type'] = 'application/json' # 设置内容类型(仅用于验证请求)
verify_response = self.session.post(url, headers=verify_headers, data=verify_data,
cookies=self.cookie) # 发送验证请求
if verify_response.cookies: # 如果响应中有新的Cookie
self.cookie.update(verify_response.cookies) # 合并Cookie而不是替换
response = verify_response.json() # 解析JSON响应
if response['code'] == 1: # 验证成功
self.token = response['data']['token'] # 保存token
self.session_id = response['data']['session_id'] # 保存会话ID
return response['code'] == 1, response['msg'] # 返回结果
except subprocess.TimeoutExpired: # 超时异常
logger.error("Node.js脚本执行超时超过10秒") # 记录错误
return False, "Node.js脚本执行超时" # 返回失败
except Exception as e: # 其他异常
logger.error(f"执行Node.js脚本异常: {str(e)}") # 记录错误
return False, f"执行Node.js脚本异常: {str(e)}" # 返回失败
if not verify_data: # 检查输出是否为空
return False, "Node.js脚本未输出任何内容" # 返回失败
def save_cookies(self, filepath="jucha_cookies.pkl"): # 保存Cookie到文件
with open(filepath, "wb") as f: # 以二进制写入模式打开文件
pickle.dump(self.cookie, f) # 序列化保存Cookie
logger.info(f"已保存Cookie到文件: {filepath}")
# 保存到Redis
try:
import redis
from app.config import config
redis_client = redis.Redis(
host=config.REDIS_HOST,
port=config.REDIS_PORT,
password=config.REDIS_PASSWORD,
db=config.REDIS_DB,
decode_responses=True
)
# 将cookie转换为字典
cookie_dict = {}
# 检查self.cookie的类型
if isinstance(self.cookie, dict):
# 如果是字典,直接使用
cookie_dict = self.cookie
logger.info(f"Cookie是字典类型直接使用")
elif hasattr(self.cookie, '__iter__'):
# 如果是可迭代对象,遍历处理
for cookie in self.cookie:
# 检查cookie对象是否有name和value属性
if hasattr(cookie, 'name') and hasattr(cookie, 'value'):
cookie_dict[cookie.name] = cookie.value
logger.info(f"Cookie是可迭代对象处理后得到: {cookie_dict}")
else:
logger.warning(f"Cookie类型不支持: {type(self.cookie)}")
redis_client.set('domain_tool:jucha_cookies', str(cookie_dict))
except Exception as e:
logger.error(f"保存Cookie到Redis异常: {str(e)}")
pass
def load_cookies(self, filepath="jucha_cookies.pkl"): # 从文件加载Cookie
try: # 尝试加载
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
loaded_cookie = pickle.load(f) # 反序列化加载Cookie
# 检查加载的cookie类型
if isinstance(loaded_cookie, dict):
# 如果是字典转换为RequestsCookieJar
cookie_jar = requests.cookies.RequestsCookieJar()
for name, value in loaded_cookie.items():
cookie_jar.set(name, value)
self.cookie = cookie_jar
else:
self.cookie = loaded_cookie
except Exception as e: # 加载失败
logger.error(f"加载Cookie失败: {e}")
self.cookie = requests.cookies.RequestsCookieJar() # 创建空的CookieJar
def load_juming_cookies(self, filepath="juming_cookies.pkl"): # 从文件加载聚名网Cookie
try: # 尝试加载
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
self.juming_cookie = pickle.load(f) # 反序列化加载Cookie
except: # 加载失败
self.juming_cookie = requests.cookies.RequestsCookieJar() # 创建空的CookieJar
def auth_login(self): # 使用聚名网账号登录聚查网
params = { # 请求参数
'platform': 'juming', # 平台标识
}
res = self.session.get( # 发送GET请求
url=f'{self.base_url}/home/login/get_auth_url', # 获取授权URL
params=params, # 请求参数
headers=self.headers, # 请求头
cookies=self.juming_cookie # 聚名网Cookie
)
if res.cookies: # 如果响应中有Cookie
self.cookie.update(res.cookies) # 更新Cookie
response = res.json() # 解析JSON响应
if response['code'] == 1: # 获取授权URL成功
url = f'%s&tiao=1' % response['data']['auth_url'] # 构建跳转URL
# 合并聚名网Cookie和聚查网Cookie
combined_cookies = requests.cookies.RequestsCookieJar()
combined_cookies.update(self.juming_cookie)
combined_cookies.update(self.cookie)
response = self.session.get( # 发送GET请求
url=url, # 跳转URL
headers=self.headers, # 请求头
cookies=combined_cookies, # 合并后的Cookie
allow_redirects=False # 不自动重定向
)
if response.cookies: # 如果响应中有Cookie
# 安全更新Cookie避免冲突
for name, value in response.cookies.items():
self.cookie[name] = value # 直接赋值覆盖同名Cookie
if response.status_code == 302: # 重定向状态码
url = response.headers['location'] # 获取重定向URL
response = self.session.get( # 发送GET请求
url=url, # 重定向URL
headers=self.headers, # 请求头
allow_redirects=False # 不自动重定向
)
if response.cookies: # 如果响应中有Cookie
# 安全更新Cookie避免冲突
for name, value in response.cookies.items():
self.cookie[name] = value # 直接赋值覆盖同名Cookie
if response.status_code == 302: # 再次重定向
return True, '登录成功' # 返回成功
return False, '聚名登录过期' # 返回失败
return False, response['msg'] # 返回失败
def _build_check_data(self, domain: str, route: str, xm_codes, data: dict = None) -> dict: # 构建检测请求数据
if data is not None: # 如果data不为空
return data # 直接返回data
return { # 构建新的请求数据
'_csrf': '', # CSRF令牌
'ymlb': domain, # 域名
'xm_codes[]': xm_codes, # 检测代码
'type': '2' if route in ['beian', 'safe'] else '1', # 类型备案和安全检测为2WHOIS为1
'route': route, # 路由
}
def _make_check_request(self, domain: str, route: str, xm_codes, data: dict = None) -> tuple: # 发起检测请求
headers = self._get_route_headers(route) # 获取对应的请求头
data = self._build_check_data(domain, route, xm_codes, data) # 构建请求数据
url = f'{self.base_url}/home_item/check' # 检测URL
response = self._check_request(url, data, headers, self.cookie) # 发送请求
return response, data # 返回响应和数据
def _handle_check_response(self, response: dict, domain: str, route: str, xm_codes, data: dict,
callback) -> tuple: # 处理检测响应
need_retry, retry_data, final_response = self._check_and_handle_captcha( # 检查并处理验证码
response, domain, route, xm_codes, data
)
if need_retry: # 需要重试
return callback(domain=domain, data=retry_data) # 递归调用回调
# 检查final_response是否为字典
if not isinstance(final_response, dict):
return False, f'响应格式错误: {final_response}', '' # 返回失败
if final_response['code'] == 1: # 检测成功
return self._get_search_result(final_response, domain, route) # 获取查询结果传入domain
return False, final_response.get('msg', '未知错误'), '' # 返回失败
def _get_search_result(self, response: dict, domain: str, route: str) -> tuple: # 获取查询结果
data = { # 构建查询数据
'_csrf': '', # CSRF令牌
'domain': domain, # 域名注意使用domain而不是ymlb
'rwid': response['data']['rwid'], # 任务ID
'type': '2' if route in ['beian', 'safe'] else '1', # 类型
'route': 'baian' if route == 'beian' else route, # 路由
}
headers = self._get_route_headers(route) # 获取对应的请求头
url = f'{self.base_url}/home/item/search' if route != 'whois' else f'{self.base_url}/home/item/search_one' # 查询URL
search_response = self._check_request(url, data, headers, self.cookie) # 发送查询请求
# 检查search_response是否为字典
if not isinstance(search_response, dict):
return False, f'响应格式错误: {search_response}', '' # 返回失败
if search_response.get('code', -1) != 1: # 查询失败
return False, search_response.get('msg', '查询失败'), '' # 返回失败
if route == 'whois': # WHOIS查询
# 安全获取WHOIS状态
whois_zt = ''
try:
data1 = search_response.get('data', {})
if isinstance(data1, dict):
data2 = data1.get('data', {})
if isinstance(data2, dict):
whois = data2.get('whois', {})
if isinstance(whois, dict):
data3 = whois.get('data', {})
if isinstance(data3, dict):
data4 = data3.get('data', {})
if isinstance(data4, dict):
whois_zt = data4.get('zt', '')
except Exception as e:
logger.error(f"获取WHOIS状态失败: {e}")
return ( # 返回WHOIS结果
True, # 成功
search_response['msg'], # 消息
whois_zt # WHOIS状态
)
elif route == 'beian': # 备案查询
# 安全获取备案数据
beian_data = {}
beian_msg = ''
try:
data1 = search_response.get('data', {})
if isinstance(data1, dict):
data2 = data1.get('data', {})
if isinstance(data2, dict):
beian = data2.get('beian', {})
if isinstance(beian, dict):
data3 = beian.get('data', {})
if isinstance(data3, dict):
beian_data = data3.get('data', {})
beian_msg = data3.get('msg', '')
except Exception as e:
logger.error(f"获取备案数据失败: {e}")
is_dict = isinstance(beian_data, dict) # 是否为字典
return ( # 返回备案结果
True, # 成功
search_response['msg'], # 消息
( # 备案信息元组
beian_data.get('sj', '') if is_dict else '', # 备案时间
beian_data.get('lx', '') if is_dict else '', # 备案类型
beian_data.get('sy', '') if is_dict else '', # 备案首页地址
beian_msg, # 备案状态
)
)
elif route == 'safe': # 安全检测
# 安全获取安全检测数据
safe_data = {}
try:
data1 = search_response.get('data', {})
if isinstance(data1, dict):
safe_data = data1.get('data', {})
except Exception as e:
logger.error(f"获取安全检测数据失败: {e}")
check_items = [ # 检测项配置列表
('qqjc', 'QQ检测'),
('weixin', '微信检测'),
('qiang', '被墙检测'),
('dyjc', '抖音检测'),
('bdjc', '百度检测'),
('llqjcgg', '谷歌检测'),
('llqjchh', '火狐检测'),
]
results = [] # 结果列表
for item_key, _ in check_items: # 遍历检测项
try:
code = 1
msg = ''
item_data = safe_data.get(item_key, {})
if isinstance(item_data, dict):
data1 = item_data.get('data', {})
if isinstance(data1, dict):
code = int(data1.get('data', 1)) # 获取检测码
msg = data1.get('msg', '') # 获取消息
if code == 3: # 如果检测码为3
msg = '拦截' # 设置为拦截
results.append((code, msg)) # 添加到结果列表
except Exception as e:
logger.error(f"获取安全检测项 {item_key} 失败: {e}")
results.append((1, '查询失败')) # 添加失败结果
return ( # 返回安全检测结果
True, # 成功
search_response['msg'], # 消息
tuple(results) # 安全检测结果元组
)
return False, '未知路由', '' # 返回失败
def check_whois_domain(self, domain: str, data=None): # 查询域名WHOIS信息
try: # 异常处理
response, check_data = self._make_check_request(domain, 'whois', 'whois', data) # 发起检测请求
return self._handle_check_response(response, domain, 'whois', 'whois', check_data,
self.check_whois_domain) # 处理响应
except Exception as e: # 异常处理
logger.error(f"check_domain异常: {e}") # 记录错误
return False, str(e), '' # 返回失败
def beian_check_domain(self, domain: str, data=None): # 查询域名备案信息
try: # 异常处理
response, check_data = self._make_check_request(domain, 'beian', 'beian', data) # 发起检测请求
return self._handle_check_response(response, domain, 'beian', 'beian', check_data,
self.beian_check_domain) # 处理响应
except Exception as e: # 异常处理
logger.error(f"beian_check_domain异常: {e}") # 记录错误
return False, str(e), [] # 返回失败
def safe_check_domain(self, domain: str, data=None): # 查询域名安全信息
try: # 异常处理
xm_codes = [ # 检测代码列表
'qqjc', # QQ检测
'weixin', # 微信检测
'dyjc', # 抖音检测
'qiang', # 被墙检测
'bdjc', # 百度检测
'llqjcgg', # 谷歌检测
'llqjchh', # 火狐检测
]
response, check_data = self._make_check_request(domain, 'safe', xm_codes, data) # 发起检测请求
return self._handle_check_response(response, domain, 'safe', xm_codes, check_data,
self.safe_check_domain) # 处理响应
except Exception as e: # 异常处理
logger.error(f"safe_check_domain异常: {e}") # 记录错误
return False, str(e), [] # 返回失败
# # 测试代码
# if __name__ == '__main__':
# j = JC() # 创建JC实例
# j.load_cookies()
# j.load_juming_cookies() # 加载聚名网Cookie
# logger.info("开始登录...")
# login_result = j.auth_login() # 使用聚名网账号登录聚查网
# logger.info(f"登录结果: {login_result}")
# if not login_result[0]:
# logger.error("登录失败,无法继续测试")
# exit(1)
# # 测试WHOIS查询
# logger.info("测试WHOIS查询:")
# result = j.check_whois_domain('921229.com')
# logger.info(f"WHOIS查询结果: {result}")
# logger.info(j.cookie)
# # # 测试备案查询
# # logger.info("测试备案查询:")
# # result = j.beian_check_domain('baidu.com')
# # logger.info(f"备案查询结果: {result}")
# #
# # 测试安全检测
# logger.info("测试安全检测:")
# result = j.safe_check_domain('576777.com')
# logger.info(f"安全检测结果: {result}")
#
# # j.save_cookies() # 保存Cookie

View File

@@ -0,0 +1,497 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :juming.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/3/19 15:51
@explain : 聚名网API封装类 - 提供登录、域名查询、删除域名列表等功能
'''
import base64 # Base64编码模块
import hashlib # 哈希算法模块
import json # JSON处理模块
import os # 操作系统接口模块
import pickle # 序列化模块
import random # 随机数生成模块
import subprocess # 子进程管理模块
import time # 时间处理模块
from functools import partial
from typing import Optional, Tuple, Dict, List, Any # 类型提示
import requests # HTTP请求库
from loguru import logger # 日志记录库
from requests.cookies import RequestsCookieJar # Cookie处理
# 常量定义
BASE_URL = "https://www.juming.com" # 聚名网基础URL
CAPTCHA_MAX_RETRY = 5 # 验证码最大重试次数
REQUEST_TIMEOUT = 10 # 请求超时时间(秒)
DOWNLOAD_TIMEOUT = 60 # 下载超时时间(秒)
SEED_TIME_OFFSET = 286 # 种子时间偏移量
SLIDE_OFFSET = 290 # 滑块偏移量
SLIDE_DURATION = 611 # 滑动持续时间
class MySubprocessPopen(subprocess.Popen): # 自定义子进程类
def __init__(self, *args, **kwargs): # 初始化方法
kwargs['encoding'] = "UTF-8" # 设置编码为UTF-8
super().__init__(*args, **kwargs) # 调用父类初始化
subprocess.Popen = MySubprocessPopen # 替换默认的Popen类
os.environ["EXECJS_RUNTIME"] = "Node" # 设置JavaScript运行时为Node.js
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8', startupinfo=startupinfo)
def resolve_node_executable() -> str:
current_dir = os.path.dirname(__file__)
project_dir = os.path.dirname(current_dir)
bundled_node = os.path.join(project_dir, "tools", "node-v20.19.4-win-x64", "node.exe")
if os.path.exists(bundled_node):
return bundled_node
return "node"
def calculate_seed(t: int) -> str: # 计算验证码种子值
now = int(time.time()) # 获取当前时间戳
c = now - t # 计算时间差
seed = base64.b64encode(str(c).encode()).decode() # Base64编码
return seed # 返回种子字符串
def random_fingerprint() -> str: # 生成随机设备指纹
data = str(random.random()) + str(time.time()) # 组合随机数和时间戳
return hashlib.sha256(data.encode()).hexdigest() # 返回SHA256哈希值
def glwb(s: Optional[str]) -> str: # JS文本过滤函数
"""
过滤特殊字符防止XSS攻击
Args:
s: 待过滤的字符串
Returns:
过滤后的字符串
"""
if s is None or not isinstance(s, str): # 检查输入是否有效
return '' # 返回空字符串
a_nr = s # 复制字符串
a_nr = a_nr.replace('"', '&#34;') # 替换双引号
a_nr = a_nr.replace("'", '&#39;') # 替换单引号
a_nr = a_nr.replace('<', '&#60;') # 替换小于号
a_nr = a_nr.replace('>', '&#62;') # 替换大于号
a_nr = a_nr.replace('\\', '&#92;') # 替换反斜杠
return a_nr # 返回过滤后的字符串
def md5_19(text: str) -> str: # 计算MD5并返回前19位
"""
计算MD5哈希值并截取前19位
Args:
text: 待哈希的文本
Returns:
MD5哈希值的前19位
"""
if not text: # 检查输入是否为空
return '' # 返回空字符串
md5 = hashlib.md5(text.encode('utf-8')).hexdigest() # 生成32位MD5
return md5[:19] # 返回前19位
def encrypt_password(loginToken: str, password: str) -> str: # 加密密码
"""
使用双重MD5加密密码
Args:
loginToken: 登录令牌
password: 明文密码
Returns:
加密后的密码
"""
filtered_pwd = glwb(password) # 过滤密码中的特殊字符
step1 = f'[jiami{filtered_pwd}mima]' # 拼接固定盐值
md5_step1 = md5_19(step1) # 第一次MD5加密
step2 = loginToken + md5_step1 # 拼接登录令牌
final_result = md5_19(step2) # 第二次MD5加密
return final_result # 返回最终加密结果
class JM(object): # 聚名网API封装类
token: str # 验证码token
loginToken: str = '' # 登录令牌
session_id: str # 会话ID
fingerprint: str # 设备指纹
captchaId: str # 验证码ID
encryptionPublicKey: str # 加密公钥
cookie: RequestsCookieJar = {} # Cookie存储
# 默认请求头
headers = {
'Host': 'www.juming.com', # 主机名
'sec-ch-ua-platform': '"Windows"', # 平台标识
'x-requested-with': 'XMLHttpRequest', # AJAX请求标识
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0', # 用户代理
'accept': '*/*', # 接受所有类型
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Microsoft Edge";v="146"', # 浏览器标识
'sec-ch-ua-mobile': '?0', # 是否移动端
'origin': 'https://www.juming.com', # 请求源
'sec-fetch-site': 'same-origin', # 请求站点
'sec-fetch-mode': 'cors', # 请求模式
'sec-fetch-dest': 'empty', # 请求目标
'referer': 'https://www.juming.com/', # 来源页面
'accept-language': 'zh-CN,zh;q=0.9', # 接受的语言
'priority': 'u=1, i', # 请求优先级
'Content-Type': 'application/x-www-form-urlencoded', # 内容类型
}
def __init__(self, proxies: Optional[Dict] = None): # 初始化方法
self.session = requests.Session() # 创建会话对象
self.session.timeout = REQUEST_TIMEOUT # 设置超时时间
self.session.proxies = proxies # 设置代理
self.base_url = BASE_URL # 设置基础URL
logger.debug(proxies)
def captcha_init(self) -> Tuple[bool, str]: # 初始化滑块验证码
"""
初始化滑块验证码
Returns:
Tuple[bool, str]: (是否成功, 消息)
"""
self.fingerprint = random_fingerprint() # 生成随机指纹
data = { # 构建请求数据
"request_id": self.fingerprint, # 请求ID
"scene": "default", # 场景
"seed": calculate_seed(SEED_TIME_OFFSET) # 种子值
}
logger.info(data)
self.headers['Content-Type'] = 'application/x-www-form-urlencoded' # 设置内容类型
url = f"{self.base_url}/captcha/init" # 初始化URL
# self.cookie.clear()
# self.cookie.update({'acw_sc__v2': '69d948ed49117d89433d4316a79ada6f00d68f26'})
# logger.info(self.cookie)
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json() # 发送请求
logger.info(response)
if response['code'] == 1: # 初始化成功
self.captchaId = response['data']['captchaId'] # 保存验证码ID
self.encryptionPublicKey = response['data']['encryptionPublicKey'] # 保存加密公钥
return response['code'] == 1, response['msg'] # 返回结果
def captcha_verify(self) -> Tuple[bool, str]: # 验证滑块验证码
"""
验证滑块验证码
Returns:
Tuple[bool, str]: (是否成功, 消息)
"""
data = { # 构建验证数据
"offset": SLIDE_OFFSET, # 滑块偏移量
"duration": SLIDE_DURATION, # 滑动持续时间
"trail": [ # 滑动轨迹
{"x": 0, "y": 0, "time": 0}, # 起点
{"x": 0, "y": 0, "time": 17}, # 第1个点
{"x": 1, "y": 0, "time": 89}, # 第2个点
{"x": 5, "y": 0, "time": 106}, # 第3个点
{"x": 14, "y": 0, "time": 123}, # 第4个点
{"x": 28, "y": 0, "time": 139}, # 第5个点
{"x": 47, "y": 0, "time": 156}, # 第6个点
{"x": 74, "y": 0, "time": 173}, # 第7个点
{"x": 102, "y": 0, "time": 189}, # 第8个点
{"x": 131, "y": 0, "time": 206}, # 第9个点
{"x": 159, "y": 0, "time": 223}, # 第10个点
{"x": 184, "y": 0, "time": 239}, # 第11个点
{"x": 205, "y": 0, "time": 256}, # 第12个点
{"x": 227, "y": 1, "time": 273}, # 第13个点
{"x": 248, "y": 0, "time": 289}, # 第14个点
{"x": 263, "y": 0, "time": 306}, # 第15个点
{"x": 276, "y": 0, "time": 323}, # 第16个点
{"x": 286, "y": 0, "time": 339}, # 第17个点
{"x": 290, "y": 0, "time": 356}, # 第18个点
{"x": 290, "y": 0, "time": 373}, # 第19个点
{"x": 290, "y": 0, "time": 389}, # 第20个点
{"x": 290, "y": 0, "time": 406}, # 第21个点
{"x": 290, "y": 0, "time": 423}, # 第22个点
{"x": 290, "y": 0, "time": 478}, # 终点
],
"fingerprint": self.fingerprint, # 设备指纹
"captchaId": self.captchaId, # 验证码ID
"serverPublicKey": self.encryptionPublicKey # 服务器公钥
}
data_str = json.dumps(data, separators=(",", ":")) # 转换为JSON字符串压缩格式
current_dir = os.path.dirname(__file__)
jsFile_path = current_dir+"/sdk_leg_env.js" # JavaScript脚本路径
if not os.path.exists(jsFile_path): # 检查脚本文件是否存在
logger.error(f"Node.js脚本文件不存在: {jsFile_path}") # 记录错误
return False, f"缺少签名脚本: {jsFile_path}" # 返回失败
try: # 异常处理
with subprocess.Popen( # 执行Node.js脚本
[resolve_node_executable(), jsFile_path], # 命令和参数
stdin=subprocess.PIPE, # 标准输入
stdout=subprocess.PIPE, # 标准输出
stderr=subprocess.PIPE, # 标准错误
text=True, # 文本模式
encoding="utf-8", # UTF-8编码
errors="ignore" # 忽略编码错误
) as proc: # 进程上下文
stdout, stderr = proc.communicate(data_str, timeout=REQUEST_TIMEOUT) # 传入参数并获取输出
verify_data = stdout.strip() # 去除空白字符
if stderr: # 如果有错误输出
logger.error(f'genData_stderr->[{stderr}]') # 记录错误
if proc.returncode != 0: # 检查退出码
return False, f"Node.js脚本执行失败退出码: {proc.returncode}, 错误信息: {stderr}" # 返回失败
url = f"{self.base_url}/captcha/verify" # 验证URL
self.headers['Content-Type'] = 'application/json' # 设置内容类型
response = self.session.post(url, headers=self.headers, data=verify_data, cookies=self.cookie).json() # 发送验证请求
if response['code'] == 1: # 验证成功
self.token = response['data']['token'] # 保存token
self.session_id = response['data']['session_id'] # 保存会话ID
return response['code'] == 1, response['msg'] # 返回结果
except subprocess.TimeoutExpired: # 超时异常
logger.error("Node.js脚本执行超时") # 记录错误
return False, "Node.js脚本执行超时" # 返回失败
except Exception as e: # 其他异常
logger.error(f"执行Node.js脚本异常: {str(e)}") # 记录错误
return False, f"执行Node.js脚本异常: {str(e)}" # 返回失败
if not verify_data: # 检查输出是否为空
return False, "Node.js脚本未输出任何内容" # 返回失败
def user_zh_wxdl_ewm(self) -> Tuple[bool, str, Optional[str]]: # 获取微信登录二维码
"""
获取微信登录二维码
Returns:
Tuple[bool, str, Optional[str]]: (是否成功, 消息, 二维码URL)
"""
url = f"{self.base_url}/user_zh/wxdl_ewm" # 二维码URL
try: # 异常处理
response = self.session.post(url, headers=self.headers).json() # 发送请求
self.token = response['data']["token"] # 保存token
return response['code'] == 1, response['msg'], response['data']['url'] if response['code'] == 1 else None # 返回结果
except Exception as e: # 异常处理
return False, f"请求失败: {str(e)}", None # 返回失败
def user_zh_p_login(self, email: str, password: str, data: Optional[Dict] = None) -> Tuple[bool, str]: # 账号密码登录
"""
使用账号密码登录
Args:
email: 邮箱账号
password: 明文密码
data: 携带的请求数据用于递归刷新token时传参
Returns:
Tuple[bool, str]: (是否成功, 消息)
"""
try: # 异常处理
captcha_success = False # 验证码成功标志
captcha_msg = f"滑块验证码失败,{CAPTCHA_MAX_RETRY}次内未成功" # 默认失败消息
for _ in range(CAPTCHA_MAX_RETRY): # 循环尝试验证滑块
if data is None: # 如果没有传入data
captcha_success, captcha_msg = self.captcha_init() # 初始化验证码
else: # 如果已有data
captcha_success = True # 直接视为验证通过
if captcha_success: # 验证码初始化成功
if data is None: # 只有第一次需要验证滑块
captcha_success, captcha_msg = self.captcha_verify() # 验证滑块
if captcha_success: # 滑块验证成功
if data is None: # 首次登录
data = { # 构建登录数据
"re_mm": encrypt_password(self.loginToken, password), # 加密密码
"re_yx": email, # 邮箱
"fs": "tl", # 登录方式
"dltoken": self.loginToken, # 登录令牌
"token": json.dumps( # token压缩格式JSON
{"code": self.token, "sessionId": self.session_id},
separators=(",", ":")
)
}
login_url = f"{self.base_url}/user_zh/p_login" # 登录URL
response = self.session.post( # 发送登录请求
url=login_url,
headers=self.headers,
json=data,
cookies=self.cookie,
)
logger.info(response.text)
result = response.json() # 解析响应
code = result.get("code") # 获取状态码
msg = result.get("msg", "登录接口未返回消息") # 获取消息
if code == 1: # 登录成功
self.cookie = response.cookies # 保存Cookie
return True, msg # 返回成功
elif code == -118: # token过期
self.loginToken = result.get("token", "") # 更新登录令牌
data["re_mm"] = encrypt_password(self.loginToken, password) # 重新加密密码
data["dltoken"] = self.loginToken # 更新令牌
return self.user_zh_p_login(email, password, data) # 递归重新登录
else: # 其他错误
return False, msg # 返回失败
return captcha_success, captcha_msg # 返回验证码结果
except Exception as e: # 异常处理
return False, f"请求失败: {str(e)}" # 返回失败
def ykj_get_list(self, page: int = 1, page_size: int = 50, data: Optional[Dict] = None) -> Tuple[bool, Any]: # 获取一口价域名列表
"""
获取一口价域名列表
Args:
page: 页码
page_size: 每页数量
data: 额外数据
Returns:
Tuple[bool, Any]: (是否成功, HTML内容或错误消息)
"""
data = { # 构建请求数据
'psize': page_size, # 每页数量
'page': page, # 页码
} if data is None else data # 如果有data则使用data
url = f"{self.base_url}/ykj/get_list" # 列表URL
try: # 异常处理
self.headers['Content-Type'] = 'application/x-www-form-urlencoded' # 设置内容类型
res = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie) # 发送请求
response = res.json() # 解析响应
if response['code'] == -401: # 需要验证码
captcha_response = False, f'滑块验证码失败,{CAPTCHA_MAX_RETRY}次内未成功' # 默认失败
for _ in range(CAPTCHA_MAX_RETRY): # 循环尝试验证
captcha_response = self.captcha_init() # 初始化验证码
if captcha_response[0]: # 初始化成功
captcha_response = self.captcha_verify() # 验证验证码
if captcha_response[0]: # 验证成功
data = {
'psize': page_size, # 每页数量
'page': page, # 页码
# 构建token数据
'token': json.dumps({ # token压缩格式JSON
'code': self.token,
'sessionId': self.session_id,
}, separators=(",", ":"))
}
return self.ykj_get_list(page, page_size,data=data) # 递归调用
return captcha_response[0], captcha_response[1] # 返回验证码结果
elif response['code'] == 1 and response.get('data') == 'yzmhuaok': # 验证码通过
self.cookie = res.cookies # 更新Cookie
return self.ykj_get_list(page, page_size) # 递归调用
elif response['code'] == 1 and response.get('html'): # 成功获取HTML
return response['code'] == 1, response['html'] # 返回HTML
return False, response['msg'] # 返回失败
except Exception as e: # 异常处理
return False, f"请求失败: {str(e)}" # 返回失败
def new_cha_del(self, date: str) -> List[str]: # 获取删除域名列表
"""
获取指定日期的删除域名列表
Args:
date: 日期格式YYYY-MM-DD
Returns:
List[str]: 域名列表
"""
url = f"{self.base_url}/newcha/del_down?scsj={date}" # 下载URL
try: # 异常处理
response = self.session.get(url, headers=self.headers, cookies=self.cookie, allow_redirects=False) # 发送请求
url = response.headers['Location'] # 获取重定向URL
response = self.session.get(url, timeout=DOWNLOAD_TIMEOUT) # 下载文件
response_text = response.content.decode('utf-8') # 解码内容
lines = response_text.strip().splitlines() # 按行分割
return lines # 返回域名列表
except Exception as e: # 异常处理
logger.error(f"<获取删除域名列表错误>: {str(e)}") # 记录错误
return [] # 返回空列表
def save_cookies(self, filepath: str = "juming_cookies.pkl") -> None: # 保存Cookie到文件
"""
保存Cookie到文件和Redis
Args:
filepath: 文件路径
"""
with open(filepath, "wb") as f: # 以二进制写入模式打开文件
pickle.dump(self.cookie, f) # 序列化保存
# 保存到Redis
try:
import redis
from app.config import config
redis_client = redis.Redis(
host=config.REDIS_HOST,
port=config.REDIS_PORT,
password=config.REDIS_PASSWORD,
db=config.REDIS_DB,
decode_responses=True
)
# 将cookie转换为字典
cookie_dict = {}
for cookie in self.cookie:
cookie_dict[cookie.name] = cookie.value
redis_client.set('domain_tool:juming_cookies', str(cookie_dict))
except Exception as e:
logger.error(f"保存Cookie到Redis失败: {str(e)}")
pass
def load_cookies(self, filepath: str = "juming_cookies.pkl") -> None: # 从文件加载Cookie
"""
从文件加载Cookie
Args:
filepath: 文件路径
"""
try: # 异常处理
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
self.cookie = pickle.load(f) # 反序列化加载
except: # 异常处理
self.cookie = requests.cookies.RequestsCookieJar() # 创建空Cookie
# if __name__ == '__main__': # 主程序入口
# m = JM() # 创建JM实例
#
# # 账号密码登录(请替换为您的账号密码)
# email = 'chaofanai1998@gmail.com' # 邮箱账号
# password = 'llzz123,./' # 密码
#
# logger.info(f"开始登录: {email}") # 记录日志
# login_result = m.user_zh_p_login(email, password) # 执行登录
# logger.info(f"登录结果: {login_result}") # 记录结果
#
# if not login_result[0]: # 登录失败
# logger.error("登录失败,程序退出") # 记录错误
# exit(1) # 退出程序
#
# m.load_cookies() # 加载Cookie
#
# # 获取一口价域名
# import re # 导入正则表达式模块
#
# for page in range(1, 2): # 循环获取页面
# success, html = m.ykj_get_list(page) # 获取列表
# if success: # 获取成功
# pattern_ym = r"<a class='yda1 ydz' ym='([^']*)'" # 匹配域名
# results = re.findall(pattern_ym, html) # 查找所有域名
# logger.info(results)
# logger.info(f"第{page}页,找到{len(results)}个域名") # 记录结果
# if not results: # 如果没有域名
# logger.info("没有更多域名,停止获取") # 记录日志
# break # 退出循环
# time.sleep(5) # 等待5秒
#
# # 获取删除域名
# deleted_domains = m.new_cha_del("2026-03-11") # 获取删除域名
# logger.info(f"找到{len(deleted_domains)}个删除域名") # 记录结果
#
# m.save_cookies() # 保存Cookie

Binary file not shown.

View File

@@ -0,0 +1,236 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :juziseo.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/10 15:00
@explain : 桔子SEO API封装类 - 提供登录、域名查询等功能
'''
# 导入标准库
import os # 操作系统接口
import pickle # 序列化反序列化
import re
import time # 时间处理
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad
import base64
# 导入第三方库
import requests # HTTP请求库
from loguru import logger # 日志记录
from requests.cookies import RequestsCookieJar # Cookie管理
from detect.geetest2 import Geetest2
# AES-CBC 加密
def aes_cbc_encrypt(data, key=b'pvjxzjmzwawfscft', iv=b'qvibva1wg0uxwjeu'):
cipher = AES.new(key, AES.MODE_CBC, iv)
padded_data = pad(data, AES.block_size) # PKCS7填充
encrypted = cipher.encrypt(padded_data)
return base64.b64encode(encrypted).decode()
class Juziseo:
"""
桔子SEO API封装类
"""
cookie: RequestsCookieJar = {} # 桔子SEO Cookie
# 通用请求头配置
headers = {
'accept': 'application/json, text/javascript, */*; q=0.01', # 接受的内容类型
'accept-language': 'zh-CN,zh;q=0.9', # 接受的语言
'content-type': 'application/x-www-form-urlencoded', # 内容类型
'origin': 'https://seo.juziseo.com', # 请求源
'priority': 'u=1, i', # 请求优先级
'referer': 'https://seo.juziseo.com/login', # 来源页面
'sec-ch-ua': '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"', # 浏览器标识
'sec-ch-ua-mobile': '?0', # 是否移动端
'sec-ch-ua-platform': '"Windows"', # 操作系统平台
'sec-fetch-dest': 'empty', # 请求目标
'sec-fetch-mode': 'cors', # 请求模式
'sec-fetch-site': 'same-origin', # 请求站点
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36',
# 用户代理
'x-requested-with': 'XMLHttpRequest', # AJAX请求标识
}
def __init__(self, proxies: dict = None):
"""
初始化桔子SEO类
:param proxies: 代理配置
"""
self.validate = None
self.challenge = None
self.gt = None
self.session = requests.Session() # 创建会话对象
self.session.proxies = proxies # 设置代理
self.session.timeout = 10 # 设置超时时间10秒
self.base_url = "https://seo.juziseo.com" # 设置基础URL
def save_cookies(self, filepath="juziseo_cookies.pkl"):
"""
保存Cookie到文件和Redis
:param filepath: 文件路径
"""
with open(filepath, "wb") as f: # 以二进制写入模式打开文件
pickle.dump(self.cookie, f) # 序列化保存Cookie
logger.info(f"已保存桔子SEO Cookie到 {filepath}")
# 保存到Redis
try:
import redis
from app.config import config
redis_client = redis.Redis(
host=config.REDIS_HOST,
port=config.REDIS_PORT,
password=config.REDIS_PASSWORD,
db=config.REDIS_DB,
decode_responses=True
)
# 将cookie转换为字典
cookie_dict = {}
for cookie in self.cookie:
cookie_dict[cookie.name] = cookie.value
redis_client.set('domain_tool:juziseo_cookies', str(cookie_dict))
except Exception as e:
pass
def load_cookies(self, filepath="juziseo_cookies.pkl"):
"""
从文件加载Cookie
:param filepath: 文件路径
"""
try: # 尝试加载
with open(filepath, "rb") as f: # 以二进制读取模式打开文件
self.cookie = pickle.load(f) # 反序列化加载Cookie
# logger.info(f"已从 {filepath} 加载桔子SEO Cookie")
except: # 加载失败
self.cookie = requests.cookies.RequestsCookieJar() # 创建空的CookieJar
logger.warning(f"加载桔子SEO Cookie失败创建空Cookie")
def start_msg_captcha_servlet(self):
url = f"{self.base_url}/class/gtcode/msg/StartMsgCaptchaServlet.php"
response = self.session.get(url, headers=self.headers, cookies=self.cookie).json()
logger.info(response)
self.gt = response['gt']
self.challenge = response['challenge']
return response['success'] == 1
def get_captcha(self):
GETT2 = Geetest2()
code = GETT2.get_tp(gt=self.gt, challenge=self.challenge, type_='auto')
logger.info(code)
if code['result'] == 'success':
self.challenge = code['challenge']
self.gt = code['gt']
self.validate = code['validate']
return True
else:
logger.error(f"获取桔子SEO验证码失败: {code.get('msg', '未知错误')}")
return False
def login(self, username, password):
"""
登录桔子SEO
:param username: 账号
:param password: 密码
:return: tuple - (是否成功, 消息)
"""
try:
for _ in range(3):
if self.start_msg_captcha_servlet():
if self.get_captcha():
break
data = {
'return_url': '/',
'user_name': aes_cbc_encrypt(bytes(username, 'utf-8')),
'password': aes_cbc_encrypt(bytes(password, 'utf-8')),
'geetest_challenge': self.challenge,
'geetest_validate': self.validate,
'geetest_seccode': self.validate + '|jordan',
'_post_type': 'ajax',
}
logger.debug(data)
# 发送登录请求
url = f"{self.base_url}/account/ajax/login_process/"
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie)
logger.info(response.text)
# 解析响应
result = response.json()
if result.get('errno') == 1:
# 登录成功保存Cookie
self.cookie = response.cookies
self.save_cookies()
logger.info("桔子SEO登录成功")
return True, "登录成功"
else:
logger.error(f"桔子SEO登录失败: {result.get('msg', '未知错误')}")
return False, result.get('msg', '登录失败')
except Exception as e:
logger.error(f"桔子SEO登录异常: {e}")
return False, f"登录失败: {str(e)}"
def check_history(self, domain: str, sensitive_words: list = None):
# 检测域名历史是否存在敏感词
if sensitive_words is None:
sensitive_words = []
data = {
'qrtypeindex': '1',
'domains': domain,
'_post_type': 'ajax',
}
url = f"{self.base_url}/snapshot/save/"
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json()
# logger.info(response)
if response['errno'] == 1:
url = response['rsm']['url']
response_html = self.session.get(url, headers=self.headers, cookies=self.cookie).content.decode('utf-8')
title_sensitive_words_match = re.search(r'标题敏感词\D*(\d+)', response_html, re.S)
title_suspected_sensitive_words_match = re.search(r'标题有疑似敏感词\D*(\d+)', response_html, re.S)
content_sensitive_words_match = re.search(r'内容敏感词\D*(\d+)', response_html, re.S)
baidu_sensitive_words_match = re.search(r'百度历史收录敏感\D*(\d+)', response_html, re.S)
if bool(title_sensitive_words_match or title_suspected_sensitive_words_match or content_sensitive_words_match or baidu_sensitive_words_match):
return False, "存在敏感词"
subdomain_match = re.search(r'子域名:\D*(\d+)', response_html, re.S)
if subdomain_match:
return False, f"存在子域名: {subdomain_match.group(1)}"
for sensitive_word in sensitive_words:
if sensitive_word in response_html:
return False, f"存在敏感词: {sensitive_word}"
return True, 'success'
return False, str(response.get('err', '请求失败'))
def check_external_link(self, domain: str, sensitive_words: list = None):
# 外链查询域名是否存在敏感词
if sensitive_words is None:
sensitive_words = []
data = {
'qrtypeindex': '1',
'domains': domain,
'_post_type': 'ajax',
}
url = f"{self.base_url}/domain_rank/save_domain/"
response = self.session.post(url, headers=self.headers, data=data, cookies=self.cookie).json()
logger.info(response)
if response['errno'] == 1:
url = response['rsm']['url']
response_html = self.session.get(url, headers=self.headers, cookies=self.cookie).content.decode('utf-8')
subdomain_match = re.search(r'子域名:\D*(\d+)', response_html, re.S)
if subdomain_match:
return False, f"存在子域名: {subdomain_match.group(1)}"
for sensitive_word in sensitive_words:
if sensitive_word in response_html:
return False, f"存在敏感词: {sensitive_word}"
return True, 'success'
return False, str(response.get('err', '请求失败'))

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,53 @@
# -*- coding: utf-8 -*-
import base64,re
import time, os,datetime, sched
import cv2
from io import BytesIO
from PIL import Image, ImageDraw, ImageFont
import requests,re
import numpy as np
class quekou():
def __init__(self):
self.t = int(time.time() * 1000)
pass
def save_png(self,path,con):
with open(path, "wb") as fp:
fp.write(con)
def get_distance2(self,bg,tp):
bg = base64.b64decode(bg) # base64转二进制
tp = base64.b64decode(tp) # base64转二进制
res = self.det.slide_match(tp, bg, simple_target=True)
res=res['target'][0]
return res
def get_distance(self, bg, tp):
bg = base64.b64decode(bg) # base64转二进制
tp = base64.b64decode(tp) # base64转二进制
''' bg: 背景图片 tp: 缺口图片 out:输出图片 '''
# 读取背景图片和缺口图片
bg_img = Image.open(BytesIO(bg)) # 背景图片
tp_img = Image.open(BytesIO(tp)) # 缺口图片
bg_edge = cv2.Canny(np.array(bg_img), 100, 200)
tp_edge = cv2.Canny(np.array(tp_img), 100, 200)
# 转换图片格式
bg_pic = cv2.cvtColor(bg_edge, cv2.COLOR_GRAY2RGB)
tp_pic = cv2.cvtColor(tp_edge, cv2.COLOR_GRAY2RGB)
# 缺口匹配
res = cv2.matchTemplate(bg_pic, tp_pic, cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res) # 寻找最优匹配
# 绘制方框
tl = max_loc # 左上角点的坐标
return tl[0]

View File

@@ -0,0 +1,85 @@
# encoding: utf-8
import requests
import random
from loguru import logger
msg="提示:返回出现'ip overtime','forbidden'等信息说明本机IP被目标网站限制,请使用/更换代理IP或更换电脑即可。"
warning="仅用于学习交流,请勿用于非法用途,违者后果自负!"
logger.warning(warning)
logger.info(msg)
UA = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36",
"Mozilla/5.0 (Macintosh; U; PPC Mac OS X; pl-PL; rv:1.0.1) Gecko/20021111 Chimera/0.6",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.8 (KHTML, like Gecko, Safari) Cheshire/1.0.UNOFFICIAL",
"Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.8.1b2) Gecko/20060821 BonEcho/2.0b2 (Debian-1.99+2.0b2+dfsg-1)",
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0",
"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)",
"Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36",
"Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
"Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; The World)",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; AcooBrowser; .NET CLR 1.1.4322; .NET CLR 2.0.50727)",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Acoo Browser; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)",
"Mozilla/4.0 (compatible; MSIE 7.0; AOL 9.5; AOLBuild 4337.35; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)",
"Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)",
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; Media Center PC 6.0)",
"Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 1.0.3705; .NET CLR 1.1.4322)",
"Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 3.0.04506.30)",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN) AppleWebKit/523.15 (KHTML, like Gecko, Safari/419.3) Arora/0.3 (Change: 287 c9dfb30)",
"Mozilla/5.0 (X11; U; Linux; en-US) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) Arora/0.6",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2pre) Gecko/20070215 K-Ninja/2.1.1",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9) Gecko/20080705 Firefox/3.0 Kapiko/3.0",
"Mozilla/5.0 (X11; Linux i686; U;) Gecko/20070322 Kazehakase/0.4.5",
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko Fedora/1.9.0.8-1.fc10 Kazehakase/0.5.6",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/535.20 (KHTML, like Gecko) Chrome/19.0.1036.7 Safari/535.20",
"Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; fr) Presto/2.9.168 Version/11.52",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 TaoBrowser/2.0 Safari/536.11",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.71 Safari/537.1 LBBROWSER",
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; LBBROWSER)",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E; LBBROWSER)",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.84 Safari/535.11 LBBROWSER",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; QQBrowser/7.0.3698.400)",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E)",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SV1; QQDownload 732; .NET4.0C; .NET4.0E; 360SE)",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E)",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1",
"Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; zh-cn) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5",
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b13pre) Gecko/20110307 Firefox/4.0b13pre",
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11",
"Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.16 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36",
"Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36",
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0.6",
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1468.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36",
"Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36"
]
def randomUA():
return {"User-Agent": random.choice(UA)}
# if __name__ == '__main__':
#
# print(randomUA())

View File

@@ -0,0 +1,75 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :register.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/3/27 22:55
@explain : 域名注册状态检测工具
'''
from datetime import datetime, timezone, timedelta # 日期时间处理,用于时区转换
import requests # HTTP请求库
from loguru import logger # 日志记录
def check_register(domain: str, postfix: str = 'com',proxies: dict = None):
'''
检测注册状态
:param domain: 待检测域名(不包含后缀)
:param postfix: 域名后缀com、net默认com
:param proxies: 代理默认None
:return: 注册状态2可注册3已注册-1检测失败过期时间格式YYYY-MM-DD HH:MM:SS为空
'''
url = f'https://rdap.verisign.com/{postfix}/v1/domain/{domain}' # 构建RDAP查询URL
# url = f'https://www.baidu.com' # 构建RDAP查询URL
# 添加重试机制
max_retries = 3
for retry in range(max_retries):
try:
response = requests.get(url, timeout=20, proxies=proxies) # 发送GET请求设置超时5秒
if response.status_code == 200: # HTTP 200表示域名已注册
json_data = response.json() # 解析JSON响应
for item in json_data['events']: # 遍历事件列表
if item['eventAction'] == 'expiration': # 如果是过期时间事件
utc_time = datetime.fromisoformat(item['eventDate'].replace('Z', '+00:00')) # 解析UTC时间
beijing_tz = timezone(timedelta(hours=8)) # 创建北京时区UTC+8
beijing_time = utc_time.astimezone(beijing_tz) # 将UTC时间转换为北京时间
return 3, beijing_time.strftime("%Y-%m-%d %H:%M:%S") # 返回已注册状态和过期时间
elif response.status_code == 404: # HTTP 404表示域名不存在可注册
# 可注册状态
return 2, '' # 返回可注册状态,过期时间为空
logger.error(response.status_code)
except requests.exceptions.RequestException as e:
# logger.warning(f"注册状态检测第{retry+1}次失败: {domain}, 错误: {e}")
if retry < max_retries - 1:
import time
time.sleep(2) # 等待2秒后重试
proxies=None
else:
# 达到最大重试次数,返回检测失败
logger.error(f"注册状态检测失败: {domain}, 已达到最大重试次数")
return -1, '' # -1表示检测失败过期时间为空
# def get_proxy():
#
# url='http://api.ch12361.com/getProxy.php?group=A&count=1'
# response = requests.get(url).json()
# username=response['username']
# password=response['password']
# ip=response['ip']
# port=response['port']
#
# proxy_url = f"http://{username}:{password}@{ip}:{port}"
# logger.info(proxy_url)
# return {'http':proxy_url,'https':proxy_url}
#
# t=datetime.now()
# logger.info(check_register('75az.com',proxies=get_proxy())) # 测试查询nnsk.com域名的注册状态
# logger.info(datetime.now()-t)

15108
domainCheck/detect/sdk_leg.js Normal file

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,897 @@
process_ = process;
require_ = require;
delete Buffer;
// delete process;
delete require;
delete global;
delete module;
delete exports;
delete __filename;
delete __dirname;
delete SharedArrayBuffer;
AsObj = {
// print: console.log,
print: function () { },
// print_:console.log,
}
no_print = ['Boolean','String','parseFloat','Array','Object','prepareStackTrace_'];
function watch(object, WatchName) {
const handler = {
get(target, property, receiver) {
if (
property !== 'isNaN' &&
property !== 'encodeURI' &&
property !== "Uint8Array" &&
property !== 'undefined' &&
property !== 'JSON' &&
property !== 'Number' &&
!no_print.includes(property) &&
property !== Symbol.for('nodejs.util.inspect.custom') &&
typeof property !== 'symbol'
) {
if (property === 'global') {
return undefined;
}
if (property === 'Buffer') {
return undefined;
}
if (property === 'process') {
return undefined;
}
if (WatchName === 'config_data') {
debugger
}
if (WatchName.indexOf('.prototype') != -1 && target[property] != undefined) {
return Reflect.get(target, property, receiver);
}
AsObj.print(
"方法:", "get",
"对象:", WatchName,
"属性:", property,
"属性类型:", typeof property,
"属性值:", typeof target[property] == 'object' ? "object" : target[property],
"属性值类型:", typeof target[property]
);
}
if (WatchName === 'top') {
return window;
}
return Reflect.get(target, property, receiver);
},
set(target, property, value, receiver) {
if (WatchName.indexOf('.prototype') != -1 && value != undefined) {
return Reflect.set(target, property, value, receiver);
}
AsObj.print(
"方法:", "set",
"对象:", WatchName,
"属性:", property,
"属性类型:", typeof property,
"属性值:", typeof value == 'object' ? "object" : value,
"属性值类型:", typeof target[property]
);
return Reflect.set(target, property, value, receiver);
},
// in操作 检测
has(target, property) {
AsObj.print(
"代理对象:", WatchName,
"方法:", "has",
"检查属性:", property,
"结果:", typeof target[property] == 'object' ? "object" : target[property],
);
return Reflect.has(target, property);
},
// Object.key 检测
ownKeys(target) {
AsObj.print(
"方法:", "ownKeys",
"对象:", target+''
);
return Reflect.ownKeys(target);
}
};
return new Proxy(object, handler);
}
// function watch(object, WatchName) {
// return object
// }
// 保护函数toString检测
const safeFunction = function safeFunction(func) {
//处理安全函数
Function.prototype.$call = Function.prototype.call;
const $toString = Function.toString;
const myFunction_toString_symbol = Symbol('('.concat('', ')'));
const myToString = function myToString() {
return typeof this === 'function' && this[myFunction_toString_symbol] || $toString.$call(this);
}
const set_native = function set_native(func, key, value) {
Object.defineProperty(func, key, {
"enumerable": false,
"configurable": true,
"writable": true,
"value": value
});
}
delete Function.prototype['toString'];
set_native(Function.prototype, "toString", myToString);
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
const safe_Function = function safe_Function(func) {
set_native(func, myFunction_toString_symbol, "function" + (func.name ? " " + func.name : "") + "() { [native code] }");
}
return safe_Function(func)
}
//创建函数,并代理上
const makeFunction = function makeFunction(name) {
v_log = AsObj.print;
// 使用 Function 保留函数名
func = new Function("v_log", `
return function ${name}() {
v_log('函数${name}传参-->', arguments);
};
`)(v_log); // 传递 v_log 到动态函数
safeFunction(func);
func = watch(func,`${name}`);
func.prototype = watch(func.prototype, `${name}.prototype`);
return func;
}
!(function () {
"use strict";
const $toString = Function.toString;
const myFunction_toString_symbol = Symbol('('.concat('', ')_', (Math.random() + '').toString(36)));
const mytoString = function () {
return typeof this == 'function' && this[myFunction_toString_symbol] || $toString.call(this);
};
function set_native(func, key, value) {
Object.defineProperty(func, key, {
"enumerable": false,
"configurable": true,
"writable": true,
"value": value
})
};
delete Function.prototype['toString'];
set_native(Function.prototype, "toString", mytoString);
set_native(Function.prototype.toString, myFunction_toString_symbol, "function toString() { [native code] }");
this.func_set_native = function (func) {
set_native(func, myFunction_toString_symbol, `function ${myFunction_toString_symbol, func.name || ''}() { [native code] }`)
}
}).call(globalThis);
// 重写全局对象原型链
function setTostringAndstringTag(obj) {
Object.defineProperties(obj.prototype, {
[Symbol.toStringTag]: {
configurable: true,
value: obj.name
}
});
safeFunction(obj);
};
// 创建标签原型
function createTagProto(propObj,portotypeObj) {
let res = propObj + ' = ' + 'function ' + propObj + '() { throw new TypeError("Illegal constructor"); };\n';
res += 'setTostringAndstringTag(' + propObj + ',null);\n';
if (portotypeObj) {
for (let key in portotypeObj) {
res += propObj + '.prototype.' + portotypeObj[key] + '= function ' + portotypeObj[key] + '() {AsObj.print("'+propObj+'.prototype.' + portotypeObj[key] + '原型方法(需在实例对象上补该方法)::",arguments)};\n';
res += 'globalThis.func_set_native(' + propObj + '.prototype.' + portotypeObj[key] + ');\n';
}
}
eval(res);
}
Object.defineProperties(globalThis, {
[Symbol.toStringTag]: {
configurable: true,
value: 'Window'
}
});
for (let key in globalThis) {
if (typeof globalThis[key] === 'function') {
safeFunction(globalThis[key])
}
}
for (let key in console) {
if (typeof console[key] === 'function') {
safeFunction(console[key])
}
}
createTagProto('EventTarget',['addEventListener']);
createTagProto('WindowProperties');
createTagProto('Window');
window = globalThis;
window.__proto__ = Window.prototype;
window.__proto__.__proto__ = WindowProperties.prototype;
window.__proto__.__proto__.__proto__ = EventTarget.prototype;
Window.__proto__ = EventTarget;
Object.defineProperty(window, 'WindowProperties', {
get: function () {
return undefined;
}
})
function randoms(min, max) {
return Math.floor(Math.random() * (max - min + 1) + min)
}
function getRandomValues(buf) {
var min = 0,
max = 255;
if (buf instanceof Uint16Array) {
max = 65535;
} else if (buf instanceof Uint32Array) {
max = 4294967295;
}
for (var element in buf) {
buf[element] = randoms(min, max);
}
return buf;
}
self = window.self = window;
frames = window.frames = window;
top = window.top = window;
parent = window.parent = window;
global = window.global = window;
Object.defineProperty(window, "global", {
configurable:false,
enumerable: true,
set: undefined,
get: function global(){
return window
}
})
Object.defineProperty(window, "top", {
configurable:false,
enumerable: true,
set: undefined,
get: function top(){
return window
}
})
Object.defineProperty(window, "self", {
configurable:false,
enumerable: true,
set: undefined,
get: function self(){
return window
}
})
Object.defineProperty(window, "parent", {
configurable:false,
enumerable: true,
set: undefined,
get: function parent(){
return window
}
})
Object.defineProperty(window, "frames", {
configurable:false,
enumerable: true,
set: undefined,
get: function frames(){
return window
}
})
innerWidth = 1536
innerHeight = 715
outerWidth = 1536
outerHeight = 824
devicePixelRatio = 1.25;
screenLeft = 0;
screenX = 0;
screenTop = 0;
screenY = 0;
opener = null;
isSecureContext = true;
crypto = {
getRandomValues:getRandomValues
};
createTagProto('DOMStringMap')
createTagProto('HTMLHeadElement',['insertBefore','removeChild'])
createTagProto('HTMLBodyElement',['addEventListener','appendChild','removeChild'])
createTagProto('HTMLHtmlElement',['getAttribute'])
createTagProto('HTMLDocument')
createTagProto('Document',['browsingTopics','appendChild','querySelector','evaluate','querySelectorAll','removeChild','requestStorageAccess','requestStorageAccessFor','hasStorageAccess','getElementsByTagName','hasPrivateToken','createElement','hasRedemptionRecord','hasFocus'])
createTagProto('Node')
document = {};
document.__proto__ = HTMLDocument.prototype;
document.__proto__.__proto__ = Document.prototype;
document.__proto__.__proto__.__proto__ = Node.prototype;
document.__proto__.__proto__.__proto__.__proto__ = EventTarget.prototype;
HTMLDocument.__proto__ = Document;
HTMLDocument.__proto__.__proto__ = Node;
HTMLDocument.__proto__.__proto__.__proto__ = EventTarget;
Document.__proto__ = Node;
Document.__proto__.__proto__ = EventTarget;
Node.__proto__ = EventTarget;
createTagProto('Plugin');
createTagProto('PluginArray');
plugins0 = {
name: 'PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type: 'application/pdf',
},
'1':{
type:'text/pdf'
}
}
plugins0['0'].enabledPlugin = plugins0;
plugins0['1'].enabledPlugin = plugins0;
plugins1 = {
name: 'Chrome PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type:'application/pdf'
},
'1': {
type:'text/pdf'
}
}
plugins1['0'].enabledPlugin = plugins1;
plugins1['1'].enabledPlugin = plugins1;
plugins2 = {
name: 'Chromium PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type:'application/pdf'
},
'1': {
type:'text/pdf'
}
}
plugins2['0'].enabledPlugin = plugins2;
plugins2['1'].enabledPlugin = plugins2;
plugins3 = {
name: 'Microsoft Edge PDF Viewer',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0':{
type:'application/pdf'
},
'1': {
type:'text/pdf'
}
}
plugins3['0'].enabledPlugin = plugins3;
plugins3['1'].enabledPlugin = plugins3;
plugins4 = {
name: 'WebKit built-in PDF',
filename: 'internal-pdf-viewer',
description:'Portable Document Format',
length: 2,
'0': {
type:'application/pdf'
},
'1':{
type:'text/pdf'
}
}
plugins4['0'].enabledPlugin = plugins4;
plugins4['1'].enabledPlugin = plugins4;
plugins = {
length: 5,
'0': plugins0,
'1': plugins1,
'2': plugins2,
'3': plugins3,
'4': plugins4,
namedItem : function (name) {
AsObj.print('Plugin-namedItem:', name)
},
item: function (index) {
AsObj.print('Plugin-item:', index)
return watch(plugins0,'item-'+index);
},
refresh: function () {
AsObj.print('Plugin-refresh:',arguments)
},
}
plugins.__proto__ = PluginArray.prototype;
MimeTypeArray = function MimeTypeArray() {
this.length = 2;
this['0'] = {
suffixes: 'pdf',
type: 'application/pdf',
description:"Portable Document Format",
enabledPlugin: plugins0
};
this['1'] = {
suffixes: 'pdf',
type: 'text/pdf',
description:"Portable Document Format",
enabledPlugin: plugins0
};
};
MimeTypeArray.prototype.toString = function () { return '[object MimeTypeArray]'; }
MimeTypeArray.toString = function () { return 'function MimeTypeArray() { [native code] }'; }
Object.defineProperties(MimeTypeArray.prototype, { [Symbol.toStringTag]: { value: 'MimeTypeArray' } })
MimeTypeArrayc = new MimeTypeArray();
MimeTypeArrayc[Symbol.iterator] = function* () {
for (let key in this) {
yield this[key];
}
}
// 创建电池管理器对象原型
const BatteryManager = {
level: 1,
charging: true,
chargingTime: 0,
dischargingTime: null,
onchargingchange: null,
onlevelchange: null,
toString: function toString() {
return `BatteryManager {
charging: ${this.charging},
level: ${this.level},
chargingTime: ${this.chargingTime},
dischargingTime: ${this.dischargingTime}
}`
}
}
window.BatteryManager = BatteryManager;
Promise2 = {
then: function () {
return this;
},
catch: function (){},
};
createTagProto('Bluetooth');
createTagProto('Navigator');
Navigator.prototype.hardwareConcurrency = 8;
Navigator.prototype.userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36';
Navigator.prototype.appVersion = '5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36'
Navigator.prototype.appName = 'Netscape';
Navigator.prototype.appCodeName = 'Mozilla';
Navigator.prototype.vendor = 'Google Inc.';
Navigator.prototype.maxTouchPoints = 10;
Navigator.prototype.platform = 'Win32';
Navigator.prototype.adAuctionComponents = function adAuctionComponents() {
AsObj.print('adAuctionComponents:::', arguments)
}
safeFunction(Navigator.prototype.adAuctionComponents)
Navigator.prototype.runAdAuction = function runAdAuction() {
AsObj.print('runAdAuction:::', arguments)
}
safeFunction(Navigator.prototype.runAdAuction)
Navigator.prototype.canLoadAdAuctionFencedFrame = makeFunction('canLoadAdAuctionFencedFrame')
Navigator.prototype.deprecatedReplaceInURN = makeFunction('deprecatedReplaceInURN')
Navigator.prototype.deprecatedURNToURL = makeFunction('deprecatedURNToURL')
Navigator.prototype.joinAdInterestGroup = makeFunction('joinAdInterestGroup')
Navigator.prototype.leaveAdInterestGroup = makeFunction('leaveAdInterestGroup')
Navigator.prototype.updateAdInterestGroups = makeFunction('updateAdInterestGroups')
Navigator.prototype.connection = watch({
downlink: 9.1,
effectiveType: '4g',
rtt: 0,
saveData: false,
},'connection')
Navigator.prototype.language = 'zh-CN';
Navigator.prototype.languages = ["zh-CN"];
Navigator.prototype.plugins = plugins;
Navigator.prototype.webdriver = false;
Navigator.prototype.cookieEnabled = true;
Navigator.prototype.onLine = true;
Navigator.prototype.doNotTrack = null;
Navigator.prototype.bluetooth = {};
Navigator.prototype.product = 'Gecko'
Navigator.prototype.deviceMemory = 8
Navigator.prototype.mediaDevices = watch({
enumerateDevices: function enumerateDevices() {
return new Promise((resolve, reject) => {
const offer = [
{deviceId: '', kind: 'audioinput', label: '', groupId: ''},
{deviceId: '', kind: 'videoinput', label: '', groupId: ''},
{deviceId: '', kind: 'audiooutput', label: '', groupId: ''},
]
resolve(offer);
});
},
getUserMedia: function getUserMedia() {
AsObj.print('getUserMedia:::', arguments)
}
},'mediaDevices')
Navigator.prototype.storage = {
estimate: function estimate() {
AsObj.print('estimate:::', arguments)
return new Promise((resolve, reject) => {
const offer = {
usage: 0, // 1GB
quota: 2147483648, // 1GB,
usageDetails: {caches: 512, indexedDB: 2855}
};
resolve(offer);
});
}
}
Navigator.prototype.webkitPersistentStorage = watch({},'webkitPersistentStorage')
Navigator.prototype.webkitTemporaryStorage = watch({
queryUsageAndQuota: function queryUsageAndQuota() {
AsObj.print('queryUsageAndQuota:::', arguments)
return new Promise((resolve, reject) => {
const offer = {
usage: 1024 * 1024 * 1024, // 1GB
quota: 1024 * 1024 * 1024, // 1GB
};
resolve(offer);
});
}
},'webkitTemporaryStorage')
Navigator.prototype.bluetooth.__proto__ = Bluetooth.prototype;
Navigator.prototype.javaEnabled = function javaEnabled() {
return false
};
safeFunction(Navigator.prototype.javaEnabled)
Navigator.prototype.getBattery = function getBattery() {
AsObj.print('getBattery:::', arguments)
return Promise.resolve({
__proto__: BatteryManager,
// 动态参数配置(示例值)
level: 1,
charging: true,
dischargingTime: null // 2小时放电时间
})
}
safeFunction(Navigator.prototype.getBattery)
Navigator.prototype.registerProtocolHandler = function registerProtocolHandler() {
AsObj.print('registerProtocolHandler:::',arguments)
}
safeFunction(Navigator.prototype.registerProtocolHandler)
Navigator.prototype.mimeTypes = watch(MimeTypeArrayc,'mimeTypes');
Navigator.prototype.geolocation = {
getCurrentPosition: function getCurrentPosition() {
return Promise2;
}
}
Navigator.prototype.pdfViewerEnabled = true;
Navigator.prototype.doNotTrack = null;
Navigator.prototype.keyboard = watch({
getLayoutMap: function getLayoutMap() {
AsObj.print('Navigator.prototype.keyboard:', arguments)
return {
then: function () {
// arguments[0](watch({
// size: 48,
// values: function () {
// return ['k', 'g', '2', '0', 'v', 'a', '`', 'l', '\\', "'", 'w', '8', 'm', 'h', '.', '7', '1', 'p', 'd', 'f', 'o', 'q', 'c', 'n', '[', 'z', 'y', '3', '6', '5', 'x', '/', '\\', ',', '-', '4', 'b', 't', '9', 's', 'i', 'u', '=', 'j', ';', 'r', ']', 'e']
// }
// }, 'navigator.keyboard.getLayoutMap.then'))
return {
catch: function () {
arguments[0]({
message:'getLayoutMap() must be called from a top-level browsing context or allowed by the permission policy.'
})
}
}
}
}
}
},'navigator.keyboard')
Navigator.prototype.permissions = watch({
query: function query() {
arg_obj = arguments[0];
if (arg_obj.name === 'audio_capture') {
return new Promise((resolve, reject) => {
const offer = {
state: 'prompt',
onchange: null,
name:arg_obj.name
};
resolve(offer);
});
}
if (arg_obj.name === 'microphone') {
return {
then: function () {
arguments[0](watch({
state: 'denied',
onchange: null,
name: 'audio_capture'
},'permissions.query.microphone'));
return {catch:function(){}}
},
catch:function(){}
}
}
if (arg_obj.name === 'camera') {
return {
then: function () {
arguments[0](watch({
state: 'prompt',
onchange: null,
name: 'video_capture'
},'permissions.query.camera'));
return {catch:function(){}}
},
catch:function(){}
}
}
AsObj.print('permissions.query:::', arguments)
}
},'permissions')
Navigator.prototype.productSub = '20030107'
Navigator.prototype.getGamepads = function getGamepads() {
AsObj.print('getGamepads:::', arguments)
return [null,null,null,null]
}
safeFunction(Navigator.prototype.getGamepads)
Navigator.prototype.sendBeacon = makeFunction('sendBeacon')
Navigator.prototype.deprecatedRunAdAuctionEnforcesKAnonymity = false
Navigator.prototype.gpu = watch({
getPreferredCanvasFormat: function getPreferredCanvasFormat() {
AsObj.print('gpu.getPreferredCanvasFormat:', arguments)
return 'bgra8unorm'
},
wgslLanguageFeatures: watch({
size: 7,
values: function values() {
debugger
AsObj.print('wgslLanguageFeatures.values')
return ['packed_4x8_integer_dot_product', 'unrestricted_pointer_parameters', 'subgroup_uniformity', 'subgroup_id', 'pointer_composite_access', 'readonly_and_readwrite_storage_textures', 'uniform_buffer_standard_layout']
},
}, 'gpu.wgslLanguageFeatures'),
requestAdapter: function requestAdapter() {
AsObj.print('gpu.requestAdapter:', arguments)
return {
then: function () {
arguments[0](watch({
features: watch({
size: 19,
values: function () {
return ['depth32float-stencil8', 'rg11b10ufloat-renderable', 'bgra8unorm-storage', 'texture-formats-tier1', 'texture-compression-bc', 'dual-source-blending', 'core-features-and-limits', 'float32-filterable', 'indirect-first-instance', 'float32-blendable', 'depth-clip-control', 'texture-compression-bc-sliced-3d', 'timestamp-query', 'texture-formats-tier2', 'clip-distances', 'shader-f16', 'primitive-index', 'texture-component-swizzle', 'subgroups']
}
}, 'gpu.requestAdapter.features'),
info: watch({ vendor: 'intel', architecture: 'gen-11', device: '', description: '', subgroupMinSize: 16 }, 'gpu.requestAdapter.info'),
limits: watch({
maxBufferSize: 2147483648,
maxStorageBufferBindingSize:2147483644
}, 'gpu.requestAdapter.limits'),
catch:function(){}
}, 'gpu.requestAdapter'));
return {
catch: function () {
return {
then: function () {
arguments[0]()
return {catch:function(){}}
}
}
}
};
},
catch: function () {
}
}
}
},'navigator.gpu')
Navigator.prototype.userAgentData = watch({
brands:[
{
"brand": "Google Chrome",
"version": "143"
},
{
"brand": "Chromium",
"version": "143"
},
{
"brand": "Not A(Brand",
"version": "24"
}
],
mobile: false,
platform: "Windows",
getHighEntropyValues: function getHighEntropyValues() {
if (arguments[0] + '' === 'architecture,bitness,model,platformVersion,uaFullVersion,wow64') {
return new Promise((resolve, reject) => {
const offer = {
"architecture": "x86",
"bitness": "64",
"brands": [
{
"brand": "Not:A-Brand",
"version": "99"
},
{
"brand": "Google Chrome",
"version": "145"
},
{
"brand": "Chromium",
"version": "145"
}
],
"mobile": false,
"model": "",
"platform": "Windows",
"platformVersion": "10.0.0",
"uaFullVersion": "145.0.7632.117",
"wow64": false
};
resolve(offer);
});
}
AsObj.print('getHighEntropyValues:::', arguments)
}
},'userAgentData')
navigator = {};
navigator.__proto__ = Navigator.prototype;
createTagProto('Location');
location = {
"ancestorOrigins": {},
"href": "https://www.neimanmarcus.com/",
"origin": "https://www.neimanmarcus.com",
"protocol": "https:",
"host": "www.neimanmarcus.com",
"hostname": "www.neimanmarcus.com",
"port": "",
"pathname": "/",
"search": "",
"hash": ""
};
location.__proto__ = Location.prototype;
location.toString = function toString() {
return this.href;
}
createTagProto('Screen');
Screen.prototype = Object.assign(Screen.prototype, {
availHeight: 824,
availLeft: 0,
availTop: 0,
availWidth: 1536,
colorDepth: 32,
height: 864,
isExtended: true,
onchange: null,
pixelDepth: 24,
width: 1536,
orientation: {
angle: 0,
type: "landscape-primary",
onchange: null
}
})
screen = {};
screen.__proto__ = Screen.prototype;
createTagProto('History',['replaceState']);
history = {};
history.__proto__ = History.prototype;
chrome = {
loadTimes: function loadTimes() { },
csi: function csi() { },
app: {
InstallState: { DISABLED: 'disabled', INSTALLED: 'installed', NOT_INSTALLED: 'not_installed' },
RunningState: { CANNOT_RUN: 'cannot_run', READY_TO_RUN: 'ready_to_run', RUNNING: 'running' },
getDetails:function getDetails(){},
getIsInstalled:function getIsInstalled(){},
installState:function installState(){},
isInstalled: false,
runningState: function runningState(){}
},
}
createTagProto('Storage');
local = {
};
localStorage = {
getItem: function getItem(key) {
AsObj.print("localStorage.getItem::", arguments);
if (!local[key]) {
return null;
}
return local[key];
},
setItem: function setItem(key, value) {
AsObj.print("localStorage.setItem::", arguments);
local[key] = value;
},
clear: function clear() {
local = {};
},
removeItem: function removeItem(key) {
AsObj.print("localStorage.removeItem::", arguments);
delete local[key];
}
}
localStorage.__proto__ = Storage.prototype;
sessionStorage = {
getItem: function getItem(key) {
AsObj.print("sessionStorage.getItem::", arguments);
if (!local[key]) {
return null;
}
return local[key];
},
setItem: function setItem(key, value) {
AsObj.print("sessionStorage.setItem::", arguments);
local[key] = value;
},
clear: function clear() {
local = {};
},
removeItem: function removeItem(key) {
AsObj.print("sessionStorage.removeItem::", arguments);
delete local[key];
}
}
sessionStorage.__proto__ = Storage.prototype;
// window = watch(window, 'window');
// global = watch(global, 'global');
// globalThis = watch(globalThis, 'globalThis');
// self = watch(self, 'self');
// crypto = watch(crypto, 'crypto');
// performance = watch(performance, 'performance');
// document = watch(document, 'document');
// navigator = watch(navigator, 'navigator');
// location = watch(location, 'location');
// screen = watch(screen, 'screen');
// history = watch(history, 'history');
// localStorage = watch(localStorage, 'localStorage');
// sessionStorage = watch(sessionStorage, 'sessionStorage');
// chrome = watch(chrome, 'chrome');
require_('./sdk_leg.js');
let input = '';
// 收集数据
process.stdin.on('data', chunk => {
input += chunk;
});
process.stdin.on('end', async () => {
var config_data = JSON.parse(input);
var cryptoManager = await CaptchaSDKCorecc();
var encryptData = await buildEncryptedVerifyRequestcc(config_data, cryptoManager);
console.log(JSON.stringify(encryptData));
process.exit(0);
})

View File

@@ -0,0 +1,20 @@
{
"detect_juziseo": false,
"detect_360_site": true,
"detect_aizhan": true,
"detect_baidu_site": true,
"detect_order": [
"detect_register",
"detect_baidu_site",
"detect_360_site",
"detect_chinaz",
"detect_aizhan",
"detect_wayback",
"detect_jucha",
"detect_juziseo"
],
"detect_register": true,
"detect_wayback": true,
"detect_chinaz": true,
"detect_jucha": false
}

2181
domainCheck/detect_worker.py Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,3 @@
{
"suffixes": ".com,.net"
}

194702
domainCheck/domains.txt Normal file

File diff suppressed because it is too large Load Diff

BIN
domainCheck/favicon.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

BIN
domainCheck/favicon2.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

View File

@@ -0,0 +1,270 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :init_database.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/9 21:20
@explain : 数据库初始化脚本
'''
import psycopg2
from loguru import logger
from app.config import config
def init_database():
"""
初始化数据库表结构
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
logger.info("数据库连接成功")
# 创建domains表
create_domains_table = """
CREATE TABLE IF NOT EXISTS domains (
id SERIAL PRIMARY KEY,
domain VARCHAR(255) UNIQUE NOT NULL,
tld VARCHAR(50) NOT NULL,
source_type INTEGER NOT NULL,
use_status INTEGER DEFAULT 0,
detect_status INTEGER DEFAULT 0,
register_status INTEGER DEFAULT 0,
has_beian BOOLEAN DEFAULT FALSE,
company_type VARCHAR(100),
website_url VARCHAR(255),
beian_year INTEGER,
snapshot_years TEXT,
expire_date TIMESTAMP,
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cur.execute(create_domains_table)
logger.info("创建domains表成功")
# 添加新字段
try:
# 添加review_status字段
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS review_status INTEGER DEFAULT 0")
# 添加detect_time字段
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS detect_time TIMESTAMP")
# 添加backlink_count字段
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0")
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS jucha_status INTEGER DEFAULT 0")
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS juziseo_status INTEGER DEFAULT 0")
# 尝试修改has_beian字段类型
try:
# 先检查字段类型
cur.execute("SELECT data_type FROM information_schema.columns WHERE table_name = 'domains' AND column_name = 'has_beian'")
result = cur.fetchone()
if result and result[0] != 'integer':
# 先删除默认值
cur.execute("ALTER TABLE domains ALTER COLUMN has_beian DROP DEFAULT")
# 修改字段类型
cur.execute("ALTER TABLE domains ALTER COLUMN has_beian TYPE INTEGER USING CASE WHEN has_beian THEN 2 ELSE 3 END")
# 设置新的默认值
cur.execute("ALTER TABLE domains ALTER COLUMN has_beian SET DEFAULT 1")
except Exception as e:
logger.warning(f"修改has_beian字段类型失败: {e}")
logger.info("添加新字段成功")
except Exception as e:
logger.warning(f"添加新字段失败: {e}")
# 回滚当前事务
conn.rollback()
# 只添加新字段不修改has_beian字段类型
try:
# 添加review_status字段
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS review_status INTEGER DEFAULT 0")
# 添加detect_time字段
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS detect_time TIMESTAMP")
# 添加backlink_count字段
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS backlink_count INTEGER DEFAULT 0")
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS jucha_status INTEGER DEFAULT 0")
cur.execute("ALTER TABLE domains ADD COLUMN IF NOT EXISTS juziseo_status INTEGER DEFAULT 0")
logger.info("添加部分新字段成功")
except Exception as e2:
logger.warning(f"添加新字段失败: {e2}")
# 添加字段注释
cur.execute("COMMENT ON COLUMN domains.id IS '主键ID'")
cur.execute("COMMENT ON COLUMN domains.domain IS '域名'")
cur.execute("COMMENT ON COLUMN domains.tld IS '顶级域名'")
cur.execute("COMMENT ON COLUMN domains.source_type IS '来源类型'")
cur.execute("COMMENT ON COLUMN domains.use_status IS '使用状态'")
cur.execute("COMMENT ON COLUMN domains.detect_status IS '检测状态'")
cur.execute("COMMENT ON COLUMN domains.register_status IS '注册状态'")
cur.execute("COMMENT ON COLUMN domains.review_status IS '人工复核状态'")
cur.execute("COMMENT ON COLUMN domains.has_beian IS '是否有备案历史'")
cur.execute("COMMENT ON COLUMN domains.company_type IS '单位性质'")
cur.execute("COMMENT ON COLUMN domains.website_url IS '网站首页网址'")
cur.execute("COMMENT ON COLUMN domains.beian_year IS '备案年份'")
cur.execute("COMMENT ON COLUMN domains.snapshot_years IS '快照年份'")
cur.execute("COMMENT ON COLUMN domains.expire_date IS '过期时间'")
cur.execute("COMMENT ON COLUMN domains.detect_time IS '检测时间'")
cur.execute("COMMENT ON COLUMN domains.jucha_status IS '聚查检测状态0未检测1已检测'")
cur.execute("COMMENT ON COLUMN domains.juziseo_status IS '桔子检测状态0未检测1已检测'")
cur.execute("COMMENT ON COLUMN domains.create_time IS '创建时间'")
cur.execute("COMMENT ON COLUMN domains.update_time IS '更新时间'")
logger.info("添加domains表字段注释成功")
# 创建detect_tasks表
create_tasks_table = """
CREATE TABLE IF NOT EXISTS detect_tasks (
id SERIAL PRIMARY KEY,
domain_id INTEGER REFERENCES domains(id),
task_type INTEGER NOT NULL,
status INTEGER DEFAULT 0,
priority INTEGER DEFAULT 0,
retry_count INTEGER DEFAULT 0,
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cur.execute(create_tasks_table)
logger.info("创建detect_tasks表成功")
# 添加字段注释
cur.execute("COMMENT ON COLUMN detect_tasks.id IS '主键ID'")
cur.execute("COMMENT ON COLUMN detect_tasks.domain_id IS '域名ID'")
cur.execute("COMMENT ON COLUMN detect_tasks.task_type IS '任务类型'")
cur.execute("COMMENT ON COLUMN detect_tasks.status IS '任务状态'")
cur.execute("COMMENT ON COLUMN detect_tasks.priority IS '优先级'")
cur.execute("COMMENT ON COLUMN detect_tasks.retry_count IS '重试次数'")
cur.execute("COMMENT ON COLUMN detect_tasks.create_time IS '创建时间'")
cur.execute("COMMENT ON COLUMN detect_tasks.update_time IS '更新时间'")
logger.info("添加detect_tasks表字段注释成功")
# 创建domain_blacklist表
create_blacklist_table = """
CREATE TABLE IF NOT EXISTS domain_blacklist (
id SERIAL PRIMARY KEY,
domain VARCHAR(255) UNIQUE NOT NULL,
reason TEXT,
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cur.execute(create_blacklist_table)
logger.info("创建domain_blacklist表成功")
# 添加字段注释
cur.execute("COMMENT ON COLUMN domain_blacklist.id IS '主键ID'")
cur.execute("COMMENT ON COLUMN domain_blacklist.domain IS '域名'")
cur.execute("COMMENT ON COLUMN domain_blacklist.reason IS '黑名单原因'")
cur.execute("COMMENT ON COLUMN domain_blacklist.create_time IS '创建时间'")
logger.info("添加domain_blacklist表字段注释成功")
# 创建sensitive_words表
create_sensitive_words_table = """
CREATE TABLE IF NOT EXISTS sensitive_words (
id SERIAL PRIMARY KEY,
word VARCHAR(255) UNIQUE NOT NULL,
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cur.execute(create_sensitive_words_table)
logger.info("创建sensitive_words表成功")
# 添加缺失的字段
try:
# 添加category字段
cur.execute("ALTER TABLE sensitive_words ADD COLUMN IF NOT EXISTS category VARCHAR(50) DEFAULT 'default'")
# 添加priority字段
cur.execute("ALTER TABLE sensitive_words ADD COLUMN IF NOT EXISTS priority INTEGER DEFAULT 1")
logger.info("添加sensitive_words表字段成功")
except Exception as e:
logger.warning(f"添加sensitive_words表字段失败: {e}")
# 添加字段注释
try:
cur.execute("COMMENT ON COLUMN sensitive_words.id IS '主键ID'")
cur.execute("COMMENT ON COLUMN sensitive_words.word IS '敏感词'")
cur.execute("COMMENT ON COLUMN sensitive_words.category IS '分类'")
cur.execute("COMMENT ON COLUMN sensitive_words.priority IS '优先级'")
cur.execute("COMMENT ON COLUMN sensitive_words.create_time IS '创建时间'")
logger.info("添加sensitive_words表字段注释成功")
except Exception as e:
logger.warning(f"添加sensitive_words表字段注释失败: {e}")
# 创建domain_detections表
create_detections_table = """
CREATE TABLE IF NOT EXISTS domain_detections (
id SERIAL PRIMARY KEY,
domain_id INTEGER REFERENCES domains(id),
baidu_history JSONB,
baidu_site JSONB,
qihu360_site JSONB,
google_site JSONB,
chinaz_info JSONB,
aizhan_info JSONB,
juziseo_info JSONB,
jucha_info JSONB,
same_url BOOLEAN,
is_chinese_title BOOLEAN,
backlink_count_gt_10 BOOLEAN,
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
update_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cur.execute(create_detections_table)
logger.info("创建domain_detections表成功")
cur.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_domain_detections_domain_id ON domain_detections(domain_id)")
# 为已存在的domain_detections表添加新字段
try:
# 添加same_url字段
cur.execute("ALTER TABLE domain_detections ADD COLUMN IF NOT EXISTS same_url BOOLEAN")
# 添加is_chinese_title字段
cur.execute("ALTER TABLE domain_detections ADD COLUMN IF NOT EXISTS is_chinese_title BOOLEAN")
# 添加backlink_count_gt_10字段
cur.execute("ALTER TABLE domain_detections ADD COLUMN IF NOT EXISTS backlink_count_gt_10 BOOLEAN")
logger.info("为domain_detections表添加新字段成功")
except Exception as e:
logger.warning(f"为domain_detections表添加新字段失败: {e}")
# 回滚当前事务
conn.rollback()
# 添加字段注释
cur.execute("COMMENT ON COLUMN domain_detections.id IS '主键ID'")
cur.execute("COMMENT ON COLUMN domain_detections.domain_id IS '域名ID'")
cur.execute("COMMENT ON COLUMN domain_detections.baidu_history IS '百度历史'")
cur.execute("COMMENT ON COLUMN domain_detections.baidu_site IS '百度site'")
cur.execute("COMMENT ON COLUMN domain_detections.qihu360_site IS '360 site'")
cur.execute("COMMENT ON COLUMN domain_detections.google_site IS 'Google site'")
cur.execute("COMMENT ON COLUMN domain_detections.chinaz_info IS '站长之家信息'")
cur.execute("COMMENT ON COLUMN domain_detections.aizhan_info IS '爱站网信息'")
cur.execute("COMMENT ON COLUMN domain_detections.juziseo_info IS '桔子SEO信息'")
cur.execute("COMMENT ON COLUMN domain_detections.jucha_info IS '聚查信息'")
cur.execute("COMMENT ON COLUMN domain_detections.same_url IS '首网址和备案网址是否一样'")
cur.execute("COMMENT ON COLUMN domain_detections.is_chinese_title IS 'title是否简体中文'")
cur.execute("COMMENT ON COLUMN domain_detections.backlink_count_gt_10 IS '快照历史友链数量是否大于10'")
cur.execute("COMMENT ON COLUMN domain_detections.create_time IS '创建时间'")
cur.execute("COMMENT ON COLUMN domain_detections.update_time IS '更新时间'")
logger.info("添加domain_detections表字段注释成功")
# 提交事务
conn.commit()
logger.info("数据库初始化完成")
except Exception as e:
logger.error(f"数据库初始化失败: {e}")
if conn:
conn.rollback()
finally:
if cur:
cur.close()
if conn:
conn.close()
if __name__ == "__main__":
init_database()

Binary file not shown.

Binary file not shown.

Binary file not shown.

1
domainCheck/logo.svg Normal file

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 12 KiB

1
domainCheck/new_logo.svg Normal file
View File

@@ -0,0 +1 @@
<svg t="1776041645024" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="2611" width="200" height="200"><path d="M462.9 858.3c-96 0-192.1-35.4-266.9-106.6C41.5 604.5 35.5 359 182.7 204.5 254 129.6 350.2 87 453.5 84.5c103.3-2.4 201.5 35.4 276.3 106.7 74.9 71.3 117.5 167.5 120 270.8s-35.4 201.5-106.7 276.3c-75.8 79.8-177.9 120-280.2 120z m0.3-730.7c-2.8 0-5.7 0-8.6 0.1-91.8 2.2-177.3 40.1-240.6 106.6C83.2 371.5 88.5 589.6 225.8 720.4S581.2 845.9 712 708.6c63.4-66.5 97-153.7 94.8-245.5-2.2-91.8-40.1-177.3-106.6-240.6-64.5-61.4-148.3-94.9-237-94.9z" fill="#1195FE" p-id="2612"></path><path d="M862.1 928.3c-17.7 0-35.5-6.5-49.3-19.7l-150.1-143 19.3-15.4c12.6-10 24.3-20.7 34.8-31.7 10.5-11 20.6-23.2 30-36.3l14.5-20.1 150.1 143c13.8 13.1 21.6 30.9 22.1 50s-6.5 37.2-19.6 51c-14.1 14.8-33 22.2-51.8 22.2zM727.8 768l114.8 109.4c11.3 10.8 29.2 10.3 40-1s10.3-29.2-1-40L766.8 727c-6.1 7.4-12.3 14.5-18.7 21.2-6.5 6.8-13.2 13.4-20.3 19.8zM268.8 551.8c-9.8-23.6-15.2-49-15.9-75.3-1.4-56.1 19.2-109.5 58-150.1 1.2-1.3 2.4-2.5 3.7-3.8 38.3-38.3 89.1-60.1 143.5-61.4 30.3-0.7 59.2 5 85.5 16l32.6-32.6c-35.8-17.9-76.3-27.6-119.1-26.6-67.7 1.6-130.7 29.3-177.3 78.6-46.7 49-71.5 113.3-69.9 180.9 0.9 37.8 10 74.1 26.4 106.9l32.5-32.6zM657 390.1c9.9 23.5 15.7 49.1 16.3 76.1 2.8 115.9-89.2 212.5-205.1 215.3-30.1 0.7-59.4-4.9-86.4-16.2l-32.6 32.6c36.7 18.5 77.7 27.8 120 26.8 67.6-1.6 128.5-29.8 172.9-74.2 47.3-47.3 76-113.2 74.3-185.3-0.9-38.7-10.6-75.1-26.9-107.6L657 390.1z" fill="#1195FE" p-id="2613"></path><path d="M287.283 615.504l188.401-62.199 6.05 18.327-188.4 62.199zM430.89 361.097l188.4-62.199 6.051 18.327-188.4 62.199z" fill="#1195FE" p-id="2614"></path><path d="M472.411 564.972l-27.36-196.516 19.117-2.661 27.36 196.515z" fill="#1195FE" p-id="2615"></path><path d="M623.8 314m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2616"></path><path d="M623.8 360.8c-25.8 0-46.8-21-46.8-46.8 0-25.8 21-46.8 46.8-46.8 25.8 0 46.8 21 46.8 46.8 0 25.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7s20.7-9.3 20.7-20.7c0-11.4-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2617"></path><path d="M449.7 363.9m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2618"></path><path d="M449.7 410.7c-25.8 0-46.8-21-46.8-46.8 0-25.8 21-46.8 46.8-46.8 25.8 0 46.8 21 46.8 46.8 0 25.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7 11.4 0 20.7-9.3 20.7-20.7 0-11.5-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2619"></path><path d="M302.4 628.7m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2620"></path><path d="M302.4 675.5c-25.8 0-46.8-21-46.8-46.8 0-25.8 21-46.8 46.8-46.8 25.8 0 46.8 21 46.8 46.8 0 25.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7 11.4 0 20.7-9.3 20.7-20.7 0-11.4-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2621"></path><path d="M474.7 560.5m-33.8 0a33.8 33.8 0 1 0 67.6 0 33.8 33.8 0 1 0-67.6 0Z" fill="#FFFFFF" p-id="2622"></path><path d="M474.7 607.3c-25.8 0-46.8-21-46.8-46.8s21-46.8 46.8-46.8 46.8 21 46.8 46.8-21 46.8-46.8 46.8z m0-67.5c-11.4 0-20.7 9.3-20.7 20.7 0 11.4 9.3 20.7 20.7 20.7s20.7-9.3 20.7-20.7c0-11.4-9.3-20.7-20.7-20.7z" fill="#1195FE" p-id="2623"></path></svg>

After

Width:  |  Height:  |  Size: 3.2 KiB

View File

@@ -0,0 +1,406 @@
# 域名库系统设计方案
## 1. 项目结构设计
```
domainScanDemo/
├── app/ # 应用主目录
│ ├── __init__.py
│ ├── main.py # 主入口
│ ├── ui/ # 可视化界面
│ │ ├── __init__.py
│ │ ├── main_window.py # 主窗口
│ │ ├── domain_import.py # 域名导入界面
│ │ ├── domain_filter.py # 域名筛选界面
│ │ └── sensitive_words.py # 敏感词配置界面
│ ├── core/ # 核心功能
│ │ ├── __init__.py
│ │ ├── domain_collector.py # 域名收集器
│ │ ├── domain_processor.py # 域名处理器
│ │ ├── detect_engine.py # 检测引擎
│ │ ├── task_scheduler.py # 任务调度器
│ │ └── export_manager.py # 导出管理器
│ ├── detectors/ # 检测插件
│ │ ├── __init__.py
│ │ ├── base.py # 基础检测类
│ │ ├── rdap_detector.py # RDAP检测
│ │ ├── wayback_detector.py # 时光机检测
│ │ ├── baidu_detector.py # 百度检测
│ │ ├── qihu360_detector.py # 360检测
│ │ ├── google_detector.py # Google检测
│ │ ├── chinaz_detector.py # 站长之家检测
│ │ ├── aizhan_detector.py # 爱站网检测
│ │ ├── juziseo_detector.py # 桔子SEO检测
│ │ └── jucha_detector.py # 聚查检测
│ ├── utils/ # 工具类
│ │ ├── __init__.py
│ │ ├── domain_utils.py # 域名工具
│ │ ├── http_utils.py # HTTP工具
│ │ ├── database.py # 数据库操作
│ │ └── config.py # 配置管理
│ └── config/ # 配置文件
│ ├── __init__.py
│ ├── settings.py # 系统设置
│ └── sensitive_words.txt # 敏感词列表
├── scripts/ # 脚本工具
│ ├── __init__.py
│ ├── domain_import.py # 域名导入脚本
│ ├── detect_runner.py # 检测运行脚本
│ └── stats_report.py # 统计报告脚本
├── tests/ # 测试代码
│ ├── __init__.py
│ ├── test_domain_utils.py # 域名工具测试
│ ├── test_detectors.py # 检测器测试
│ └── test_database.py # 数据库测试
├── requirements.txt # 依赖包
├── README.md # 项目说明
└── .env # 环境变量
```
## 2. 核心功能模块设计
### 2.1 域名收集模块
**功能**:从多个来源收集域名数据
**实现**
- **聚名网爬虫**:抓取一口价和过期删除域名
- **搜索引擎采集**:通过关键词搜索收集域名
- **企业目录抓取**:从企业黄页等网站抓取域名
- **Zone File 解析**:解析域名区域文件
- **TXT 导入**:支持从文本文件导入域名
- **第三方 API**:集成域名数据 API
**技术要点**
- 使用 `Scrapy` 框架进行网站爬取
- 支持代理池,避免 IP 被封禁
- 多线程并行采集,提高效率
- 域名标准化处理(小写、去空格、去协议、只留主域)
### 2.2 域名处理模块
**功能**:处理收集到的域名,进行标准化和去重
**实现**
- 域名标准化:转换为小写,去除空格和协议
- 后缀过滤:只保留 `.com``.net`
- 去重处理:使用布隆过滤器和数据库唯一索引
- 来源标记:记录域名来源和批次信息
- 任务创建:为新域名创建检测任务
**技术要点**
- 使用 Redis 布隆过滤器快速去重
- 批量处理,减少数据库操作
- 事务处理,确保数据一致性
### 2.3 检测引擎模块
**功能**:执行域名检测任务,包括注册状态、风险评估等
**实现**
- **基础检测**
- 注册状态检测RDAP优先其次WHOIS
- 黑名单缓存检查
- 时光机快照年份采集
- 敏感词匹配(快照内容)
- **深度检测**
- 百度历史/Site
- 360 Site
- Google Site
- 站长之家/爱站
- 桔子SEO历史记录
- 聚查WHOIS/备案/拦截)
- 桔子SEO外链查询
**技术要点**
- 插件化设计,支持添加新的检测模块
- 优先级队列,按成本和重要性排序
- 失败重试机制,提高检测成功率
- 并发处理,提高检测速度
### 2.4 任务调度模块
**功能**:管理检测任务队列,分配任务到检测节点
**实现**
- 任务队列管理:使用 RabbitMQ 存储任务
- 任务分配:根据节点负载分配任务
- 状态跟踪:实时监控任务执行状态
- 断点续跑:支持系统重启后继续执行任务
- 失败重试:自动重试失败的任务
**技术要点**
- 使用 Celery 作为任务队列
- 分布式部署,支持多服务器
- 实时监控和告警
- 任务优先级管理
### 2.5 导出管理模块
**功能**:根据运营筛选条件导出域名
**实现**
- 多条件组合筛选
- 预览功能
- TXT 导出
- 批量更新使用状态
- 批量更新人工审核状态
**技术要点**
- 高效的数据库查询优化
- 支持大文件导出
- 导出进度实时显示
- 导出历史记录
### 2.6 可视化界面模块
**功能**:提供用户友好的操作界面
**实现**
- **主窗口**:系统概览和功能导航
- **域名导入**:支持多种方式导入域名
- **域名筛选**:多条件组合筛选
- **敏感词配置**:管理敏感词列表
- **系统设置**:配置系统参数
**技术要点**
- 使用 PySide6 构建界面
- 响应式设计,支持不同屏幕尺寸
- 多线程处理,避免界面卡顿
- 实时数据更新
## 3. 数据库设计
**数据库**PostgreSQL 15+
**核心表结构**
| 表名 | 功能 | 关键字段 |
|------|------|----------|
| `domains` | 域名主表 | domain, tld, use_status, detect_status, register_status, source_type |
| `domain_detections` | 检测结果 | domain_id, baidu_history, baidu_site, is_simplified, qihu360_site, google_site |
| `domain_sources` | 来源信息 | domain_id, source_type, source_url, batch_id |
| `detect_tasks` | 检测任务 | domain_id, task_type, status, priority, retry_count |
| `sensitive_words` | 敏感词 | word, category, level |
| `domain_blacklist` | 黑名单 | domain, reason, blacklist_time, source |
| `system_config` | 系统配置 | config_key, config_value, description |
| `detect_nodes` | 检测节点 | node_name, ip_address, status, last_heartbeat |
| `task_stats` | 任务统计 | date, total_tasks, success_tasks, failed_tasks |
| `domain_stats` | 域名统计 | date, total_domains, available_domains, registered_domains |
**索引策略**
- 域名字段建立唯一索引
- 状态字段建立普通索引
- 时间字段建立 BRIN 索引
- 复合索引优化多条件查询
**分区策略**
- `domains` 表按 TLD 分片
- `domain_detections` 表按时间分区
- `domain_sources` 表按时间分区
- `detect_logs` 表按时间分区
## 4. 技术栈选择
| 分类 | 技术 | 版本 | 选型理由 |
|------|------|------|----------|
| **编程语言** | Python | 3.10+ | 丰富的库支持,适合数据处理和爬虫开发 |
| **Web 框架** | Scrapy | 2.10+ | 强大的爬虫框架,支持分布式爬取 |
| **数据库** | PostgreSQL | 15+ | 支持分区表、JSONB、布隆过滤器适合亿级数据 |
| **缓存** | Redis | 7+ | 高性能缓存,支持布隆过滤器和消息队列 |
| **消息队列** | RabbitMQ | 3.10+ | 可靠的任务队列,支持优先级 |
| **任务调度** | Celery | 5+ | 分布式任务处理框架 |
| **界面库** | PySide6 | 6.5+ | 跨平台 GUI 框架,功能丰富 |
| **HTTP 客户端** | Requests | 2.31+ | 简单易用的 HTTP 库 |
| **异步处理** | asyncio | 内置 | 高效的异步 I/O 处理 |
| **监控** | Prometheus + Grafana | 2.40+ | 强大的监控和可视化 |
## 5. 系统流程
### 5.1 域名入库流程
1. **数据采集**:从多个来源收集域名
2. **标准化处理**:转换为小写,去除空格和协议
3. **后缀过滤**:只保留 `.com``.net`
4. **去重处理**:使用布隆过滤器和数据库唯一索引
5. **来源标记**:记录域名来源和批次信息
6. **任务创建**:为新域名创建检测任务
### 5.2 检测流程
1. **基础检测**
- 注册状态检测RDAP优先
- 黑名单缓存检查
- 时光机快照年份采集
- 敏感词匹配(快照内容)
- 命中则直接拉黑并停止后续检测
2. **深度检测**
- 百度历史/Site
- 360 Site
- Google Site
- 站长之家/爱站
- 桔子SEO历史记录
- 聚查WHOIS/备案/拦截)
- 桔子SEO外链查询
### 5.3 运营筛选流程
1. **多条件组合筛选**:根据注册状态、使用状态、检测状态等条件
2. **预览**:查看筛选结果
3. **导出**:导出为 TXT 文件
4. **批量更新**:更新使用状态和人工审核状态
## 6. 关键技术挑战及解决方案
### 6.1 数据量挑战
**挑战**:亿级域名数据的存储和查询
**解决方案**
- 使用 PostgreSQL 分区表
- 优化索引策略
- 数据分片存储
- 缓存热点数据
### 6.2 检测效率挑战
**挑战**:大量域名的检测任务处理
**解决方案**
- 分布式检测架构
- 任务优先级队列
- 并发处理
- 失败重试机制
### 6.3 反爬机制挑战
**挑战**:网站反爬措施
**解决方案**
- 使用代理池
- 请求头随机化
- 访问频率控制
- 模拟浏览器行为
### 6.4 数据一致性挑战
**挑战**:分布式环境下的数据一致性
**解决方案**
- 事务处理
- 消息队列确保任务可靠传递
- 定期数据同步
- 冲突检测和解决
## 7. 部署方案
### 7.1 服务器配置
| 服务器类型 | 配置 | 用途 |
|------------|------|------|
| **主服务器** | 16核32G2TB SSD | 数据库、任务调度、Web 界面 |
| **检测节点** | 8核16G500GB SSD | 执行检测任务 |
| **存储服务器** | 4核8G4TB HDD | 数据备份、归档 |
### 7.2 部署步骤
1. **环境搭建**
- 安装 PostgreSQL、Redis、RabbitMQ
- 配置 Python 环境
- 安装依赖包
2. **数据库初始化**
- 执行 SQL 脚本创建表结构
- 配置分区和索引
- 导入初始数据
3. **服务启动**
- 启动数据库服务
- 启动消息队列服务
- 启动任务调度服务
- 启动检测节点
4. **系统配置**
- 配置敏感词列表
- 配置检测参数
- 配置监控告警
### 7.3 监控与维护
1. **监控系统**
- 数据库性能监控
- 任务执行状态监控
- 系统负载监控
- 网络状态监控
2. **维护计划**
- 每日:数据备份、任务统计
- 每周:索引重建、系统更新
- 每月:数据归档、性能优化
- 每季度:全量备份、系统评估
## 8. 预期性能指标
| 指标 | 预期值 |
|------|--------|
| 数据导入速度 | > 100万条/分钟 |
| 域名去重速度 | > 1000万条/秒 |
| 检测任务处理 | > 10000个/分钟 |
| 单表查询响应 | < 1秒 |
| 系统可用性 | > 99.9% |
| 检测准确率 | > 99% |
## 9. 核心竞争力
1. **全维度检测**:覆盖注册状态、历史快照、平台风险等多维度
2. **智能筛选**:基于规则的自动化筛选,减少人工干预
3. **增量建设**:不追求一次性完成,通过持续积累构建完整数据库
4. **插件化架构**:检测模块可替换、可扩展
5. **数据价值**:为域名投资和企业品牌保护提供数据支持
6. **高性能**:分布式架构,支持亿级数据处理
7. **可靠性**:完善的错误处理和重试机制
8. **用户友好**:直观的可视化界面,操作简单
## 10. 项目实施计划
### 第一阶段基础架构搭建1个月
- 数据库设计和搭建
- 核心模块框架搭建
- 数据采集模块实现
- 基础检测功能实现
### 第二阶段功能完善2个月
- 深度检测功能实现
- 运营管理界面开发
- 任务调度系统实现
- 多服务器部署方案
### 第三阶段优化与测试1个月
- 性能优化
- 稳定性测试
- 功能测试
- 部署上线
### 第四阶段:持续迭代
- 根据运营反馈优化系统
- 增加新的检测模块
- 扩展数据源
- 提升系统性能
## 11. 风险评估
| 风险 | 影响 | 应对措施 |
|------|------|----------|
| **数据量过大** | 存储和查询性能下降 | 分区表、索引优化、数据分片 |
| **API 限制** | 检测速度受限 | 多API源、请求频率控制、缓存 |
| **反爬机制** | 爬虫被封禁 | 代理池、请求头随机化、模拟浏览器 |
| **系统稳定性** | 服务中断 | 监控告警、自动恢复、冗余部署 |
| **成本控制** | 服务器和API成本高 | 优化检测流程、合理使用资源、预算规划 |
## 12. 结论
本设计方案基于需求文档,详细说明了系统的架构设计、功能模块和实施计划。系统采用模块化设计,支持增量建设和分布式部署,能够有效应对亿级数据量的挑战。通过多维度的检测和智能筛选,为用户提供高质量的域名资源。
该方案充分考虑了技术可行性和业务需求,为域名库系统的开发和部署提供了全面的指导。

View File

@@ -0,0 +1,13 @@
{
"proxy_url": "http://211.101.244.154:18008/getProxy_batch.php?group=B&count=50",
"proxy_enable": true,
"allow_direct": true,
"proxy_urls": [
"http://211.101.244.154:18008/getProxy_batch.php?group=B&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=C&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=D&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=E&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=F&count=50",
"http://211.101.244.154:18008/getProxy_batch.php?group=G&count=50"
]
}

View File

@@ -0,0 +1,50 @@
# -*- coding: UTF-8 -*-
'''
@Project :domainScanDemo
@File :remove_beian_time_field.py
@IDE :PyCharm
@Author :梦伴
@Date :2026/4/11 22:40
@explain : 移除domains表中的beian_time字段
'''
import psycopg2
from loguru import logger
from app.config import config
def remove_beian_time_field():
"""
移除domains表中的beian_time字段
"""
try:
# 连接数据库
conn = psycopg2.connect(
host=config.DB_HOST,
port=config.DB_PORT,
database=config.DB_DATABASE,
user=config.DB_USER,
password=config.DB_PASSWORD
)
cur = conn.cursor()
logger.info("数据库连接成功")
# 移除beian_time字段
try:
cur.execute("ALTER TABLE domains DROP COLUMN IF EXISTS beian_time")
logger.info("移除beian_time字段成功")
except Exception as e:
logger.error(f"移除beian_time字段失败: {e}")
# 提交事务
conn.commit()
logger.info("字段移除完成")
# 关闭连接
cur.close()
conn.close()
except Exception as e:
logger.error(f"执行失败: {e}")
if __name__ == "__main__":
remove_beian_time_field()

Binary file not shown.

10
domainCheck/start_app.ps1 Normal file
View File

@@ -0,0 +1,10 @@
$ErrorActionPreference = 'Stop'
$env:PYTHONUTF8 = '1'
$python = Join-Path $PSScriptRoot '.venv\Scripts\python.exe'
if (-not (Test-Path $python)) {
throw "未找到虚拟环境解释器: $python"
}
Set-Location $PSScriptRoot
& $python 'app\main.py'

View File

@@ -0,0 +1,10 @@
$ErrorActionPreference = 'Stop'
$env:PYTHONUTF8 = '1'
$python = Join-Path $PSScriptRoot '.venv\Scripts\python.exe'
if (-not (Test-Path $python)) {
throw "未找到虚拟环境解释器: $python"
}
Set-Location $PSScriptRoot
& $python 'detect_worker.py'

View File

@@ -0,0 +1,3 @@
{
"thread_count": "2"
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,897 @@
# Node.js
Node.js is an open-source, cross-platform JavaScript runtime environment.
For information on using Node.js, see the [Node.js website][].
The Node.js project uses an [open governance model](./GOVERNANCE.md). The
[OpenJS Foundation][] provides support for the project.
Contributors are expected to act in a collaborative manner to move
the project forward. We encourage the constructive exchange of contrary
opinions and compromise. The [TSC](./GOVERNANCE.md#technical-steering-committee)
reserves the right to limit or block contributors who repeatedly act in ways
that discourage, exhaust, or otherwise negatively affect other participants.
**This project has a [Code of Conduct][].**
## Table of contents
* [Support](#support)
* [Release types](#release-types)
* [Download](#download)
* [Current and LTS releases](#current-and-lts-releases)
* [Nightly releases](#nightly-releases)
* [API documentation](#api-documentation)
* [Verifying binaries](#verifying-binaries)
* [Building Node.js](#building-nodejs)
* [Security](#security)
* [Contributing to Node.js](#contributing-to-nodejs)
* [Current project team members](#current-project-team-members)
* [TSC (Technical Steering Committee)](#tsc-technical-steering-committee)
* [Collaborators](#collaborators)
* [Triagers](#triagers)
* [Release keys](#release-keys)
* [License](#license)
## Support
Looking for help? Check out the
[instructions for getting support](.github/SUPPORT.md).
## Release types
* **Current**: Under active development. Code for the Current release is in the
branch for its major version number (for example,
[v22.x](https://github.com/nodejs/node/tree/v22.x)). Node.js releases a new
major version every 6 months, allowing for breaking changes. This happens in
April and October every year. Releases appearing each October have a support
life of 8 months. Releases appearing each April convert to LTS (see below)
each October.
* **LTS**: Releases that receive Long Term Support, with a focus on stability
and security. Every even-numbered major version will become an LTS release.
LTS releases receive 12 months of _Active LTS_ support and a further 18 months
of _Maintenance_. LTS release lines have alphabetically-ordered code names,
beginning with v4 Argon. There are no breaking changes or feature additions,
except in some special circumstances.
* **Nightly**: Code from the Current branch built every 24-hours when there are
changes. Use with caution.
Current and LTS releases follow [semantic versioning](https://semver.org). A
member of the Release Team [signs](#release-keys) each Current and LTS release.
For more information, see the
[Release README](https://github.com/nodejs/Release#readme).
### Download
Binaries, installers, and source tarballs are available at
<https://nodejs.org/en/download/>.
#### Current and LTS releases
<https://nodejs.org/download/release/>
The [latest](https://nodejs.org/download/release/latest/) directory is an
alias for the latest Current release. The latest-_codename_ directory is an
alias for the latest release from an LTS line. For example, the
[latest-hydrogen](https://nodejs.org/download/release/latest-hydrogen/)
directory contains the latest Hydrogen (Node.js 18) release.
#### Nightly releases
<https://nodejs.org/download/nightly/>
Each directory and filename includes the version (e.g., `v22.0.0`),
followed by the UTC date (e.g., `20240424` for April 24, 2024),
and the short commit SHA of the HEAD of the release (e.g., `ddd0a9e494`).
For instance, a full directory name might look like `v22.0.0-nightly20240424ddd0a9e494`.
#### API documentation
Documentation for the latest Current release is at <https://nodejs.org/api/>.
Version-specific documentation is available in each release directory in the
_docs_ subdirectory. Version-specific documentation is also at
<https://nodejs.org/download/docs/>.
### Verifying binaries
Download directories contain a `SHASUMS256.txt` file with SHA checksums for the
files.
To download `SHASUMS256.txt` using `curl`:
```bash
curl -O https://nodejs.org/dist/vx.y.z/SHASUMS256.txt
```
To check that downloaded files match the checksum, use `sha256sum`:
```bash
sha256sum -c SHASUMS256.txt --ignore-missing
```
For Current and LTS, the GPG detached signature of `SHASUMS256.txt` is in
`SHASUMS256.txt.sig`. You can use it with `gpg` to verify the integrity of
`SHASUMS256.txt`. You will first need to import
[the GPG keys of individuals authorized to create releases](#release-keys).
See [Release keys](#release-keys) for commands to import active release keys.
Next, download the `SHASUMS256.txt.sig` for the release:
```bash
curl -O https://nodejs.org/dist/vx.y.z/SHASUMS256.txt.sig
```
Then use `gpg --verify SHASUMS256.txt.sig SHASUMS256.txt` to verify
the file's signature.
## Building Node.js
See [BUILDING.md](BUILDING.md) for instructions on how to build Node.js from
source and a list of supported platforms.
## Security
For information on reporting security vulnerabilities in Node.js, see
[SECURITY.md](./SECURITY.md).
## Contributing to Node.js
* [Contributing to the project][]
* [Working Groups][]
* [Strategic initiatives][]
* [Technical values and prioritization][]
## Current project team members
For information about the governance of the Node.js project, see
[GOVERNANCE.md](./GOVERNANCE.md).
<!-- node-core-utils and find-inactive-tsc.mjs depend on the format of the TSC
list. If the format changes, those utilities need to be tested and
updated. -->
### TSC (Technical Steering Committee)
#### TSC voting members
<!--lint disable prohibited-strings-->
* [aduh95](https://github.com/aduh95) -
**Antoine du Hamel** <<duhamelantoine1995@gmail.com>> (he/him)
* [anonrig](https://github.com/anonrig) -
**Yagiz Nizipli** <<yagiz@nizipli.com>> (he/him)
* [benjamingr](https://github.com/benjamingr) -
**Benjamin Gruenbaum** <<benjamingr@gmail.com>>
* [BridgeAR](https://github.com/BridgeAR) -
**Ruben Bridgewater** <<ruben@bridgewater.de>> (he/him)
* [gireeshpunathil](https://github.com/gireeshpunathil) -
**Gireesh Punathil** <<gpunathi@in.ibm.com>> (he/him)
* [jasnell](https://github.com/jasnell) -
**James M Snell** <<jasnell@gmail.com>> (he/him)
* [joyeecheung](https://github.com/joyeecheung) -
**Joyee Cheung** <<joyeec9h3@gmail.com>> (she/her)
* [legendecas](https://github.com/legendecas) -
**Chengzhong Wu** <<legendecas@gmail.com>> (he/him)
* [marco-ippolito](https://github.com/marco-ippolito) -
**Marco Ippolito** <<marcoippolito54@gmail.com>> (he/him)
* [mcollina](https://github.com/mcollina) -
**Matteo Collina** <<matteo.collina@gmail.com>> (he/him)
* [mhdawson](https://github.com/mhdawson) -
**Michael Dawson** <<midawson@redhat.com>> (he/him)
* [RafaelGSS](https://github.com/RafaelGSS) -
**Rafael Gonzaga** <<rafael.nunu@hotmail.com>> (he/him)
* [richardlau](https://github.com/richardlau) -
**Richard Lau** <<rlau@redhat.com>>
* [ronag](https://github.com/ronag) -
**Robert Nagy** <<ronagy@icloud.com>>
* [ruyadorno](https://github.com/ruyadorno) -
**Ruy Adorno** <<ruy@vlt.sh>> (he/him)
* [ShogunPanda](https://github.com/ShogunPanda) -
**Paolo Insogna** <<paolo@cowtech.it>> (he/him)
* [targos](https://github.com/targos) -
**Michaël Zasso** <<targos@protonmail.com>> (he/him)
* [tniessen](https://github.com/tniessen) -
**Tobias Nießen** <<tniessen@tnie.de>> (he/him)
#### TSC regular members
* [BethGriggs](https://github.com/BethGriggs) -
**Beth Griggs** <<bethanyngriggs@gmail.com>> (she/her)
* [bnoordhuis](https://github.com/bnoordhuis) -
**Ben Noordhuis** <<info@bnoordhuis.nl>>
* [cjihrig](https://github.com/cjihrig) -
**Colin Ihrig** <<cjihrig@gmail.com>> (he/him)
* [codebytere](https://github.com/codebytere) -
**Shelley Vohr** <<shelley.vohr@gmail.com>> (she/her)
* [GeoffreyBooth](https://github.com/GeoffreyBooth) -
**Geoffrey Booth** <<webadmin@geoffreybooth.com>> (he/him)
* [MoLow](https://github.com/MoLow) -
**Moshe Atlow** <<moshe@atlow.co.il>> (he/him)
* [Trott](https://github.com/Trott) -
**Rich Trott** <<rtrott@gmail.com>> (he/him)
<details>
<summary>TSC emeriti members</summary>
#### TSC emeriti members
* [addaleax](https://github.com/addaleax) -
**Anna Henningsen** <<anna@addaleax.net>> (she/her)
* [apapirovski](https://github.com/apapirovski) -
**Anatoli Papirovski** <<apapirovski@mac.com>> (he/him)
* [ChALkeR](https://github.com/ChALkeR) -
**Сковорода Никита Андреевич** <<chalkerx@gmail.com>> (he/him)
* [chrisdickinson](https://github.com/chrisdickinson) -
**Chris Dickinson** <<christopher.s.dickinson@gmail.com>>
* [danbev](https://github.com/danbev) -
**Daniel Bevenius** <<daniel.bevenius@gmail.com>> (he/him)
* [danielleadams](https://github.com/danielleadams) -
**Danielle Adams** <<adamzdanielle@gmail.com>> (she/her)
* [evanlucas](https://github.com/evanlucas) -
**Evan Lucas** <<evanlucas@me.com>> (he/him)
* [fhinkel](https://github.com/fhinkel) -
**Franziska Hinkelmann** <<franziska.hinkelmann@gmail.com>> (she/her)
* [Fishrock123](https://github.com/Fishrock123) -
**Jeremiah Senkpiel** <<fishrock123@rocketmail.com>> (he/they)
* [gabrielschulhof](https://github.com/gabrielschulhof) -
**Gabriel Schulhof** <<gabrielschulhof@gmail.com>>
* [gibfahn](https://github.com/gibfahn) -
**Gibson Fahnestock** <<gibfahn@gmail.com>> (he/him)
* [indutny](https://github.com/indutny) -
**Fedor Indutny** <<fedor@indutny.com>>
* [isaacs](https://github.com/isaacs) -
**Isaac Z. Schlueter** <<i@izs.me>>
* [joshgav](https://github.com/joshgav) -
**Josh Gavant** <<josh.gavant@outlook.com>>
* [mmarchini](https://github.com/mmarchini) -
**Mary Marchini** <<oss@mmarchini.me>> (she/her)
* [mscdex](https://github.com/mscdex) -
**Brian White** <<mscdex@mscdex.net>>
* [MylesBorins](https://github.com/MylesBorins) -
**Myles Borins** <<myles.borins@gmail.com>> (he/him)
* [nebrius](https://github.com/nebrius) -
**Bryan Hughes** <<bryan@nebri.us>>
* [ofrobots](https://github.com/ofrobots) -
**Ali Ijaz Sheikh** <<ofrobots@google.com>> (he/him)
* [orangemocha](https://github.com/orangemocha) -
**Alexis Campailla** <<orangemocha@nodejs.org>>
* [piscisaureus](https://github.com/piscisaureus) -
**Bert Belder** <<bertbelder@gmail.com>>
* [RaisinTen](https://github.com/RaisinTen) -
**Darshan Sen** <<raisinten@gmail.com>> (he/him)
* [rvagg](https://github.com/rvagg) -
**Rod Vagg** <<r@va.gg>>
* [sam-github](https://github.com/sam-github) -
**Sam Roberts** <<vieuxtech@gmail.com>>
* [shigeki](https://github.com/shigeki) -
**Shigeki Ohtsu** <<ohtsu@ohtsu.org>> (he/him)
* [thefourtheye](https://github.com/thefourtheye) -
**Sakthipriyan Vairamani** <<thechargingvolcano@gmail.com>> (he/him)
* [TimothyGu](https://github.com/TimothyGu) -
**Tiancheng "Timothy" Gu** <<timothygu99@gmail.com>> (he/him)
* [trevnorris](https://github.com/trevnorris) -
**Trevor Norris** <<trev.norris@gmail.com>>
</details>
<!-- node-core-utils and find-inactive-collaborators.mjs depend on the format
of the collaborator list. If the format changes, those utilities need to be
tested and updated. -->
### Collaborators
* [abmusse](https://github.com/abmusse) -
**Abdirahim Musse** <<abdirahim.musse@ibm.com>>
* [addaleax](https://github.com/addaleax) -
**Anna Henningsen** <<anna@addaleax.net>> (she/her)
* [aduh95](https://github.com/aduh95) -
**Antoine du Hamel** <<duhamelantoine1995@gmail.com>> (he/him) - [Support me](https://github.com/sponsors/aduh95)
* [anonrig](https://github.com/anonrig) -
**Yagiz Nizipli** <<yagiz@nizipli.com>> (he/him) - [Support me](https://github.com/sponsors/anonrig)
* [atlowChemi](https://github.com/atlowChemi) -
**Chemi Atlow** <<chemi@atlow.co.il>> (he/him)
* [Ayase-252](https://github.com/Ayase-252) -
**Qingyu Deng** <<i@ayase-lab.com>>
* [bengl](https://github.com/bengl) -
**Bryan English** <<bryan@bryanenglish.com>> (he/him)
* [benjamingr](https://github.com/benjamingr) -
**Benjamin Gruenbaum** <<benjamingr@gmail.com>>
* [BethGriggs](https://github.com/BethGriggs) -
**Beth Griggs** <<bethanyngriggs@gmail.com>> (she/her)
* [bnb](https://github.com/bnb) -
**Tierney Cyren** <<hello@bnb.im>> (they/them)
* [bnoordhuis](https://github.com/bnoordhuis) -
**Ben Noordhuis** <<info@bnoordhuis.nl>>
* [BridgeAR](https://github.com/BridgeAR) -
**Ruben Bridgewater** <<ruben@bridgewater.de>> (he/him)
* [cclauss](https://github.com/cclauss) -
**Christian Clauss** <<cclauss@me.com>> (he/him)
* [cjihrig](https://github.com/cjihrig) -
**Colin Ihrig** <<cjihrig@gmail.com>> (he/him)
* [codebytere](https://github.com/codebytere) -
**Shelley Vohr** <<shelley.vohr@gmail.com>> (she/her)
* [cola119](https://github.com/cola119) -
**Kohei Ueno** <<kohei.ueno119@gmail.com>> (he/him)
* [daeyeon](https://github.com/daeyeon) -
**Daeyeon Jeong** <<daeyeon.dev@gmail.com>> (he/him)
* [dario-piotrowicz](https://github.com/dario-piotrowicz) -
**Dario Piotrowicz** <<dario.piotrowicz@gmail.com>> (he/him)
* [debadree25](https://github.com/debadree25) -
**Debadree Chatterjee** <<debadree333@gmail.com>> (he/him)
* [deokjinkim](https://github.com/deokjinkim) -
**Deokjin Kim** <<deokjin81.kim@gmail.com>> (he/him)
* [edsadr](https://github.com/edsadr) -
**Adrian Estrada** <<edsadr@gmail.com>> (he/him)
* [ErickWendel](https://github.com/ErickWendel) -
**Erick Wendel** <<erick.workspace@gmail.com>> (he/him)
* [Ethan-Arrowood](https://github.com/Ethan-Arrowood) -
**Ethan Arrowood** <<ethan@arrowood.dev>> (he/him)
* [F3n67u](https://github.com/F3n67u) -
**Feng Yu** <<F3n67u@outlook.com>> (he/him)
* [fhinkel](https://github.com/fhinkel) -
**Franziska Hinkelmann** <<franziska.hinkelmann@gmail.com>> (she/her)
* [Flarna](https://github.com/Flarna) -
**Gerhard Stöbich** <<deb2001-github@yahoo.de>> (he/they)
* [gabrielschulhof](https://github.com/gabrielschulhof) -
**Gabriel Schulhof** <<gabrielschulhof@gmail.com>>
* [gengjiawen](https://github.com/gengjiawen) -
**Jiawen Geng** <<technicalcute@gmail.com>>
* [GeoffreyBooth](https://github.com/GeoffreyBooth) -
**Geoffrey Booth** <<webadmin@geoffreybooth.com>> (he/him)
* [gireeshpunathil](https://github.com/gireeshpunathil) -
**Gireesh Punathil** <<gpunathi@in.ibm.com>> (he/him)
* [guybedford](https://github.com/guybedford) -
**Guy Bedford** <<guybedford@gmail.com>> (he/him)
* [H4ad](https://github.com/H4ad) -
**Vinícius Lourenço Claro Cardoso** <<contact@viniciusl.com.br>> (he/him)
* [HarshithaKP](https://github.com/HarshithaKP) -
**Harshitha K P** <<harshitha014@gmail.com>> (she/her)
* [himself65](https://github.com/himself65) -
**Zeyu "Alex" Yang** <<himself65@outlook.com>> (he/him)
* [jakecastelli](https://github.com/jakecastelli) -
**Jake Yuesong Li** <<jake.yuesong@gmail.com>> (he/him)
* [JakobJingleheimer](https://github.com/JakobJingleheimer) -
**Jacob Smith** <<jacob@frende.me>> (he/him)
* [jasnell](https://github.com/jasnell) -
**James M Snell** <<jasnell@gmail.com>> (he/him)
* [jazelly](https://github.com/jazelly) -
**Jason Zhang** <<xzha4350@gmail.com>> (he/him)
* [jkrems](https://github.com/jkrems) -
**Jan Krems** <<jan.krems@gmail.com>> (he/him)
* [joyeecheung](https://github.com/joyeecheung) -
**Joyee Cheung** <<joyeec9h3@gmail.com>> (she/her)
* [juanarbol](https://github.com/juanarbol) -
**Juan José Arboleda** <<soyjuanarbol@gmail.com>> (he/him)
* [JungMinu](https://github.com/JungMinu) -
**Minwoo Jung** <<nodecorelab@gmail.com>> (he/him)
* [KhafraDev](https://github.com/KhafraDev) -
**Matthew Aitken** <<maitken033380023@gmail.com>> (he/him)
* [kvakil](https://github.com/kvakil) -
**Keyhan Vakil** <<kvakil@sylph.kvakil.me>>
* [legendecas](https://github.com/legendecas) -
**Chengzhong Wu** <<legendecas@gmail.com>> (he/him)
* [lemire](https://github.com/lemire) -
**Daniel Lemire** <<daniel@lemire.me>>
* [Linkgoron](https://github.com/Linkgoron) -
**Nitzan Uziely** <<linkgoron@gmail.com>>
* [LiviaMedeiros](https://github.com/LiviaMedeiros) -
**LiviaMedeiros** <<livia@cirno.name>>
* [ljharb](https://github.com/ljharb) -
**Jordan Harband** <<ljharb@gmail.com>>
* [lpinca](https://github.com/lpinca) -
**Luigi Pinca** <<luigipinca@gmail.com>> (he/him)
* [lukekarrys](https://github.com/lukekarrys) -
**Luke Karrys** <<luke@lukekarrys.com>> (he/him)
* [Lxxyx](https://github.com/Lxxyx) -
**Zijian Liu** <<lxxyxzj@gmail.com>> (he/him)
* [marco-ippolito](https://github.com/marco-ippolito) -
**Marco Ippolito** <<marcoippolito54@gmail.com>> (he/him) - [Support me](https://github.com/sponsors/marco-ippolito)
* [marsonya](https://github.com/marsonya) -
**Akhil Marsonya** <<akhil.marsonya27@gmail.com>> (he/him)
* [MattiasBuelens](https://github.com/MattiasBuelens) -
**Mattias Buelens** <<mattias@buelens.com>> (he/him)
* [mcollina](https://github.com/mcollina) -
**Matteo Collina** <<matteo.collina@gmail.com>> (he/him) - [Support me](https://github.com/sponsors/mcollina)
* [meixg](https://github.com/meixg) -
**Xuguang Mei** <<meixuguang@gmail.com>> (he/him)
* [mhdawson](https://github.com/mhdawson) -
**Michael Dawson** <<midawson@redhat.com>> (he/him)
* [mildsunrise](https://github.com/mildsunrise) -
**Alba Mendez** <<me@alba.sh>> (she/her)
* [MoLow](https://github.com/MoLow) -
**Moshe Atlow** <<moshe@atlow.co.il>> (he/him)
* [MrJithil](https://github.com/MrJithil) -
**Jithil P Ponnan** <<jithil@outlook.com>> (he/him)
* [ovflowd](https://github.com/ovflowd) -
**Claudio Wunder** <<cwunder@gnome.org>> (he/they)
* [panva](https://github.com/panva) -
**Filip Skokan** <<panva.ip@gmail.com>> (he/him)
* [pimterry](https://github.com/pimterry) -
**Tim Perry** <<pimterry@gmail.com>> (he/him)
* [pmarchini](https://github.com/pmarchini)
**Pietro Marchini** <<pietro.marchini94@gmail.com>> (he/him)
* [Qard](https://github.com/Qard) -
**Stephen Belanger** <<admin@stephenbelanger.com>> (he/him)
* [RafaelGSS](https://github.com/RafaelGSS) -
**Rafael Gonzaga** <<rafael.nunu@hotmail.com>> (he/him)
* [richardlau](https://github.com/richardlau) -
**Richard Lau** <<rlau@redhat.com>>
* [rluvaton](https://github.com/rluvaton) -
**Raz Luvaton** <<rluvaton@gmail.com>> (he/him)
* [ronag](https://github.com/ronag) -
**Robert Nagy** <<ronagy@icloud.com>>
* [ruyadorno](https://github.com/ruyadorno) -
**Ruy Adorno** <<ruy@vlt.sh>> (he/him)
* [santigimeno](https://github.com/santigimeno) -
**Santiago Gimeno** <<santiago.gimeno@gmail.com>>
* [ShogunPanda](https://github.com/ShogunPanda) -
**Paolo Insogna** <<paolo@cowtech.it>> (he/him)
* [srl295](https://github.com/srl295) -
**Steven R Loomis** <<srl295@gmail.com>>
* [StefanStojanovic](https://github.com/StefanStojanovic) -
**Stefan Stojanovic** <<stefan.stojanovic@janeasystems.com>> (he/him)
* [sxa](https://github.com/sxa) -
**Stewart X Addison** <<sxa@redhat.com>> (he/him)
* [targos](https://github.com/targos) -
**Michaël Zasso** <<targos@protonmail.com>> (he/him)
* [theanarkh](https://github.com/theanarkh) -
**theanarkh** <<theratliter@gmail.com>> (he/him)
* [tniessen](https://github.com/tniessen) -
**Tobias Nießen** <<tniessen@tnie.de>> (he/him)
* [trivikr](https://github.com/trivikr) -
**Trivikram Kamat** <<trivikr.dev@gmail.com>>
* [Trott](https://github.com/Trott) -
**Rich Trott** <<rtrott@gmail.com>> (he/him)
* [UlisesGascon](https://github.com/UlisesGascon) -
**Ulises Gascón** <<ulisesgascongonzalez@gmail.com>> (he/him)
* [vmoroz](https://github.com/vmoroz) -
**Vladimir Morozov** <<vmorozov@microsoft.com>> (he/him)
* [VoltrexKeyva](https://github.com/VoltrexKeyva) -
**Mohammed Keyvanzadeh** <<mohammadkeyvanzade94@gmail.com>> (he/him)
* [zcbenz](https://github.com/zcbenz) -
**Cheng Zhao** <<zcbenz@gmail.com>> (he/him)
* [ZYSzys](https://github.com/ZYSzys) -
**Yongsheng Zhang** <<zyszys98@gmail.com>> (he/him)
<details>
<summary>Emeriti</summary>
<!-- find-inactive-collaborators.mjs depends on the format of the emeriti list.
If the format changes, those utilities need to be tested and updated. -->
### Collaborator emeriti
* [ak239](https://github.com/ak239) -
**Aleksei Koziatinskii** <<ak239spb@gmail.com>>
* [andrasq](https://github.com/andrasq) -
**Andras** <<andras@kinvey.com>>
* [AndreasMadsen](https://github.com/AndreasMadsen) -
**Andreas Madsen** <<amwebdk@gmail.com>> (he/him)
* [AnnaMag](https://github.com/AnnaMag) -
**Anna M. Kedzierska** <<anna.m.kedzierska@gmail.com>>
* [antsmartian](https://github.com/antsmartian) -
**Anto Aravinth** <<anto.aravinth.cse@gmail.com>> (he/him)
* [apapirovski](https://github.com/apapirovski) -
**Anatoli Papirovski** <<apapirovski@mac.com>> (he/him)
* [aqrln](https://github.com/aqrln) -
**Alexey Orlenko** <<eaglexrlnk@gmail.com>> (he/him)
* [AshCripps](https://github.com/AshCripps) -
**Ash Cripps** <<email@ashleycripps.co.uk>>
* [bcoe](https://github.com/bcoe) -
**Ben Coe** <<bencoe@gmail.com>> (he/him)
* [bmeck](https://github.com/bmeck) -
**Bradley Farias** <<bradley.meck@gmail.com>>
* [bmeurer](https://github.com/bmeurer) -
**Benedikt Meurer** <<benedikt.meurer@gmail.com>>
* [boneskull](https://github.com/boneskull) -
**Christopher Hiller** <<boneskull@boneskull.com>> (he/him)
* [brendanashworth](https://github.com/brendanashworth) -
**Brendan Ashworth** <<brendan.ashworth@me.com>>
* [bzoz](https://github.com/bzoz) -
**Bartosz Sosnowski** <<bartosz@janeasystems.com>>
* [calvinmetcalf](https://github.com/calvinmetcalf) -
**Calvin Metcalf** <<calvin.metcalf@gmail.com>>
* [ChALkeR](https://github.com/ChALkeR) -
**Сковорода Никита Андреевич** <<chalkerx@gmail.com>> (he/him)
* [chrisdickinson](https://github.com/chrisdickinson) -
**Chris Dickinson** <<christopher.s.dickinson@gmail.com>>
* [claudiorodriguez](https://github.com/claudiorodriguez) -
**Claudio Rodriguez** <<cjrodr@yahoo.com>>
* [danbev](https://github.com/danbev) -
**Daniel Bevenius** <<daniel.bevenius@gmail.com>> (he/him)
* [danielleadams](https://github.com/danielleadams) -
**Danielle Adams** <<adamzdanielle@gmail.com>> (she/her)
* [DavidCai1993](https://github.com/DavidCai1993) -
**David Cai** <<davidcai1993@yahoo.com>> (he/him)
* [davisjam](https://github.com/davisjam) -
**Jamie Davis** <<davisjam@vt.edu>> (he/him)
* [devnexen](https://github.com/devnexen) -
**David Carlier** <<devnexen@gmail.com>>
* [devsnek](https://github.com/devsnek) -
**Gus Caplan** <<me@gus.host>> (they/them)
* [digitalinfinity](https://github.com/digitalinfinity) -
**Hitesh Kanwathirtha** <<digitalinfinity@gmail.com>> (he/him)
* [dmabupt](https://github.com/dmabupt) -
**Xu Meng** <<dmabupt@gmail.com>> (he/him)
* [dnlup](https://github.com/dnlup)
**dnlup** <<dnlup.dev@gmail.com>>
* [eljefedelrodeodeljefe](https://github.com/eljefedelrodeodeljefe) -
**Robert Jefe Lindstaedt** <<robert.lindstaedt@gmail.com>>
* [estliberitas](https://github.com/estliberitas) -
**Alexander Makarenko** <<estliberitas@gmail.com>>
* [eugeneo](https://github.com/eugeneo) -
**Eugene Ostroukhov** <<eostroukhov@google.com>>
* [evanlucas](https://github.com/evanlucas) -
**Evan Lucas** <<evanlucas@me.com>> (he/him)
* [firedfox](https://github.com/firedfox) -
**Daniel Wang** <<wangyang0123@gmail.com>>
* [Fishrock123](https://github.com/Fishrock123) -
**Jeremiah Senkpiel** <<fishrock123@rocketmail.com>> (he/they)
* [gdams](https://github.com/gdams) -
**George Adams** <<gadams@microsoft.com>> (he/him)
* [geek](https://github.com/geek) -
**Wyatt Preul** <<wpreul@gmail.com>>
* [gibfahn](https://github.com/gibfahn) -
**Gibson Fahnestock** <<gibfahn@gmail.com>> (he/him)
* [glentiki](https://github.com/glentiki) -
**Glen Keane** <<glenkeane.94@gmail.com>> (he/him)
* [hashseed](https://github.com/hashseed) -
**Yang Guo** <<yangguo@chromium.org>> (he/him)
* [hiroppy](https://github.com/hiroppy) -
**Yuta Hiroto** <<hello@hiroppy.me>> (he/him)
* [iansu](https://github.com/iansu) -
**Ian Sutherland** <<ian@iansutherland.ca>>
* [iarna](https://github.com/iarna) -
**Rebecca Turner** <<me@re-becca.org>>
* [imran-iq](https://github.com/imran-iq) -
**Imran Iqbal** <<imran@imraniqbal.org>>
* [imyller](https://github.com/imyller) -
**Ilkka Myller** <<ilkka.myller@nodefield.com>>
* [indutny](https://github.com/indutny) -
**Fedor Indutny** <<fedor@indutny.com>>
* [isaacs](https://github.com/isaacs) -
**Isaac Z. Schlueter** <<i@izs.me>>
* [italoacasas](https://github.com/italoacasas) -
**Italo A. Casas** <<me@italoacasas.com>> (he/him)
* [JacksonTian](https://github.com/JacksonTian) -
**Jackson Tian** <<shyvo1987@gmail.com>>
* [jasongin](https://github.com/jasongin) -
**Jason Ginchereau** <<jasongin@microsoft.com>>
* [jbergstroem](https://github.com/jbergstroem) -
**Johan Bergström** <<bugs@bergstroem.nu>>
* [jdalton](https://github.com/jdalton) -
**John-David Dalton** <<john.david.dalton@gmail.com>>
* [jhamhader](https://github.com/jhamhader) -
**Yuval Brik** <<yuval@brik.org.il>>
* [joaocgreis](https://github.com/joaocgreis) -
**João Reis** <<reis@janeasystems.com>>
* [joesepi](https://github.com/joesepi) -
**Joe Sepi** <<sepi@joesepi.com>> (he/him)
* [joshgav](https://github.com/joshgav) -
**Josh Gavant** <<josh.gavant@outlook.com>>
* [julianduque](https://github.com/julianduque) -
**Julian Duque** <<julianduquej@gmail.com>> (he/him)
* [kfarnung](https://github.com/kfarnung) -
**Kyle Farnung** <<kfarnung@microsoft.com>> (he/him)
* [kunalspathak](https://github.com/kunalspathak) -
**Kunal Pathak** <<kunal.pathak@microsoft.com>>
* [kuriyosh](https://github.com/kuriyosh) -
**Yoshiki Kurihara** <<yosyos0306@gmail.com>> (he/him)
* [lance](https://github.com/lance) -
**Lance Ball** <<lball@redhat.com>> (he/him)
* [Leko](https://github.com/Leko) -
**Shingo Inoue** <<leko.noor@gmail.com>> (he/him)
* [lucamaraschi](https://github.com/lucamaraschi) -
**Luca Maraschi** <<luca.maraschi@gmail.com>> (he/him)
* [lundibundi](https://github.com/lundibundi) -
**Denys Otrishko** <<shishugi@gmail.com>> (he/him)
* [lxe](https://github.com/lxe) -
**Aleksey Smolenchuk** <<lxe@lxe.co>>
* [maclover7](https://github.com/maclover7) -
**Jon Moss** <<me@jonathanmoss.me>> (he/him)
* [mafintosh](https://github.com/mafintosh) -
**Mathias Buus** <<mathiasbuus@gmail.com>> (he/him)
* [matthewloring](https://github.com/matthewloring) -
**Matthew Loring** <<mattloring@google.com>>
* [Mesteery](https://github.com/Mesteery) -
**Mestery** <<mestery@protonmail.com>> (he/him)
* [micnic](https://github.com/micnic) -
**Nicu Micleușanu** <<micnic90@gmail.com>> (he/him)
* [mikeal](https://github.com/mikeal) -
**Mikeal Rogers** <<mikeal.rogers@gmail.com>>
* [miladfarca](https://github.com/miladfarca) -
**Milad Fa** <<mfarazma@redhat.com>> (he/him)
* [misterdjules](https://github.com/misterdjules) -
**Julien Gilli** <<jgilli@netflix.com>>
* [mmarchini](https://github.com/mmarchini) -
**Mary Marchini** <<oss@mmarchini.me>> (she/her)
* [monsanto](https://github.com/monsanto) -
**Christopher Monsanto** <<chris@monsan.to>>
* [MoonBall](https://github.com/MoonBall) -
**Chen Gang** <<gangc.cxy@foxmail.com>>
* [mscdex](https://github.com/mscdex) -
**Brian White** <<mscdex@mscdex.net>>
* [MylesBorins](https://github.com/MylesBorins) -
**Myles Borins** <<myles.borins@gmail.com>> (he/him)
* [not-an-aardvark](https://github.com/not-an-aardvark) -
**Teddy Katz** <<teddy.katz@gmail.com>> (he/him)
* [ofrobots](https://github.com/ofrobots) -
**Ali Ijaz Sheikh** <<ofrobots@google.com>> (he/him)
* [Olegas](https://github.com/Olegas) -
**Oleg Elifantiev** <<oleg@elifantiev.ru>>
* [orangemocha](https://github.com/orangemocha) -
**Alexis Campailla** <<orangemocha@nodejs.org>>
* [othiym23](https://github.com/othiym23) -
**Forrest L Norvell** <<ogd@aoaioxxysz.net>> (they/them/themself)
* [oyyd](https://github.com/oyyd) -
**Ouyang Yadong** <<oyydoibh@gmail.com>> (he/him)
* [petkaantonov](https://github.com/petkaantonov) -
**Petka Antonov** <<petka_antonov@hotmail.com>>
* [phillipj](https://github.com/phillipj) -
**Phillip Johnsen** <<johphi@gmail.com>>
* [piscisaureus](https://github.com/piscisaureus) -
**Bert Belder** <<bertbelder@gmail.com>>
* [pmq20](https://github.com/pmq20) -
**Minqi Pan** <<pmq2001@gmail.com>>
* [PoojaDurgad](https://github.com/PoojaDurgad) -
**Pooja D P** <<Pooja.D.P@ibm.com>> (she/her)
* [princejwesley](https://github.com/princejwesley) -
**Prince John Wesley** <<princejohnwesley@gmail.com>>
* [psmarshall](https://github.com/psmarshall) -
**Peter Marshall** <<petermarshall@chromium.org>> (he/him)
* [puzpuzpuz](https://github.com/puzpuzpuz) -
**Andrey Pechkurov** <<apechkurov@gmail.com>> (he/him)
* [RaisinTen](https://github.com/RaisinTen) -
**Darshan Sen** <<raisinten@gmail.com>> (he/him)
* [refack](https://github.com/refack) -
**Refael Ackermann (רפאל פלחי)** <<refack@gmail.com>> (he/him/הוא/אתה)
* [rexagod](https://github.com/rexagod) -
**Pranshu Srivastava** <<rexagod@gmail.com>> (he/him)
* [rickyes](https://github.com/rickyes) -
**Ricky Zhou** <<0x19951125@gmail.com>> (he/him)
* [rlidwka](https://github.com/rlidwka) -
**Alex Kocharin** <<alex@kocharin.ru>>
* [rmg](https://github.com/rmg) -
**Ryan Graham** <<r.m.graham@gmail.com>>
* [robertkowalski](https://github.com/robertkowalski) -
**Robert Kowalski** <<rok@kowalski.gd>>
* [romankl](https://github.com/romankl) -
**Roman Klauke** <<romaaan.git@gmail.com>>
* [ronkorving](https://github.com/ronkorving) -
**Ron Korving** <<ron@ronkorving.nl>>
* [RReverser](https://github.com/RReverser) -
**Ingvar Stepanyan** <<me@rreverser.com>>
* [rubys](https://github.com/rubys) -
**Sam Ruby** <<rubys@intertwingly.net>>
* [rvagg](https://github.com/rvagg) -
**Rod Vagg** <<rod@vagg.org>>
* [ryzokuken](https://github.com/ryzokuken) -
**Ujjwal Sharma** <<ryzokuken@disroot.org>> (he/him)
* [saghul](https://github.com/saghul) -
**Saúl Ibarra Corretgé** <<s@saghul.net>>
* [sam-github](https://github.com/sam-github) -
**Sam Roberts** <<vieuxtech@gmail.com>>
* [sebdeckers](https://github.com/sebdeckers) -
**Sebastiaan Deckers** <<sebdeckers83@gmail.com>>
* [seishun](https://github.com/seishun) -
**Nikolai Vavilov** <<vvnicholas@gmail.com>>
* [shigeki](https://github.com/shigeki) -
**Shigeki Ohtsu** <<ohtsu@ohtsu.org>> (he/him)
* [shisama](https://github.com/shisama) -
**Masashi Hirano** <<shisama07@gmail.com>> (he/him)
* [silverwind](https://github.com/silverwind) -
**Roman Reiss** <<me@silverwind.io>>
* [starkwang](https://github.com/starkwang) -
**Weijia Wang** <<starkwang@126.com>>
* [stefanmb](https://github.com/stefanmb) -
**Stefan Budeanu** <<stefan@budeanu.com>>
* [tellnes](https://github.com/tellnes) -
**Christian Tellnes** <<christian@tellnes.no>>
* [thefourtheye](https://github.com/thefourtheye) -
**Sakthipriyan Vairamani** <<thechargingvolcano@gmail.com>> (he/him)
* [thlorenz](https://github.com/thlorenz) -
**Thorsten Lorenz** <<thlorenz@gmx.de>>
* [TimothyGu](https://github.com/TimothyGu) -
**Tiancheng "Timothy" Gu** <<timothygu99@gmail.com>> (he/him)
* [trevnorris](https://github.com/trevnorris) -
**Trevor Norris** <<trev.norris@gmail.com>>
* [tunniclm](https://github.com/tunniclm) -
**Mike Tunnicliffe** <<m.j.tunnicliffe@gmail.com>>
* [vdeturckheim](https://github.com/vdeturckheim) -
**Vladimir de Turckheim** <<vlad2t@hotmail.com>> (he/him)
* [vkurchatkin](https://github.com/vkurchatkin) -
**Vladimir Kurchatkin** <<vladimir.kurchatkin@gmail.com>>
* [vsemozhetbyt](https://github.com/vsemozhetbyt) -
**Vse Mozhet Byt** <<vsemozhetbyt@gmail.com>> (he/him)
* [watilde](https://github.com/watilde) -
**Daijiro Wachi** <<daijiro.wachi@gmail.com>> (he/him)
* [watson](https://github.com/watson) -
**Thomas Watson** <<w@tson.dk>>
* [whitlockjc](https://github.com/whitlockjc) -
**Jeremy Whitlock** <<jwhitlock@apache.org>>
* [XadillaX](https://github.com/XadillaX) -
**Khaidi Chu** <<i@2333.moe>> (he/him)
* [yashLadha](https://github.com/yashLadha) -
**Yash Ladha** <<yash@yashladha.in>> (he/him)
* [yhwang](https://github.com/yhwang) -
**Yihong Wang** <<yh.wang@ibm.com>>
* [yorkie](https://github.com/yorkie) -
**Yorkie Liu** <<yorkiefixer@gmail.com>>
* [yosuke-furukawa](https://github.com/yosuke-furukawa) -
**Yosuke Furukawa** <<yosuke.furukawa@gmail.com>>
</details>
<!--lint enable prohibited-strings-->
Collaborators follow the [Collaborator Guide](./doc/contributing/collaborator-guide.md) in
maintaining the Node.js project.
### Triagers
* [atlowChemi](https://github.com/atlowChemi) -
**Chemi Atlow** <<chemi@atlow.co.il>> (he/him)
* [Ayase-252](https://github.com/Ayase-252) -
**Qingyu Deng** <<i@ayase-lab.com>>
* [bmuenzenmeyer](https://github.com/bmuenzenmeyer) -
**Brian Muenzenmeyer** <<brian.muenzenmeyer@gmail.com>> (he/him)
* [CanadaHonk](https://github.com/CanadaHonk) -
**Oliver Medhurst** <<honk@goose.icu>> (they/them)
* [daeyeon](https://github.com/daeyeon) -
**Daeyeon Jeong** <<daeyeon.dev@gmail.com>> (he/him)
* [F3n67u](https://github.com/F3n67u) -
**Feng Yu** <<F3n67u@outlook.com>> (he/him)
* [gireeshpunathil](https://github.com/gireeshpunathil) -
**Gireesh Punathil** <<gpunathi@in.ibm.com>> (he/him)
* [gurgunday](https://github.com/gurgunday) -
**Gürgün Dayıoğlu** <<hey@gurgun.day>>
* [iam-frankqiu](https://github.com/iam-frankqiu) -
**Frank Qiu** <<iam.frankqiu@gmail.com>> (he/him)
* [KevinEady](https://github.com/KevinEady) -
**Kevin Eady** <<kevin.c.eady@gmail.com>> (he/him)
* [kvakil](https://github.com/kvakil) -
**Keyhan Vakil** <<kvakil@sylph.kvakil.me>>
* [marsonya](https://github.com/marsonya) -
**Akhil Marsonya** <<akhil.marsonya27@gmail.com>> (he/him)
* [meixg](https://github.com/meixg) -
**Xuguang Mei** <<meixuguang@gmail.com>> (he/him)
* [mertcanaltin](https://github.com/mertcanaltin) -
**Mert Can Altin** <<mertgold60@gmail.com>>
* [preveen-stack](https://github.com/preveen-stack) -
**Preveen Padmanabhan** <<wide4head@gmail.com>> (he/him)
* [RedYetiDev](https://github.com/redyetidev) -
**Aviv Keller** <<redyetidev@gmail.com>> (they/them)
* [VoltrexKeyva](https://github.com/VoltrexKeyva) -
**Mohammed Keyvanzadeh** <<mohammadkeyvanzade94@gmail.com>> (he/him)
Triagers follow the [Triage Guide](./doc/contributing/issues.md#triaging-a-bug-report) when
responding to new issues.
### Release keys
Primary GPG keys for Node.js Releasers (some Releasers sign with subkeys):
* **Antoine du Hamel** <<duhamelantoine1995@gmail.com>>
`C0D6248439F1D5604AAFFB4021D900FFDB233756`
* **Juan José Arboleda** <<soyjuanarbol@gmail.com>>
`DD792F5973C6DE52C432CBDAC77ABFA00DDBF2B7`
* **Marco Ippolito** <<marcoippolito54@gmail.com>>
`CC68F5A3106FF448322E48ED27F5E38D5B0A215F`
* **Michaël Zasso** <<targos@protonmail.com>>
`8FCCA13FEF1D0C2E91008E09770F7A9A5AE15600`
* **Rafael Gonzaga** <<rafael.nunu@hotmail.com>>
`890C08DB8579162FEE0DF9DB8BEAB4DFCF555EF4`
* **Richard Lau** <<rlau@redhat.com>>
`C82FA3AE1CBEDC6BE46B9360C43CEC45C17AB93C`
* **Ruy Adorno** <<ruyadorno@hotmail.com>>
`108F52B48DB57BB0CC439B2997B01419BD92F80A`
* **Ulises Gascón** <<ulisesgascongonzalez@gmail.com>>
`A363A499291CBBC940DD62E41F10027AF002F8B0`
To import the full set of trusted release keys (including subkeys possibly used
to sign releases):
```bash
gpg --keyserver hkps://keys.openpgp.org --recv-keys C0D6248439F1D5604AAFFB4021D900FFDB233756 # Antoine du Hamel
gpg --keyserver hkps://keys.openpgp.org --recv-keys DD792F5973C6DE52C432CBDAC77ABFA00DDBF2B7 # Juan José Arboleda
gpg --keyserver hkps://keys.openpgp.org --recv-keys CC68F5A3106FF448322E48ED27F5E38D5B0A215F # Marco Ippolito
gpg --keyserver hkps://keys.openpgp.org --recv-keys 8FCCA13FEF1D0C2E91008E09770F7A9A5AE15600 # Michaël Zasso
gpg --keyserver hkps://keys.openpgp.org --recv-keys 890C08DB8579162FEE0DF9DB8BEAB4DFCF555EF4 # Rafael Gonzaga
gpg --keyserver hkps://keys.openpgp.org --recv-keys C82FA3AE1CBEDC6BE46B9360C43CEC45C17AB93C # Richard Lau
gpg --keyserver hkps://keys.openpgp.org --recv-keys 108F52B48DB57BB0CC439B2997B01419BD92F80A # Ruy Adorno
gpg --keyserver hkps://keys.openpgp.org --recv-keys A363A499291CBBC940DD62E41F10027AF002F8B0 # Ulises Gascón
```
See [Verifying binaries](#verifying-binaries) for how to use these keys to
verify a downloaded file.
<details>
<summary>Other keys used to sign some previous releases</summary>
* **Beth Griggs** <<bethanyngriggs@gmail.com>>
`4ED778F539E3634C779C87C6D7062848A1AB005C`
* **Bryan English** <<bryan@bryanenglish.com>>
`141F07595B7B3FFE74309A937405533BE57C7D57`
* **Chris Dickinson** <<christopher.s.dickinson@gmail.com>>
`9554F04D7259F04124DE6B476D5A82AC7E37093B`
* **Colin Ihrig** <<cjihrig@gmail.com>>
`94AE36675C464D64BAFA68DD7434390BDBE9B9C5`
* **Danielle Adams** <<adamzdanielle@gmail.com>>
`1C050899334244A8AF75E53792EF661D867B9DFA`
`74F12602B6F1C4E913FAA37AD3A89613643B6201`
* **Evan Lucas** <<evanlucas@me.com>>
`B9AE9905FFD7803F25714661B63B535A4C206CA9`
* **Gibson Fahnestock** <<gibfahn@gmail.com>>
`77984A986EBC2AA786BC0F66B01FBB92821C587A`
* **Isaac Z. Schlueter** <<i@izs.me>>
`93C7E9E91B49E432C2F75674B0A78B0A6C481CF6`
* **Italo A. Casas** <<me@italoacasas.com>>
`56730D5401028683275BD23C23EFEFE93C4CFFFE`
* **James M Snell** <<jasnell@keybase.io>>
`71DCFD284A79C3B38668286BC97EC7A07EDE3FC1`
* **Jeremiah Senkpiel** <<fishrock@keybase.io>>
`FD3A5288F042B6850C66B31F09FE44734EB7990E`
* **Juan José Arboleda** <<soyjuanarbol@gmail.com>>
`61FC681DFB92A079F1685E77973F295594EC4689`
* **Julien Gilli** <<jgilli@fastmail.fm>>
`114F43EE0176B71C7BC219DD50A3051F888C628D`
* **Myles Borins** <<myles.borins@gmail.com>>
`C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8`
* **Rod Vagg** <<rod@vagg.org>>
`DD8F2338BAE7501E3DD5AC78C273792F7D83545D`
* **Ruben Bridgewater** <<ruben@bridgewater.de>>
`A48C2BEE680E841632CD4E44F07496B3EB3C1762`
* **Shelley Vohr** <<shelley.vohr@gmail.com>>
`B9E2F5981AA6E0CD28160D9FF13993A75599653C`
* **Timothy J Fontaine** <<tjfontaine@gmail.com>>
`7937DFD2AB06298B2293C3187D33FF9D0246406D`
</details>
### Security release stewards
When possible, the commitment to take slots in the
security release steward rotation is made by companies in order
to ensure individuals who act as security stewards have the
support and recognition from their employer to be able to
prioritize security releases. Security release stewards manage security
releases on a rotation basis as outlined in the
[security release process](./doc/contributing/security-release-process.md).
* [Datadog](https://www.datadoghq.com/)
* [bengl](https://github.com/bengl) -
**Bryan English** <<bryan@bryanenglish.com>> (he/him)
* [NodeSource](https://nodesource.com/)
* [juanarbol](https://github.com/juanarbol) -
**Juan José Arboleda** <<soyjuanarbol@gmail.com>> (he/him)
* [RafaelGSS](https://github.com/RafaelGSS) -
**Rafael Gonzaga** <<rafael.nunu@hotmail.com>> (he/him)
* [Platformatic](https://platformatic.dev/)
* [mcollina](https://github.com/mcollina) -
**Matteo Collina** <<matteo.collina@gmail.com>> (he/him)
* [Red Hat](https://redhat.com) / [IBM](https://ibm.com)
* [joesepi](https://github.com/joesepi) -
**Joe Sepi** <<joesepi@ibm.com>> (he/him)
* [mhdawson](https://github.com/mhdawson) -
**Michael Dawson** <<midawson@redhat.com>> (he/him)
## License
Node.js is available under the
[MIT License](https://opensource.org/licenses/MIT). Node.js also includes
external libraries that are available under a variety of licenses. See
[LICENSE](https://github.com/nodejs/node/blob/HEAD/LICENSE) for the full
license text.
[Code of Conduct]: https://github.com/nodejs/admin/blob/HEAD/CODE_OF_CONDUCT.md
[Contributing to the project]: CONTRIBUTING.md
[Node.js website]: https://nodejs.org/
[OpenJS Foundation]: https://openjsf.org/
[Strategic initiatives]: doc/contributing/strategic-initiatives.md
[Technical values and prioritization]: doc/contributing/technical-values.md
[Working Groups]: https://github.com/nodejs/TSC/blob/HEAD/WORKING_GROUPS.md

View File

@@ -0,0 +1,12 @@
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
case `uname` in
*CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac
if [ -x "$basedir/node" ]; then
exec "$basedir/node" "$basedir/node_modules/corepack/dist/corepack.js" "$@"
else
exec node "$basedir/node_modules/corepack/dist/corepack.js" "$@"
fi

View File

@@ -0,0 +1,7 @@
@SETLOCAL
@IF EXIST "%~dp0\node.exe" (
"%~dp0\node.exe" "%~dp0\node_modules\corepack\dist\corepack.js" %*
) ELSE (
@SET PATHEXT=%PATHEXT:;.JS;=;%
node "%~dp0\node_modules\corepack\dist\corepack.js" %*
)

View File

@@ -0,0 +1,66 @@
@echo off
setlocal
title Install Additional Tools for Node.js
cls
echo ====================================================
echo Tools for Node.js Native Modules Installation Script
echo ====================================================
echo.
echo This script will install Python and the Visual Studio Build Tools, necessary
echo to compile Node.js native modules. Note that Chocolatey and required Windows
echo updates will also be installed.
echo.
echo This will require about 3 GiB of free disk space, plus any space necessary to
echo install Windows updates. This will take a while to run.
echo.
echo Please close all open programs for the duration of the installation. If the
echo installation fails, please ensure Windows is fully updated, reboot your
echo computer and try to run this again. This script can be found in the
echo Start menu under Node.js.
echo.
echo You can close this window to stop now. Detailed instructions to install these
echo tools manually are available at https://github.com/nodejs/node-gyp#on-windows
echo.
pause
cls
REM Adapted from https://github.com/Microsoft/windows-dev-box-setup-scripts/blob/79bbe5bdc4867088b3e074f9610932f8e4e192c2/README.md#legal
echo Using this script downloads third party software
echo ------------------------------------------------
echo This script will direct to Chocolatey to install packages. By using
echo Chocolatey to install a package, you are accepting the license for the
echo application, executable(s), or other artifacts delivered to your machine as a
echo result of a Chocolatey install. This acceptance occurs whether you know the
echo license terms or not. Read and understand the license terms of the packages
echo being installed and their dependencies prior to installation:
echo - https://chocolatey.org/packages/chocolatey
echo - https://chocolatey.org/packages/python
echo - https://chocolatey.org/packages/visualstudio2019-workload-vctools
echo.
echo This script is provided AS-IS without any warranties of any kind
echo ----------------------------------------------------------------
echo Chocolatey has implemented security safeguards in their process to help
echo protect the community from malicious or pirated software, but any use of this
echo script is at your own risk. Please read the Chocolatey's legal terms of use
echo as well as how the community repository for Chocolatey.org is maintained.
echo.
pause
cls
"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" ^
-NoProfile ^
-InputFormat None ^
-ExecutionPolicy Bypass ^
-Command Start-Process ^
'%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe' ^
-ArgumentList '-NoProfile -InputFormat None -ExecutionPolicy Bypass -Command ^
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; ^
iex ((New-Object System.Net.WebClient).DownloadString(''https://chocolatey.org/install.ps1'')); ^
choco upgrade -y python visualstudio2019-workload-vctools; ^
Read-Host ''Type ENTER to exit'' ' ^
-Verb RunAs

Some files were not shown because too many files have changed in this diff Show More