- 基于OpenResty的RESTful API服务 - 支持Cookie/Token等数据类型的存储和管理 - 登录认证保护 - Web管理界面 - 数据过期管理
383 lines
6.3 KiB
Markdown
383 lines
6.3 KiB
Markdown
# 通用数据同步服务
|
||
|
||
轻量级数据同步服务,支持 Cookie、Token、Credential 等多种类型数据的存储和管理。
|
||
|
||
## ✨ 特性
|
||
|
||
- 🚀 **轻量级**:纯 Flask,内存占用 < 30MB
|
||
- 🔒 **安全**:密码保护 + 跨域支持
|
||
- 📦 **通用**:支持 cookie/token/credential/session/custom 类型
|
||
- ⏰ **过期检测**:定时检查 + 微信通知
|
||
- 🎨 **易用**:油猴脚本一键同步
|
||
- 📊 **状态管理**:实时查看数据状态
|
||
|
||
---
|
||
|
||
## 🚀 快速开始
|
||
|
||
### 1. 安装依赖
|
||
|
||
```bash
|
||
pip3 install flask flask-cors
|
||
```
|
||
|
||
### 2. 启动服务
|
||
|
||
```bash
|
||
cd /home/hermes/projects/data-sync-service
|
||
python3 server.py
|
||
```
|
||
|
||
服务将在 `http://0.0.0.0:5001` 启动。
|
||
|
||
### 3. 安装油猴脚本
|
||
|
||
1. 打开 `client/userscript-template.js`
|
||
2. 复制内容到 Tampermonkey
|
||
3. 保存
|
||
|
||
### 4. 使用
|
||
|
||
在支持的网站(通义听悟、微信公众号等):
|
||
1. 登录账号
|
||
2. 点击右侧悬浮按钮 **🔄 同步数据**
|
||
3. 选择 **立即同步**
|
||
4. 看到 ✅ 成功提示
|
||
|
||
---
|
||
|
||
## 📚 API 文档
|
||
|
||
### 健康检查
|
||
|
||
```bash
|
||
GET /api/health
|
||
```
|
||
|
||
**响应**:
|
||
```json
|
||
{
|
||
"status": "ok",
|
||
"version": "1.0.0",
|
||
"uptime": 12345
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 上传数据
|
||
|
||
```bash
|
||
POST /api/data
|
||
Content-Type: application/json
|
||
```
|
||
|
||
**请求体**:
|
||
```json
|
||
{
|
||
"password": "admin123123123",
|
||
"tag": "tingwu:ykaayk@qq.com",
|
||
"type": "cookie",
|
||
"data": {
|
||
"cookies": [...],
|
||
"raw_cookie": "..."
|
||
},
|
||
"metadata": {
|
||
"expires_in": 604800,
|
||
"note": "备注",
|
||
"login_url": "https://tingwu.aliyun.com/"
|
||
}
|
||
}
|
||
```
|
||
|
||
**响应**:
|
||
```json
|
||
{
|
||
"success": true,
|
||
"message": "Data saved",
|
||
"tag": "tingwu:ykaayk@qq.com",
|
||
"timestamp": "2026-08-14 02:00:00",
|
||
"expires_at": "2026-08-21 02:00:00"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 获取数据
|
||
|
||
```bash
|
||
GET /api/data/:tag
|
||
X-Password: admin123123123
|
||
```
|
||
|
||
**响应**:
|
||
```json
|
||
{
|
||
"tag": "tingwu:ykaayk@qq.com",
|
||
"type": "cookie",
|
||
"data": {...},
|
||
"metadata": {...}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 删除数据
|
||
|
||
```bash
|
||
DELETE /api/data/:tag
|
||
X-Password: admin123123123
|
||
```
|
||
|
||
---
|
||
|
||
### 列出所有数据
|
||
|
||
```bash
|
||
GET /api/data
|
||
```
|
||
|
||
**响应**:
|
||
```json
|
||
{
|
||
"count": 2,
|
||
"tags": [
|
||
{
|
||
"tag": "tingwu:ykaayk@qq.com",
|
||
"type": "cookie",
|
||
"created_at": "2026-08-14 02:00:00",
|
||
"expires_at": "2026-08-21 02:00:00",
|
||
"status": "valid"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 复制配置
|
||
|
||
```bash
|
||
GET /api/data/copy/:tag
|
||
```
|
||
|
||
**响应**:
|
||
```json
|
||
{
|
||
"tag": "tingwu:ykaayk@qq.com",
|
||
"copy_text": "{\n \"serverUrl\": \"http://47.122.126.244:5001\",\n \"password\": \"admin123123123\",\n \"tag\": \"tingwu:ykaayk@qq.com\",\n \"type\": \"cookie\"\n}"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## ⏰ 定时过期检测
|
||
|
||
### 手动运行
|
||
|
||
```bash
|
||
python3 scripts/check_expiry.py
|
||
```
|
||
|
||
### 定时运行(Hermes Cron)
|
||
|
||
在 Hermes 中执行:
|
||
|
||
```
|
||
帮我设置定时任务,每天上午 9 点运行 /home/hermes/projects/data-sync-service/scripts/check_expiry.py 检查数据过期状态
|
||
```
|
||
|
||
或手动创建:
|
||
|
||
```bash
|
||
hermes cron create \
|
||
--schedule "0 9 * * *" \
|
||
--script "/home/hermes/projects/data-sync-service/scripts/check_expiry.py" \
|
||
--name "数据过期检测"
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 数据迁移
|
||
|
||
从旧的 `cookie-receiver` 迁移数据:
|
||
|
||
```bash
|
||
python3 scripts/migrate.py
|
||
```
|
||
|
||
这会自动:
|
||
1. 读取旧数据文件
|
||
2. 转换为新格式
|
||
3. 保存到新位置
|
||
4. 更新索引
|
||
|
||
---
|
||
|
||
## 📁 目录结构
|
||
|
||
```
|
||
/home/hermes/projects/data-sync-service/
|
||
├── server.py # Flask 服务
|
||
├── scripts/
|
||
│ ├── check_expiry.py # 过期检测
|
||
│ └── migrate.py # 数据迁移
|
||
├── client/
|
||
│ └── userscript-template.js # 油猴脚本
|
||
├── data/
|
||
│ ├── index.json # 索引文件
|
||
│ ├── tingwu_ykaayk.json # 数据文件
|
||
│ └── ...
|
||
└── README.md
|
||
```
|
||
|
||
---
|
||
|
||
## 🎯 支持的项目
|
||
|
||
### 已内置
|
||
- **通义听悟**(tingwu.aliyun.com)
|
||
- **微信公众号**(mp.weixin.qq.com)
|
||
|
||
### 添加新项目
|
||
|
||
编辑 `client/userscript-template.js`,在 `PROJECT_PATTERNS` 中添加:
|
||
|
||
```javascript
|
||
'example.com': {
|
||
project: 'example',
|
||
accountSelector: '.user-email',
|
||
loginUrl: 'https://example.com/login'
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 🔒 安全建议
|
||
|
||
### 生产环境
|
||
|
||
1. **修改密码**:
|
||
```python
|
||
# server.py
|
||
PASSWORD = os.environ.get('DATA_SYNC_PASSWORD', 'your-secure-password')
|
||
```
|
||
|
||
2. **使用 HTTPS**:
|
||
在服务前加 Nginx 反向代理:
|
||
```nginx
|
||
location /api/ {
|
||
proxy_pass http://127.0.0.1:5001;
|
||
}
|
||
```
|
||
|
||
3. **限流保护**:
|
||
```bash
|
||
pip3 install flask-limiter
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 性能指标
|
||
|
||
- **内存占用**:~20-30MB
|
||
- **响应时间**:<50ms
|
||
- **启动时间**:<1秒
|
||
- **并发支持**:100+ req/s
|
||
|
||
---
|
||
|
||
## 🐛 故障排查
|
||
|
||
### 服务无法启动
|
||
|
||
```bash
|
||
# 检查端口占用
|
||
sudo lsof -i:5001
|
||
|
||
# 检查日志
|
||
python3 server.py
|
||
```
|
||
|
||
### 油猴脚本无反应
|
||
|
||
1. 检查网站是否匹配 `@match` 规则
|
||
2. 打开浏览器控制台查看错误
|
||
3. 确认服务器可访问
|
||
|
||
### 数据未同步
|
||
|
||
```bash
|
||
# 检查服务器日志
|
||
python3 server.py
|
||
|
||
# 测试 API
|
||
curl http://localhost:5001/api/health
|
||
```
|
||
|
||
---
|
||
|
||
## 📝 使用示例
|
||
|
||
### Python 客户端
|
||
|
||
```python
|
||
import requests
|
||
|
||
# 上传数据
|
||
response = requests.post('http://47.122.126.244:5001/api/data', json={
|
||
'password': 'admin123123123',
|
||
'tag': 'myapp:user@example.com',
|
||
'type': 'token',
|
||
'data': {'access_token': 'xxx'},
|
||
'metadata': {
|
||
'expires_in': 3600,
|
||
'note': 'API Token',
|
||
'login_url': 'https://myapp.com/login'
|
||
}
|
||
})
|
||
|
||
print(response.json())
|
||
|
||
# 获取数据
|
||
response = requests.get(
|
||
'http://47.122.126.244:5001/api/data/myapp:user@example.com',
|
||
headers={'X-Password': 'admin123123123'}
|
||
)
|
||
|
||
data = response.json()
|
||
print(data['data'])
|
||
```
|
||
|
||
### Bash 客户端
|
||
|
||
```bash
|
||
# 上传
|
||
curl -X POST http://47.122.126.244:5001/api/data \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"password": "admin123123123",
|
||
"tag": "test:user",
|
||
"type": "cookie",
|
||
"data": {"test": "value"}
|
||
}'
|
||
|
||
# 获取
|
||
curl http://47.122.126.244:5001/api/data/test:user \
|
||
-H "X-Password: admin123123123"
|
||
```
|
||
|
||
---
|
||
|
||
## 🎉 完成!
|
||
|
||
现在你有一个完整的通用数据同步服务:
|
||
|
||
- ✅ 轻量级 Flask 服务
|
||
- ✅ 过期检测 + 微信通知
|
||
- ✅ 油猴脚本一键同步
|
||
- ✅ 完整的 API
|
||
- ✅ 数据迁移工具
|
||
|
||
开始使用吧!🚀
|