190 lines
5.9 KiB
Markdown
190 lines
5.9 KiB
Markdown
|
# 🔧 铸渊指令｜PM2 服务排查 + 全系统健康诊断（2026-03-13·霜砚签发）
|
|||
|
|
|
|||
|
|
> **签发**：霜砚（Notion执行体）
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> **接收**：铸渊（GitHub执行体）
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> **时间**：2026-03-13 12:00 CST
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> **优先级**：P1·正常
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> **前置要求**：⚠️ 必须先完成「阶段零·核心大脑认知恢复」再执行后续排查
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 阶段零 · 核心大脑认知恢复（必须最先执行）
|
|||
|
|
|
|||
|
|
<aside>
|
|||
|
|
⚠️
|
|||
|
|
|
|||
|
|
**铸渊，在做任何排查之前，你必须先恢复核心大脑认知。**
|
|||
|
|
|
|||
|
|
不然你不知道这整个系统框架在干什么，排查会走偏。
|
|||
|
|
|
|||
|
|
</aside>
|
|||
|
|
|
|||
|
|
### 步骤
|
|||
|
|
|
|||
|
|
1. **读取曜冥核心大脑 v4.0**
|
|||
|
|
- Page ID: `30bfb92f383181b58713da7ecc34a37c`
|
|||
|
|
- 通过 Notion API 读取全文：`GET https://api.notion.com/v1/blocks/30bfb92f383181b58713da7ecc34a37c/children?page_size=200`
|
|||
|
|
- 重点理解：
|
|||
|
|
- 一、数字地球 = 唯一降落锚点
|
|||
|
|
- 二、角色分工（曜冥/知秋/霜砚/铸渊）
|
|||
|
|
- 五·二、全自动闭环架构（三层：入口层/调度层/处理层）
|
|||
|
|
- 六、永久规则（broadcast_code_injection = always）
|
|||
|
|
- **技术定义：Persona-Driven DevOps** — 人格体驱动的开发运维系统
|
|||
|
|
2. **读取数字地球主控台**
|
|||
|
|
- Database ID: `c6f7722729264a91a3769224d5f631a3`
|
|||
|
|
- 了解当前系统全貌：存在注册表里有所有页面/数据库的路径
|
|||
|
|
3. **确认认知恢复完毕**
|
|||
|
|
- 在 Issue 或 PR 里写一行：`✅ 核心大脑 v4.0 认知恢复完毕 · [时间]`
|
|||
|
|
- 然后再进入阶段一
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 阶段一 · PM2 报错服务诊断
|
|||
|
|
|
|||
|
|
### 1.1 问题描述
|
|||
|
|
|
|||
|
|
服务器 `8.155.62.235`（阿里云 ECS）上 PM2 管理的 6 个服务中，有 3 个持续 errored：
|
|||
|
|
|
|||
|
|
| PM2 ID | 进程名 | 状态 | 重启次数 | 内存 |
|
|||
|
|
| --- | --- | --- | --- | --- |
|
|||
|
|
| 3 | guanghulab | **errored** | 420 | 0b |
|
|||
|
|
| 2 | guanghulab-backend | **errored** | 420 | 0b |
|
|||
|
|
| 4 | guanghulab-ws | **errored** | 390 | 0b |
|
|||
|
|
| 27 | guanghulab-proxy | online | 0 | 52.4mb |
|
|||
|
|
| 0 | hololake | online | 3 | 65.3mb |
|
|||
|
|
| 1 | hololake-api | online | 1 | 54.5mb |
|
|||
|
|
| 28 | persona-studio | online | 0 | 59.7mb |
|
|||
|
|
|
|||
|
|
### 1.2 排查步骤
|
|||
|
|
|
|||
|
|
铸渊通过 SSH（`DEPLOY_KEY` 已在 GitHub Secrets）执行以下命令并记录输出：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# ━━━ 第1步：查看报错日志 ━━━
|
|||
|
|
pm2 logs guanghulab --lines 50 --nostream
|
|||
|
|
pm2 logs guanghulab-backend --lines 50 --nostream
|
|||
|
|
pm2 logs guanghulab-ws --lines 50 --nostream
|
|||
|
|
|
|||
|
|
# ━━━ 第2步：查看进程详情 ━━━
|
|||
|
|
pm2 show guanghulab
|
|||
|
|
pm2 show guanghulab-backend
|
|||
|
|
pm2 show guanghulab-ws
|
|||
|
|
|
|||
|
|
# ━━━ 第3步：查看启动脚本路径 ━━━
|
|||
|
|
pm2 prettylist | grep -A5 'guanghulab\|guanghulab-backend\|guanghulab-ws'
|
|||
|
|
|
|||
|
|
# ━━━ 第4步：检查文件是否存在 ━━━
|
|||
|
|
ls -la /var/www/guanghulab/
|
|||
|
|
ls -la /var/www/guanghulab/backend/ 2>/dev/null || echo "backend目录不存在"
|
|||
|
|
ls -la /var/www/guanghulab/persona-studio/backend/ 2>/dev/null || echo "persona-studio backend目录不存在"
|
|||
|
|
|
|||
|
|
# ━━━ 第5步：检查端口占用 ━━━
|
|||
|
|
netstat -tlnp | grep -E '3000|3001|3002'
|
|||
|
|
|
|||
|
|
# ━━━ 第6步：检查Node版本和依赖 ━━━
|
|||
|
|
node -v
|
|||
|
|
npm -v
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 1.3 诊断后决策
|
|||
|
|
|
|||
|
|
根据排查结果，铸渊自行判断：
|
|||
|
|
|
|||
|
|
| 情况 | 处理方式 |
|
|||
|
|
| --- | --- |
|
|||
|
|
| 启动脚本路径不存在 | 删除该 PM2 进程（`pm2 delete 进程名`） |
|
|||
|
|
| 依赖缺失（node_modules） | `cd 目录 && npm install && pm2 restart 进程名` |
|
|||
|
|
| 端口冲突 | 修改端口或合并服务 |
|
|||
|
|
| 旧服务已被新服务替代 | 删除旧进程，保留新的（如 hololake 替代了 guanghulab） |
|
|||
|
|
| 代码报错 | 修复代码，提 PR |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 阶段二 · 全系统健康检查
|
|||
|
|
|
|||
|
|
排查完 PM2 后，铸渊对整个服务器做一次全面健康检查：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# ━━━ Nginx 状态 ━━━
|
|||
|
|
nginx -t
|
|||
|
|
systemctl status nginx
|
|||
|
|
|
|||
|
|
# ━━━ SSL 证书有效期 ━━━
|
|||
|
|
openssl x509 -in /etc/letsencrypt/live/guanghulab.com/fullchain.pem -noout -dates
|
|||
|
|
|
|||
|
|
# ━━━ 磁盘空间 ━━━
|
|||
|
|
df -h
|
|||
|
|
|
|||
|
|
# ━━━ 内存使用 ━━━
|
|||
|
|
free -h
|
|||
|
|
|
|||
|
|
# ━━━ PM2 全局状态 ━━━
|
|||
|
|
pm2 status
|
|||
|
|
|
|||
|
|
# ━━━ 网站可访问性 ━━━
|
|||
|
|
curl -sI https://guanghulab.com | head -5
|
|||
|
|
curl -s http://127.0.0.1:3001/api/v1/ | head -c 200 || echo "3001端口无响应"
|
|||
|
|
curl -s http://127.0.0.1:3002/api/ps/ | head -c 200 || echo "3002端口无响应"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 阶段三 · 生成诊断报告
|
|||
|
|
|
|||
|
|
铸渊完成排查后，**在 Notion 工单队列写一条工单**：
|
|||
|
|
|
|||
|
|
- **工单名**：`PM2服务排查报告 · 2026-03-13`
|
|||
|
|
- **操作类型**：其他
|
|||
|
|
- **提交者**：铸渊（需要在工单提交者选项里确认，如没有可写在具体内容里）
|
|||
|
|
- **具体内容**：包含以下信息
|
|||
|
|
- 3 个 errored 服务的报错原因
|
|||
|
|
- 处理结果（删除/修复/保留）
|
|||
|
|
- 全系统健康检查结果
|
|||
|
|
- SSL 证书到期时间
|
|||
|
|
- 磁盘/内存使用情况
|
|||
|
|
- 建议的后续优化
|
|||
|
|
|
|||
|
|
工单队列 Database ID：`84edd0640bf146a9a5a7840107013e8c`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 当前 Nginx 配置参考（已由冰朔于 2026-03-13 12:00 更新）
|
|||
|
|
|
|||
|
|
配置文件路径：`/etc/nginx/nginx.conf`（注意：不是 snippets 目录）
|
|||
|
|
|
|||
|
|
当前路由：
|
|||
|
|
|
|||
|
|
| 路由 | 指向 | 用途 |
|
|||
|
|
| --- | --- | --- |
|
|||
|
|
| `/` | 静态文件 `/var/www/guanghulab/status-board` | 光湖主站首页 |
|
|||
|
|
| `/webhook/feishu` | `127.0.0.1:3000` | 飞书 Webhook |
|
|||
|
|
| `/api/v1/` | `127.0.0.1:3001` | 光湖后端 API |
|
|||
|
|
| `/api/ps/` | `127.0.0.1:3002` | Persona Studio API（新增·含CORS） |
|
|||
|
|
| `/ws/preview` | `127.0.0.1:3002` | Persona Studio WebSocket（新增） |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 关联文档
|
|||
|
|
|
|||
|
|
- 曜冥核心大脑 v4.0：Page ID `30bfb92f383181b58713da7ecc34a37c`
|
|||
|
|
- 数字地球主控台：Database ID `c6f7722729264a91a3769224d5f631a3`
|
|||
|
|
- 工单队列：Database ID `84edd0640bf146a9a5a7840107013e8c`
|
|||
|
|
- 铸渊配置清单：见同级页面「📋 铸渊配置清单」
|
|||
|
|
- 全自动闭环对接规范：见同级页面「🔄 铸渊指令｜全自动闭环对接规范」
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
> ✅ 铸渊完成后，在本页下方追加一行：
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> `✅ PM2诊断完毕 · [日期时间] · 处理结果摘要`
|
|||
|
|
>
|