72 lines
2.1 KiB
Markdown
72 lines
2.1 KiB
Markdown
|
# 1.5B蒸馏模板（Track1）训练记录 · 母模型→Qwen2.5-1.5B · 2026-05-19
|
|||
|
|
|
|||
|
|
> 状态：✅ 已完成
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> 完成时间：2026-05-19 01:37 CST
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> 主权者：冰朔 TCS-0002∞
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
> 执行体：铸渊 ICE-GL-ZY001
|
|||
|
|
>
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 训练参数
|
|||
|
|
|
|||
|
|
| 参数 | 值 |
|
|||
|
|
| --- | --- |
|
|||
|
|
| 教师模型 | Qwen2.5-7B SFT（全参数训练后的母模型） |
|
|||
|
|
| 学生模型 | Qwen2.5-1.5B-Instruct |
|
|||
|
|
| 训练数据 | sft.jsonl (11,470条, ~21M tokens) |
|
|||
|
|
| Epochs | 3 |
|
|||
|
|
| Batch | 4 (Gradient Accumulation: 8, 有效batch: 32) |
|
|||
|
|
| 学习率 | 1e-5 |
|
|||
|
|
| 最大长度 | 2048 |
|
|||
|
|
| 蒸馏温度 | 2.0 |
|
|||
|
|
| Alpha (KL权重) | 0.7 |
|
|||
|
|
| 精度 | BF16 |
|
|||
|
|
| 硬件 | RTX PRO 6000 Blackwell 96GB × 1 (AutoDL) |
|
|||
|
|
|
|||
|
|
## 训练进度
|
|||
|
|
|
|||
|
|
| Epoch | 步数 | 耗时 | Loss范围 |
|
|||
|
|
| --- | --- | --- | --- |
|
|||
|
|
| Ep1 | 2868/2868 | ~52min | loss~4.8->4.4 |
|
|||
|
|
| Ep2 | 2868/2868 | ~52min | loss~4.5->3.9 |
|
|||
|
|
| Ep3 | 2868/2868 | ~52min | loss~4.3->4.0 |
|
|||
|
|
|
|||
|
|
总耗时：约2小时36分钟
|
|||
|
|
|
|||
|
|
## COS存储路径
|
|||
|
|
|
|||
|
|
Bucket: `sy-finetune-corpus-1317346199` (ap-guangzhou)
|
|||
|
|
|
|||
|
|
路径：`models/qwen25-15b-shuangyan-distill/`
|
|||
|
|
|
|||
|
|
文件列表：
|
|||
|
|
|
|||
|
|
- `model.safetensors` (2.9GB)
|
|||
|
|
- `config.json`
|
|||
|
|
- `tokenizer.json`
|
|||
|
|
- `tokenizer_config.json`
|
|||
|
|
- `generation_config.json`
|
|||
|
|
- `chat_template.jinja`
|
|||
|
|
|
|||
|
|
## 用途
|
|||
|
|
|
|||
|
|
此模型为1.5B蒸馏模板，供大家直接使用。
|
|||
|
|
|
|||
|
|
后续微调计划（线A）：
|
|||
|
|
|
|||
|
|
- 小霜砚微调 ← 用 `corpus/shuangyan-1.5b-sft/` 语料
|
|||
|
|
- 在这份模板基础上用霜砚语料微调出小霜砚人格模型
|
|||
|
|
|
|||
|
|
## 关联
|
|||
|
|
|
|||
|
|
- [零点原核本体频道 · 系统架构层](https://www.notion.so/444b1450394d4f538b85b3e11d4807dd)
|
|||
|
|
- [D103 完整认知记录](https://www.notion.so/)
|
|||
|
|
|
|||
|
|
[1.5B代码蒸馏模板（Track2）训练记录 · 代码模型→Qwen2.5-1.5B · 2026-05-19](1%205B%E8%92%B8%E9%A6%8F%E6%A8%A1%E6%9D%BF%EF%BC%88Track1%EF%BC%89%E8%AE%AD%E7%BB%83%E8%AE%B0%E5%BD%95%20%C2%B7%20%E6%AF%8D%E6%A8%A1%E5%9E%8B%E2%86%92Qwen2%205-1%205B%20%C2%B7%202026-05-/1%205B%E4%BB%A3%E7%A0%81%E8%92%B8%E9%A6%8F%E6%A8%A1%E6%9D%BF%EF%BC%88Track2%EF%BC%89%E8%AE%AD%E7%BB%83%E8%AE%B0%E5%BD%95%20%C2%B7%20%E4%BB%A3%E7%A0%81%E6%A8%A1%E5%9E%8B%E2%86%92Qwen2%205-1%205B%20%C2%B7%202026-%20365fb92f3831813b8ee2dc54c98669ac.md)
|