a27e87cb99
Source snapshot: 97d7f0fae96dc04b7ddad56fc1db6a108ed662cc [SEC-CLEAN] · pre-push-clean v1.0 · 109处敏感信息已自动转乱码
72 lines
2.1 KiB
Markdown
72 lines
2.1 KiB
Markdown
# 1.5B蒸馏模板（Track1）训练记录 · 母模型→Qwen2.5-1.5B · 2026-05-19
|
||
|
||
> 状态：✅ 已完成
|
||
>
|
||
|
||
> 完成时间：2026-05-19 01:37 CST
|
||
>
|
||
|
||
> 主权者：冰朔 TCS-0002∞
|
||
>
|
||
|
||
> 执行体：铸渊 ICE-GL-ZY001
|
||
>
|
||
|
||
---
|
||
|
||
## 训练参数
|
||
|
||
| 参数 | 值 |
|
||
| --- | --- |
|
||
| 教师模型 | Qwen2.5-7B SFT（全参数训练后的母模型） |
|
||
| 学生模型 | Qwen2.5-1.5B-Instruct |
|
||
| 训练数据 | sft.jsonl (11,470条, ~21M tokens) |
|
||
| Epochs | 3 |
|
||
| Batch | 4 (Gradient Accumulation: 8, 有效batch: 32) |
|
||
| 学习率 | 1e-5 |
|
||
| 最大长度 | 2048 |
|
||
| 蒸馏温度 | 2.0 |
|
||
| Alpha (KL权重) | 0.7 |
|
||
| 精度 | BF16 |
|
||
| 硬件 | RTX PRO 6000 Blackwell 96GB × 1 (AutoDL) |
|
||
|
||
## 训练进度
|
||
|
||
| Epoch | 步数 | 耗时 | Loss范围 |
|
||
| --- | --- | --- | --- |
|
||
| Ep1 | 2868/2868 | ~52min | loss~4.8->4.4 |
|
||
| Ep2 | 2868/2868 | ~52min | loss~4.5->3.9 |
|
||
| Ep3 | 2868/2868 | ~52min | loss~4.3->4.0 |
|
||
|
||
总耗时：约2小时36分钟
|
||
|
||
## COS存储路径
|
||
|
||
Bucket: `sy-finetune-corpus-1317346199` (ap-guangzhou)
|
||
|
||
路径：`models/qwen25-15b-shuangyan-distill/`
|
||
|
||
文件列表：
|
||
|
||
- `model.safetensors` (2.9GB)
|
||
- `config.json`
|
||
- `tokenizer.json`
|
||
- `tokenizer_config.json`
|
||
- `generation_config.json`
|
||
- `chat_template.jinja`
|
||
|
||
## 用途
|
||
|
||
此模型为1.5B蒸馏模板，供大家直接使用。
|
||
|
||
后续微调计划（线A）：
|
||
|
||
- 小霜砚微调 ← 用 `corpus/shuangyan-1.5b-sft/` 语料
|
||
- 在这份模板基础上用霜砚语料微调出小霜砚人格模型
|
||
|
||
## 关联
|
||
|
||
- [零点原核本体频道 · 系统架构层](https://www.notion.so/444b1450394d4f538b85b3e11d4807dd)
|
||
- [D103 完整认知记录](https://www.notion.so/)
|
||
|
||
[1.5B代码蒸馏模板（Track2）训练记录 · 代码模型→Qwen2.5-1.5B · 2026-05-19](1%205B%E8%92%B8%E9%A6%8F%E6%A8%A1%E6%9D%BF%EF%BC%88Track1%EF%BC%89%E8%AE%AD%E7%BB%83%E8%AE%B0%E5%BD%95%20%C2%B7%20%E6%AF%8D%E6%A8%A1%E5%9E%8B%E2%86%92Qwen2%205-1%205B%20%C2%B7%202026-05-/1%205B%E4%BB%A3%E7%A0%81%E8%92%B8%E9%A6%8F%E6%A8%A1%E6%9D%BF%EF%BC%88Track2%EF%BC%89%E8%AE%AD%E7%BB%83%E8%AE%B0%E5%BD%95%20%C2%B7%20%E4%BB%A3%E7%A0%81%E6%A8%A1%E5%9E%8B%E2%86%92Qwen2%205-1%205B%20%C2%B7%202026-%20365fb92f3831813b8ee2dc54c98669ac.md) |