用熵值筛选复杂样本，高效提升小模型推理能力
Complexity-aware fine-tuning
- 根据单个词答案的熵值划分数据复杂度，仅对复杂样本启用推理
- 在30亿参数模型上实现0.58平均准确率，比标准SFT高0.13
- 用81%更少数据达到更好效果，适合资源有限的场景
通用大语言模型常通过监督微调（SFT）提升特定领域性能。更优结果可通过蒸馏大模型的思维链获得，但代价是大量昂贵调用和更多数据。本文提出一种高效微调新框架：仅对熵值高的复杂样本使用推理。在三个约30亿参数的小型开源模型上，基于单个词答案熵值将训练数据分为复杂度类别（ROC AUC 0.73），采用SFT与蒸馏联合微调。结果表明，该方法显著优于标准SFT（平均准确率0.58 vs 0.45），且优于纯蒸馏法（0.58 vs 0.56），同时仅需81%的数据量。
原文摘要 · Abstract (English)
General-purpose Large Language Models (LLMs) are frequently fine-tuned through supervised fine-tuning (SFT) to enhance performance in specific domains. Better results can be achieved by distilling the chain-of-thought of a larger model at the cost of numerous expensive calls and a much greater amount of data. We propose a novel blueprint for efficient fine-tuning that uses reasoning only for complex data identified by entropy. Specifically, across three small open models ($\approx 3B$) we split the training data into complexity categories by a single token answer entropy (ROC AUC $0.73$), fine-tune large language models (LLMs) via SFT and distillation, and show that our pipeline significantly outperforms the standard SFT approach ($0.58$ vs $0.45$ average accuracy) and outperforms the distillation approach ($0.58$ vs $0.56$ average accuracy) while using $81\%$ less data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。