用熵差筛选数据，让小样本训练也能高效适配不同任务。
InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning
- 通过基线模型与微调后模型的熵差识别关键数据。
- 仅用10%数据就实现数学推理提升17%、通用指令52%。
- 自动适应推理或通用任务，无需手动调参。
监督微调是适配大语言模型的核心方法，但全量数据训练成本高且收益递减。现有数据选择方法存在严重领域偏差：针对通用指令优化的方法在推理任务上表现差，反之亦然。我们发现，基模型与极小量指令微调后的校准模型之间的熵差具有规律性——熵差最低的样本在各领域均表现最优，但这一规律呈现领域自适应特征：推理任务偏好熵增（认知扩展），通用任务偏好熵减（认知压缩）。为此提出InstructDiff框架，通过预热校准、双向NLL过滤和熵排序，将熵差作为统一的领域自适应选择标准。大量实验表明，InstructDiff在数学推理上相比全数据训练提升17%相对性能，在通用指令任务上提升52%，且仅使用10%数据，优于已有基线。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is fundamental to adapting large language models, yet training on complete datasets incurs prohibitive costs with diminishing returns. Existing data selection methods suffer from severe domain specificity: techniques optimized for general instruction-following fail on reasoning tasks, and vice versa. We observe that measuring entropy differences between base models and minimally instruction-tuned calibrated models reveals a pattern -- samples with the lowest differential entropy consistently yield optimal performance across domains, yet this principle manifests domain-adaptively: reasoning tasks favor entropy increase (cognitive expansion), while general tasks favor entropy decrease (cognitive compression). We introduce InstructDiff, a unified framework that operationalizes differential entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropy-based ranking. Extensive experiments show that InstructDiff achieves 17\% relative improvement over full data training on mathematical reasoning and 52\% for general instruction-following, outperforming prior baselines while using only 10\% of the data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。