通过抑制梯度冲突提升大模型训练效率，仅用10%数据达到全量训练效果
SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training
- 基于信息冲突分析设计惩罚机制，动态优化数据选择
- 在8个基准上用10%数据达到或超过全量训练性能
- 支持早停和代理模型，显著降低训练成本
指令微调中的信息驱动数据选择具有吸引力：最大化Fisher信息矩阵的行列式可形成单调子模目标，使贪心算法在基数预算下获得(1-1/e)近似。然而实践中，缓解梯度冲突、即单样本梯度间的不一致，是导致边际信息增益衰减缓慢的关键因素，阻碍了信息损失的显著减少。我们通过ε-分解形式化这一现象，将子模性偏离量化为冲突统计的函数，得出随冲突减小而收紧的数据依赖近似因子。基于此分析，提出SPICE——一种兼顾信息最大化与冲突惩罚的感知式选择器，支持早停和代理模型以提高效率。实验表明，SPICE选取的子集对数行列式信息高于原有标准，且这些信息增益转化为实际性能提升：在LLaMA2-7B和Qwen2-7B上，使用10%数据即可匹配或超越8种方法（包括全数据微调），实现显著更低的训练成本。
原文摘要 · Abstract (English)
Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling greedy algorithms to achieve a $(1-1/e)$ approximation under a cardinality budget. In practice, however, we identify alleviating gradient conflicts, misalignment between per-sample gradients, is a key factor that slows down the decay of marginal log-determinant information gains, thereby preventing significant loss of information. We formalize this via an $\varepsilon$-decomposition that quantifies the deviation from ideal submodularity as a function of conflict statistics, yielding data-dependent approximation factors that tighten as conflicts diminish. Guided by this analysis, we propose SPICE, a conflict-aware selector that maximizes information while penalizing misalignment, and that supports early stopping and proxy models for efficiency. Empirically, SPICE selects subsets with higher log-determinant information than original criteria, and these informational gains translate into performance improvements: across 8 benchmarks with LLaMA2-7B and Qwen2-7B, SPICE uses only 10% of the data, yet matches or exceeds 6 methods including full-data tuning. This achieves performance improvements with substantially lower training cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。