用梯度信噪比筛选指令数据，提升训练效率与模型质量。
Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning
- 通过梯度信噪比衡量样本价值，动态捕捉模型不确定性。
- 在相同算力下收敛更快，优于随机采样和主流基线方法。
- 无需额外标注，适合大规模指令微调场景使用。
指令微调是适配大语言模型的标准范式，但现代指令数据集规模大、噪声多且冗余，全量微调成本高且常不必要。现有数据筛选方法要么构建昂贵的梯度数据存储，要么依赖弱代理生成静态分数，普遍忽略模型演化中的不确定性，错失了大模型可解释性的重要来源。本文提出GRADFILTERING，一种无目标依赖、感知不确定性的数据筛选框架，利用小型GPT-2代理结合LoRA集成，将每个样本的梯度聚合为梯度信噪比（G-SNR）作为评估指标。实验表明，该方法在多数基于大模型评分的评测及人工评估中表现匹配或超越随机子集与强基线。此外，在相同计算预算下，经GRADFILTERING筛选的数据子集收敛速度更快，体现出不确定性感知评分的优势。
原文摘要 · Abstract (English)
Instruction tuning is a standard paradigm for adapting large language models (LLMs), but modern instruction datasets are large, noisy, and redundant, making full-data fine-tuning costly and often unnecessary. Existing data selection methods either build expensive gradient datastores or assign static scores from a weak proxy, largely ignoring evolving uncertainty, and thus missing a key source of LLM interpretability. We propose GRADFILTERING, an objective-agnostic, uncertainty-aware data selection framework that utilizes a small GPT-2 proxy with a LoRA ensemble and aggregates per-example gradients into a Gradient Signal-to-Noise Ratio (G-SNR) utility. Our method matches or surpasses random subsets and strong baselines in most LLM-as-a-judge evaluations as well as in human assessment. Moreover, GRADFILTERING-selected subsets converge faster than competitive filters under the same compute budget, reflecting the benefit of uncertainty-aware scoring.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。