arXiv:2602.17835cs.LG2026-02被引 10

用更小模型高效模拟大模型梯度影响,提升数据筛选效果。

Influence-Preserving Proxies for Gradient-Based Data Selection in LLM Fine-tuning

  • 从目标大模型直接生成可保留影响信息的轻量代理模型
  • 1.5B代理在Qwen3-4B上表现优于1.7B现成代理,3B模型计算成本减半
  • 适合需要高效微调数据筛选的大模型研究者

监督微调(SFT)高度依赖于选择能提升下游性能的训练数据。基于梯度的数据筛选方法如TracIn和影响函数虽有效,但计算开销随模型参数量增长而急剧上升,难以应用于千亿参数级大语言模型(LLMs)。现有方案常采用现成的小模型作为代理,但其学习动态不明确、规模不可调,且无法与目标模型在梯度影响估计上对齐。为此,本文提出Iprox,一种两阶段框架:首先通过低秩压缩保留目标模型的影响信息,再通过梯度与输出对齐,构建灵活控制计算成本同时保持目标模型影响特性的代理模型。在多个大模型家族和任务上的实验表明,Iprox持续优于现成代理与基线方法。在Qwen3-4B上,1.5B的Iprox代理表现超越1.7B的现成代理;在Llama3.2上,相比完整3B模型,计算成本降低超50%的同时性能更优。结果证明,Iprox可有效生成保影响代理,使梯度驱动的数据筛选在大模型中更具可扩展性。

原文摘要 · Abstract (English)

Supervised fine-tuning (SFT) relies critically on selecting training data that most benefits a model's downstream performance. Gradient-based data selection methods such as TracIn and Influence Functions leverage influence to identify useful samples, but their computational cost scales poorly, making them impractical for multi-billion-parameter large language models (LLMs). A common alternative is to use off-the-shelf smaller models as proxies, but they remain suboptimal since their learning dynamics are unclear, their sizes cannot be flexibly adjusted, and they cannot be further aligned with the target model in terms of gradient-based influence estimation. To address these challenges, we introduce Iprox, a two-stage framework that derives influence-preserving proxies directly from the target model. It first applies a low-rank compression stage to preserve influence information of the target model, and then an aligning stage to align both model gradients and logits, thereby constructing proxies that flexibly control computational cost while retaining the target model's influence. Experimental results across diverse LLM families and evaluation tasks show that Iprox consistently outperforms off-the-shelf proxies and baseline methods. On Qwen3-4B, a 1.5B proxy constructed with Iprox achieves stronger performance than the larger 1.7B off-the-shelf proxy. Notably, on Llama3.2, Iprox achieves better performance than baselines while reducing computational cost by more than half relative to the full 3B model. These results show that Iprox provides effective influence-preserving proxies, making gradient-based data selection more scalable for LLMs.

数据筛选大模型微调代理模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。