arXiv:2412.09538cs.LGstat.ML2024-12被引 32

提出轨迹敏感的数据影响评估方法,揭示训练中数据顺序的关键作用。

Capturing the Temporal Dependence of Training Data Influence

  • 定义轨迹特定的留一法影响,考虑数据顺序与模型优化路径。
  • 发现早期和后期数据对最终模型影响更大,存在明显影响阶段分化。
  • 提出数据值嵌入技术,高效近似计算,适合数据筛选与训练优化。

传统数据影响估计方法(如影响函数)假设学习算法对训练数据顺序不变,但现代基础模型使用随机算法和多阶段课程训练时,对数据顺序敏感,违背此假设。这导致影响函数无法回答关键问题:如何捕捉数据影响随优化轨迹的变化?为此,我们形式化了轨迹特定留一法(LOO)影响,量化在特定训练迭代中移除某数据点的影响,考虑数据实际出现顺序与模型参数演化路径。精确计算该影响存在巨大计算挑战。为此,我们提出数据值嵌入技术,通过构建包含数据与模型参数累积交互的数据嵌入,仅用一次点积即可高效近似LOO影响。该嵌入捕获数据顺序信息,揭示数据影响存在明显阶段特征:早期和后期数据对最终模型影响显著更大。这些洞察可指导数据选择时机,降低计算开销,为数据管理研究开辟新方向。

原文摘要 · Abstract (English)

Traditional data influence estimation methods, like influence function, assume that learning algorithms are permutation-invariant with respect to training data. However, modern training paradigms, especially for foundation models using stochastic algorithms and multi-stage curricula, are sensitive to data ordering, thus violating this assumption. This mismatch renders influence functions inadequate for answering a critical question in machine learning: How can we capture the dependence of data influence on the optimization trajectory during training? To address this gap, we formalize the concept of trajectory-specific leave-one-out (LOO) influence, which quantifies the impact of removing a data point from a specific iteration during training, accounting for the exact sequence of data encountered and the model's optimization trajectory. However, exactly evaluating the trajectory-specific LOO presents a significant computational challenge. To address this, we propose data value embedding, a novel technique enabling efficient approximation of trajectory-specific LOO. Specifically, we compute a training data embedding that encapsulates the cumulative interactions between data and the evolving model parameters. The LOO can then be efficiently approximated through a simple dot-product between the data value embedding and the gradient of the given test data. As data value embedding captures training data ordering, it offers valuable insights into model training dynamics. In particular, we uncover distinct phases of data influence, revealing that data points in the early and late stages of training exert a greater impact on the final model. These insights translate into actionable strategies for managing the computational overhead of data selection by strategically timing the selection process, potentially opening new avenues in data curation research.

数据影响训练轨迹数据嵌入模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。