arXiv:2605.15675cs.LGcs.AI2026-05

提出考虑样本间互动的影响力函数,更准确识别群体影响。

Interaction-Aware Influence Functions for Group Attribution

  • 引入二阶展开,加入样本对间的相互作用项
  • 在六组数据-模型组合中显著优于传统方法
  • 适合需要精准筛选训练数据的场景

影响力函数用于估算移除某个训练样本对目标函数(如保留损失)的影响。标准做法是将组内各成员的个体影响力相加,但无法捕捉样本之间的协同或冗余关系。本文提出一种交互感知的影响力函数,通过在训练参数处对目标函数进行二阶展开,得到一个包含成对交互项的估计器,该交互项反映两个样本对目标影响的一致性。在六个数据集-模型组合(涵盖逻辑回归、MLP和ResNet-9)上,该方法在留组重训练任务中表现显著优于一阶影响力。此外,在使用Llama-3.1-8B进行指令微调时,作为贪心选择规则,其在七个下游任务中的五个上超越基于影响力和表示相似性的基线,而标准影响力选择在此情境下甚至不如随机选择。

原文摘要 · Abstract (English)

Influence functions approximate how removing a training example changes a quantity of interest, called the target function, such as a held-out loss. To estimate the influence of a group of examples, the standard practice is to sum the individual influences of its members. However, this sum does not capture how examples jointly affect the target: a pair of examples may be redundant or complementary, but the sum cannot distinguish these cases. We propose an interaction-aware influence function that characterizes how interactions between examples influence the target. By expanding the target to second order around the trained parameters, we obtain an estimator that augments the standard sum with a pairwise interaction term that captures the alignment between two examples' effects on the target. We empirically evaluate our estimator in two settings. First, on six dataset-model pairs spanning logistic regression, MLPs, and ResNet-9, our estimator tracks leave-group-out retraining substantially better than first-order influence across all settings. Second, when used as a greedy selection rule for instruction-tuning data on Llama-3.1-8B, it beats prior influence-based and representation-similarity baselines on five of seven downstream tasks, in a regime where standard influence-based selection underperforms random selection.

影响力分析数据筛选机器学习解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。