用更聪明的采样方法，快速准确地找出训练数据对模型的影响。
Faithful and Fast Influence Function via Advanced Sampling
- 基于特征和预测值分布设计新采样策略，提升代表性。
- 计算时间减少30.1%，内存降低42.2%，或F1得分提升2.5%。
- 适合需要高效解释模型行为的研究者与工程师。
如何解释训练数据对黑箱模型的影响？影响函数（IFs）通过梯度和海森矩阵提供事后解释，但计算全量数据的海森矩阵成本高昂，需可行替代方案。常用随机采样小样本虽快，但因样本配置方差大，导致估计结果不一致。为此，我们提出两种基于特征和逻辑值（logits）分布的先进采样技术，通过考虑数据的随机分布特性，选取小而具代表性的子集，显著提升影响函数估计精度。在类别移除实验中验证：以F1分数衡量模型遗忘被移除类别的能力，同时保持其余类别推理一致性。结果表明，本方法相较基线可节省30.1%计算时间、降低42.2%内存使用，或使F1得分提升2.5%。
原文摘要 · Abstract (English)
How can we explain the influence of training data on black-box models? Influence functions (IFs) offer a post-hoc solution by utilizing gradients and Hessians. However, computing the Hessian for an entire dataset is resource-intensive, necessitating a feasible alternative. A common approach involves randomly sampling a small subset of the training data, but this method often results in highly inconsistent IF estimates due to the high variance in sample configurations. To address this, we propose two advanced sampling techniques based on features and logits. These samplers select a small yet representative subset of the entire dataset by considering the stochastic distribution of features or logits, thereby enhancing the accuracy of IF estimations. We validate our approach through class removal experiments, a typical application of IFs, using the F1-score to measure how effectively the model forgets the removed class while maintaining inference consistency on the remaining classes. Our method reduces computation time by 30.1% and memory usage by 42.2%, or improves the F1-score by 2.5% compared to the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。