提出新方法精准评估约束学习中数据的影响。
Directional Influence Function: Estimating Training Data Influence in Constrained Learning
- 基于变分不等式建模约束优化,显式纳入可行性要求
- 在约束线性回归和公平性约束的CNN上,准确预测删样后损失变化
- 适用于需要可解释性和鲁棒性的约束学习场景
随着约束学习日益普遍,模型需在公平性、安全性、鲁棒性及物理逻辑约束下训练。理解训练样本对模型参数的影响对可解释性和鲁棒性至关重要。经典影响函数(IF)通过局部敏感性分析估计样本贡献,但在约束环境下失效:数据扰动会同时改变目标函数和可行域,导致结果违反约束。为此,我们提出方向影响函数(DIF),将约束学习的最优性条件建模为变分不等式(VI),分析训练数据扰动对VI的影响。我们在约束线性回归上验证DIF,发现其能精确复现留一法重训结果,而传统IF与基于惩罚的IF存在显著偏差。进一步应用于公平性约束的CNN,DIF准确预测了删样后的测试损失变化,且与实际重训结果高度一致。结果表明,DIF是约束学习中高效可靠的样本归因工具。
原文摘要 · Abstract (English)
As constrained learning becomes increasingly common, models are trained under explicit feasibility requirements to enforce fairness, safety, robustness, regulariza- tion, and physics or logic constraints. Understanding how training samples in- fluence the model solution (e.g., learned parameters) is crucial for interpretability and robustness. The classical influence function (IF) estimates sample contribu- tions via local sensitivity analysis, measuring how the solution changes when a specific training sample is perturbed or removed. However, IF becomes unreli- able in constrained settings: data perturbations can reshape both the objective and the feasible region, leading to estimates that violate feasibility. In response, we propose the Directional Influence Function (DIF), a novel estimator that explicitly incorporates these constraints into influence estimation. DIF formulates the opti- mality conditions of constrained learning as a variational inequality (VI) and ana- lyzes how perturbing training data affects this VI. We validate DIF on constrained linear regression and demonstrate that it recovers leave-one-out retraining results, whereas IF and penalty-based IF exhibit significant bias. We further apply DIF to fairness-constrained CNNs, where DIF accurately predicts test loss changes under data removal and aligns closely with actual retraining. Our results establish DIF as an efficient and reliable tool for data attribution in constrained learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。