arXiv:2510.03361cs.CVcs.AI2025-10

让模型自动生成预测依据,直接关联训练样本。

Provenance Networks: End-to-End Exemplar-Based Explainability

  • 预测时自动引用最相关的训练样本作为解释
  • 可检测数据异常、验证输入是否在训练集中
  • 适合需要透明可信的AI应用场景

我们提出证明溯源网络(Provenance Networks),一种新型神经网络模型,旨在实现端到端、以训练数据驱动的可解释性。与传统事后解释方法不同,该模型在正常运行过程中即学习将每个预测结果与其支持的训练样本直接关联,将可解释性嵌入架构本身。其机制类似于学习后的KNN,每个输出由特征空间中相关性加权的具体示例支撑。该方法有助于系统研究记忆与泛化之间的权衡,验证输入是否出现在训练集中,辅助识别错误标注或异常数据点,提升对输入扰动的鲁棒性,并支持识别生成新数据点的相似输入。通过联合优化主任务与可解释性目标,该模型提供传统深度网络无法获得的行为洞察。尽管引入额外计算开销且目前仅适用于中等规模数据集,但其为现有可解释性技术提供了互补方案,尤其应对现代深度学习中的模型不透明、幻觉和数据贡献归属难题,从而提升神经模型的透明度、鲁棒性和可信度。

原文摘要 · Abstract (English)

We introduce provenance networks, a novel class of neural models designed to provide end-to-end, training-data-driven explainability. Unlike conventional post-hoc methods, provenance networks learn to link each prediction directly to its supporting training examples as part of the model's normal operation, embedding interpretability into the architecture itself. Conceptually, the model operates similarly to a learned KNN, where each output is justified by concrete exemplars weighted by relevance in the feature space. This approach facilitates systematic investigations of the trade-off between memorization and generalization, enables verification of whether a given input was included in the training set, aids in the detection of mislabeled or anomalous data points, enhances resilience to input perturbations, and supports the identification of similar inputs contributing to the generation of a new data point. By jointly optimizing the primary task and the explainability objective, provenance networks offer insights into model behavior that traditional deep networks cannot provide. While the model introduces additional computational cost and currently scales to moderately sized datasets, it provides a complementary approach to existing explainability techniques. In particular, it addresses critical challenges in modern deep learning, including model opaqueness, hallucination, and the assignment of credit to data contributors, thereby improving transparency, robustness, and trustworthiness in neural models.

可解释性神经网络数据溯源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。