神经网络实际依赖距离而非强度来做出判断。
Neural Networks Use Distance Metrics
- 通过扰动内部激活的距离与强度，发现模型对距离敏感
- 小距离扰动严重影响性能，大强度扰动影响较小
- 适合研究模型决策机制的学者阅读
我们提供了实证证据，表明使用ReLU和绝对值激活函数的神经网络会学习基于距离的表征。在训练好的模型中，我们独立操纵内部激活的距离和强度特性，发现两种架构对微小的距离扰动极为敏感，而对较大的强度扰动则保持鲁棒性。这些发现挑战了当前将神经网络激活视为强度主导的主流解释，并为理解其学习与决策过程提供了新视角。
原文摘要 · Abstract (English)
We present empirical evidence that neural networks with ReLU and Absolute Value activations learn distance-based representations. We independently manipulate both distance and intensity properties of internal activations in trained models, finding that both architectures are highly sensitive to small distance-based perturbations while maintaining robust performance under large intensity-based perturbations. These findings challenge the prevailing intensity-based interpretation of neural network activations and offer new insights into their learning and decision-making processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。