神经网络天然偏好距离表示，可提升模型性能。
Neural Networks Learn Distance Metrics
- 用几何框架分析神经网络的远近表示机制
- 实验表明距离表示比强度表示性能更优
- 适合关注模型表征设计的研究者
神经网络可能自然倾向于基于距离的表示方式，即激活值越小，表示与学习到的原型越接近。这与依赖激活值大小的强度表示方法形成对比。为验证该假设，我们对六种基于MNIST的架构变体进行了实验，强制其学习距离或强度表示。结果表明，底层表示方式显著影响模型性能。我们提出一种新的几何框架来解释这些发现，并引入基于马氏距离方程的OffsetL2新架构以进一步验证该框架。本工作强调了在神经网络设计中考虑距离学习的重要性。
原文摘要 · Abstract (English)
Neural networks may naturally favor distance-based representations, where smaller activations indicate closer proximity to learned prototypes. This contrasts with intensity-based approaches, which rely on activation magnitudes. To test this hypothesis, we conducted experiments with six MNIST architectural variants constrained to learn either distance or intensity representations. Our results reveal that the underlying representation affects model performance. We develop a novel geometric framework that explains these findings and introduce OffsetL2, a new architecture based on Mahalanobis distance equations, to further validate this framework. This work highlights the importance of considering distance-based learning in neural network design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。