提出一种新型表示分析方法，可高效比较不同网络层的特征结构
Normalized Space Alignment: A Versatile Metric for Representation Analysis
- 通过对比点云间距离来衡量特征空间相似性
- 支持作为可微损失函数，提升模型训练稳定性
- 适用于结构保持、鲁棒性分析等多场景
我们提出一种针对神经网络表示的流形分析技术——归一化空间对齐（Normalized Space Alignment, NSA）。NSA 比较源自同一源且大小相同的两组点云之间的成对距离，即使它们维度不同亦可适用。NSA 可作为分析工具或可微损失函数，提供一种稳健的跨层、跨模型表示比较与对齐方法，满足相似性度量与神经网络损失函数的必要条件。我们展示了 NSA 的多功能性：作为表示空间分析指标、结构保持型损失函数以及鲁棒性分析工具。NSA 不仅计算高效，还能在小批量训练中近似全局结构差异，适用于多种神经网络训练范式。
原文摘要 · Abstract (English)
We introduce a manifold analysis technique for neural network representations. Normalized Space Alignment (NSA) compares pairwise distances between two point clouds derived from the same source and having the same size, while potentially possessing differing dimensionalities. NSA can act as both an analytical tool and a differentiable loss function, providing a robust means of comparing and aligning representations across different layers and models. It satisfies the criteria necessary for both a similarity metric and a neural network loss function. We showcase NSA's versatility by illustrating its utility as a representation space analysis metric, a structure-preserving loss function, and a robustness analysis tool. NSA is not only computationally efficient but it can also approximate the global structural discrepancy during mini-batching, facilitating its use in a wide variety of neural network training paradigms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。