提出低交互秩理论,统一解释多乘积双编码器的设计与局限。
Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads

- 基于交互谱定义函数复杂度,分解近似误差
- 样本复杂度由编码器复杂度之和决定而非乘积
- 白化可恢复可解释的语义轴,适用于视觉语言模型
多乘积双编码器将输入对的输出表示为各自编码的内积,独立出现在算子学习、二分匹配、对比视觉语言模型和检索等领域。然而,缺乏统一理论指导基本设计:应保留多少交互模式、如何归一化编码器、何时应避免该结构。本文引入低交互秩函数类,其内在复杂度由交互谱衡量。在该框架下,近似误差分解为谱截断项与编码器实现项;样本复杂度取决于两个编码器复杂度之和,而非乘积;基于谱衰减的可用性准则决定了架构能否成功。该框架揭示了核心可识别性问题:编码器仅在线性规范对称性下定义,导致学习坐标任意。我们证明归一化即规范固定,白化将交互模式确定至置换与符号,解释了对比维度不可解释性,并提供可重构解法。在合成核、算子学习和CLIP模型上的实验验证了理论预测:谱衰减速率符合预期,白化恢复真实模式,独立训练的CLIP模型间仅存在单一旋转关系,经白化去除后显现出可解释的概念轴。
原文摘要 · Abstract (English)
A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings. This architecture has been developed independently in operator learning, bipartite matching, contrastive vision-language models, retrieval, and other areas, yet no unified theory guides the basic design decisions: how many interaction modes to represent, how to normalize the encoders, and when the architecture should be avoided. We provide such a foundation by introducing the class of functions of low interaction rank, a class whose intrinsic complexity is measured by its interaction spectrum. Within this framework, approximation error decomposes into a spectral truncation term and an encoder-realization term; sample complexity is governed by the sum of the two encoder complexities rather than their product; and a usability criterion based on spectral decay determines when the architecture can succeed. The same framework exposes a central identifiability problem: the encoders are defined only up to a linear gauge symmetry that leaves the learned coordinates arbitrary. We show that normalization is gauge fixing and that whitening pins the interaction modes up to permutation and sign, thereby explaining the uninterpretability of contrastive dimensions and providing a constructive remedy. Experiments on synthetic kernels, operator learning, and CLIP models validate the theoretical predictions: spectral decay rates match the predicted scaling, whitening recovers the true modes, and independently trained CLIP models are related by a single rotation which, after removal by whitening, exposes interpretable concept axes. The code of this paper is provided at https://github.com/RS2002/Mul-Net .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。