揭示双编码器绑定能力的理论极限，指出其失败源于激励与结构设计。
The Limits of Binding in Dual Encoders
- 基于理想编码框架，证明绑定失败由可验证假设导致而非必然。
- 深度限制：嵌套深度每增1，交换边际按指数衰减，CLIP规模下仅支持单数字深度。
- 关键发现：当前部署模型绑定能力仅达理论上限的25%-35%，主因是训练激励不足。
双编码器模型如CLIP通过独立计算图像与文本向量的内积评分，难以实现角色绑定，常在区分'红色汽车和蓝色狗'与'蓝色汽车和红色狗'时表现接近随机。本文在Kang等人提出的理想编码框架下，首先证明相关公理可满足，因此所有不可能性皆源于附加的、可检验的假设。接着证明三类障碍：深度上，递归角色绑定的交换边际遵循精确公式 $m(D) = 2b^{-D}$，有限维版本在显式集中估计下成立；可解深度随维度对数增长，在CLIP规模下仅为个位数，对应普通语言嵌套深度。目标上，无架构约束的节流定理表明，对比目标对绑定的全部奖励受限于训练中将句式与其交换版本对比的速率，该速率在网页尺度下趋近零；完全反转绑定成本仅为此速率乘以平均绑定边际，模拟验证成立。几何上，提出紧致平滑-绑定边界：两交换相关句式越靠近共同改写锚点，绑定边际越小，且存在精确常数。在18个已部署文本编码器上测量纯文本诊断，所有模型均位于理论上限的25%-35%之间，诱导的每项上限与SugarCrepe子集难度高度相关（r=0.99）。因此，当前双编码器的绑定失败并非维度或平滑度限制，而是激励机制与代码结构限制，其深度上限在修正后依然存在。
原文摘要 · Abstract (English)
Dual-encoder models such as CLIP score an image-caption pair by a single inner product of two independently computed unit vectors, and fail at binding, often scoring near chance when asked to distinguish "a red car and a blue dog" from "a blue car and a red dog". We give a mathematical account of when this failure is necessary and when it is contingent. Working within the ideal-encoder framework proposed by Kang et al., we first show the relevant axioms are satisfiable, so every impossibility must enter through an added, checkable hypothesis. We then prove three such obstructions. Depth: for recursive role-binding codes the swap margin obeys an exact law $m(D) = 2b^{-D}$ in the nesting depth D, with a finite-dimension version holding up to one explicitly flagged concentration estimate; the resolvable depth grows only logarithmically in the dimension and is single-digit at CLIP scale, the nesting depth of ordinary language. Objective: architecture-free throttle theorems showing that the contrastive objective's entire reward for binding is bounded by the rate at which training contrasts a caption against its own swap, a rate that vanishes at web scale, and that exactly reversed binding costs only that rate times the mean binding margin; both are verified in simulation. Geometry: a tight smoothness-binding frontier: the closer the two swap-related captions must embed to a shared paraphrase anchor, the smaller the binding margin can be, with an exact constant. Measuring its text-only diagnostic across 18 deployed text encoders, every model sits at roughly 25-35% of its ceiling, and the induced per-item ceiling tracks SugarCrepe's subset difficulty at r = 0.99. Binding failure in deployed dual encoders is thus not a dimension or smoothness limit today, but an incentive and code-structure limit, with a proved depth ceiling that remains once those are fixed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。