揭示稀疏Sinkhorn层中支持图如何控制梯度传播，给出设计准则。
Support Topology and Gradient Mixing in Sinkhorn Layers
- 构建固定支持的微分传输计算框架，分析梯度传播机制。
- 证明仅当列间两跳重叠时，才能保证全局收缩性，否则收缩系数趋近1。
- 适用于需可微传输的模型设计，如注意力机制改进与高效推理。
稀疏Sinkhorn层通过固定支持图限制令牌间的传输。本文建立固定支持微积分体系，表明每个行列环在列势扰动上诱导一个行随机算子（模常数）。其转置可传播零质量反向传播余切。有限环算子使用两个不同半步传输方案；在平衡固定点退化为单一步长的两步行走。推导出相应的评分项和边缘源项，并利用Dobrushin收缩与极小化理论，界定了同质及源驱动尾部余切。主要结果刻画了支持结构与边缘分布何时能统一保证一步收缩：运输多面体的所有可行面上必须具有成对两跳列重叠。否则，存在特定评分方向使收缩系数任意接近1。将分析扩展至有序支持调度，推导出分区热浴层、坐标扫描、强制共享质量及寄存器增强支持的验证证书。这些结果为可微传输层的支持设计提供数学标准，但仅限于固定支持商梯度分量的保证。
原文摘要 · Abstract (English)
Sparse Sinkhorn layers use a fixed support graph to restrict transport between tokens. How does this graph control gradient propagation through the scaling iterations. We develop a fixed-support calculus showing that each row-column cycle induces a row-stochastic operator on column-potential perturbations modulo constants. Its transpose propagates zero-mass reverse-mode cotangents. The finite-cycle operator uses two distinct half-step transport plans; at a balanced fixed point it reduces to a two-step walk determined by a single plan. We derive the accompanying score and marginal source terms and use Dobrushin contraction and minorization to bound homogeneous and source-driven tail cotangents. Our main result characterizes when support and marginals guarantee one-step contraction uniformly over finite scores: every feasible face of the transportation polytope must have pairwise two-hop column overlap. Otherwise, suitable score directions make the contraction coefficient arbitrarily close to one. We extend this analysis to ordered support schedules and derive certificates for partition heat-bath layers, coordinate sweeps, forced shared mass, and register-augmented supports. These results provide mathematical criteria for support design in differentiable transport layers, with guarantees restricted to the fixed-support quotient-gradient component.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。