提出新框架,量化跨模态对齐的难易程度与非传递性。
Sheaf-Laplacian Obstruction and Projection Hardness for Cross-Modal Compatibility on a Modality-Independent Site
- 基于局部邻域定义投影难度与层流拉普拉斯阻碍量。
- 实证显示分步对齐比直接映射效率更高,宽度差距达二次方级。
- 适用于多模态模型设计者,尤其关注对齐复杂性的研究者。
跨模态表示在对齐难易程度上存在差异,且兼容性通常不具有传递性:通过中间模态对齐可能比直接映射更简单。本文引入一种固定邻域站点上的参考形式化框架,定义两个定向不变量。投影难度 $H_{a\to b}(\varepsilon)$ 表示在嵌套的Lipschitz控制族中达到误差 $\varepsilon$ 所需的最小复杂度;对于给定的局部投影族,层流拉普拉斯阻碍量 $C_{a\to b}(\varepsilon)$ 是使参数局部拟合达到相同误差所需的最小参数变化量。在恒等约束下,阻碍量等于向量值参数场的图Dirichlet能量;层流形式将零能量解释为成功拼接,并可推广至边依赖传输与异质参数空间。本文揭示了阻碍量与站点谱间隙及全局映射误差过剩的关系,并构造了ReLU实例,展示非传递兼容性及分步与直接映射宽度间的二次分离。受控合成校准验证了预测的难度分离、由割引发的阻碍缩放以及对固定站点图的敏感性。
原文摘要 · Abstract (English)
Cross-modal representations vary in how easily they can be aligned, and compatibility is generally non-transitive: two modalities may align through an intermediate modality at lower complexity than through a direct map. We introduce a reference formalism that evaluates all modalities on a fixed neighborhood site and defines two directed invariants. Projection hardness \(H_{a\to b}(\varepsilon)\) is the minimum complexity within a nested Lipschitz-controlled family required to reach error \(\varepsilon\). For a declared local projection family, sheaf-Laplacian obstruction \(C_{a\to b}(\varepsilon)\) is the minimum variation of locally fitted projection parameters required to reach the same error. Under identity restrictions, obstruction is the graph Dirichlet energy of a vector-valued parameter field; the sheaf formulation identifies zero energy with successful gluing and extends to edge-dependent transports and heterogeneous parameter spaces. We relate obstruction to the site spectral gap and excess global-map error, and construct ReLU examples showing non-transitive compatibility and a quadratic separation between staged and direct width. Controlled synthetic calibrations recover the predicted hardness separation, cut-induced obstruction scaling, and sensitivity to the fixed site graph.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。