arXiv:2606.06100cs.CV2026-06被引 1

用双曲空间建模视觉关系,提升模型组合推理能力。

HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning

论文配图:HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning
图 1 · 摘自论文原文
  • 在双曲面上构建连续视觉关系图,避免离散标签干扰。
  • 训练时提升生成式VQA准确率至61.03%,推理时使组合评分提高6.25个百分点。
  • 适合关注视觉关系建模与组合推理的开发者和研究者。

视觉语言模型在需要理解物体间关系的组合推理任务上表现不佳。现有方法通过引入外部场景图生成器提供的三元组<对象, 关系, 对象>来改善,但发现离散文本标签与连续视觉特征冲突,导致GQA准确率从60.38%下降至58.86%。本文提出HyperVis,完全绕过场景图语义瓶颈:从N个无类别区域提议出发,通过空间偏置交叉注意力计算密集的O(N²)视觉关系张量,投影至Lorentz双曲面,并利用交并比驱动的蕴含锥和外角排斥实现层次结构。实验表明,HyperVis在两方面起作用:(1)作为训练正则化器,双曲关系损失优化了LoRA表示,使生成式VQA(GQA)达到61.03%,优于无关系损失的57.21%;(2)作为推理时的关系编码器,双曲前缀标记将SugarCrepe得分提升至79.94%,较基线+6.25个百分点。学习到的曲率κ=4.0,远高于以往接近零的值,表明连续视觉特征确实需要强曲率空间。欧氏空间对照实验显示,关系管道在平坦空间中正则化效果类似(GQA 60.81%),但组合性优势专属于双曲空间(SugarCrepe +4.58pp),且蕴含损失在欧氏训练中约高6倍。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) struggle with compositional reasoning that requires understanding inter-object relationships. A natural remedy is to inject explicit scene graph triplets $\langle s, p, o \rangle$ from an off-the-shelf scene graph generator (SGG), but we show this backfires: discrete text labels collide with the continuous visual modality, degrading GQA accuracy from 60.38\% to 58.86\%. We propose \textbf{HyperVis}, which bypasses the SGG semantic bottleneck entirely. From $N$ class-agnostic region proposals, we compute a dense $O(N^2)$ visual relation tensor via spatially-biased cross-attention, project it onto a Lorentz hyperboloid, and enforce hierarchy through spatial physics, namely IoA-driven entailment cones and exterior-angle repulsion. We discover that HyperVis contributes in two complementary ways: (1) as a \emph{training-time regularizer}, the hyperbolic relational losses shape LoRA representations that improve generative VQA (GQA 61.03\% vs.\ 57.21\% for LoRA fine-tuning without relational losses, recovering and surpassing the baseline); and (2) as an \emph{inference-time relational encoder}, hyperbolic prefix tokens boost discriminative compositional scoring (SugarCrepe 79.94\%, $+$6.25pp over baseline). The learned curvature stabilises at $κ{=}4.0$, an order of magnitude above prior hyperbolic VLMs where $κ$ typically collapses toward zero, indicating that continuous visual features genuinely require the exponential volume of strongly curved space. A controlled Euclidean ablation confirms this decomposition: the relational pipeline regularises LoRA comparably in flat space (GQA 60.81\%), but the compositionality gain is specifically hyperbolic (SugarCrepe $+$4.58pp over Euclidean), with entailment loss ${\sim}6{\times}$ higher in Euclidean training. Codes are available at TBA.

视觉关系双曲几何组合推理VLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。