arXiv:2605.22658cs.CVcs.LG2026-05中稿 · CVPR被引 2

用稀疏自编码器实现推理与视觉的可解释对齐,提升分割模型透明度。

SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation

论文配图:SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation
图 1 · 摘自论文原文
  • 通过稀疏自编码器将思维链与视觉特征映射到共享概念空间
  • 在五个基准上达到或超越现有最优性能,且概念质量与分割精度强相关
  • 适合关注模型可解释性与推理对齐的研究者和开发者

尽管大语言模型具备强大的组合推理能力,现有推理分割流程无法透明地将推理过程与视觉感知关联。当前方法如潜在查询对齐虽为端到端但为黑箱,而文本定位读出仅可读不可解释,常作为无约束的后处理步骤。为此,我们提出SegCompass,一种基于稀疏自编码器(SAE)的端到端模型,建立显式、可解释且可微的对齐路径。给定图像-指令对,先生成思维链(CoT)轨迹,再利用SAE将CoT与视觉标记映射至高维稀疏概念空间。查询码本从中选取显著概念,经槽映射空间定位为多槽热图,引导最终掩码解码器。整个模型联合训练,结合强化学习优化推理路径与标准分割监督。该SAE驱动接口提供“白盒”连接,远比潜在查询可追踪,也比文本读出更连贯。在五个挑战性基准上的大量实验表明,SegCompass达到或超越当前最优性能。关键的是,可视化与定量分析显示所学稀疏概念质量与最终掩码准确率高度相关,证实其优异表现源于增强且可检验的对齐机制。代码已开源。

原文摘要 · Abstract (English)

While large language models provide strong compositional reasoning, existing reasoning segmentation pipelines fail to transparently connect this reasoning to visual perception. Current methods, such as latent query alignment, are end-to-end yet opaque "black boxes". Conversely, textual localization readout is merely readable, not truly interpretable, often functioning as an unconstrained post-hoc step. To bridge this interpretability gap, we propose SegCompass, an end-to-end model that leverages a Sparse Autoencoder (SAE) to forge an explicit, interpretable, and differentiable alignment pathway. Given an image-instruction pair, SegCompass first generates a chain-of-thought (CoT) trace. The core of our method is an SAE that maps both the CoT and visual tokens into a shared, high-dimensional sparse concept space. A query codebook selects salient concepts from this space, which are then spatially grounded by a slot mapper into a multi-slot heatmap that guides the final mask decoder. The entire model is trained jointly, unifying reinforcement learning for the reasoning path with standard segmentation supervision. This SAE-driven interface provides a "white-box" connection that is significantly more traceable than latent queries and more coherent than textual readouts. Extensive experiments on five challenging benchmarks demonstrate that SegCompass matches or surpasses state-of-the-art performance. Crucially, our visual and quantitative analyses show a strong correlation between the quality of the learned sparse concepts and final mask accuracy, confirming that SegCompass achieves superior results through its enhanced and inspectable alignment. Code is available at https://github.com/ZhenyuLU-Heliodore/SegCompass.

可解释性稀疏自编码器推理分割视觉对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。