arXiv:2605.22679cs.CVcs.LG2026-05

不增加维度就能拆解视觉语言模型的语义,让抽象表示变可读。

Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models

论文配图:Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
图 1 · 摘自论文原文
  • 通过自适应旋转构建可逆变换,用稀疏瓶颈提取语义坐标。
  • 在CLIP和BLIP上实现高保真重建与强可解释性,优于传统方法。
  • 适合想理解模型内部机制的研究者或开发者使用。

视觉语言模型虽能学习强大多模态嵌入,但其内部语义仍不透明。尽管稀疏自编码器(SAEs)可提取可解释特征,却需扩展表示维度,破坏原始几何结构并引入冗余。我们提出CEDAR(基于自适应旋转的概念嵌入解纠缠),一种后处理方法,可在不增加维度的情况下揭示预训练嵌入的组合结构。通过学习带top-k稀疏瓶颈的可逆变换,CEDAR将语义信息集中于轴对齐的解纠缠坐标。在类似CLIP的架构中,各坐标可对应文本概念;对于生成模型如BLIP,可解码为自然语言描述。实验表明,CEDAR在重建与稀疏性之间达到良好权衡,且解释更符合人类感知。结果表明,视觉语言表示中的表观纠缠可通过合适的基变换解决,无需过完备扩展。

原文摘要 · Abstract (English)

Vision-language models learn powerful multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract interpretable features, they rely on expanding the representation dimension, which compromises the original geometry and introduces redundancy. We introduce CEDAR (Conceptual Embedding Disentanglement via Adaptive Rotation), a post-hoc method that reveals the compositional structure of pretrained embeddings without increasing dimensionality. By learning an invertible transformation with a top-$k$ sparsity bottleneck, CEDAR concentrates semantic information into axis-aligned disentangled coordinates. In CLIP-like architecture, individual coordinates can be interpreted with textual concepts, while for generative models such as BLIP, they can be decoded into natural language descriptions. Experiments demonstrate that CEDAR achieves a competitive reconstruction-sparsity trade-off while producing explanations that are more interpretable and better aligned with human perception. Our results suggest that the apparent entanglement in vision-language representations can be resolved through a suitable change of basis, eliminating the need for overcomplete expansions.

视觉语言嵌入解纠缠可解释性稀疏性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。