通过级联稀疏自编码器，让多模态大模型学会分层视觉概念。
Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMs
- 用第二层SAE直接学习第一层解码器权重，实现概念的层级抽象。
- 在多个数据集上提升概念层次一致性，优于现有最先进方法。
- 适合研究模型可解释性或做概念控制的开发者使用。
多模态大语言模型（MLLMs）在视觉语言任务中表现优异，但其内部视觉表征难以解释。稀疏自编码器（SAEs）能将密集激活分解为稀疏可解释特征，但现有架构多恢复扁平特征字典，不擅长显式组织多层次概念。本文提出级联稀疏自编码器（CSAEs），用于学习MLLM中的分层视觉概念。CSAEs不采用嵌套或堆叠方式，而是将第一层SAE的解码器权重作为输入，训练第二层SAE进行更高层次抽象，从而学习“概念的概念”。该设计避免了嵌套带来的共享前缀耦合、套娃式层次和简单堆叠的瓶颈问题。在Qwen3-VL、Gemma-3和LLaVA等多个视觉数据集上的实验表明，CSAEs在层次概念连贯性方面显著优于当前最先进的SAE基线。概念引导实验进一步证明，所学概念组支持对MLLM输出的有效群体干预。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representations remain difficult to interpret. Sparse Autoencoders (SAEs) provide a scalable way to decompose dense model activations into sparse, interpretable features. However, existing SAE architectures primarily recover flat feature dictionaries and are less suited for explicit multi-level concept organization. In this paper, we introduce cascaded sparse autoencoders (CSAEs) for learning hierarchical visual concepts in MLLMs. Rather than nesting or stacking SAE sparse activation codes, CSAEs train a second-level SAE directly on the decoder weights of the first-level SAE, treating learned low-level feature directions as inputs for higher-level abstraction. This design enables CSAEs to learn "concepts of concepts" while avoiding drawbacks from the shared-prefix coupling of nesting, Matryoshka-style hierarchies and the bottlenecks of naively stacked SAEs. Experiments across Qwen3-VL, Gemma-3, and LLaVA on multiple visual datasets show that CSAEs improve interpretability in terms of hierarchical concept coherence over state-of-the-art SAE baselines. Results on concept steering further demonstrate that the learned concept groups support effective group-level interventions in MLLM outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。