动作后训练让视觉语言模型深度理解能力下降，尤其在深层逐渐崩溃。
From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability
- 通过对比基线模型与动作后训练模型，分析各层深度解码能力变化。
- 后训练模型在深层的深度解码能力显著下降，形成‘悬崖’效应。
- 问题根源是深层MLP模块干扰，修复它可恢复大部分解码性能。
构建视觉-语言-动作模型（VLA）时的动作后训练会如何影响视觉语言模型（VLM）的空间理解能力？我们探测了空间几何理解的基础能力——深度感知，在一组权重匹配的开源基线模型（Molmo2-ER）与动作后训练模型（MolmoAct2-LIBERO）的每一解码层中的表现。结果发现，无论哪一层，VLA的深度解码能力均弱于基线模型，形成持续存在的“地板”差距；更关键的是，这种退化不均匀：基线模型的深度解码能力随层数加深而提升，而VLA却在深层急剧下降，形成额外的“悬崖”现象。通过因果消融实验发现，该悬崖效应主要源于深层MLP模块的干扰——移除深层MLP后，终端解码能力恢复近八成，而注意力模块的消融或在基线模型中执行相同操作则无类似效果。模块级分解揭示：基线模型将深度信息集中存储在累积的MLP写入中，而动作后训练破坏了这一机制，导致深层深度解码能力崩溃。
原文摘要 · Abstract (English)
How much of a vision-language model's (VLM) spatial understanding remains after the action post-training process of building a vision-language-action model (VLA)? We probe depth perception, a primitive of spatiogeometric understanding, from every decoder layer of a weight-matched open-source base VLM/VLA pair: Molmo2-ER and MolmoAct2-LIBERO. First, the VLA decodes depth worse at every layer, a persistent gap we call the floor. Second, the degradation is not uniform: while the base VLM's depth decodability improves through its final layers, the VLA's collapses, an additional late-layer drop we call the cliff. We causally localize the cliff to late-layer MLP interference: ablating the late-layer MLP writes recovers the majority of the terminal decodability cliff, while matched attention ablations and the same intervention in the weight-matched base VLM produce no comparable recovery. A module-level decomposition explains this dissociation: the base VLM carries depth most accessibly in accumulated MLP writes, whereas action post-training collapses depth decodability in the late accumulated writes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。