通过递归重用参数，让大模型在不增大体积的情况下更高效、更灵活。
Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models
- 用递归连接器融合多层特征，保持视觉与语言模态差异。
- 递归损失确保每轮迭代性能提升，深度增加结果更好。
- 适合资源受限设备，也支持算力充足时持续优化输出。
大型多模态模型在视觉-语言任务中表现卓越，但其庞大参数量在训练和推理中常未被充分利用。本文提出递归式Transformer架构RecursiveVLM，通过递归精炼实现更强的多模态表征，且不增加模型规模。核心创新包括：(i) 递归连接器，通过融合中间层隐藏状态并应用模态特定投影，对齐不同递归步骤的特征，尊重视觉与语言标记的统计差异；(ii) 单调递归损失，监督每一步递归，保证性能随递归深度单调提升。该设计将递归变为按需精炼机制：在资源受限设备上以少量循环获得强结果，在算力充足时逐步提升输出质量。实验表明，相较标准Transformer提升+3%，比原始递归基线提升+7%，证明战略性递归是实现高效、可部署自适应多模态模型的有效路径。
原文摘要 · Abstract (English)
Large Multimodal Models (LMMs) have achieved remarkable success in vision-language tasks, yet their vast parameter counts are often underutilized during both training and inference. In this work, we embrace the idea of looping back to move forward: reusing model parameters through recursive refinement to extract stronger multimodal representations without increasing model size. We propose RecursiveVLM, a recursive Transformer architecture tailored for LMMs. Two key innovations enable effective looping: (i) a Recursive Connector that aligns features across recursion steps by fusing intermediate-layer hidden states and applying modality-specific projections, respecting the distinct statistical structures of vision and language tokens; (ii) a Monotonic Recursion Loss that supervises every step and guarantees performance improves monotonically with recursion depth. This design transforms recursion into an on-demand refinement mechanism: delivering strong results with few loops on resource-constrained devices and progressively improving outputs when more computation resources are available. Experiments show consistent gains of +3% over standard Transformers and +7% over vanilla recursive baselines, demonstrating that strategic looping is a powerful path toward efficient, deployment-adaptive LMMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。