让小模型学会思考，通过动态选择记忆与探索来提升性能。
Empowering Small VLMs to Think with Dynamic Memorization and Exploration
- 动态切换监督微调与强化学习，平衡记忆与探索
- 在多个专有任务上显著提升小模型表现
- 适合资源有限但需可靠推理的小模型应用
小规模视觉语言模型（SVLM）在专有任务中极具潜力。赋予其推理能力是提升其性能与可靠性的重要一步。然而，现有训练范式如监督微调（SFT）和可验证奖励强化学习（RLVR），对基础模型容量要求过高，超出了SVLM的能力范围，导致直接应用无法有效引入推理能力。一种自然思路是结合SFT与RLVR，利用互补性降低对模型容量的依赖。但核心挑战在于处理固有的权衡：过度依赖SFT会导致模型伪推理痕迹的机械记忆，而过度强调RLVR则引发不稳定探索（即优势坍缩）。为此，我们提出DyME，一种新型训练范式，在每个优化步骤动态选择基于SFT的记忆或基于RLVR的探索。通过确保每次更新都参与权衡，DyME成为稳定且独立有效的策略。此外，我们还引入协同的视觉监督机制（含视觉检查器与修正器），在优化过程中注入动态增强的图像引导。跨多个领域的实验表明，DyME能持续保持平衡，显著提升专有任务性能。结果确立了DyME作为赋能小模型可靠推理能力的实用高效方案。
原文摘要 · Abstract (English)
Small-scale Vision-Language Models (SVLMs) are exceptionally well-suited for proprietary tasks. Equipping them with thinking capabilities is a critical step to enhance their performance and reliability in these specific domains. However, existing training paradigms, including Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Reward (RLVR), impose substantial demands on the base VLM, exceeding the capacity of SVLMs. Consequently, directly applying these paradigms to SVLMs fails to instill the desired thinking abilities. A natural solution is to combine SFT and RLVR, leveraging their complementarity to reduce the dependence on model capacity. Yet the core challenge lies in managing the inherent trade-off: excessive reliance on SFT can force the model to memorize pseudo thinking traces, while over-emphasizing RLVR can lead to unstable exploration (i.e., advantage collapse). To address this, we propose DyME, a novel training paradigm that Dynamically selects between Memorization (via SFT) and Exploration (via RLVR) at each optimization step. By ensuring that every update contributes to the trade-off, DyME serves as a robust, standalone strategy that stabilizes SVLM learning. Complementing this paradigm, we further introduce a synergistic Visual Supervision mechanism (comprising a visual checker and refiner) designed to inject dynamically enhanced, image-grounded guidance during optimization. Extensive experiments across diverse domains demonstrate that DyME consistently achieves this balance, and thus delivers substantial performance improvements on specialized tasks. These results establish DyME as a practical and effective solution for empowering SVLMs with reliable thinking capabilities. GitHub: https://github.com/HKUST-LongGroup/DyME
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。