通过锚点演化提升多模态大模型推理能力,解决幻觉与性能瓶颈问题。
AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolution

- 用真实数据锚点定位模型缺陷,生成高质量训练数据。
- 先加框架监督再逐步去除,实现推理能力内化,提升10.3%性能。
- 适合追求高可靠推理的多模态模型研究者使用。
通过监督微调(SFT)和强化学习(RL)进行后训练,对提升多模态大语言模型(MLLM)的推理能力至关重要,但现有方法常因静态数据限制而陷入性能瓶颈。尽管当前方法采用自反思或自演化来突破边界,仍受低质量合成数据导致的认知漂移与幻觉推理路径影响。为此,我们提出锚点演化(AnE)新范式,结合真值锚点数据筛选与模型演化,实现可信且稳定的性能提升。具体地,提出真值锚点扩展(Truth Anchor Expansion),通过轨迹回放定位模型失效边界,并利用真实数据库检索高保真锚点,确保数据质量;随后引入支架剥离机制(Scaffold-Stripping Mechanism),先以支架增强监督缓解直接微调的学习复杂性与分布漂移,再通过强化学习剥离支架模板,使推理路径转化为模型内在能力。在八个多模态推理基准测试中,该方法显著推进模型性能边界,相比基础模型平均提升10.3%,达到当前最优水平。代码将公开。
原文摘要 · Abstract (English)
Post-training via Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) is crucial for enhancing reasoning in Multimodal Large Language Models (MLLMs), yet existing paradigms often reach a performance bottleneck due to the limitations of static data. While current methods leverage self-reflection or self-evolution to push these boundaries, they still suffer from cognitive drift and hallucinated reasoning paths caused by low-quality synthetic data. To address these challenges, we propose Anchor Evolution (AnE), a new paradigm that integrates truth-anchored data curation and model evolution, achieving faithful and steady performance gains at the reasoning frontier. Specifically, we propose Truth Anchor Expansion, which pinpoints the model failing frontier via trajectory rollouts and leverages ground-truth databases to retrieve high-fidelity anchors for faithful data curation. Subsequently, we introduce the Scaffold-Stripping Mechanism to internalize reasoning capabilities. This mechanism first anchors reasoning paths via scaffold-augmented supervision to mitigate the learning complexity and distribution drift of direct SFT on raw data, then leverages RL to strip the scaffold template, thereby effectively transitioning the reasoning paths into intrinsic model capabilities. Experimental results on multimodal reasoning benchmarks show that our method substantially advances the model performance frontier, improving the base model by 10.3\% across eight multimodal benchmarks and achieving state-of-the-art results. The code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。