用大脑回溯预测机制,让模型无监督地自我反思,提升视觉推理能力。
BUS: Brain-Inspired Unsupervised Self-Reflection via Backward Prediction for Multimodal Reasoning

- 借鉴人类大脑回溯预测机制,设计无监督自反思框架
- 在多个基准上提升5.8%~8.0%,无需标注数据
- 可插件式适配主流微调方法,适合复杂视觉任务研究者
当前视觉语言模型在需要一致且细粒度推理的复杂视觉任务中表现不足。现有方法虽尝试训练模型进行自省推理,但依赖大量标注数据,且测试时缺乏显式反思行为。受神经科学启发,本文发现主流视觉语言模型具备类似人脑的回溯预测能力——即预测某个未来状态可能由哪些当前状态导致。为此提出一种名为脑启发无监督自反思(BUS)的标签无关训练框架,利用该能力增强复杂视觉任务中的反思推理。BUS通过回溯预测实现推理过程的自验证,提供无监督学习信号,摆脱对标注数据的依赖,同时提升推理性能。作为模型无关的插件模块,兼容监督微调(SFT)和强化学习(RL)。以Qwen3-VL-8B为基础,分别在HR-Bench-8K(+8.0%)、HR-Bench-4K(+7.7%)、V* Bench(+6.3%)和MME-RealWorld-Lite(+5.8%)上取得显著提升,证明回溯预测是推动反思推理的关键。
原文摘要 · Abstract (English)
Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods aim to train models to facilitate self-reflective reasoning, i.e., reviewing and improving the generated reasoning. However, they require large volumes of annotated data and lack explicit reflective behavior during test time. By contrast, humans perform explicit and efficient self-reflection through mechanisms such as backward prediction, i.e., predicting which current states are likely to precede a given future state. Inspired by neuroscience, this work proposes a novel solution to address these challenges. We first observe and investigate the phenomenon that mainstream VLMs can perform backward prediction, similar to the human brain. A label-free training framework named Brain-inspired Unsupervised Self-reflection (BUS) is proposed to leverage and exploit backward prediction capability to enhance reflective reasoning in complex visual tasks. BUS enables self-verification of reflective reasoning based on backward prediction, providing explicit learning signals under unsupervised conditions. In this way, BUS eliminates reliance on annotated data while improving reasoning performance. Designed as a model-agnostic plug-in, our framework is compatible with popular fine-tuning methods, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Initialized from Qwen3-VL-8B, it improves HR-Bench-8K (+8.0%), HR-Bench-4K (+7.7%), V* Bench (+6.3%), and MME-RealWorld-Lite (+5.8%), proving backward prediction is key to advancing reflective reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。