将视觉感知与推理分离,提升大模型测试时扩展能力。
SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs
- 分两阶段处理:先定位关键视觉区域,再基于区域推理
- 在低预算下比基线高4.6分,节省200倍上下文令牌
- 适合需要高效推理或视觉错误敏感的场景
尽管取得进展,视觉语言模型(VLMs)在测试时动态扩展上下文长度仍不稳定。无结构的视觉推理链将感知与推理纠缠,导致长而混乱的上下文,微小感知错误可能引发严重答案偏差。推理还需昂贵的手动奖励强化学习。为此,我们提出SPARC(分离感知与推理回路),一种模块化框架,明确解耦视觉感知与推理。受大脑感官到认知处理顺序启发,SPARC采用两阶段流程:模型先执行显式视觉搜索定位问题相关区域,再基于这些区域进行推理生成最终答案。该分离支持独立测试时扩展和不对称算力分配(如在分布偏移下优先处理感知),并可选择性优化(如仅改进瓶颈感知阶段)。通过全局搜索使用低分辨率图像、仅对选定区域进行高分辨率处理,实现压缩上下文,降低视觉令牌数与计算量。SPARC在多个挑战性视觉推理任务上超越单一模型基线和强视觉对齐方法,例如使Qwen3VL 4B在$V^*$ VQA基准上提升6.7分,并在分布外设置中以200倍更低的令牌预算超过“思考图像”方法4.6分。
原文摘要 · Abstract (English)
Despite recent successes, test-time scaling -- i.e., dynamically expanding the token budget during inference as needed -- remains brittle for vision-language models (VLMs). Unstructured visual reasoning chains entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Reasoning also requires expensive reinforcement learning with hand-crafted rewards. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), and supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance). It also accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing visual token count and compute. SPARC outperforms monolithic baselines and strong visual-grounding approaches across challenging visual reasoning tasks, such as improving Qwen3VL 4B on the $V^*$ VQA benchmark by 6.7 points and surpassing "thinking with images" by 4.6 points in an OOD setting with a $200\times$ lower token budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。