让视觉语言模型学会用树状结构推理，更准更简洁。
STELAR-VISION: Self-Topology-Aware Efficient Learning for Aligned Reasoning in Vision
- 用合成数据生成多种推理拓扑，训练模型灵活使用树/图结构。
- 在多个评测中准确率提升7.3%以上，输出长度减少且精度损失小。
- 适合需要高效精准多步推理的复杂视觉理解任务。
视觉语言模型（VLMs）在推理方面取得显著进展，但在复杂多模态任务中仍表现不佳，常产生冗长输出。其主要瓶颈在于过度依赖链式思维（CoT）推理，而许多任务更适合树状或图状结构。为此，我们提出STELAR-Vision训练框架，核心为TopoAug——一种生成多样化拓扑结构的合成数据流水线。通过监督微调与强化学习，对Qwen2VL模型进行后训练，兼顾准确率与效率。此外，提出Frugal Learning策略，在几乎不损失精度的前提下减少输出长度。在MATH-V和VLM-S2H上，相比基线模型准确率提升9.7%，超越更大的Qwen2VL-72B-Instruct达7.3%。在五个分布外基准测试中，优于Phi-4-Multimodal-Instruct最高28.4%，优于LLaMA-3.2-11B-Vision-Instruct最高13.2%，展现强大泛化能力。相比仅使用链式结构训练，本方法在分布内数据集上整体准确率高4.3%，并在所有分布外任务中持续领先。
原文摘要 · Abstract (English)
Vision-language models (VLMs) have made significant strides in reasoning, yet they often struggle with complex multimodal tasks and tend to generate overly verbose outputs. A key limitation is their reliance on chain-of-thought (CoT) reasoning, despite many tasks benefiting from alternative topologies like trees or graphs. To address this, we introduce STELAR-Vision, a training framework for topology-aware reasoning. At its core is TopoAug, a synthetic data pipeline that enriches training with diverse topological structures. Using supervised fine-tuning and reinforcement learning, we post-train Qwen2VL models with both accuracy and efficiency in mind. Additionally, we propose Frugal Learning, which reduces output length with minimal accuracy loss. On MATH-V and VLM-S2H, STELAR-Vision improves accuracy by 9.7% over its base model and surpasses the larger Qwen2VL-72B-Instruct by 7.3%. On five out-of-distribution benchmarks, it outperforms Phi-4-Multimodal-Instruct by up to 28.4% and LLaMA-3.2-11B-Vision-Instruct by up to 13.2%, demonstrating strong generalization. Compared to Chain-Only training, our approach achieves 4.3% higher overall accuracy on in-distribution datasets and consistently outperforms across all OOD benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。