用强化学习提升视觉语言模型的几何感知能力,效果显著优于传统微调。
Enhancing Geometric Perception in VLMs via Translator-Guided Reinforcement Learning
- 设计可独立评估几何感知的基准与自动数据生成工具
- 通过自然语言转领域语言的翻译器实现细粒度奖励信号,提升模型性能
- 在跨域任务中表现更优,适合需要精准几何理解的研究者
视觉语言模型(VLM)常因对基本图示元素感知不足而难以进行几何推理。为此,我们提出GeoPerceive基准,包含配对的图示实例与领域特定语言(DSL)表示,并构建高效自动数据生成管道,实现几何感知的独立评估。为利用该数据增强VLM的几何感知能力,我们提出GeoDPO框架,采用由合成数据训练的自然语言到DSL翻译器,将自然语言描述转化为细粒度的DSL级评分作为强化学习奖励信号。在域内与域外数据集上评估显示:尽管监督微调(SFT)仅带来轻微提升甚至在域外任务中性能下降,而GeoDPO实现显著增益——域内+26.5%、域外+8.0%、下游推理任务+39.0%。结果表明GeoDPO在性能与泛化性上远超SFT。代码已开源以保障可复现性。
原文摘要 · Abstract (English)
Vision-language models (VLMs) often struggle with geometric reasoning due to their limited perception of fundamental diagram elements. To tackle this challenge, we introduce GeoPerceive, a benchmark comprising diagram instances paired with domain-specific language (DSL) representations, along with an efficient automatic data generation pipeline. This design enables the isolated evaluation of geometric perception independently from reasoning. To exploit the data provided by GeoPerceive for enhancing the geometric perception capabilities of VLMs, we propose GeoDPO, a translator-guided reinforcement learning (RL) framework. GeoDPO employs an NL-to-DSL translator, which is trained on synthetic pairs generated by the data engine of GeoPerceive, to bridge natural language and DSL. This translator facilitates the computation of fine-grained, DSL-level scores, which serve as reward signals in reinforcement learning. We assess GeoDPO on both in-domain and out-of-domain datasets, spanning tasks in geometric perception as well as downstream reasoning. Experimental results demonstrate that, while supervised fine-tuning (SFT) offers only marginal improvements and may even impair performance in out-of-domain scenarios, GeoDPO achieves substantial gains: $+26.5\%$ on in-domain data, $+8.0\%$ on out-of-domain data, and $+39.0\%$ on downstream reasoning tasks. These findings underscore the superior performance and generalization ability of GeoDPO over SFT. All codes are released at https://github.com/Longin-Yu/GeoPerceive to ensure reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。