提出可识别几何特征的通用视觉模型，解决平面几何题更准确。
GeoDANO: Geometric VLM with Domain Agnostic Vision Encoder
- 用合成几何图-描述对训练通用视觉编码器，提升几何特征识别能力。
- 在数学题数据集上超越专用方法和GPT-4o，准确率显著提升。
- 适合需要理解几何图形的AI系统开发者或教育科技研究者。
我们提出GeoDANO，一种具备领域无关视觉编码器的几何视觉语言模型（VLM），用于求解平面几何问题。尽管已有VLM被用于几何问题求解，但其对几何特征的识别能力尚未充分评估。为此，我们构建了一个基准测试，用于评估点、线等基本几何元素及正交等关系的识别能力。初步研究表明，通用VLM中常用的OpenCLIP等视觉编码器难以检测这些特征，且跨域泛化能力差。为克服该局限，我们开发了GeoCLIP——一个基于CLIP架构、在合成几何图-文本对上训练的模型。基准测试显示，GeoCLIP在几何特征识别方面优于现有编码器。随后，我们提出GeoDANO，通过引入领域自适应策略，增强对未见图示风格的适应能力。GeoDANO在MathVerse数据集上表现超越现有专用方法及GPT-4o。代码已开源：https://github.com/ml-postech/GeoDANO。
原文摘要 · Abstract (English)
We introduce GeoDANO, a geometric vision-language model (VLM) with a domain-agnostic vision encoder, for solving plane geometry problems. Although VLMs have been employed for solving geometry problems, their ability to recognize geometric features remains insufficiently analyzed. To address this gap, we propose a benchmark that evaluates the recognition of visual geometric features, including primitives such as dots and lines, and relations such as orthogonality. Our preliminary study shows that vision encoders often used in general-purpose VLMs, e.g., OpenCLIP, fail to detect these features and struggle to generalize across domains. To overcome the limitation, we develop GeoCLIP, a CLIP-based model trained on synthetic geometric diagram--caption pairs. Benchmark results show that GeoCLIP outperforms existing vision encoders in recognizing geometric features. We then propose our VLM, GeoDANO, which augments GeoCLIP with a domain adaptation strategy for unseen diagram styles. GeoDANO outperforms specialized methods for plane geometry problems and GPT-4o on MathVerse. The implementation is available at https://github.com/ml-postech/GeoDANO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。