自动生成复杂几何多模态数据集，提升视觉与符号推理对齐能力。
Synthesizing Multimodal Geometry Datasets from Scratch and Enabling Visual Alignment via Plotting Code
- 分三阶段生成：符号种子、验证实例化、代码绘图，保证图文一致
- 新数据集GeoCode结构更复杂，数学正确性经多阶段验证
- 通过代码预测强化视觉-符号对齐，模型性能在多个基准显著提升
多模态几何推理要求模型同时理解视觉图示并进行结构化符号推断，但现有视觉-语言模型因训练数据有限且视觉-符号对齐薄弱，难以处理复杂几何构造。我们提出一个从零开始合成复杂多模态几何问题的流水线，构建名为GeoCode的数据集。该方法将问题生成分解为符号种子构建、基于验证的实体化以及代码驱动的图示渲染三个阶段，确保结构、文本、推理与图像的一致性。利用GeoCode中提供的绘图代码，我们进一步引入代码预测作为显式对齐目标，将视觉理解转化为监督式结构化预测任务。GeoCode相比现有基准展现出更高的结构复杂度和推理难度，同时通过多阶段验证保持数学正确性。大量实验表明，基于GeoCode训练的模型在多个几何基准上均取得稳定提升，验证了数据集与对齐策略的有效性。代码将公开于https://github.com/would1920/GeoCode。
原文摘要 · Abstract (English)
Multimodal geometry reasoning requires models to jointly understand visual diagrams and perform structured symbolic inference, yet current vision--language models struggle with complex geometric constructions due to limited training data and weak visual--symbolic alignment. We propose a pipeline for synthesizing complex multimodal geometry problems from scratch and construct a dataset named \textbf{GeoCode}, which decouples problem generation into symbolic seed construction, grounded instantiation with verification, and code-based diagram rendering, ensuring consistency across structure, text, reasoning, and images. Leveraging the plotting code provided in GeoCode, we further introduce code prediction as an explicit alignment objective, transforming visual understanding into a supervised structured prediction task. GeoCode exhibits substantially higher structural complexity and reasoning difficulty than existing benchmarks, while maintaining mathematical correctness through multi-stage validation. Extensive experiments show that models trained on GeoCode achieve consistent improvements on multiple geometry benchmarks, demonstrating both the effectiveness of the dataset and the proposed alignment strategy. The code will be available at https://github.com/would1920/GeoCode.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。