让模型通过生成代码来还原图表，实现可验证的精准视觉推理。
RECODE: Reasoning Through Code Generation for Visual Question Answering
- 将图像反向重构为可执行代码，实现符号化推理
- 在多个图表问答数据集上超越现有方法
- 适合需要高精度逻辑推断的研究者和开发者
多模态大语言模型在处理图表、示意图等结构化视觉内容时，因基于像素的感知缺乏验证机制而难以进行精确推理。为此，我们提出利用‘反渲染’——即从图像逆向生成可执行代码——作为可验证视觉推理的新范式。我们构建了RECODE框架，先生成多个候选程序以复现输入图像，再通过批判器选择最忠实的重建结果，并迭代优化代码。该过程将模糊的感知任务转化为可验证的符号问题，支持后续精确计算与逻辑推理。在CharXiv、ChartQA和Geometry3K等多个视觉推理基准测试中，RECODE显著优于不使用代码或仅用代码辅助绘线/裁剪的方法。本工作证明，将视觉感知锚定在可执行代码中，为更准确、可验证的多模态推理提供了新路径。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) struggle with precise reasoning for structured visuals like charts and diagrams, as pixel-based perception lacks a mechanism for verification. To address this, we propose to leverage derendering -- the process of reverse-engineering visuals into executable code -- as a new modality for verifiable visual reasoning. Specifically, we propose RECODE, an agentic framework that first generates multiple candidate programs to reproduce the input image. It then uses a critic to select the most faithful reconstruction and iteratively refines the code. This process not only transforms an ambiguous perceptual task into a verifiable, symbolic problem, but also enables precise calculations and logical inferences later on. On various visual reasoning benchmarks such as CharXiv, ChartQA, and Geometry3K, RECODE significantly outperforms methods that do not leverage code or only use code for drawing auxiliary lines or cropping. Our work demonstrates that grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。