arXiv:2602.16455cs.CV2026-02被引 6

用像素级反馈提升图表解析准确率,避免漏读错位。

Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing

  • 通过生成像素定位并自我可视化来修正视觉错误
  • 在新构建的ChartP-Bench上达到92.3%解析准确率
  • 适合需要高精度视觉理解的图表分析任务

尽管大型视觉语言模型在文本推理和自修正方面表现出色,但在以视觉感知为核心的复杂任务(如图表解析)中效果有限。现有模型常因图表信息过密而出现数据遗漏、对齐错误和幻觉等问题。受人类阅读复杂图表时用手指作为视觉锚点的启发,我们提出视觉自精炼(Visual Self-Refine, VSR)新范式:让模型生成像素级定位结果,可视化后回传自身,实现对视觉误差的直观检视与修正。我们在图表解析领域实例化为ChartVSR,将解析过程分为两个阶段:精炼阶段通过视觉反馈迭代验证所有数据点的像素定位准确性;解码阶段则利用这些经验证的定位作为精确视觉锚点完成最终结构化数据提取。为克服现有基准局限,我们还构建了新的高挑战性评测集ChartP-Bench。实验表明,VSR是一种通用视觉反馈机制,为提升多种以视觉为中心的任务精度提供了新方向。

原文摘要 · Abstract (English)

While Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities for reasoning and self-correction at the textual level, these strengths provide minimal benefits for complex tasks centered on visual perception, such as Chart Parsing. Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and hallucination. Inspired by the human strategy of using a finger as a ``visual anchor'' to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR). The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors. We instantiate the VSR paradigm in the domain of Chart Parsing by proposing ChartVSR. This model decomposes the parsing process into two stages: a Refine Stage, where it iteratively uses visual feedback to ensure the accuracy of all data points' Pixel-level Localizations, and a Decode Stage, where it uses these verified localizations as precise visual anchors to parse the final structured data. To address the limitations of existing benchmarks, we also construct ChartP-Bench, a new and highly challenging benchmark for chart parsing. Our work also highlights VSR as a general-purpose visual feedback mechanism, offering a promising new direction for enhancing accuracy on a wide range of vision-centric tasks.

图表解析视觉反馈像素定位LVLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。