arXiv:2512.14735q-fin.CPcs.AI2025-12

用对抗生成构建金融图像理解金字塔,提升模型推理能力

PyFi: Toward Pyramid-like Financial Image Understanding for VLMs via Adversarial Agents

  • 通过对抗多智能体生成600K金融问答对,形成由浅入深的推理层级
  • 在金融视觉任务上,模型准确率提升19.52%和8.06%(相比基线)
  • 适合研究金融视觉推理、多步问题分解与可扩展数据构建的学者

本文提出PyFi框架,实现视觉语言模型(VLMs)在金融图像理解中的渐进式推理。核心是PyFi-600K数据集,包含600,000个金融问答对,按从基础感知到复杂金融理解的层级组织。该数据通过无人工标注的合成方式生成,采用基于蒙特卡洛树搜索(MCTS)的多智能体对抗机制(PyFi-adv),其中挑战者与求解者智能体协作生成逐步深入的推理链。利用此数据集,我们对先进VLMs进行细粒度、分层评估。将Qwen2.5-VL-3B和Qwen2.5-VL-7B在金字塔结构的问题链上微调后,模型能通过逐步分解复杂问题,分别实现平均准确率提升19.52%和8.06%。所有代码、数据与模型已开源。

原文摘要 · Abstract (English)

This paper proposes PyFi, a novel framework for pyramid-like financial image understanding that enables vision language models (VLMs) to reason through question chains in a progressive, simple-to-complex manner. At the core of PyFi is PyFi-600K, a dataset comprising 600K financial question-answer pairs organized into a reasoning pyramid: questions at the base require only basic perception, while those toward the apex demand increasing levels of capability in financial visual understanding and expertise. This data is scalable because it is synthesized without human annotations, using PyFi-adv, a multi-agent adversarial mechanism under the Monte Carlo Tree Search (MCTS) paradigm, in which, for each image, a challenger agent competes with a solver agent by generating question chains that progressively probe deeper capability levels in financial visual reasoning. Leveraging this dataset, we present fine-grained, hierarchical, and comprehensive evaluations of advanced VLMs in the financial domain. Moreover, fine-tuning Qwen2.5-VL-3B and Qwen2.5-VL-7B on the pyramid-structured question chains enables these models to answer complex financial questions by decomposing them into sub-questions with gradually increasing reasoning demands, yielding average accuracy improvements of 19.52% and 8.06%, respectively, on the dataset. All resources of code, dataset and models are available at: https://github.com/AgenticFinLab/PyFi .

金融视觉多步推理对抗生成数据合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。