arXiv:2601.04582cs.CL2026-01Conference of the …被引 1

用强化学习提升文本生成图表的准确性和可执行性。

Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization

  • 基于多目标奖励的强化学习框架,联合优化文本、代码和图表质量。
  • 图表质量提升22%,代码执行成功率从78%升至97%。
  • 适用于需要高精度可视化生成的研究与工业场景。

文本到可视化(Text2Vis)系统将表格数据的自然语言查询转化为简洁答案和可执行的可视化结果。尽管闭源大模型能生成有效代码,但生成的图表常缺乏语义对齐与清晰度,这些特性只能在代码执行后评估。开源模型表现更差,常生成不可执行或视觉效果不佳的输出。虽然监督微调可提升代码可执行性,但无法改善整体可视化质量,因传统微调损失无法捕捉执行后的反馈。为此,我们提出RL-Text2Vis,首个用于文本到可视化生成的强化学习框架。基于组相对策略优化(GRPO),该方法设计新型多目标奖励函数,利用执行后反馈同时优化文本准确性、代码有效性与可视化质量。在Qwen2.5(7B和14B)模型上训练,RL-Text2Vis在Text2Vis基准上相比GPT-4o实现22%的图表质量相对提升,并将代码执行成功率从78%提高至97%(相对于零样本基线)。模型显著优于强零样本及监督基线,在VIS-Eval与NVBench等域外数据集上也展现良好泛化能力。结果表明GRPO是结构化多模态推理生成的有效策略。代码已开源:https://github.com/vis-nlp/RL-Text2Vis。

原文摘要 · Abstract (English)

Text-to-Visualization (Text2Vis) systems translate natural language queries over tabular data into concise answers and executable visualizations. While closed-source LLMs generate functional code, the resulting charts often lack semantic alignment and clarity, qualities that can only be assessed post-execution. Open-source models struggle even more, frequently producing non-executable or visually poor outputs. Although supervised fine-tuning can improve code executability, it fails to enhance overall visualization quality, as traditional SFT loss cannot capture post-execution feedback. To address this gap, we propose RL-Text2Vis, the first reinforcement learning framework for Text2Vis generation. Built on Group Relative Policy Optimization (GRPO), our method uses a novel multi-objective reward that jointly optimizes textual accuracy, code validity, and visualization quality using post-execution feedback. By training Qwen2.5 models (7B and 14B), RL-Text2Vis achieves a 22% relative improvement in chart quality over GPT-4o on the Text2Vis benchmark and boosts code execution success from 78% to 97% relative to its zero-shot baseline. Our models significantly outperform strong zero-shot and supervised baselines and also demonstrate robust generalization to out-of-domain datasets like VIS-Eval and NVBench. These results establish GRPO as an effective strategy for structured, multimodal reasoning in visualization generation. We release our code at https://github.com/vis-nlp/RL-Text2Vis.

文本生成强化学习可视化多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。