用代码生成动态图表，让大模型学会做会动的可视化。
OpusAnimation: Code-Based Dynamic Chart Generation
- 基于代码-视觉联合奖励训练，提升模型生成动态图表能力。
- 在三个任务上平均性能领先开源模型8.31%，仅30亿参数媲美闭源模型。
- 首个动态图表生成基准与高质量数据集，适合研究多模态生成者。
动态图表生成（DCG）旨在生成由代码渲染的动画可视化图表。尽管多模态大模型（MLLM）在静态图表生成与理解方面取得显著进展，其在动态图表生成与理解方面的潜力仍待挖掘。为此，我们提出DCG-Bench（动态图表生成基准），首个从三个维度评估MLLM动态图表生成能力的基准：简单文本到图表、详细文本到图表和视频到图表任务。我们构建了DCG-8K数据集，包含8,000个指令-代码-视频三元组及对应的问答对，用于代码与视频层面的评估。基于该数据集，我们设计两阶段训练方案，提出联合代码-视觉奖励机制，训练出专家级模型Qwen2.5-VL-DCG-3B。实验结果表明，现有MLLM在视觉到图表任务中表现不足，而我们的模型在三项任务上平均性能超越最佳开源模型8.31%，且仅使用30亿参数即达到与闭源模型相当水平，验证了训练方法的有效性。代码与数据集将公开。
原文摘要 · Abstract (English)
Dynamic Chart Generation (DCG) involves producing code-rendered animated visualizations as charts. While recent advances in multi-modal large language models (MLLMs) have significantly improved their capability on static chart generation and comprehension, MLLMs' potential for handling dynamic chart generation and understanding remains underexplored. To bridge this research gap, we introduce DCG-Bench (Dynamic Chart Generation Benchmark), the first benchmark evaluating MLLM's capability on dynamic chart generation tasks from three dimensions: Simple Text-to-Chart, Detailed Text-to-Chart, and Video-to-Chart tasks. We construct DCG-8K, a high-quality DCG dataset with annotations covering instruction-code-video triplets and QA pairs for both code and video evaluation. Based on DCG-8K, we explored a two-stage training recipe, proposing Joint-Code-Visual Reward for group relative policy optimization to construct expert MLLM Qwen2.5-VL-DCG-3B for the DCG task. Our benchmarking result reveals shortcomings of existing MLLMs in the visual-to-chart task, and our model beats the best open-sourced MLLM with an average 8.31% performance gain across three tasks, and shows on par performance against proprietary models with only 3B parameters, proving the effectiveness of our training recipe. Our code and dataset will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。