评测视频评论创作的创意能力，提出新框架提升AI幽默与讽刺生成。
GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art
- 设计多模态基准GODBench，融合视频与文本评估创意表达。
- 现有模型在生成幽默讽刺评论上表现不佳，平均得分低于40分。
- 提出波浪式思维框架RoT，显著提升创意生成质量，适合内容创作者参考。
视频评论艺术通过传递幽默、讽刺或情感共鸣增强用户参与度，需要对文化与语境细节有深刻理解。尽管多模态大语言模型（MLLMs）和思维链（CoT）在数理编程等任务中表现出强推理能力，但在生成如共鸣笑话、深刻讽刺等创造性内容方面仍存在明显不足。现有评测数据集在模态覆盖和类别多样性上受限，难以全面评估视频评论创作中的创造力。为此，我们提出GODBench——一个融合视频与文本模态的新基准，系统评估MLLMs生成评论艺术的能力。受物理波传播规律启发，我们设计了多步推理框架Ripple of Thought（RoT），以增强MLLMs的创造性表达。大量实验表明，当前MLLMs与CoT方法在理解和生成创意视频评论方面仍面临巨大挑战。相比之下，RoT展现出显著改进效果，验证了其在推动基于MLLM的创造性生成方面的潜力。GODBench已公开于https://github.com/stan-lei/GODBench-ACL2025。
原文摘要 · Abstract (English)
Video Comment Art enhances user engagement by providing creative content that conveys humor, satire, or emotional resonance, requiring a nuanced and comprehensive grasp of cultural and contextual subtleties. Although Multimodal Large Language Models (MLLMs) and Chain-of-Thought (CoT) have demonstrated strong reasoning abilities in STEM tasks (e.g. mathematics and coding), they still struggle to generate creative expressions such as resonant jokes and insightful satire. Moreover, existing benchmarks are constrained by their limited modalities and insufficient categories, hindering the exploration of comprehensive creativity in video-based Comment Art creation. To address these limitations, we introduce GODBench, a novel benchmark that integrates video and text modalities to systematically evaluate MLLMs' abilities to compose Comment Art. Furthermore, inspired by the propagation patterns of waves in physics, we propose Ripple of Thought (RoT), a multi-step reasoning framework designed to enhance the creativity of MLLMs. Extensive experiments reveal that existing MLLMs and CoT methods still face significant challenges in understanding and generating creative video comments. In contrast, RoT provides an effective approach to improve creative composing, highlighting its potential to drive meaningful advancements in MLLM-based creativity. GODBench is publicly available at https://github.com/stan-lei/GODBench-ACL2025.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。