让AI像营销专家一样,从广告视频中推理出说服策略。
AD-MIR: Bridging the Gap from Perception to Persuasion in Advertising Video Understanding via Structured Reasoning
- 分两阶段解析:先构建结构化视觉数据库,再用迭代提问推断营销策略。
- 在AdsQA上比最强通用模型高9.5%(宽松准确率),关键靠帧级证据验证。
- 适合研究广告理解、可解释性AI或数字营销的开发者和学者。
广告视频的多模态理解对于揭示视觉叙事与抽象说服策略之间的复杂关系至关重要。然而,现有智能体虽擅长通用搜索,却难以弥合像素级感知与高层营销逻辑之间的认知鸿沟。为此,我们提出AD-MIR框架,通过两阶段架构解码广告意图。第一阶段“结构感知记忆构建”将原始视频转化为结构化数据库,融合语义检索与精确关键词匹配,优先保留品牌细节(如标志、屏幕文字),动态过滤无关背景,定位核心主体。第二阶段“结构化推理代理”模拟营销专家,通过迭代式问询循环分解叙事,推断隐含的说服手法。关键在于其基于证据的自我校正机制,严格对照具体视频帧验证推论,缺乏视觉支持时自动回溯。在AdsQA基准上的评估表明,AD-MIR达到当前最优性能,严格准确率超越最强通用代理DVD 1.8%,宽松准确率高出9.5%。结果表明,有效的广告理解需将抽象营销策略显式锚定于像素级证据。代码已公开于https://github.com/Little-Fridge/AD-MIR。
原文摘要 · Abstract (English)
Multimodal understanding of advertising videos is essential for interpreting the intricate relationship between visual storytelling and abstract persuasion strategies. However, despite excelling at general search, existing agents often struggle to bridge the cognitive gap between pixel-level perception and high-level marketing logic. To address this challenge, we introduce AD-MIR, a framework designed to decode advertising intent via a two-stage architecture. First, in the Structure-Aware Memory Construction phase, the system converts raw video into a structured database by integrating semantic retrieval with exact keyword matching. This approach prioritizes fine-grained brand details (e.g., logos, on-screen text) while dynamically filtering out irrelevant background noise to isolate key protagonists. Second, the Structured Reasoning Agent mimics a marketing expert through an iterative inquiry loop, decomposing the narrative to deduce implicit persuasion tactics. Crucially, it employs an evidence-based self-correction mechanism that rigorously validates these insights against specific video frames, automatically backtracking when visual support is lacking. Evaluation on the AdsQA benchmark demonstrates that AD-MIR achieves state-of-the-art performance, surpassing the strongest general-purpose agent, DVD, by 1.8% in strict and 9.5% in relaxed accuracy. These results underscore that effective advertising understanding demands explicitly grounding abstract marketing strategies in pixel-level evidence. The code is available at https://github.com/Little-Fridge/AD-MIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。