通过自适应剪枝与语义图融合，大幅减少视频问答的令牌消耗。
Less is More: Token-Efficient Video-QA via Adaptive Frame-Pruning and Semantic Graph Integration
- 自适应帧剪枝消除视觉冗余，智能聚类关键帧
- 引入轻量语义图补偿信息，令牌减少最高达82.2%
- 提升模型鲁棒性，用更少帧实现更优准确率
多模态大模型在视频问答任务中的应用受限于处理大量视频帧带来的高令牌开销。尽管关键帧选择是主流缓解策略，我们发现其存在显著的时间冗余问题，即‘视觉回声’，这会导致上下文稀释并可能降低性能。为此，我们提出一种结合自适应帧剪枝（AFP）与轻量文本语义图的新框架。AFP通过自适应聚类剔除视觉回声，语义图则提供低成本语义补全。在LongVideoBench和Video-MME上的实验表明，该方法最多可减少82.2%的输入令牌。更重要的是，通过生成简洁高质量提示，该框架不仅提升效率，还显著增强上游选择器的鲁棒性与准确性，表现优于甚至远超使用更多帧的基线模型。
原文摘要 · Abstract (English)
The practical application of Multimodal Large Language Models (MLLMs) to Video Question Answering (Video-QA) is severely hindered by the high token cost of processing numerous video frames. While keyframe selection is the dominant strategy for mitigating this, we identify a critical flaw: even state-of-the-art selectors produce prompts suffering from significant temporal redundancy, a challenge unique to video that we term 'visual echoes'. This issue leads to context dilution and can paradoxically degrade performance. To address this dual challenge, we propose a novel refinement framework that synergistically combines Adaptive Frame-Pruning(AFP) with a lightweight text-based semantic graph. AFP intelligently prunes 'visual echoes' by adaptively clustering frames, while the semantic graph provides crucial, low-cost semantic compensation. Conducting extensive experiments on the LongVideoBench and Video-MME benchmarks against multiple state-of-the-art selectors, our approach demonstrates a drastic reduction in total input tokens by up to 82.2%. Crucially, by creating a concise, high-quality prompt, our framework not only enhances efficiency but also demonstrates a remarkable ability to robustify and improve the accuracy of upstream selectors, achieving results that are highly competitive with, and often superior to, baselines that use vastly more frames.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。