arXiv:2412.10435cs.CVcs.AI2024-12KDD被引 7

用轻量级预筛选降低视频质量评估的算力消耗,效果不降反升。

COEF-VQ: Cost-Efficient Video Quality Understanding through a Cascaded Multimodal LLM Framework

  • 先用轻量模型按不确定度筛选,只让高难度视频进大模型处理。
  • 线上测试降低不当内容观看率9.9%,且不影响用户活跃度。
  • 适合资源受限场景下需高精度视频理解的平台应用。

随着多模态大语言模型(MLLM)技术的发展,其在视频理解任务中展现出潜力。然而,在线部署时面临巨大的GPU资源消耗问题。本文提出COEF-VQ,一种新型级联式多模态框架,旨在提升短视频平台上的视频质量理解能力的同时优化计算效率。该方法引入基于熵的预筛选阶段,由轻量模型评估样本不确定性,并仅将高不确定性样本传递至计算开销较大的MLLM进行最终判断。通过优先分析复杂样本,框架显著减少GPU使用量,同时保持全量部署的分类性能。我们在短视频平台的视频管理平台(VMP)上部署该框架,在两个内部任务中开展实验。结果显示,离线评估中性能显著提升;线上A/B测试中,不当内容视频观看率下降9.9%,且未影响用户参与度。上线后持续监测验证了其真实世界效果。

原文摘要 · Abstract (English)

Recently, with the emergence of recent Multimodal Large Language Model (MLLM) technology, it has become possible to exploit its video understanding capability on different classification tasks. In practice, we face the difficulty of huge requirements for GPU resource if we need to deploy MLLMs online. In this paper, we propose COEF-VQ, a novel cascaded MLLM framework designed to enhance video quality understanding on the short-video platform while optimizing computational efficiency. Our approach integrates an entropy-based pre-filtering stage, where a lightweight model assesses uncertainty and selectively filters cases before passing them to the more computationally intensive MLLM for final evaluation. By prioritizing high-uncertainty samples for deeper analysis, our framework significantly reduces GPU usage while maintaining the strong classification performance of a full MLLM deployment. To demonstrate the effectiveness of COEF-VQ, we deploy this new framework onto the video management platform (VMP) at the short-video platform, and perform a series of detailed experiments on two in-house tasks related to video quality understanding. We show that COEF-VQ leads to substantial performance gains from the offline evaluation in these two tasks and effectively enhances platform safety with limit resource consumption, significantly reducing inappropriate content video view rate by 9.9% in a online A/B test without affecting engagement. Post-launch monitoring confirmed sustained improvements, validating its real-world impact.

视频理解多模态效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。