评测视觉语言模型对五类视频篡改的鲁棒性，揭示大模型不等于更安全。
MVTamperBench: Evaluating Robustness of Vision-Language Models
- 构建真实场景下的视频篡改基准，涵盖旋转、遮挡、替换等五类攻击
- 测试45个模型在17000+篡改片段上表现，发现参数量大也不一定更抗干扰
- 适合关注视频安全、虚假信息检测的研究者和平台内容审核团队
多模态大语言模型（MLLMs）是近期视觉-语言模型（VLMs）的重要进展，在视频理解方面取得显著突破。然而，其对对抗性篡改与操纵的脆弱性仍缺乏系统研究。为此，我们提出 extbf{MVTamperBench}，一个针对五类常见篡改技术（旋转、遮挡、替换、重复、删除）的基准测试，基于真实场景如监控干扰、社交媒体编辑和虚假信息传播。该基准包含约3.4K原始视频，扩展为超过17,000个篡改片段，覆盖19项具体视频篡改任务，考验模型在空间与时间连贯性上的检测能力。我们评估了来自15+模型家族的45个最新MLLMs。结果表明，不同篡改类型间鲁棒性差异显著，且参数量大并不意味着更强的抗篡改能力。MVTamperBench为开发安全关键应用中的抗篡改MLLM设立了新标准，适用于检测点击诱饵、防止有害内容传播及媒体平台政策执行。代码与数据已公开，推动可信视频理解研究。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs), are recent advancement of Vision-Language Models (VLMs) that have driven major advances in video understanding. However, their vulnerability to adversarial tampering and manipulations remains underexplored. To address this gap, we introduce \textbf{MVTamperBench}, a benchmark that systematically evaluates MLLM robustness against five prevalent tampering techniques: rotation, masking, substitution, repetition, and dropping; based on real-world visual tampering scenarios such as surveillance interference, social media content edits, and misinformation injection. MVTamperBench comprises ~3.4K original videos, expanded into over ~17K tampered clips covering 19 distinct video manipulation tasks. This benchmark challenges models to detect manipulations in spatial and temporal coherence. We evaluate 45 recent MLLMs from 15+ model families. We reveal substantial variability in resilience across tampering types and show that larger parameter counts do not necessarily guarantee robustness. MVTamperBench sets a new benchmark for developing tamper-resilient MLLM in safety-critical applications, including detecting clickbait, preventing harmful content distribution, and enforcing policies on media platforms. We release all code, data, and benchmark to foster open research in trustworthy video understanding. Code: https://amitbcp.github.io/MVTamperBench/ Data: https://huggingface.co/datasets/Srikant86/MVTamperBench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。