首个视频大模型可信度评估平台，揭示安全与公平短板
VMDT: Decoding the Trustworthiness of Video Foundation Models
- 构建VMDT统一评测框架，覆盖五大可信维度
- 开源视频生成模型均无法识别有害请求且易产有害内容
- 模型越大越不公且隐私风险高，但对抗鲁棒性反而提升
随着基础模型日益复杂，其可信度保障愈发关键；然而，相较于文本和图像，视频模态仍缺乏全面的可信度基准。本文提出VMDT（Video-Modal Decoding Trust），首个统一平台，用于评估文本到视频（T2V）和视频到文本（V2T）模型在安全、幻觉、公平性、隐私及对抗鲁棒性五个关键维度的表现。通过对7个T2V模型和19个V2T模型的广泛评估，发现所有开源T2V模型均无法识别有害查询，常生成有害视频，且不公平性高于图像模型。在V2T模型中，公平性和隐私风险随模型规模上升，而幻觉和对抗鲁棒性改善，但整体性能仍较低。值得注意的是，安全性能与模型规模无相关性，表明规模之外的因素主导当前安全水平。研究凸显了构建更鲁棒、可信视频基础模型的紧迫性，VMDT为衡量和追踪进展提供了系统框架。代码已公开于https://sunblaze-ucb.github.io/VMDT-page/。
原文摘要 · Abstract (English)
As foundation models become more sophisticated, ensuring their trustworthiness becomes increasingly critical; yet, unlike text and image, the video modality still lacks comprehensive trustworthiness benchmarks. We introduce VMDT (Video-Modal DecodingTrust), the first unified platform for evaluating text-to-video (T2V) and video-to-text (V2T) models across five key trustworthiness dimensions: safety, hallucination, fairness, privacy, and adversarial robustness. Through our extensive evaluation of 7 T2V models and 19 V2T models using VMDT, we uncover several significant insights. For instance, all open-source T2V models evaluated fail to recognize harmful queries and often generate harmful videos, while exhibiting higher levels of unfairness compared to image modality models. In V2T models, unfairness and privacy risks rise with scale, whereas hallucination and adversarial robustness improve -- though overall performance remains low. Uniquely, safety shows no correlation with model size, implying that factors other than scale govern current safety levels. Our findings highlight the urgent need for developing more robust and trustworthy video foundation models, and VMDT provides a systematic framework for measuring and tracking progress toward this goal. The code is available at https://sunblaze-ucb.github.io/VMDT-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。