系统梳理多模态模型的对抗攻击类型，助力从业者识别风险。
Adversarial Attacks in Multimodal Systems: A Practitioner's Survey
- 按文本、图像、视频、音频四类归纳对抗攻击方法
- 揭示多模态模型因融合多种模态而放大攻击威胁
- 面向实践者，提供可操作的威胁认知与防护参考
多模态模型的出现是人工智能的重大进步，单一模型可理解文本、图像、视频和音频等多种模态。开源多模态模型使这些突破更易获取。然而，由于跨模态对抗攻击的广泛存在，这些模型继承了所有模态的脆弱性，导致对抗威胁被放大。尽管已有大量研究探讨各模态内或跨模态的攻击，但针对实践者的攻击类型综述在多模态领域仍属空白。随着更多机器学习从业者在实际应用中采用、微调并部署开源模型，理解威胁格局并采取预防措施至关重要。本文填补该空白，系统调研针对文本、图像、视频和音频四类模态的对抗攻击。本综述呈现了多模态对抗威胁的发展演变，并据我们所知，是首个对多模态威胁格局的全面总结。
原文摘要 · Abstract (English)
The introduction of multimodal models is a huge step forward in Artificial Intelligence. A single model is trained to understand multiple modalities: text, image, video, and audio. Open-source multimodal models have made these breakthroughs more accessible. However, considering the vast landscape of adversarial attacks across these modalities, these models also inherit vulnerabilities of all the modalities, and ultimately, the adversarial threat amplifies. While broad research is available on possible attacks within or across these modalities, a practitioner-focused view that outlines attack types remains absent in the multimodal world. As more Machine Learning Practitioners adopt, fine-tune, and deploy open-source models in real-world applications, it's crucial that they can view the threat landscape and take the preventive actions necessary. This paper addresses the gap by surveying adversarial attacks targeting all four modalities: text, image, video, and audio. This survey provides a view of the adversarial attack landscape and presents how multimodal adversarial threats have evolved. To the best of our knowledge, this survey is the first comprehensive summarization of the threat landscape in the multimodal world.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。