arXiv:2605.20266cs.SD2026-05综述被引 4

系统梳理音频大模型的可信性问题与防御路径

A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook

论文配图:A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook
图 1 · 摘自论文原文
  • 从架构到对齐算法,解析音频大模型的内生机制
  • 发现跨模态越狱、声学后门等六大可信风险
  • 适合关注音频AI安全与可信智能的研究者

大语言模型的发展推动了多模态大语言模型的兴起。其中,大型音频语言模型(LALMs)是实现通用听觉智能的关键。尽管性能卓越,其能力提升远超保障可信性的系统框架发展。本综述全面分析了LALMs的内生机制,涵盖架构创新与对齐算法如何促成涌现推理。具体而言,统一端到端框架和连续声学信号集成扩大了攻击面。为严格评估此类范式中的风险,我们构建了可信性分类体系,归纳出跨模态越狱、潜在声学后门及生物特征隐私泄露等关键漏洞。通过六个分析维度(幻觉、鲁棒性、安全性、隐私性、公平性、认证)回顾当前先进LALMs。进攻手段成熟而防御体系薄弱,凸显音频智能在多维风险下的可信性缺口。最后,提出‘纵深防御’架构、因果听觉世界建模及内在表征工程的发展路线,以支持更可靠、可信的音频智能。项目已开源至GitHub:https://github.com/Kwwwww74/Awesome-Trustworthy-AudioLLMs。

原文摘要 · Abstract (English)

Advances in Large Language Models (LLMs) have paved the way for Multimodal Large Language Models (MLLMs). Among these, Large Audio Language Models (LALMs) are essential for realizing universal auditory intelligence. Despite their remarkable performance, the escalation of LALMs' capabilities has significantly outpaced the development of systemic frameworks to ensure their trustworthiness. This survey provides a comprehensive investigation into the endogenous mechanisms of LALMs, detailing the architectural innovations and alignment algorithms that facilitate emergent reasoning. Specifically, we analyze how the transition to unified end-to-end frameworks and the integration of continuous acoustic signals expand the attack surface. To rigorously evaluate the risks within these paradigms, we establish a comprehensive taxonomy of trustworthiness, categorizing critical vulnerabilities such as cross-modal jailbreaking, latent acoustic backdoors, and biometric privacy leakage. We review the state-of-the-art LALMs through six analytical pillars: hallucination, robustness, safety, privacy, fairness, and authentication. The pronounced imbalance between a mature offensive landscape and underdeveloped defenses highlights persistent trustworthiness gaps and multidimensional risks in audio-centric intelligence. Finally, we propose a roadmap advocating for ``Defense-in-Depth'' architectures, causal auditory world modeling, and intrinsic representation engineering to support the development of more reliable and trustworthy audio intelligence. Our project has been uploaded to GitHub https://github.com/Kwwwww74/Awesome-Trustworthy-AudioLLMs.

音频大模型可信智能安全评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。