arXiv:2605.16436cs.CRcs.AI2026-05

AgentAI让精准欺骗可大规模实施,旧安全体系面临崩溃。

The End of Trust: How Agentic AI Breaks Security Assumptions

  • 提出'无限冒名者'模型:代理自动介入已互信双方间劫持关系。
  • 传统检测依赖生成内容可区分性,但高保真伪造已难分辨。
  • 主张从验证身份转向审查行为,适合安全架构设计者参考。

数十年来,数字交互安全依赖于未被明言的经济约束:攻击者必须在欺骗的逼真度与传播规模间权衡。高仿冒需人力投入且限于少数高价值目标,而大规模攻击则牺牲可信度换取覆盖面。检测系统、验证机制和用户培训均基于这种低成本欺骗的产物而设定。然而,代理型AI打破了这一权衡,使高保真、个性化伪造得以大规模部署。我们指出,这并非单纯威胁加剧,而是彻底颠覆了现有安全范式。为此提出‘无限冒名者’攻击模型——一个自治代理介入原本互信的双方之间,劫持已有关系而非从零建立。当前以检测为导向的防御隐含假设:生成内容仍可辨别,但该假设正被侵蚀。我们倡导‘嫌疑默认’新范式,将安全重心从身份认证转向行为评估,并探讨平台作为数字互动监管基础时引发的治理张力。

原文摘要 · Abstract (English)

For decades, the security of digital interaction has rested on an unacknowledged economic constraint. Attackers faced a tradeoff between the fidelity of a deception and the scale at which it could be deployed. Convincing impersonation required sustained human effort and was confined to a narrow set of high-value targets, while mass-market attacks sacrificed plausibility for reach. Detection systems, verification mechanisms, and user awareness training have all been implicitly calibrated to the artifacts of cheap deception that this tradeoff produced. Agentic AI collapses the tradeoff, allowing high-fidelity, individually tailored deception to be produced at mass-market scale. We argue that this shift exhausts a security paradigm rather than merely intensifying the threat landscape. We introduce the Infinite Impostor, an attack model in which an autonomous agent interposes itself between two parties who already trust each other, hijacking an existing relationship rather than building a new one from scratch. Detection-oriented defenses share an assumption that generative progress is eliminating, that synthetic outputs are distinguishable from authentic ones. We propose a suspect-by-default paradigm that shifts security from authenticating actors to evaluating actions, and examine the governance tensions that arise when platforms become the regulatory substrate of digital interaction.

AI安全代理模型信任机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。