多数神经网络几乎总是能生成任意输出，存在安全漏洞风险。
On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
- 分析主流神经架构的满射性，发现其几乎总能覆盖所有输出
- 证明GPT类模型和扩散模型可对任意输出反推输入
- 揭示模型对越狱攻击的固有脆弱性，适合关注安全的研究者
给定一个训练好的神经网络，是否能通过某个输入生成任意指定的输出？等价于该网络是否为满射函数。在生成模型中，满射性意味着任何输出（包括有害或不良内容）原则上都可被生成，引发模型安全与越狱攻击的担忧。本文证明，现代神经架构中的许多核心组件，如预层归一化网络和线性注意力模块，几乎总是满射的。作为推论，广泛使用的生成框架，包括GPT风格的Transformer和采用确定性常微分方程求解器的扩散模型，均支持对任意输出的逆映射。通过研究这些现代常用架构的满射性，我们提出一个形式化框架，揭示了它们对一类广泛对抗攻击的不可避免的脆弱性。
原文摘要 · Abstract (English)
Given a trained neural network, can any specified output be generated by some input? Equivalently, does the network correspond to a function that is surjective? In generative models, surjectivity implies that any output, including harmful or undesirable content, can in principle be generated by the networks, raising concerns about model safety and jailbreak vulnerabilities. In this paper, we prove that many fundamental building blocks of modern neural architectures, such as networks with pre-layer normalization and linear-attention modules, are almost always surjective. As corollaries, widely used generative frameworks, including GPT-style transformers and diffusion models with deterministic ODE solvers, admit inverse mappings for arbitrary outputs. By studying surjectivity of these modern and commonly used neural architectures, we contribute a formalism that sheds light on their unavoidable vulnerability to a broad class of adversarial attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。