语言模型输出自带椭圆签名,可防伪造验证来源
Every Language Model Has a Forgery-Resistant Signature
- 利用模型输出在高维椭球面的几何特性作为唯一指纹
- 无参数访问下无法伪造该签名,现有方法均不可行
- 无需输入或完整权重,适合模型溯源与内容验证
闭源语言模型通过公开API广泛使用,催生了模型溯源与细节提取的取证需求。本文发现,语言模型输出遵循一个不为人知的几何约束:所有输出的对数概率(logprobs)位于一个高维椭球面上。这一椭球签名可作为模型的唯一标识,用于识别输出来源。该签名具有四大特性:难以伪造(无参数时无法生成符合椭球的logprobs)、自然存在(所有语言模型均具备)、自包含(无需输入或完整权重即可检测)、紧凑冗余(每个logprob输出均可独立检测)。我们提出一种从小模型中提取椭球的方法,并讨论其在大规模模型上的实施难题。最后,基于此签名设计了一种类似对称密钥认证的模型输出验证协议。
原文摘要 · Abstract (English)
The ubiquity of closed-weight language models with public-facing APIs has generated interest in forensic methods, both for extracting hidden model details (e.g., parameters) and for identifying models by their outputs. One successful approach to these goals has been to exploit the geometric constraints imposed by the language model architecture and parameters. In this work, we show that a lesser-known geometric constraint -- namely, that language model outputs lie on the surface of a high-dimensional ellipse -- functions as a signature for the model and can be used to identify the source model of a given output. This ellipse signature has unique properties that distinguish it from existing model-output association methods like language model fingerprints. In particular, the signature is hard to forge: without direct access to model parameters, it is practically infeasible to produce log-probabilities (logprobs) on the ellipse using currently known methods. Secondly, the signature is naturally occurring, since all language models have these elliptical constraints. Thirdly, the signature is self-contained, in that it is detectable without access to the model inputs or the full weights. Finally, the signature is compact and redundant, as it is independently detectable in each logprob output from the model. We evaluate a novel technique for extracting the ellipse from small models and discuss the practical hurdles that make it infeasible for production-scale models. Finally, we use ellipse signatures to propose a protocol for language model output verification, analogous to cryptographic symmetric-key message authentication systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。