通过分析生成时间，可窃取大模型的架构与部署细节。
Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing
- 仅用令牌生成时间推断模型架构与优化策略。
- 成功识别谷歌Gemini Flash 2.5使用128K上下文草稿模型进行推测解码。
- 对Llama模型的架构猜测前10名中正确率超90%。
本文提出LeakyLMs，一种利用远程API调用时的逐令牌生成时间来泄露生产级语言模型的私有信息的新攻击方法。这是首个证明仅通过生成时间即可推断关键模型与部署细节的工作。攻击分为两部分：第一部分检测推理优化技术，如推测解码，并识别其草稿模型上下文长度；实验显示谷歌Gemini Flash 2.5采用推测解码，草稿上下文窗口约128K tokens。第二部分恢复核心架构参数，包括变换器层数、隐状态维度和注意力头数。该方法基于现代NVIDIA GPU上的延迟特性构建精准的时间模型，通过搜索架构空间实现推断。在Llama系列模型上，近似正确配置在前10个猜测中出现超过90%。
原文摘要 · Abstract (English)
This work presents LeakyLMs, a set of attacks that leak proprietary model, architecture, and deployment information from production language models. LeakyLMs is the first to demonstrate that key model and deployment details can be inferred using only token generation timing, even when interacting through remote APIs. LeakyLMs introduces two core attacks. The first attack targets inference optimizations and deployment strategies. For example, our attack detects whether a provider uses speculative decoding, a widely deployed inference-time optimization, and further identifies the context length of the draft model used in the pipeline. Our measurements show that Google Gemini Flash 2.5 uses speculative decoding with a draft context window of approximately 128K tokens. The second attack recovers key architectural properties, including the number of transformer layers, hidden dimension size, and number of attention heads. To achieve this, LeakyLMs builds a detailed and accurate model of token-generation timing on modern NVIDIA GPUs, characterizing how latency scales with model configuration and hardware parameters. The attack then performs a search over the architecture space using this timing model. In experiments with Llama models, the near-correct architectural configuration appears in the top-10 guesses more than 90% of the time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。