通过分析生成时的词元间隔时间,可无感识别大模型身份。
LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysis
- 利用词元生成的时间间隔差异,构建模型指纹
- 在不同网络环境下对16个小型模型和10个专有大模型测试准确率高
- 无需访问模型权重,适用于加密流量场景
随着大语言模型(LLMs)在各领域广泛应用,识别部署或交互的模型对系统安全与可信性至关重要。现有验证方法依赖输出分析,易受对抗攻击,且多为事后检测,常需模型权重支持。本文提出一种被动、非侵入式实时指纹技术,即使在网络加密条件下仍有效。基于语言模型自回归生成特性——逐词元生成并依赖前序内容——产生独特的时序模式,如同节奏或心跳。我们发现,测量连续词元间的时间间隔(ITTs)可高精度区分不同模型。构建深度学习管道,通过网络流量分析捕捉此类时序特征,在16个小型语言模型(SLMs)和10个专有大模型上评估,涵盖本地主机(GPU/CPU)、局域网(LAN)、远程网络及虚拟私有网络(VPN)等多种部署场景。实验表明,该方法在多种网络条件下均保持高准确性,为真实世界中的模型识别开辟新路径,助力更安全可信的模型部署。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) become increasingly integrated into many technological ecosystems across various domains and industries, identifying which model is deployed or being interacted with is critical for the security and trustworthiness of the systems. Current verification methods typically rely on analyzing the generated output to determine the source model. However, these techniques are susceptible to adversarial attacks, operate in a post-hoc manner, and may require access to the model weights to inject a verifiable fingerprint. In this paper, we propose a novel passive and non-invasive fingerprinting technique that operates in real-time and remains effective even under encrypted network traffic conditions. Our method leverages the intrinsic autoregressive generation nature of language models, which generate text one token at a time based on all previously generated tokens, creating a unique temporal pattern like a rhythm or heartbeat that persists even when the output is streamed over a network. We find that measuring the Inter-Token Times (ITTs)-time intervals between consecutive tokens-can identify different language models with high accuracy. We develop a Deep Learning (DL) pipeline to capture these timing patterns using network traffic analysis and evaluate it on 16 Small Language Models (SLMs) and 10 proprietary LLMs across different deployment scenarios, including local host machine (GPU/CPU), Local Area Network (LAN), Remote Network, and Virtual Private Network (VPN). The experimental results confirm that our proposed technique is effective and maintains high accuracy even when tested in different network conditions. This work opens a new avenue for model identification in real-world scenarios and contributes to more secure and trustworthy language model deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。