给大模型植入数万份可追踪指纹，且不影响性能。
Scalable Fingerprinting of Large Language Models
- 提出Perinucleus采样法，实现大规模指纹嵌入。
- 在Llama-3.1-8B中成功嵌入24,576个指纹，性能无损。
- 指纹持久且抗对抗攻击，适合模型版权保护场景。
模型指纹技术已成为模型所有者通过API访问识别共享模型的强大工具。然而，为降低误报率、防止指纹泄露，并抵御用户联盟绕过检测，我们强调‘可扩展性’至关重要——即能够向模型中嵌入大量指纹。为此，我们首次在远超以往规模的范围内实验指纹设计，并提出一种名为Perinucleus采样的新方法，可生成可扩展、持久且无害的指纹。实验证明，该方案可在Llama-3.1-8B模型中嵌入24,576个指纹，较现有方法提升两个数量级，且不损害模型性能。这些指纹在标准后训练数据上的监督微调后依然保持稳定。我们进一步分析指纹安全风险，并从理论和实证角度证明，此类可扩展指纹方案能有效缓解风险。代码已开源于https://github.com/SewoongLab/scalable-fingerprinting-of-llms。
原文摘要 · Abstract (English)
Model fingerprinting has emerged as a powerful tool for model owners to identify their shared model given API access. However, to lower false discovery rate, fight fingerprint leakage, and defend against coalitions of model users attempting to bypass detection, we argue that {\em scalability} is critical, i.e., scaling up the number of fingerprints one can embed into a model. Hence, we pose scalability as a crucial requirement for fingerprinting schemes. We experiment with fingerprint design at a scale significantly larger than previously considered, and introduce a new method, dubbed Perinucleus sampling, to generate scalable, persistent, and harmless fingerprints. We demonstrate that this scheme can add 24,576 fingerprints to a Llama-3.1-8B model -- two orders of magnitude more than existing schemes -- without degrading the model's utility. Our inserted fingerprints persist even after supervised fine-tuning on standard post-training data. We further address security risks for fingerprinting, and theoretically and empirically show how a scalable fingerprinting scheme like ours can mitigate these risks. Our code is available at https://github.com/SewoongLab/scalable-fingerprinting-of-llms
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。