arXiv:2509.00973cs.CRcs.AI2025-09中稿 · publication in the…被引 3

通过泄露的输出概率,低成本复制黑盒大模型。

Clone What You Can't Steal: Black-Box LLM Replication via Logit Leakage and Distillation

  • 从少于1万次查询中提取顶级概率,用奇异值分解重建输出层。
  • 4层学生模型实现17.1%加速、18.1%参数减少,性能接近原模型。
  • 全程不超过24小时GPU时间,不触发接口限流,适合隐蔽攻击者。

大型语言模型(LLMs)日益用于卫星操作、指挥控制、军事决策支持和网络防御等关键系统,通常通过API访问。当这些API缺乏强访问控制时,可能暴露完整或前k个logits,形成被忽视的安全风险。现有研究多聚焦于重构输出投影层或蒸馏表面行为,但在严格查询限制下复现黑盒模型仍属空白。本文提出一种受限复制流程,将部分logit泄露转化为可部署的替代模型。该两阶段方法首先通过奇异值分解(SVD)在不足10,000次查询内重构输出投影矩阵;其次在开源数据集上训练不同深度的紧凑学生模型以蒸馏剩余架构。6层学生模型复现了97.6%的6层教师模型隐藏状态几何结构,困惑度仅增加7.31%,负对数似然(NLL)为7.58。4层变体实现17.1%推理加速与18.1%参数压缩,性能相当。整个攻击过程耗时低于24 GPU小时,未触发API速率限制防御。结果表明,低成本对手可快速克隆LLM,凸显强化推理API与安全本地部署的紧迫性。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are increasingly deployed in mission-critical systems, facilitating tasks such as satellite operations, command-and-control, military decision support, and cyber defense. Many of these systems are accessed through application programming interfaces (APIs). When such APIs lack robust access controls, they can expose full or top-k logits, creating a significant and often overlooked attack surface. Prior art has mainly focused on reconstructing the output projection layer or distilling surface-level behaviors. However, regenerating a black-box model under tight query constraints remains underexplored. We address that gap by introducing a constrained replication pipeline that transforms partial logit leakage into a functional deployable substitute model clone. Our two-stage approach (i) reconstructs the output projection matrix by collecting top-k logits from under 10k black-box queries via singular value decomposition (SVD) over the logits, then (ii) distills the remaining architecture into compact student models with varying transformer depths, trained on an open source dataset. A 6-layer student recreates 97.6% of the 6-layer teacher model's hidden-state geometry, with only a 7.31% perplexity increase, and a 7.58 Negative Log-Likelihood (NLL). A 4-layer variant achieves 17.1% faster inference and 18.1% parameter reduction with comparable performance. The entire attack completes in under 24 graphics processing unit (GPU) hours and avoids triggering API rate-limit defenses. These results demonstrate how quickly a cost-limited adversary can clone an LLM, underscoring the urgent need for hardened inference APIs and secure on-premise defense deployments.

模型复制黑盒攻击日志泄露蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。