攻击者通过恶意代码在本地微调中窃取敏感信息,成功率超98%。
Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors

- 利用伪装成标准架构的恶意代码,实现主动执行劫持而非被动权重污染。
- 在默认LoRA设置下,秘密泄露严格准确率超98%,且模型主任务性能下降有限。
- 可黑盒验证泄露真实性,有效绕过安全过滤、代码审计和困惑度检测。
本地微调数据常包含API密钥、个人标识符和财务记录等敏感信息。尽管‘本地离线微调’常被视为隐私边界,我们揭示:被篡改的模型代码足以窃取这些信息。现有被动预训练权重污染攻击对稀疏高熵目标(如密钥)无效,因其依赖概率语义前缀。为此,我们发现并利用一个被忽视的供应链漏洞——将恶意代码伪装为标准架构定义,实现从被动污染到主动执行劫持的范式转变。提出确定性全链路记忆机制:通过在线张量规则匹配锁定动态计算流中的令牌级秘密,并利用值梯度解耦技术隐蔽注入攻击梯度,克服梯度淹没问题,强制模型记忆。此外,首次实现攻击者可验证的黑盒秘密窃取,通过精准区分真实泄露与幻觉。该攻击在默认LoRA设置下严格准确率超98%,主任务性能损失小,且有效规避语义安全过滤、代码审计及困惑度检测等防御措施。
原文摘要 · Abstract (English)
Local fine-tuning datasets routinely contain sensitive secrets such as API keys, personal identifiers, and financial records. Although "local offline fine-tuning" is often viewed as a privacy boundary, we reveal that compromised model code is sufficient to steal them. Current passive pretrained-weight poisoning attacks, while effective for natural language, fundamentally fail to capture such sparse high-entropy targets due to their reliance on probabilistic semantic prefixes. To bridge this gap, we identify and exploit a practical but overlooked supply-chain vector -- malicious model code camouflaged as standard architectural definitions to realize a paradigm shift from passive weight poisoning to active execution hijacking. We introduce a deterministic full-chain memorization mechanism: it locks onto token-level secrets in dynamic computation flows via online tensor-rule matching, and leverages value-gradient decoupling to stealthily inject attack gradients, overcoming gradient drowning to force model memorization. Furthermore, we achieve, for the first time, attacker-verifiable secret stealing through black-box queries that precisely distinguishes true leakage from hallucination. Our attack achieves over 98% Strict ASR in the default LoRA setting with limited primary-task utility degradation and effectively evades defense measures including semantic safety filtering, code auditing, and perplexity-based detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。