arXiv:2506.18053cs.CRcs.AI2025-06

破解隐私保护中的模型混淆,发现它能隐藏用户输入却不影响整体性能。

Mechanistic Interpretability in the Presence of Architectural Obfuscation

  • 用混淆技术打乱模型内部表示,测试其对可解释性的影响。
  • 混淆后注意力头激活模式紊乱,但整体计算结构仍完整。
  • 适合关注隐私与可解释性平衡的研究者或安全系统设计者。

架构混淆(如打乱隐藏状态张量、线性变换嵌入表或重映射标记)近年被用作轻量级隐私保护方案,替代复杂的加密机制。尽管已有研究证明此类方法易受重建攻击,但其对机制可解释性的影响尚未系统研究。本文以从零训练的 GPT-2-small 模型为例,采用典型混淆映射,在混淆映射私有、原始基底未知(模拟诚实但好奇的服务器)的前提下,使用 logit-lens 归因、因果路径修补和注意力头消融等方法定位并操控已知电路。结果表明,混淆显著改变注意力头内的激活模式,但保持层间计算图结构不变。这导致因果追踪失去与基线语义的对齐,标记级 logit 归因噪声过大无法重构用户输入。而前馈与残差路径功能依然完好,说明混淆降低了细粒度可解释性,但未影响顶层任务表现。研究提供了定量证据:架构混淆可同时(i)保留全局模型行为,(ii)阻碍用户特定内容的机制分析。该工作为未来隐私防御与鲁棒可解释性工具设计提供指引。

原文摘要 · Abstract (English)

Architectural obfuscation - e.g., permuting hidden-state tensors, linearly transforming embedding tables, or remapping tokens - has recently gained traction as a lightweight substitute for heavyweight cryptography in privacy-preserving large-language-model (LLM) inference. While recent work has shown that these techniques can be broken under dedicated reconstruction attacks, their impact on mechanistic interpretability has not been systematically studied. In particular, it remains unclear whether scrambling a network's internal representations truly thwarts efforts to understand how the model works, or simply relocates the same circuits to an unfamiliar coordinate system. We address this gap by analyzing a GPT-2-small model trained from scratch with a representative obfuscation map. Assuming the obfuscation map is private and the original basis is hidden (mirroring an honest-but-curious server), we apply logit-lens attribution, causal path-patching, and attention-head ablation to locate and manipulate known circuits. Our findings reveal that obfuscation dramatically alters activation patterns within attention heads yet preserves the layer-wise computational graph. This disconnect hampers reverse-engineering of user prompts: causal traces lose their alignment with baseline semantics, and token-level logit attributions become too noisy to reconstruct. At the same time, feed-forward and residual pathways remain functionally intact, suggesting that obfuscation degrades fine-grained interpretability without compromising top-level task performance. These results establish quantitative evidence that architectural obfuscation can simultaneously (i) retain global model behaviour and (ii) impede mechanistic analyses of user-specific content. By mapping where interpretability breaks down, our study provides guidance for future privacy defences and for robustness-aware interpretability tooling.

可解释性隐私保护模型混淆大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。