用语言结构特征标记大模型生成文本，不损失质量还能100%检测
SLAM: Structural Linguistic Activation Marking for Language Models
- 将水印写入语言结构方向而非词频，避免影响生成质量
- 在Gemma-2模型上实现100%检测率，质量损失仅1-2分（原方法7.5-11.5）
- 适合对文本自然度要求高的场景，但对改写较敏感
大模型水印需可检测且不损害文本质量，但现有方案多通过改变下一个词分布来实现，导致明显质量下降。我们提出SLAM（Structural Linguistic Activation Marking），一种白盒水印方案，将水印写入结构几何而非词频：稀疏自编码器识别残差流中编码语言结构（如语态、时态、从句顺序）的方向，并在生成时因果性地引导这些方向，保持词汇采样与语义不受约束。在Gemma-2 2B和9B模型上，SLAM实现100%检测准确率，质量损失仅为1-2奖励点——远低于KGW、EWD和Unigram的7.5-11.5。自然度与多样性在两模型上均接近未加水印水平。代价是互补鲁棒性：SLAM抗词级修改，但对重构语法的改写敏感（需牺牲质量），与依赖词频的方法相反。
原文摘要 · Abstract (English)
LLM watermarks must be detectable without compromising text quality, yet most existing schemes bias the next-token distribution and pay for detection with measurable quality loss. We present SLAM (Structural Linguistic Activation Marking), a novel white-box watermarking scheme that sidesteps this cost by writing the mark into structural geometry rather than token frequencies: sparse autoencoders identify residual-stream directions encoding linguistic structure (e.g., voice, tense, clause order), and we causally steer those directions at generation time, leaving lexical sampling and semantics unconstrained. On Gemma-2 2B and 9B, SLAM achieves 100% detection accuracy with a quality cost of only 1-2 reward points - compared to 7.5-11.5 for KGW, EWD, and Unigram - with naturalness and diversity preserved at near-unwatermarked levels across both models. The trade-off is a complementary robustness profile: SLAM resists word-level edits but is vulnerable to paraphrase that restructures syntax (at a quality cost), the converse of token-distribution methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。