同一系数在不同模型中效果迥异，暴露跨模型分词器移植的安全隐患。
When the Same Coefficients Reach Different Places: Asymmetric Realizability in Transplanting Tokenizers across Large Language Models
- 用加权组合重建分词器嵌入，发现相同系数在源与目标模型中表现不一
- 构造出仅在目标模型激活的‘破坏性令牌’，源模型中却无反应
- 揭示开放权重模型组合链中的潜在安全风险，适合关注模型安全的研究者
跨词汇表模型组合中的分词器移植通过共享词元锚点的加权组合重构仅存在于源模型的嵌入行，并将这些系数复用于目标模型。我们识别出该重构的结构几何特性：相同的系数向量在源与目标锚点空间中可实现的集合不同，即存在‘非对称可实现性’差距。在65组源-目标配对中（使用OMP），经CLP、WECHSEL和FOCUS跨操作验证，我们构造了‘破坏性令牌’：单一系数向量在源锚点空间中统计上无影响，但在目标模型中却产生高显著性重构。同一Gemma-2-2B源检查点可对抗13个来自五个模型家族的目标模型。植入方向在权重合并后仍保持干净参考。在部署者案例研究中，标准LoRA微调仅抑制匹配训练语料分布的提示，不足以防御此类攻击。测试的谱滤波器未能捕捉该不对称性。我们讨论了其在开放权重模型组合供应链中可能被滥用的风险。
原文摘要 · Abstract (English)
Tokenizer transplant in cross-vocabulary model composition reconstructs donor-only embedding rows as weighted combinations over shared lexical anchors and reuses those coefficients on the base. We identify a structural geometric property of this reconstruction: the same coefficient vector reaches different sets in the donor and base anchor spans, an \emph{asymmetric realizability} gap. Across 65 donor-base pairs under OMP, with cross-operator validation on CLP, WECHSEL, and FOCUS, we construct \textit{breaker tokens}: single coefficient vectors that remain statistically inert in the donor anchor span while producing a high-salience reconstruction in the base. The same Gemma-2-2B donor checkpoint admits this construction against 13 different downstream bases drawn from five model families. The planted direction passes weight-merging with a clean reference unchanged. In a deployer case study, standard LoRA fine-tuning suppresses the breaker primarily on prompts whose distribution matches the training corpus and is not a sufficient mitigation against this attack family in our setting. The tested spectral filters miss the asymmetry. We discuss potential misuse in the open-weight composition supply chain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。