用多语言数据训练稀疏编码器，实现更可靠的跨语言控制。
Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
- 在多语言数据上训练稀疏编码器，提升跨语言表征能力。
- 基于对齐与可分性的交集选择干预层，避免逐层试错。
- 适用于需要稳定跨语言生成的场景，如翻译与摘要任务。
稀疏自编码器（SAEs）为大语言模型提供了特征级可解释性与激活操控能力，但在多语言场景中，基于SAE的语言控制仍不可靠：多数SAE仅在纯英文数据上训练，且干预层选择依赖经验。本文通过构建一种原理化的多语言语言操控机制，解决上述问题。首先，我们证明在多语言数据上训练SAE能持续增强跨语言表征，并在不同层和模型家族中实现更可靠、高质量的语言控制。其次，提出一种无需遍历搜索的先验层选择规则——基于多语言对齐性与语言可分性的交集，精准预测有效干预深度。我们在LLaMA-3.1-8B与Gemma-2-9B上评估该方法，涵盖机器翻译与跨语言摘要（CrossSumm），采用SpBLEU、ROUGE-L、COMET、LaSE等指标。结果表明，多语言SAE结合交集选层策略，稳定了语言识别准确率与生成质量之间的权衡，为多语言SAE操控提供了可预测、基于表征的原理性框架。
原文摘要 · Abstract (English)
Sparse autoencoders (SAEs) enable feature-level mechanistic interpretability and activation steering in large language models (LLMs), but SAE-based language control remains unreliable in multilingual settings: most SAEs are trained on English-only data, and steering layers are chosen heuristically. We address these limitations by advancing a principled, mechanistic account of multilingual language steering with SAEs. First, we show that training SAEs on multilingual data consistently strengthens cross-lingual representations and yields more reliable, quality-preserving language control across layers and model families. Second, we introduce an \emph{a priori} steering layer-selection rule based on the intersection of multilingual alignment and language separability, which predicts effective intervention depths without exhaustive layerwise search. We evaluate our approach on LLaMA-3.1-8B and Gemma-2-9B across machine translation and cross-lingual summarization (CrossSumm), using SpBLEU, ROUGE-L, COMET, and LaSE. Our results show that multilingual SAEs combined with intersection-selected layers stabilize the trade-off between language identification accuracy and generation quality, providing a principled, predictive, representation-level account of multilingual SAE steering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。