通过潜空间方向控制，有效减少多语言模型的无意识语言切换。
Language steering in latent space to mitigate unintended code-switching
- 用PCA提取平行文本中的语言主方向，引导嵌入向量沿该方向变化。
- 单主成分即达95%-99%语言分类准确率，语法分布差异降低55%。
- 轻量级方案适合部署，特别适合需稳定输出单一语言的场景。
多语言大模型常出现无意识语言切换等幻觉现象，影响下游任务可靠性。本文提出潜空间语言引导方法，通过主成分分析（PCA）在并行翻译数据中识别语言方向，并在推理时沿该轴调整词嵌入以控制语言身份。该方法在保持语义不变的同时显著减少语言切换，计算开销极小，仅需少量并行数据即可校准。实验证明，在Qwen2.5和Llama-3.2模型上，仅用一个主成分即可实现95%-99%的语言分类准确率，跨多个语言对的下一词分布差异降低高达55%。基于生成的评估显示，Llama-3.2在四个语言对上代码切换指数下降63%-99%（p < 0.001）。进一步分析表明，语言表征在深层集中，且具有近乎完美的线性可分性。
原文摘要 · Abstract (English)
Multilingual Large Language Models (LLMs) often exhibit hallucinations such as unintended code-switching, reducing reliability in downstream tasks. We propose latent-space language steering, a lightweight inference-time method that identifies language directions via Principal Component Analysis (PCA) on parallel translations and steers token embeddings along these axes to control language identity. Our approach mitigates code-switching while preserving semantics with negligible computational overhead and requires only minimal parallel data for calibration. Empirically, we achieve 95-99\% language classification accuracy using a single principal component and reduce next-token distributional divergence by up to 55\% across multiple language pairs on Qwen2.5 and Llama-3.2 models. Generation-based evaluation on Llama-3.2 further demonstrates 63--99\% reduction in Code-Switching Index across four language pairs ($p < 0.001$). We further analyze the layer-wise evolution of language representations, revealing that language identity concentrates in final layers with near-perfect linear separability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。