通过引导稀疏自编码器减少大模型意外语言混杂问题
SASFT: Sparse Autoencoder-guided Supervised Finetuning to Mitigate Unexpected Code-Switching in LLMs
- 用稀疏自编码器分析发现：异常换语时目标语言特征预激活值过高
- 提出SASFT方法，训练中控制特定语言特征的预激活水平，使换语率下降超50%
- 既消除语言混杂又保持多语言能力，适合需要稳定输出的部署场景
大型语言模型虽具备出色的多语言能力，但常出现意外语言混杂（即在响应中无故切换到未预期的语言），导致可读性差、影响使用。现有研究缺乏机制分析且效果有限。本文首次利用稀疏自编码器深入分析该问题，发现当模型切换语言时，目标语言特征的预激活值显著过高。基于此，提出稀疏自编码器引导的监督微调（SASFT），在训练中引导模型维持特定语言特征的适当预激活水平。在三个语言上的五种模型实验表明，与标准监督微调相比，SASFT将意外语言混杂率降低超过50%，其中一种情况实现完全消除。同时，模型在六个多语言基准测试上的表现保持或提升，验证了其在抑制语言混杂的同时有效保留多语言能力。代码与数据已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have impressive multilingual capabilities, but they suffer from unexpected code-switching, also known as language mixing, which involves switching to unexpected languages in the model response. This problem leads to poor readability and degrades the usability of model responses. However, existing work on this issue lacks a mechanistic analysis and shows limited effectiveness. In this paper, we first provide an in-depth analysis of unexpected code-switching using sparse autoencoders and find that when LLMs switch to a language, the features of that language exhibit excessive pre-activation values. Based on our findings, we propose $\textbf{S}$parse $\textbf{A}$utoencoder-guided $\textbf{S}$upervised $\textbf{F}$ine$\textbf{t}$uning (SASFT), which teaches LLMs to maintain appropriate pre-activation values of specific language features during training. Experiments on five models across three languages demonstrate that SASFT consistently reduces unexpected code-switching by more than 50\% compared to standard supervised fine-tuning, with complete elimination in one case. Moreover, SASFT maintains or even improves the models' performance on six multilingual benchmarks, showing its effectiveness in addressing code-switching while preserving multilingual capabilities. The code and data are available at https://github.com/Aatrox103/SASFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。