发现多语言模型在混语输入时存在语言锚定偏差，提出新方法修复性能下降。
Code-Switching Reveals Language Anchoring in Multilingual LLMs
- 用语法强制混语设计诊断模型内部表示，定位语言锚点。
- 混语中目标语言框架导致性能下降，源语言框架则保持稳定。
- 提出CANVAS干预方法，在推理时对齐语言向量，提升问答准确率。
多语言大模型日益需要处理混语输入，但混语常导致性能低于单一语言模型。为理解这一现象，我们采用语法强制混语作为受控诊断场景，定位混语表征相对于源语言和目标语言表征的位置。我们提出锚定偏差（Anchor Bias）——一种几何度量，用于衡量混语隐藏状态更接近源语言还是目标语言表征。在多种多语言大模型中，锚定偏差揭示了一致的语法框架效应：源语言框架的混语保持源语言锚定，而目标语言框架的混语则向目标语言偏移，并表现出更大的问答（QA）性能下降。基于此表征模式，我们提出CANVAS（上下文锚定式神经向量对齐调控），一种推理阶段的干预方法：从输入中提取源语言画布，软性引导目标语言隐藏状态向源语言锚点对齐。CANVAS在各类多语言模型与混语条件下一致恢复了问答F1分数，表明内部锚定信号可作为缓解混语推理失败的可操作目标。
原文摘要 · Abstract (English)
Multilingual Large Language Models (MLLMs) are increasingly expected to handle Code-Switched (CS) inputs, yet mixing languages frequently degrades performance relative to source- or target-language monolingual counterparts. To understand this degradation, we use grammar-forced CS as a controlled diagnostic setting for locating CS representations relative to their source and target counterparts. We introduce Anchor Bias, a geometric measure that quantifies language anchoring, whether a CS hidden state aligns closer to its source or target language counterpart. Across diverse MLLMs, Anchor Bias reveals a consistent grammar-frame effect: source-framed CS stays source-anchored, whereas target-framed CS shifts target-ward and shows larger Question Answering (QA) degradation. Motivated by this representational pattern, we propose CANVAS (Contextual Anchor-based Neural Vector Alignment Steering), an inference-time intervention that extracts a source-side canvas from the input and softly steers target-language hidden states toward the source anchor during prefill. CANVAS consistently recovers QA F1 across MLLMs and CS conditions, showing that internal anchoring signals provide an actionable target for mitigating CS inference failures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。