arXiv:2603.26236cs.CL2026-03

发现并控制跨语言的非正式语体抽象表征

A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs

  • 用稀疏自编码器探测模型内部表示,识别非正式语体特征
  • 在深层网络中发现一个几何一致的跨语言共享语体子空间
  • 可零样本迁移至6种不同语系语言,实现形式化程度可控

尽管多语言大模型能跨语言传递事实与句法知识,但其是否将文化特异的语用风格(如俚语)作为独立的语言特异性记忆或统一的抽象概念仍不明确。我们通过在Gemma-2-9B-IT模型中使用稀疏自编码器(SAEs),在英语、希伯来语和俄语三种类型差异大的语言上进行探究。为排除词汇敏感性的干扰,构建了一个新数据集,其中每个目标词在字面与非正式语境中均出现。结果发现,尽管大部分非正式语体信号分布于语言特定特征中,但仍存在一小部分高度鲁棒的跨语言核心特征持续浮现。该共享核心构成一个几何上连贯的‘非正式语体子空间’,并在模型深层逐渐增强。关键的是,这些共享表征并非仅相关:通过激活操控,可因果性地改变所有源语言输出的形式化程度,并零样本迁移至六种未见语言(涵盖多种语系与文字)。这些结果首次提供了机制证据,表明多语言大模型将非正式语体内化为可移植的、语言无关的语用抽象。

原文摘要 · Abstract (English)

While multilingual language models successfully transfer factual and syntactic knowledge across languages, it remains unclear whether they process culture-specific pragmatic registers, such as slang, as isolated language-specific memorizations or as unified, abstract concepts. We study this by probing the internal representations of Gemma-2-9B-IT using Sparse Autoencoders (SAEs) across three typologically diverse source languages: English, Hebrew, and Russian. To definitively isolate pragmatic register processing from trivial lexical sensitivity, we introduce a novel dataset in which every target term is polysemous, appearing in both literal and informal contexts. We find that while much of the informal-register signal is distributed across language-specific features, a small but highly robust cross-linguistic core consistently emerges. This shared core forms a geometrically coherent ``informal register subspace'' that sharpens in the model's deeper layers. Crucially, these shared representations are not merely correlational: activation steering with these features causally shifts output formality across all source languages and transfers zero-shot to six unseen languages spanning diverse language families and scripts. Together, these results provide the first mechanistic evidence that multilingual LLMs internalize informal register not just as surface-level heuristics, but as a portable, language-agnostic pragmatic abstraction.

语用学跨语言表征学习稀疏编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。