arXiv:2507.14900cs.CL2025-07ACL被引 11

用神经元激活模式评估大模型跨语言对齐能力,更贴近语义本质。

From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment

  • 基于神经元激活重叠度设计新型评估方法,更贴近语义对齐本质。
  • 仅用100组平行句即达到0.9556的相关性,验证方法有效性。
  • 适合研究多语言模型语义理解与跨语言对齐的学者使用。

大型语言模型(LLMs)展现出卓越的多语言能力,但如何评估跨语言对齐仍缺乏深入研究。现有基准主要关注句子嵌入,但先前研究表明神经网络倾向于生成非平滑的表示空间,影响低资源语言的语义对齐评估。受神经科学发现启发——相似信息会激活重叠的神经元区域,我们提出一种基于神经元状态的跨语言对齐评估方法(NeuronXA),为评估多语言大模型的跨语言对齐能力提供更语义化的路径。我们在多个主流多语言模型(LLaMA、Qwen、Mistral、GLM 和 OLMo)上,通过两个迁移任务和三个多语言基准进行评估。结果表明,仅需100组平行句子对,NeuronXA在下游任务性能上的皮尔逊相关系数达0.9556,在可迁移性上的相关系数为0.8514。这证明了该方法在评估跨语言对齐与可迁移性方面的有效性,即使在小数据条件下亦然。该研究为推进跨语言对齐研究及提升多语言大模型的语义理解能力提供了新思路。

原文摘要 · Abstract (English)

Large language models (LLMs) have demonstrated remarkable multilingual capabilities, however, how to evaluate cross-lingual alignment remains underexplored. Existing alignment benchmarks primarily focus on sentence embeddings, but prior research has shown that neural models tend to induce a non-smooth representation space, which impact of semantic alignment evaluation on low-resource languages. Inspired by neuroscientific findings that similar information activates overlapping neuronal regions, we propose a novel Neuron State-Based Cross-Lingual Alignment (NeuronXA) to assess the cross-lingual a lignment capabilities of LLMs, which offers a more semantically grounded approach to assess cross-lingual alignment. We evaluate NeuronXA on several prominent multilingual LLMs (LLaMA, Qwen, Mistral, GLM, and OLMo) across two transfer tasks and three multilingual benchmarks. The results demonstrate that with only 100 parallel sentence pairs, NeuronXA achieves a Pearson correlation of 0.9556 with downstream tasks performance and 0.8514 with transferability. These findings demonstrate NeuronXA's effectiveness in assessing both cross-lingual alignment and transferability, even with a small dataset. This highlights its potential to advance cross-lingual alignment research and to improve the semantic understanding of multilingual LLMs.

跨语言对齐大模型评估神经元分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。