首次分析大模型跨语言知识边界认知，发现中间层编码关键信息
Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations
- 通过探测内部表征分析多语言下模型对已知/未知问题的识别机制
- 不同语言间认知差异呈线性结构，可无训练迁移以降低低资源语言幻觉风险
- 双语问答翻译微调能进一步提升跨语言知识边界识别能力
尽管理解大模型的知识边界对防止幻觉至关重要，但现有研究主要聚焦于英语。本文首次通过探测多语言下大模型处理已知与未知问题时的内部表征，分析其跨语言知识边界认知。实证研究揭示三个关键发现：1）知识边界感知在不同语言中均编码于中间至中上层；2）语言间认知差异呈线性结构，据此提出无需训练的对齐方法，有效实现跨语言知识边界感知能力迁移，有助于降低低资源语言的幻觉风险；3）在双语问答对翻译数据上微调，可进一步增强模型跨语言知识边界识别能力。由于缺乏标准的跨语言知识边界评测基准，我们构建了一个包含三类典型知识边界数据的多语言评估套件。代码与数据集已公开于 https://github.com/DAMO-NLP-SG/LLM-Multilingual-Knowledge-Boundaries。
原文摘要 · Abstract (English)
While understanding the knowledge boundaries of LLMs is crucial to prevent hallucination, research on the knowledge boundaries of LLMs has predominantly focused on English. In this work, we present the first study to analyze how LLMs recognize knowledge boundaries across different languages by probing their internal representations when processing known and unknown questions in multiple languages. Our empirical studies reveal three key findings: 1) LLMs' perceptions of knowledge boundaries are encoded in the middle to middle-upper layers across different languages. 2) Language differences in knowledge boundary perception follow a linear structure, which motivates our proposal of a training-free alignment method that effectively transfers knowledge boundary perception ability across languages, thereby helping reduce hallucination risk in low-resource languages; 3) Fine-tuning on bilingual question pair translation further enhances LLMs' recognition of knowledge boundaries across languages. Given the absence of standard testbeds for cross-lingual knowledge boundary analysis, we construct a multilingual evaluation suite comprising three representative types of knowledge boundary data. Our code and datasets are publicly available at https://github.com/DAMO-NLP-SG/LLM-Multilingual-Knowledge-Boundaries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。