为印度语言设计的可解释性框架，提升多语种模型在低资源语言上的理解能力。
Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
- 通过学习共享仿射变换，对齐各语言隐藏状态与目标输出分布。
- 在10种印度语言上测试，显著优于现有方法，尤其对形态丰富的语言效果突出。
- 适合关注多语言模型可解释性、特别是印地语等低资源语言研究者使用。
多语种大语言模型（LLMs）在印度等语言多样性地区日益普及，但多数可解释性工具仍局限于英语。已有研究表明，这些模型常在以英语为中心的表征空间中运行，跨语言可解释性成为迫切问题。我们提出Indic-TunedLens，一种专为印度语言设计的新型可解释性框架，通过学习共享仿射变换来调整每种目标语言的隐藏状态，使其与目标输出分布对齐，从而实现更忠实的模型表征解码。我们在10种印度语言上使用MMLU基准进行评估，结果表明该框架显著优于当前最优可解释性方法，尤其在形态丰富且资源稀少的语言上表现突出。研究揭示了多语种变换器在层间语义编码的深层机制。模型已发布于https://huggingface.co/spaces/MihirRajeshPanchal/IndicTunedLens，代码开源于https://github.com/MihirRajeshPanchal/IndicTunedLens。
原文摘要 · Abstract (English)
Multilingual large language models (LLMs) are increasingly deployed in linguistically diverse regions like India, yet most interpretability tools remain tailored to English. Prior work reveals that LLMs often operate in English centric representation spaces, making cross lingual interpretability a pressing concern. We introduce Indic-TunedLens, a novel interpretability framework specifically for Indian languages that learns shared affine transformations. Unlike the standard Logit Lens, which directly decodes intermediate activations, Indic-TunedLens adjusts hidden states for each target language, aligning them with the target output distributions to enable more faithful decoding of model representations. We evaluate our framework on 10 Indian languages using the MMLU benchmark and find that it significantly improves over SOTA interpretability methods, especially for morphologically rich, low resource languages. Our results provide crucial insights into the layer-wise semantic encoding of multilingual transformers. Our model is available at https://huggingface.co/spaces/MihirRajeshPanchal/IndicTunedLens. Our code is available at https://github.com/MihirRajeshPanchal/IndicTunedLens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。