用自然语言解释大模型行为，让黑箱决策变透明。
Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language
- 构建元模型架构，通过输入模型激活态回答其行为问题。
- 在欺骗性场景下仍能良好泛化，跨任务表现稳定。
- 适合想理解大模型决策逻辑的研究者和开发者。
随着大语言模型日益融入日常生活，其潜在的欺骗性行为凸显了对其决策过程准确解读的必要性。尽管传统探测方法在特定任务中已展现一定效果，但仍局限于狭窄范围，更全面的解释仍属缺失。为此，我们提出元模型架构：一个以‘元模型’为核心，接收‘输入模型’激活特征，并以自然语言回答关于该模型行为问题的系统。通过在特定任务类型上训练元模型，并评估其在欺骗性场景下的分布外表现，我们发现元模型对分布外任务具有良好的泛化能力，为该方向的未来研究开辟了新路径。代码已公开于 https://github.com/acostarelli/meta-models-public。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) become increasingly integrated into our daily lives, the potential harms from deceptive behavior underlie the need for faithfully interpreting their decision-making. While traditional probing methods have shown some effectiveness, they remain best for narrowly scoped tasks while more comprehensive explanations are still necessary. To this end, we investigate meta-models-an architecture using a "meta-model" that takes activations from an "input-model" and answers natural language questions about the input-model's behaviors. We evaluate the meta-model's ability to generalize by training them on selected task types and assessing their out-of-distribution performance in deceptive scenarios. Our findings show that meta-models generalize well to out-of-distribution tasks and point towards opportunities for future research in this area. Our code is available at https://github.com/acostarelli/meta-models-public .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。