让对齐技术适配现代大模型，可视化分析内部机制。
LogitLens4LLMs: Extending Logit Lens Analysis to Modern Large Language Models
- 通过组件钩子捕获注意力与MLP输出，兼容主流模型架构。
- 支持批量与交互式分析，实现低延迟的大规模层间探测。
- 开源工具包助力研究者深入理解大模型内部运作逻辑。
本文提出 LogitLens4LLMs，一个将对齐分析技术扩展至现代大型语言模型的工具包。尽管对齐分析在早期模型中已发挥重要作用，但其先前实现仅限于旧有架构。本工作突破了现有方法的局限，使该技术可应用于前沿模型（如 Qwen-2.5 与 Llama-3.1），并自动化关键分析流程。通过开发针对特定组件的钩子，完整捕获注意力机制与 MLP 输出，新实现与 HuggingFace Transformers 库完全兼容，且推理开销极低。工具包支持交互式探索与批处理，适用于大规模逐层分析。通过开源发布，我们希望推动对大模型内部机理的更深入研究。代码已公开于 https://github.com/zhenyu-02/LogitLens4LLMs。
原文摘要 · Abstract (English)
This paper introduces LogitLens4LLMs, a toolkit that extends the Logit Lens technique to modern large language models. While Logit Lens has been a crucial method for understanding internal representations of language models, it was previously limited to earlier model architectures. Our work overcomes the limitations of existing implementations, enabling the technique to be applied to state-of-the-art architectures (such as Qwen-2.5 and Llama-3.1) while automating key analytical workflows. By developing component-specific hooks to capture both attention mechanisms and MLP outputs, our implementation achieves full compatibility with the HuggingFace transformer library while maintaining low inference overhead. The toolkit provides both interactive exploration and batch processing capabilities, supporting large-scale layer-wise analyses. Through open-sourcing our implementation, we aim to facilitate deeper investigations into the internal mechanisms of large-scale language models. The toolkit is openly available at https://github.com/zhenyu-02/LogitLens4LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。