打造可扩展的跨层转换器工具库,让大模型机制解释更高效清晰。
CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
- 通过跨层共享特征提升模型表示紧凑性
- 支持分布式训练与压缩激活缓存,实现大规模训练
- 集成自动解释流程与可视化界面,适合研究者快速分析
机制可解释性旨在理解大语言模型如何表征和处理信息。基于字典学习和转换器的近期方法能用稀疏、可解释的特征及其交互表示模型计算,生成特征归因图。但这些图常规模庞大且冗余,实际可解释性受限。跨层转换器(CLTs)通过跨层共享特征并保留层特定解码,实现更紧凑的表示,但其训练与分析在大规模下仍具挑战。本文提出一个开源库,支持CLTs的端到端训练与可解释性分析。框架集成可扩展分布式训练、模型分片与压缩激活缓存,统一自动化可解释性分析流程,使用Circuit-Tracer计算归因图,并提供灵活可视化界面。该方案为基于CLT的机制可解释性提供了实用且统一的规模化解决方案。代码已开源:https://github.com/LLM-Interp/CLT-Forge。
原文摘要 · Abstract (English)
Mechanistic interpretability seeks to understand how Large Language Models (LLMs) represent and process information. Recent approaches based on dictionary learning and transcoders enable representing model computation in terms of sparse, interpretable features and their interactions, giving rise to feature attribution graphs. However, these graphs are often large and redundant, limiting their interpretability in practice. Cross-Layer Transcoders (CLTs) address this issue by sharing features across layers while preserving layer-specific decoding, yielding more compact representations, but remain difficult to train and analyze at scale. We introduce an open-source library for end-to-end training and interpretability of CLTs. Our framework integrates scalable distributed training with model sharding and compressed activation caching, a unified automated interpretability pipeline for feature analysis and explanation, attribution graph computation using Circuit-Tracer, and a flexible visualization interface. This provides a practical and unified solution for scaling CLT-based mechanistic interpretability. Our code is available at: https://github.com/LLM-Interp/CLT-Forge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。