用转换器追踪扩散模型内部语义流动，揭示图文交互机制。
DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing
- 用时序条件转换器替代MLP层，实现特征到特征的精确归因。
- 恢复出紧凑可解释的电路，揭示属性绑定与跨流传播机制。
- 适合研究生成模型机理或改进可控生成的开发者使用。
机制可解释性旨在通过分解模型计算为可解释特征和电路来理解神经网络行为。尽管基于转换器的电路追踪已实现对大语言模型的详细因果分析，但用于图像生成的多模态扩散变压器仍相对不透明。我们缺乏工具来理解语义信息如何在去噪步骤间传播，以及文本与图像表示如何在双流MM-DiT架构中交互。现有方法仅提供部分洞察：注意力图仅展示令牌间有限互动，而稀疏自编码器虽能发现可解释特征，却无法直接揭示这些特征如何经非线性MLP层变换与组合。本文将基于转换器的电路追踪扩展至多模态扩散变压器。我们训练了时序条件转换器，准确逼近FLUX.1[schnell]中MLP子层的输入-输出行为。通过用转换器替换MLP并线性化剩余计算，实现了精确的特征到特征归因，并恢复出紧凑可解释的电路。实验表明，我们的转换器在稀疏性-忠实度权衡上表现相当或略优。所得电路揭示了属性绑定与跨流语义传播的机制，并提供了系统生成错误的因果解释。此外，基于电路的干预比传统SAE引导更精准有效。结果表明，基于转换器的电路分析适用于最先进的扩散变压器，为理解与控制多模态生成模型提供了强大框架。代码见https://github.com/Artalmaz31/DifFRACT。
原文摘要 · Abstract (English)
Mechanistic interpretability seeks to explain neural network behavior by decomposing model computations into interpretable features and circuits. While transcoder-based circuit tracing has recently enabled detailed causal analyses of large language models, multimodal diffusion transformers for image generation remain comparatively opaque. We still lack tools for understanding how semantic information propagates across denoising steps and how text and image representations interact within double-stream MM-DiT architectures. Existing methods provide only partial insight: attention maps expose a limited view of token interactions, while sparse autoencoders can discover interpretable features but do not directly reveal how these features are transformed and composed through nonlinear MLP layers. In this work, we extend transcoder-based circuit tracing to multimodal diffusion transformers. We train timestep-conditioned transcoders that faithfully approximate the input-output behavior of MLP sublayers in FLUX.1[schnell]. By replacing MLPs with transcoders and linearizing the remaining computation, we obtain exact feature-to-feature attribution and recover compact, interpretable circuits. Empirically, our transcoders match or slightly outperform sparse autoencoders on the sparsity-faithfulness tradeoff. The resulting circuits reveal mechanisms underlying attribute binding and cross-stream semantic propagation, and provide causal explanations for systematic generation errors. Moreover, circuit-guided interventions are substantially more precise and effective than standard SAE-based steering. Our results demonstrate that transcoder-based circuit analysis is feasible for state-of-the-art diffusion transformers and provides a powerful framework for understanding and controlling multimodal generative models. The code is available at https://github.com/Artalmaz31/DifFRACT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。