提出FlashTrace方法，让大模型推理过程解释更高效准确。
Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs
- 用逐段聚合一次性计算多词重要性，大幅提升效率。
- 在长文本任务中速度比基线快130倍以上，且解释更可信。
- 适合需要理解复杂推理过程的研究者和开发者使用。
token归因方法通过识别输入中因果重要的词元，为语言模型输出提供直观解释。然而，随着现代大模型越来越多依赖长推理链，现有方法面临两大挑战：(1) 效率瓶颈，即在长度为N的上下文中对长度为M的目标片段进行归因需O(M*N)次操作，导致长上下文归因极慢；(2) 可信度下降，中间推理词元吸收了归因质量，导致重要性无法回溯至原始输入。为此，我们提出FlashTrace，一种高效的多词归因方法，采用分段聚合策略，在单次遍历中完成对多词目标的归因，同时保持高可信度。此外，我们设计了递归归因机制，将重要性沿中间推理链回溯至源头输入。在长上下文检索（RULER）和多步推理（MATH、MorehopQA）任务上的实验表明，FlashTrace相比现有基线实现超过130倍的速度提升，同时保持更优的可信度。我们进一步分析了递归归因动态，发现即使一次递归跳跃也能通过推理链追踪提升可信度。
原文摘要 · Abstract (English)
Token attribution methods provide intuitive explanations for language model outputs by identifying causally important input tokens. However, as modern LLMs increasingly rely on extended reasoning chains, existing schemes face two critical challenges: (1) efficiency bottleneck, where attributing a target span of M tokens within a context of length N requires O(M*N) operations, making long-context attribution prohibitively slow; and (2) faithfulness drop, where intermediate reasoning tokens absorb attribution mass, preventing importance from propagating back to the original input. To address these, we introduce FlashTrace, an efficient multi-token attribution method that employs span-wise aggregation to compute attribution over multi-token targets in a single pass, while maintaining faithfulness. Moreover, we design a recursive attribution mechanism that traces importance through intermediate reasoning chains back to source inputs. Extensive experiments on long-context retrieval (RULER) and multi-step reasoning (MATH, MorehopQA) tasks demonstrate that FlashTrace achieves over 130x speedup over existing baselines while maintaining superior faithfulness. We further analyze the dynamics of recursive attribution, showing that even a single recursive hop improves faithfulness by tracing importance through the reasoning chain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。