通过分析模型权重发现推理路径，提升大模型复杂问题解决能力
Weight-of-Thought Reasoning: Exploring Neural Network Weights for Enhanced LLM Reasoning
- 用图神经网络建模权重间的推理关系，挖掘内部动态
- 在数学、逻辑等任务上显著优于传统方法，复杂问题提升更明显
- 适合关注模型可解释性与推理机制研究的读者
大型语言模型（LLMs）在采用思维链（CoT）提示时展现出卓越的推理能力，但现有方法仅关注输出词元层面，忽视内部权重动态。本文提出权重之思（WoT）推理，通过图结构消息传递、多步推理和注意力机制，探索推理前的权重空间，构建相互连接的推理节点图。在演绎、数学、代数、组合及几何等多种推理任务上的实验表明，该方法在复杂问题上表现显著优于传统方法，同时提升了推理过程的可解释性，为增强大模型推理能力提供了新方向。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable reasoning capabilities when prompted with strategies such as Chain-of-Thought (CoT). However, these approaches focus on token-level output without considering internal weight dynamics. We introduce Weight-of-Thought (WoT) reasoning, a novel approach that examines neural network weights before inference to identify reasoning pathways. Unlike existing methods, WoT explores the weight space through graph-based message passing, multi-step reasoning processes, and attention mechanisms. Our implementation creates an interconnected graph of reasoning nodes. Experiments on diverse reasoning tasks (syllogistic, mathematical, algebraic, combinatorial, and geometric) demonstrate that WoT achieves superior performance compared to traditional methods, particularly for complex problems. This approach leads to both improved performance and greater interpretability of the reasoning process, offering a promising direction for enhancing LLM reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。