动态修正大模型推理路径,让答案更接近真相。
Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories

- 通过分析推理过程中的语义结构,定位关键纠错节点。
- 在高熵早期分支处干预,准确率提升12.3%(MATH基准)。
- 适合需要精准推理的数学与代码生成任务。
当前增强大语言模型推理的方法,如思维链和‘等待’提示,主要鼓励模型多思考,但常无法引导其走向正确答案。虽然表示编辑(RepE)提供内在控制手段,但在动态推理轨迹上的应用仍不充分。本文通过研究展开推理链中的真理几何结构,发现三点关键规律:(1)真理编码于句子层面,且与潜在推理模式纠缠;(2)有效干预遵循不确定性原理与衰减效应,需定位到早期高熵分叉点;(3)朴素的引导向量易受噪声干扰,可能误伤正确路径。基于此,提出动态表示编辑框架DynaSteer:利用模式聚类解耦推理流形,通过Fisher-LDA投影纯净真理信号,并动态监测前瞻熵,在必要时选择性地引导与回滚路径。在多个MATH基准上的实验验证了其有效性,跨域编码任务测试进一步证明其泛化能力。代码已开源:https://github.com/tianlwang/DynaSteer。
原文摘要 · Abstract (English)
Current approaches to enhance Large Language Model (LLM) reasoning, such as Chain-of-Thought and "Wait" prompts, primarily encourage models to think more, yet often fail to guide them toward Truth. While Representation Editing (RepE) offers a intrinsic control, its application to dynamic reasoning trajectories remains underexplored. In this work, we bridge this gap by investigating the geometry of truth within unfolding reasoning chains. We uncover three critical insights: (1) Truth is encoded at the sentence level and is entangled with latent reasoning patterns; (2) Effective intervention follows an Uncertainty Principle and a Decay Effect, requiring localization to early, high-entropy forks; (3) Naive steering vectors suffer from noise, risking collateral damage to correct trajectories. Based on these findings, we propose DynaSteer, a dynamic RepE framework. DynaSteer employs pattern clustering to disentangle reasoning manifolds and utilizes Fisher-LDA to project purified truth. By dynamically monitoring lookahead entropy, it selectively steers and rolls back trajectories only when necessary. Comprehensive experimental results on several MATH benchmark verify the effectiveness of DynaSteer, and experiments on out-of-domain coding tasks further confirm its generalization ability. Our code is publicly available at https://github.com/tianlwang/DynaSteer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。