用轨迹几何分析大模型推理过程，可预测和纠正答案正确性。
LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
- 将推理过程视为表示空间中的结构化路径，分步对应特定子空间。
- 晚期推理阶段正确与错误路径明显分离，可提前预测答案正确性（AUC达0.87）。
- 提出轨迹引导干预方法，可在推理中动态修正结果或控制长度。
本研究将大语言模型的链式思考生成过程建模为表示空间中的结构化轨迹。我们发现数学推理会经过功能有序、步骤特异的子空间，且随着层数加深，这些子空间逐渐可分。这种结构在基础模型中已存在，而推理训练主要加速向终止相关子空间的收敛，而非引入新表征组织。尽管早期推理步骤路径相似，正确与错误解在后期出现系统性偏离。该晚期偏差使我们能在推理中期预测最终答案正确性，最大ROC-AUC达0.87。此外，我们提出基于轨迹的引导机制，一种推理时干预框架，可依据理想轨迹实现推理修正与长度控制。上述结果确立了推理轨迹作为理解、预测与控制大模型推理行为的几何视角。
原文摘要 · Abstract (English)
This work characterizes large language models' chain-of-thought generation as a structured trajectory through representation space. We show that mathematical reasoning traverses functionally ordered, step-specific subspaces that become increasingly separable with layer depth. This structure already exists in base models, while reasoning training primarily accelerates convergence toward termination-related subspaces rather than introducing new representational organization. While early reasoning steps follow similar trajectories, correct and incorrect solutions diverge systematically at late stages. This late-stage divergence enables mid-reasoning prediction of final-answer correctness with ROC-AUC up to 0.87. Furthermore, we introduce trajectory-based steering, an inference-time intervention framework that enables reasoning correction and length control based on derived ideal trajectories. Together, these results establish reasoning trajectories as a geometric lens for interpreting, predicting, and controlling LLM reasoning behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。