用双曲几何指导大模型多步推理，提升准确率同时保持高效。
HyperGuide: Hyperbolic Guidance for Efficient Multi-Step Reasoning in Large Language Models
- 将推理过程映射到双曲空间，用距离原点远近表示解的接近程度。
- 在多个基准上显著提升深层推理链的准确率，尤其对复杂问题效果更优。
- 轻量级设计适合部署，适用于需要高精度推理的应用场景。
多步推理仍是大语言模型的核心挑战：单次生成效率高但准确性不足；树搜索方法虽能探索多路径，却计算开销大。本文提出将推理进展提炼为双曲几何信号，引导逐步生成。其思路源于观察：组合推理树中解的状态稀少，而死胡同呈指数增长。双曲空间具备原点附近体积紧凑、边界处容量指数扩张的特性，使距原点距离自然编码解的接近度，角度分离则区分需不同操作的分支。我们训练一个轻量级头模块，将模型隐藏状态投影至该空间，并通过低秩适配器交互式微调自身推理尝试以响应注入信号。在多个基准测试中，几何信号带来一致性能提升，且在更深推理链上表现更优。代码已开源：https://github.com/yuyuliu11037/HyperGuide。
原文摘要 · Abstract (English)
Multi-step reasoning remains a central challenge for large language models: single-pass generation is efficient but lacks accuracy; tree-search methods explore multiple paths but are computation-heavy. We address this gap by distilling reasoning progress into a hyperbolic geometric signal that guides step-by-step generation. Our approach is motivated by a structural observation: in combinatorial reasoning trees, solution-bearing states are few while dead ends are exponentially numerous. The hyperbolic space matches this asymmetry, with compact volume near the origin and exponentially expanding capacity toward the boundary, so that distance-to-origin naturally encodes solution proximity while angular separation distinguishes branches requiring different next operations. We train a lightweight head to project LLM hidden states into this space, then fine-tune a low-rank adapter interactively on its own reasoning attempts to act on the injected signal. Across multiple benchmarks, the geometric signal yields consistent gains, with larger improvements on deeper reasoning chains. Our code is publicly available at https://github.com/yuyuliu11037/HyperGuide.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。