通过注入行为模式，无需修改参数就能提升大模型推理能力。
InjectRBP: Steering Large Language Model Reasoning Behavior via Pattern Injection
- 用模型自身正确回答的行为模式作为模板进行注入。
- 在多个任务上提升性能，最高达8.67%。
- 适合想优化推理但无法调参的研究者和应用开发者。
推理能显著提升大语言模型的表现。尽管已有研究通过调整提示来增强推理，但这些方法多依赖直觉，缺乏对底层行为模式的系统分析。我们从行为模式角度研究模型推理机制，发现模型在面对特定问题时会呈现自适应的行为分布，且结构化注入这些模式可显著影响推理质量和结果。基于此，我们提出两种无需参数更新的优化方法：InjectCorrect 通过模仿模型自身历史正确答案的行为模式引导推理；InjectRLOpt 则从历史行为数据中学习价值函数，并利用可靠性感知软最大策略生成推理注入项。实验表明，两种方法均能在不修改模型参数的前提下，在多种推理任务上取得提升，最高分别达5.34%和8.67%。
原文摘要 · Abstract (English)
Reasoning can significantly enhance the performance of Large Language Models. While recent studies have exploited behavior-related prompts adjustment to enhance reasoning, these designs remain largely intuitive and lack a systematic analysis of the underlying behavioral patterns. Motivated by this, we investigate how models' reasoning behaviors shape reasoning from the perspective of behavioral patterns. We observe that models exhibit adaptive distributions of reasoning behaviors when responding to specific types of questions, and that structurally injecting these patterns can substantially influence the quality of the models' reasoning processes and outcomes. Building on these findings, we propose two optimization methods that require no parameter updates: InjectCorrect and InjectRLOpt. InjectCorrect guides the model by imitating behavioral patterns derived from its own past correct answers. InjectRLOpt learns a value function from historical behavior-pattern data and, via our proposed Reliability-Aware Softmax Policy, generates behavioral injectant during inference to steer the reasoning process. Our experiments demonstrate that both methods can improve model performance across various reasoning tasks without requiring any modifications to model parameters, achieving gains of up to 5.34% and 8.67%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。