用自动优化的提示词引导大模型推理，提升准确率与安全性。
ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimization
- 通过进化算法生成可引导推理行为的think-prefix提示词。
- 使模型准确率-长度权衡显著改善，安全得分从27.0%降至0.7%。
- 无需训练，适配多种任务，也兼容已有训练方法。
大型推理模型（LRMs）虽强大，但仍存在效率低、方向偏离的问题。现有无训练方法或依赖僵硬启发式规则，或仅提供描述性分析。本文提出ThinkPilot，一种无训练框架，可自动优化模型推理过程。它利用进化算法生成think-prefix（提示前缀），基于推理行为分类体系演化，引导模型获得更优表现。大量实验表明：ThinkPilot显著改善推理的准确率-长度权衡，大幅提高安全性（如将DeepSeek-R1-Distill-Qwen-32B的StrongREJECT得分从27.0%降至0.7%），并增强指令遵循能力。分析显示，think-prefix能可靠控制模型推理行为，不同任务对特定行为分布有偏好。通过自动识别并激发这些行为，ThinkPilot为对齐模型推理与任务需求提供了通用方案。数据与代码见https://github.com/teqkilla/ThinkPilot。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) are powerful, but they still suffer from inefficient and off-target reasoning. Currently, training-free methods are limited to either rigid heuristics or descriptive, non-actionable analyses. In this paper, we introduce ThinkPilot, a training-free framework that automatically optimizes LRMs reasoning. It uses an evolutionary process to generate think-prefixes, which are instructions that evolve driven by a taxonomy of reasoning behaviors to guide models toward superior performance. Extensive experiments demonstrate ThinkPilot's broad effectiveness: it significantly improves the accuracy-length trade-off for efficient reasoning, drastically improves safety (for example, cutting the StrongREJECT score of DeepSeek-R1-Distill-Qwen-32B from 27.0% to 0.7), and enhances instruction following. It also synergizes with existing training-based methods. Our analysis reveals that think-prefixes can reliably control LRMs' reasoning behaviors, and that different tasks have strong preferences for specific behavioral distributions. By automatically identifying and eliciting these behaviors, ThinkPilot provides a generalizable framework for aligning LRMs reasoning with task demands. Data and code are available at https://github.com/teqkilla/ThinkPilot
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。