让大模型只在必要时思考，大幅节省计算资源。
Ada-RS: Adaptive Rejection Sampling for Selective Thinking
- 根据生成内容质量动态筛选推理路径，避免无意义计算。
- 在电商工具调用任务中，减少80%输出token，思考频率降低95%。
- 适配多种训练策略，适合对延迟敏感的部署场景。
大语言模型在成本与延迟敏感场景中的应用日益增多。尽管思维链能提升推理能力，但对简单请求也会浪费大量 token。本文研究面向工具使用的模型的择优推理问题，提出自适应拒绝采样（Ada-RS）——一种无需依赖具体算法的样本过滤框架，用于学习高效、有选择性的推理。针对每个输入上下文，Ada-RS 使用长度惩罚的自适应奖励函数对多个采样完成进行评分，并通过随机拒绝采样保留高奖励候选（或偏好对）以供后续优化。实验证明，该方法可无缝集成至偏好对（如 DPO）或分组策略优化（如 DAPO）中。在合成的面向工具调用的电商基准上，使用 Qwen3-8B 与 LoRA 微调，相比标准算法，Ada-RS 最多减少 80% 平均输出 token，思考率降低高达 95%，同时保持或提升工具调用准确率。结果表明，训练信号的选择是实现低延迟部署下高效推理的强大杠杆。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly being deployed in cost and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to 80% and reducing thinking rate by up to 95% while maintaining or improving tool call accuracy. These results highlight that training-signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。