让AI自适应选提示策略，省资源又准
Dynamic Policy Induction for Adaptive Prompt Optimization: Bridging the Efficiency-Accuracy Gap via Lightweight Reinforcement Learning
- 用轻量强化学习自动选提示方法，只在必要时才用复杂策略
- 算术题上比自洽法少61.5%的token消耗，准确率还高
- 适合追求高效部署的AI工程师和系统设计者
大语言模型的表现高度依赖提示策略，但零样本、少样本或思维链等静态方法存在效率与准确率的固有权衡。高精度的自洽策略在简单任务上浪费大量计算资源，而轻量方法在复杂输入上又表现不佳。本文提出提示策略网络（PPN），一种基于轻量强化学习的框架，将自适应策略选择建模为单步马尔可夫决策过程。通过近端策略优化（PPO）训练，并由显式资源奖励函数引导，PPN学会仅在必要时使用高成本推理策略。在算术推理基准上的实验表明，相较于自洽法，PPN在效率-准确率帕累托前沿上表现更优，最多可减少61.5%的token开销，同时保持竞争力的准确率。本工作为低成本高效部署大模型提供了系统性框架，推动了轻量化优化技术的发展，助力可扩展、可持续的语言模型应用。
原文摘要 · Abstract (English)
The performance of Large Language Models (LLMs) depends heavily on the chosen prompting strategy, yet static approaches such as Zero-Shot, Few-Shot, or Chain-of-Thought (CoT) impose a rigid efficiency-accuracy trade-off. Highly accurate strategies like Self-Consistency (SC) incur substantial computational waste on simple tasks, while lightweight methods often fail on complex inputs. This paper introduces the Prompt Policy Network (PPN), a lightweight reinforcement learning framework that formalizes adaptive strategy selection as a single-step Markov Decision Process (MDP). The PPN, trained with Proximal Policy Optimization (PPO) and guided by a resource-explicit reward function, learns to allocate costly reasoning strategies only when necessary. Experiments on arithmetic reasoning benchmarks demonstrate that PPN achieves superior performance on the efficiency-accuracy Pareto front, delivering up to 61.5% token cost reduction compared to Self-Consistency while maintaining competitive accuracy. This work contributes a systematic, adaptive framework for cost-efficient LLM deployment, advancing the design of lightweight optimization techniques for scalable and sustainable language model applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。