用损失函数优化提示词，不用采样也能高效提升大小模型表现。
PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models
- 用分词级交叉熵做评估信号，避免采样和人工评分。
- 在BBH上平均准确率最高，AlpacaEval 2.0胜率提升超19点。
- 适用于监督与偏好任务，对小模型也高效实用。
提示词优化是提升大语言模型性能的实用替代方案，但现有方法多依赖输出采样结合自评或人工标注偏好，难以扩展，尤其对非指令微调的小模型。本文提出PMPO（概率度量提示优化），以分词级交叉熵作为直接、轻量的评估信号。通过掩码分析定位低质量提示片段，并迭代重写生成改进版本。关键在于评估阶段仅需单次前向传播最小化损失，无需输出采样或人工评分即可选优，仅用标准生成提出改写。该统一的损失驱动策略支持监督与偏好任务。在多种模型规模和数据集上，PMPO均优于先前提示优化器：在BBH上取得最高平均准确率，在GSM8K和AQUA-RAT上表现强劲，使AlpacaEval 2.0胜率提升超过19点。结果表明其高效、有效且适用广泛。
原文摘要 · Abstract (English)
Prompt optimization is a practical and widely applicable alternative to fine tuning for improving large language model performance. Yet many existing methods evaluate candidate prompts by sampling full outputs, often coupled with self critique or human annotated preferences, which limits scalability, especially for smaller models or models that are not instruction tuned. We present PMPO (Probabilistic Metric Prompt Optimization), a unified framework that uses token level cross entropy as a direct, lightweight evaluation signal. PMPO locates low quality prompt segments via a masking based analysis and iteratively rewrites them to propose improved variants. Crucially, during evaluation, PMPO selects among variants by minimizing loss in a single forward pass, eliminating output sampling and human or judge based scoring for selection while still using standard generation only to propose rewrites. This unified, loss based strategy supports both supervised and preference based tasks. Across model sizes and datasets, PMPO outperforms prior prompt optimizers: it achieves the highest average accuracy on BBH, performs strongly on GSM8K and AQUA RAT, and raises AlpacaEval 2.0 win rates by over 19 points. These results demonstrate PMPO's effectiveness, efficiency, and broad applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。