用少量人工反馈优化大模型提示词，提升生成质量
PLHF: Prompt Optimization with Few-Shot Human Feedback
- 引入人类反馈作为评估器，替代传统难定义的评分标准
- 仅需一轮人工反馈即可完成整个提示词优化流程
- 在公开与工业数据集上均优于现有提示优化方法
自动提示词优化框架旨在为大语言模型（LLMs）生成符合期望输出质量的提示词。尽管现有方法能处理固定答案类任务，但在输出质量难以通过标准参考样本对比评估时，定义评价指标变得复杂。因此，在缺乏明确指标的情况下高效有效地优化提示词成为关键挑战。为此，我们提出PLHF（Prompt Learning with Human Feedback），一种受知名强化学习人类反馈（RLHF）技术启发的少样本提示词优化框架。与简单策略不同，PLHF采用特定评估模块作为质量估计器。该方法仅需单轮人工反馈即可完成整个优化过程。在公共及工业数据集上的实证结果表明，PLHF在大模型提示词优化中优于先前的输出评分策略。
原文摘要 · Abstract (English)
Automatic prompt optimization frameworks are developed to obtain suitable prompts for large language models (LLMs) with respect to desired output quality metrics. Although existing approaches can handle conventional tasks such as fixed-solution question answering, defining the metric becomes complicated when the output quality cannot be easily assessed by comparisons with standard golden samples. Consequently, optimizing the prompts effectively and efficiently without a clear metric becomes a critical challenge. To address the issue, we present PLHF (which stands for "P"rompt "L"earning with "H"uman "F"eedback), a few-shot prompt optimization framework inspired by the well-known RLHF technique. Different from naive strategies, PLHF employs a specific evaluator module acting as the metric to estimate the output quality. PLHF requires only a single round of human feedback to complete the entire prompt optimization process. Empirical results on both public and industrial datasets show that PLHF outperforms prior output grading strategies for LLM prompt optimizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。