用AI自动生成和优化提示词,让模型更准更可信。
Reflective Prompt Tuning through Language Model Function-Calling

- 让大模型自我诊断错误并迭代改进提示词
- 在多跳推理任务上提升12.9分,且更会判断自己有没有把握
- 适合需要高精度和可解释性的复杂推理场景
大型语言模型(LLMs)已具备强大的指令遵循与复杂推理能力,使提示词成为无需参数更新即可适配模型的灵活接口。然而,提示词设计仍依赖人工,对格式、措辞和指令顺序敏感,导致效率低下。现有自动化方法多基于候选提示搜索或固定纠错流程,难以捕捉系统性错误模式。本文提出反射式提示调优(Reflective Prompt Tuning, RPT),利用大模型函数调用模拟人类提示工程师的迭代工作流。一个优化器调用诊断函数,在整个优化数据集上评估目标模型,总结重复出现的失败模式,并生成结构化报告。优化器结合该报告与历史诊断记忆,迭代修正提示词。RPT还支持置信度感知优化,利用诊断反馈中的校准信号进行最终提示选择。在三个推理任务中,RPT相比初始提示最高提升12.9分,性能媲美前沿方法,并改善了置信度校准。分析表明,其在多跳和数学推理任务中尤为有效,生成的提示修改精准对应诊断出的错误模式,显著提升任务表现与校准能力。
原文摘要 · Abstract (English)
Large language models (LLMs) have become increasingly capable of following instructions and complex reasoning, making prompting a flexible interface for adapting models without parameter updates. Yet prompt design remains labor-intensive and highly sensitive to formatting, phrasing, and instruction order, motivating automated prompt optimization methods that reduce manual effort while preserving inference-time flexibility. However, existing methods often search over prompt candidates or use fixed critique-refine pipelines driven by individual examples or small batches, limiting their ability to capture systematic error patterns and make targeted edits grounded in failure history. We propose Reflective Prompt Tuning (RPT), a framework that uses LLM function calling to simulate the iterative workflow of human prompt engineers. An LLM optimizer calls a diagnostic function that evaluates the target model over an entire optimization set, summarizes recurring failure modes, and returns a structured diagnostic report. The optimizer uses this report, together with an accumulated memory of prior reports, to revise the prompt for the next iteration. RPT further supports confidence-aware optimization by using calibration signals in diagnostic feedback and final prompt selection. Across three reasoning tasks, RPT improves over initial prompts by up to 12.9 points, remains competitive with state of the art, and improves confidence calibration. Our analyses show that RPT is especially effective on multi-hop and mathematical reasoning, producing targeted prompt revisions that align with diagnosed failure patterns and lead to gains in task performance and calibration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。