通过分段标注优化提示,让大模型更精准理解指令意图。
Prompt Segmentation and Annotation Optimisation: Controlling LLM Behaviour via Optimised Segment-Level Annotations

- 将提示拆分为可读段落并添加重要性标注
- 提升推理准确率与自洽性,不降低原提示效果
- 适合需要精确控制生成结果的研究者使用
提示工程对高效使用生成式AI至关重要,但现有优化方法常在无结构、庞大的提示空间中操作,导致计算成本高且可能扭曲原始意图。本文提出提示分段与标注优化(PSAO),一种结构化提示优化框架。PSAO将提示分解为可解释的段落(如句子),并为每段添加人类可读的标注(如{不重要}、{重要}、{非常重要}),引导大语言模型在生成时合理分配注意力、减少混淆。我们形式化定义了分段与标注规则,并证明优化后的段级标注能提升大模型输出质量,同时保留原始提示作为候选以防止性能下降。实验表明,采用标注后推理准确率和自洽性均得到提升。但如何高效识别最优分段与标注仍具挑战,留待未来研究。本工作为段级标注优化的可行性与潜力提供了概念验证。
原文摘要 · Abstract (English)
Prompt engineering is crucial for effective interaction with generative artificial intelligence systems, yet existing optimisation methods often operate over an unstructured and vast prompt space, leading to high computational costs and potential distortions of the original intent. We introduce Prompt Segmentation and Annotation Optimisation (PSAO), a structured prompt optimisation framework designed to improve prompt optimisation controllability and efficiency. PSAO decomposes a prompt into interpretable segments (e.g., sentences) and augments each with human-readable annotations (e.g., {not important}, {important}, {very important}). These annotations guide large language models (LLMs) in allocating focus and clarifying confusion during response generation. We formally define the segmentations and annotations and demonstrate that optimised segment-level annotations can lead to improved LLM responses, with the original prompt retained as a candidate in the optimisation space to prevent performance degradation. Empirical evaluations indicate that PSAO benefits from annotations in terms of improved reasoning accuracy and self-consistency. However, developing efficient methods for identifying optimal segmentations and annotations remains challenging and is reserved for future investigation. This work is intended as a proof of concept, demonstrating the feasibility and potential of segment-level annotation optimisation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。