用检索增强生成自动优化提示,提升物流文本帧检测准确率15%。
Auto-Prompting with Retrieval Guidance for Frame Detection in Logistics
- 结合检索增强与自动思维链,迭代优化任务专用提示。
- 相比基线提示,推理准确率最高提升15%,效率显著改善。
- 无需微调,适配GPT-4o、Qwen等多模型,适合物流场景应用。
提示工程在不需大量微调的情况下,使大语言模型适应复杂推理与标注任务至关重要。本文提出一种新型提示优化流程,用于物流文本中的帧检测,融合检索增强生成(RAG)、少样本提示、思维链(CoT)推理及自动思维链合成(Auto-CoT),生成高效的任务特定提示。核心是基于LLM的提示优化代理,通过检索示例、性能反馈和内部自评估迭代优化提示。在真实物流文本标注任务上评估,结果表明,经优化的提示(尤其是结合Auto-CoT与RAG)相比零样本或静态提示,推理准确率最高提升15%。系统在GPT-4o、Qwen 2.5(72B)和LLaMA 3.1(70B)等多个模型上均表现一致提升,验证了其泛化性与实用性。研究说明结构化提示优化可作为全微调的可行替代方案,为物流等领域的领域特定NLP应用提供可扩展部署路径。
原文摘要 · Abstract (English)
Prompt engineering plays a critical role in adapting large language models (LLMs) to complex reasoning and labeling tasks without the need for extensive fine-tuning. In this paper, we propose a novel prompt optimization pipeline for frame detection in logistics texts, combining retrieval-augmented generation (RAG), few-shot prompting, chain-of-thought (CoT) reasoning, and automatic CoT synthesis (Auto-CoT) to generate highly effective task-specific prompts. Central to our approach is an LLM-based prompt optimizer agent that iteratively refines the prompts using retrieved examples, performance feedback, and internal self-evaluation. Our framework is evaluated on a real-world logistics text annotation task, where reasoning accuracy and labeling efficiency are critical. Experimental results show that the optimized prompts - particularly those enhanced via Auto-CoT and RAG - improve real-world inference accuracy by up to 15% compared to baseline zero-shot or static prompts. The system demonstrates consistent improvements across multiple LLMs, including GPT-4o, Qwen 2.5 (72B), and LLaMA 3.1 (70B), validating its generalizability and practical value. These findings suggest that structured prompt optimization is a viable alternative to full fine-tuning, offering scalable solutions for deploying LLMs in domain-specific NLP applications such as logistics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。