让大模型更听话：用多智能体自动优化提示词约束
Enhancing LLM Instruction Following: An Evaluation-Driven Multi-Agentic Workflow for Prompt Instructions Optimization
- 拆分任务描述与执行约束，分别优化
- 在Llama 3.1 8B和Mixtral-8x 7B上提升合规率
- 适合需要严格遵循指令的工业级应用
大型语言模型（LLMs）常生成内容相关但不符合形式约束的输出，即概念正确但流程出错。传统提示词优化仅重写主任务描述，忽略作为响应接受标准的细粒度约束。本文提出一种新型多智能体工作流，将主任务描述与约束解耦，利用量化评分作为反馈，迭代改写并优化二者。评估表明，该方法生成的修订提示词显著提升了Llama 3.1 8B和Mixtral-8x 7B等模型的合规得分。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often generate substantively relevant content but fail to adhere to formal constraints, leading to outputs that are conceptually correct but procedurally flawed. Traditional prompt refinement approaches focus on rephrasing the description of the primary task an LLM has to perform, neglecting the granular constraints that function as acceptance criteria for its response. We propose a novel multi-agentic workflow that decouples optimization of the primary task description from its constraints, using quantitative scores as feedback to iteratively rewrite and improve them. Our evaluation demonstrates this method produces revised prompts that yield significantly higher compliance scores from models like Llama 3.1 8B and Mixtral-8x 7B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。