arXiv:2602.00997cs.AIcs.CL2026-02

通过错误分类指导提示优化,高效提升大模型性能

Error Taxonomy-Guided Prompt Optimization

  • 基于错误分类的自上而下优化方法,全局识别常见失败模式
  • 在多个任务上达到顶尖准确率,计算开销仅为现有方法1/3
  • 适合需要低资源高效调优的AI应用开发者使用

自动提示优化(APO)是一种无需修改模型权重即可提升大语言模型性能的有效方法。现有方法多依赖试错,不断测试不同提示或上下文示例,消耗大量计算资源。近期研究发现,从执行日志中提取自然语言反馈,有助于识别提示改进方向。然而,大多数方法采用自下而上的迭代方式,仅根据单个问题的反馈调整提示,容易丧失全局视角。本文提出错误分类引导的提示优化(ETGPO),采用自上而下的策略:通过收集模型错误,构建错误分类体系,并将针对高频失败模式的指导信息注入提示。在数学、问答和逻辑推理等多个基准测试中,ETGPO表现优于或相当于是当前最优方法,同时优化阶段的令牌消耗和评估预算减少约三分之二。

原文摘要 · Abstract (English)

Automatic Prompt Optimization (APO) is a powerful approach for extracting performance from large language models without modifying their weights. Many existing methods rely on trial-and-error, testing different prompts or in-context examples until a good configuration emerges, often consuming substantial compute. Recently, natural language feedback derived from execution logs has shown promise as a way to identify how prompts can be improved. However, most prior approaches operate in a bottom-up manner, iteratively adjusting the prompt based on feedback from individual problems, which can cause them to lose the global perspective. In this work, we propose Error Taxonomy-Guided Prompt Optimization (ETGPO), a prompt optimization algorithm that adopts a top-down approach. ETGPO focuses on the global failure landscape by collecting model errors, categorizing them into a taxonomy, and augmenting the prompt with guidance targeting the most frequent failure modes. Across multiple benchmarks spanning mathematics, question answering, and logical reasoning, ETGPO achieves accuracy that is comparable to or better than state-of-the-art methods, while requiring roughly one third of the optimization-phase token usage and evaluation budget.

提示优化错误分类大模型调优高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。