arXiv:2602.01711cs.AIcs.LG2026-02被引 2

用因果推理优化提示词,让大模型更稳更省。

Optimizing Prompts for Large Language Models: A Causal Approach

  • 通过双重机器学习分离提示词的因果效应,避开查询特征干扰。
  • 在数学推理等任务中,对难例表现显著优于人工和现有方法。
  • 离线训练奖励模型,实现低成本、高精度的个性化提示生成。

大语言模型日益嵌入企业工作流,但性能高度依赖提示设计。自动提示优化(APO)面临两大挑战:静态提示平均表现尚可,却无法适应多样化查询;动态方法依赖离线奖励模型,其相关性无法区分提示效果与查询特征。本文提出因果提示优化(CPO),将提示设计重构为因果估计问题。第一阶段,利用双重机器学习(DML)对提示与查询的语义嵌入进行分析,剥离提示变化的因果影响;第二阶段,基于无偏奖励信号,高效搜索针对具体查询的最优提示,无需昂贵在线评估。在数学推理、可视化与数据分析等多个基准上,CPO持续超越人工提示与先进自动化方法,尤其在难题上的鲁棒性显著提升。更重要的是,通过将评估从实时推理转为离线因果建模,CPO使每查询定制成为可能,成本仅为在线方法的极小部分。结果表明,因果推断是企业级大模型提示优化的可扩展、高可靠、低成本基础。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are increasingly embedded in enterprise workflows, yet their performance remains highly sensitive to prompt design. Automatic Prompt Optimization (APO) seeks to mitigate this instability, but existing approaches face two persistent challenges. First, commonly used prompt strategies rely on static instructions that perform well on average but fail to adapt to heterogeneous queries. Second, more dynamic approaches depend on offline reward models that are fundamentally correlational, confounding prompt effectiveness with query characteristics. We propose Causal Prompt Optimization (CPO), a framework that reframes prompt design as a problem of causal estimation. CPO operates in two stages. First, it learns an offline causal reward model by applying Double Machine Learning (DML) to semantic embeddings of prompts and queries, isolating the causal effect of prompt variations from confounding query attributes. Second, it utilizes this unbiased reward signal to guide a resource-efficient search for query-specific prompts without relying on costly online evaluation. We evaluate CPO across benchmarks in mathematical reasoning, visualization, and data analytics. CPO consistently outperforms human-engineered prompts and state-of-the-art automated optimizers. The gains are driven primarily by improved robustness on hard queries, where existing methods tend to deteriorate. Beyond performance, CPO fundamentally reshapes the economics of prompt optimization: by shifting evaluation from real-time model execution to an offline causal model, it enables high-precision, per-query customization at a fraction of the inference cost required by online methods. Together, these results establish causal inference as a scalable foundation for reliable and cost-efficient prompt optimization in enterprise LLM deployments.

提示优化因果推断大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。