通过自适应合同减少AI委托中的评估成本
Adaptive Contracts for Cost-Effective AI Delegation
- 根据初步信号选择性进行详细评估，节省资源
- 在问答和代码生成数据集上验证，效果优于固定合同
- 适合关注成本优化的AI服务采购方
当组织通过按绩效付费的合同将文本生成任务委托给AI提供商时，评估噪声会导致预期支付增加。随着评估方法日益复杂，降噪带来的经济收益常被评估成本上升所抵消。本文提出自适应合同机制，允许在观察到初步粗略信号后，选择性地执行详细评估，从而节约资源。本文有三方面贡献：首先，在自然假设或核心维度较小时，提出高效算法计算最优自适应合同，并证明一般无结构情况下的近似计算困难；其次，构建随机化自适应合同的替代模型并分析其优劣；最后，基于问答和代码生成数据集的实证研究显示，自适应策略显著优于非自适应基线。
原文摘要 · Abstract (English)
When organizations delegate text generation tasks to AI providers via pay-for-performance contracts, expected payments rise when evaluation is noisy. As evaluation methods become more elaborate, the economic benefits of decreased noise are often overshadowed by increased evaluation costs. In this work, we introduce adaptive contracts for AI delegation, which allow detailed evaluation to be performed selectively after observing an initial coarse signal in order to conserve resources. We make three sets of contributions: First, we provide efficient algorithms for computing optimal adaptive contracts under natural assumptions or when core problem dimensions are small, and prove hardness of approximation in the general unstructured case. We then formulate alternative models of randomized adaptive contracts and discuss their benefits and limitations. Finally, we empirically demonstrate the benefits of adaptivity over non-adaptive baselines using question-answering and code-generation datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。