arXiv:2602.05340stat.MLcs.LG2026-02被引 2

让实验设计更懂决策,用方向性不确定度提前停止无效实验。

Decision-Focused Sequential Experimental Design: A Directional Uncertainty-Guided Approach

  • 用方向性不确定度替代传统预测误差,指导实验选择。
  • 在真实大模型任务中,比传统方法提前停止,节省实验次数。
  • 无需求解优化问题,计算高效,适合实际部署。

我们研究预测-然后-优化范式下的顺序实验设计问题。在此范式中,预测模型的输出作为下游线性优化问题的系数向量。传统顺序实验设计旨在控制输入变量(特征),使每次实验结果(标签)带来的预测精度提升最大化。然而,在预测-然后-优化设置中,性能最终由下游优化产生的决策损失评估,而非预测误差。这种预测精度与决策损失之间的不匹配导致传统无决策意识的设计效率低下。为解决此问题,我们提出一种基于方向性的预测不确定性度量。该度量无需求解优化预言机,因此计算上可行。我们证明了由此产生的顺序设计准则具有强一致性与收敛性保证。在一大类分布下,我们展示所提的方向性不确定性设计相较于无决策意识设计可实现更早停止。这一优势在真实世界的大模型任务(如LLM作业分配)中得到进一步验证。

原文摘要 · Abstract (English)

We consider the sequential experimental design problem in the predict-then-optimize paradigm. In this paradigm, the outputs of the prediction model are used as coefficient vectors in a downstream linear optimization problem. Traditional sequential experimental design aims to control the input variables (features) so that the improvement in prediction accuracy from each experimental outcome (label) is maximized. However, in the predict-then-optimize setting, performance is ultimately evaluated based on the decision loss induced by the downstream optimization, rather than by prediction error. This mismatch between prediction accuracy and decision loss renders traditional decision-blind designs inefficient. To address this issue, we propose a directional-based metric to quantify predictive uncertainty. This metric does not require solving an optimization oracle and is therefore computationally tractable. We show that the resulting sequential design criterion enjoys strong consistency and convergence guarantees. Under a broad class of distributions, we demonstrate that our directional uncertainty-based design attains an earlier stopping time than decision-blind designs. This advantage is further supported by real-world experiments on an LLM job allocation problem.

实验设计预测-优化不确定性决策导向

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。