用扩散模型建模不确定性,直接优化决策质量。
Diffusion-DFL: Decision-focused Diffusion Models for Stochastic Optimization
- 用扩散模型生成不确定参数分布,通过采样优化决策。
- 新方法在多个数据集上决策质量优于基线模型。
- 轻量级估计器避免反向传播,节省内存和计算资源。
决策聚焦学习(DFL)通过训练预测器以优化下游决策目标,而非仅最小化预测误差,实现预测与优化的融合。现有方法多依赖确定性点预测,难以捕捉真实环境中的随机性。本文提出首个基于扩散模型的DFL方法,训练扩散模型以表征不确定参数的分布,并通过从扩散模型中采样进行随机优化来制定决策。贡献有二:首先,利用重参数化技巧构建可端到端训练的扩散DFL框架;其次,提出一种轻量级得分函数估计器,仅需少量前向扩散过程,无需对采样过程反向传播。该设计基于我们发现:对随机优化的反向传播可通过加权得分函数近似实现。实验表明,本方法在多个基准测试中持续优于强基线,显著提升决策质量。所有实验代码已公开于项目仓库:https://github.com/GT-KOALA/Diffusion_DFL。
原文摘要 · Abstract (English)
Decision-focused learning (DFL) integrates predictive modeling and optimization by training predictors to optimize the downstream decision target rather than merely minimizing prediction error. To date, existing DFL methods typically rely on deterministic point predictions, which are often insufficient to capture the intrinsic stochasticity of real-world environments. To address this challenge, we propose the first diffusion-based DFL approach, which trains a diffusion model to represent the distribution of uncertain parameters and optimizes the decision by solving a stochastic optimization with samples drawn from the diffusion model. Our contributions are twofold. First, we formulate diffusion DFL using the reparameterization trick, enabling end-to-end training through diffusion. While effective, it is memory and compute-intensive due to the need to differentiate through the diffusion sampling process. Second, we propose a lightweight score function estimator that uses only several forward diffusion passes and avoids backpropagation through the sampling. This follows from our results that backpropagating through stochastic optimization can be approximated by a weighted score function formulation. We empirically show that our diffusion DFL approach consistently outperforms strong baselines in decision quality. The source code for all experiments is available at the project repository: https://github.com/GT-KOALA/Diffusion_DFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。