让模型实时适应变化的决策目标,提升动态环境下的决策质量。
Online Decision-Focused Learning
- 用正则化使不可导的目标函数可微,结合扰动技术解决非凸问题。
- 提出两种在线算法,首次实现对动态决策学习的可证明性能保证。
- 适合需要实时优化决策的场景,如供应链、金融交易等。
决策聚焦学习(DFL)是一种日益流行的范式,用于训练输出将用于决策任务的预测模型。与仅优化预测精度不同,DFL直接最小化下游决策相关的损失。然而,现有研究仅关注数据固定且目标函数不变的场景。本文研究目标函数和数据分布随时间演化的动态环境中的DFL。该设定对在线学习构成挑战,因目标函数梯度为零或未定义,无法使用标准一阶优化方法,且通常为非凸。为此,我们(i)对目标函数进行正则化使其可微,(ii)结合扰动技术与近似最优预言机以应对非凸性。由此提出两种专为DFL设计的在线算法,并分别建立了静态与动态的遗憾界。这是首个针对在线决策聚焦问题的可证明保证。最后,在背包问题实验中验证了算法有效性,其性能优于两个基准方法。
原文摘要 · Abstract (English)
Decision-focused learning (DFL) is an increasingly popular paradigm for training predictive models whose outputs are used in decision-making tasks. Instead of merely optimizing for predictive accuracy, DFL trains models to directly minimize the loss associated with downstream decisions. However, existing studies focus solely on scenarios where a fixed batch of data is available and the objective function does not change over time. We instead investigate DFL in dynamic environments where the objective function and data distribution evolve over time. This setting is challenging for online learning because the objective function has zero or undefined gradients, which prevents the use of standard first-order optimization methods, and is generally non-convex. To address these difficulties, we (i) regularize the objective to make it differentiable and (ii) use perturbation techniques along with a near-optimal oracle to overcome non-convexity. Combining those techniques yields two original online algorithms tailored for DFL, for which we establish respectively static and dynamic regret bounds. These are the first provable guarantees for the online decision-focused problem. Finally, we showcase the effectiveness of our algorithms on a knapsack experiment, where they outperform two standard benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。