arXiv:2510.15273stat.MLcs.LG2025-10

考虑决策干扰的前瞻性在线策略,能减少长期遗憾并提升决策质量。

Foresighted Online Policy Optimization with Interference

  • 通过探索与利用交替策略,捕捉当前决策对后续影响
  • 理论证明在两种定义下实现次线性遗憾,且无需依赖统计推断
  • 适用于存在个体间干扰的在线决策场景,如城市酒店收益管理

上下文老虎机通过利用陆续到来个体的基线特征,在平衡探索与利用的基础上优化累积回报,是在线决策的关键。现有方法通常假设无干扰,即每个个体的行为仅影响自身回报。然而,在许多实际场景中该假设不成立,忽略干扰会导致短视策略,只关注即时收益,进而引发次优决策并随时间增加遗憾。为填补这一空白,本文提出前瞻性在线策略与干扰(FRONT),创新性地考虑当前决策对后续决策和回报的长期影响。FRONT采用一系列探索与利用策略,应对干扰的复杂性,确保参数推断稳健且遗憾最小化。理论上,我们建立了在线估计器的尾界,并在干扰网络满足一定条件下推导出感兴趣参数的渐近分布。进一步证明,FRONT在两种不同定义下均实现次线性遗憾,涵盖决策的即时与连带影响,并在有无统计推断的情况下均成立。通过大量模拟和真实世界的城市酒店利润应用验证了FRONT的有效性。

原文摘要 · Abstract (English)

Contextual bandits, which leverage the baseline features of sequentially arriving individuals to optimize cumulative rewards while balancing exploration and exploitation, are critical for online decision-making. Existing approaches typically assume no interference, where each individual's action affects only their own reward. Yet, such an assumption can be violated in many practical scenarios, and the oversight of interference can lead to short-sighted policies that focus solely on maximizing the immediate outcomes for individuals, which further results in suboptimal decisions and potentially increased regret over time. To address this significant gap, we introduce the foresighted online policy with interference (FRONT) that innovatively considers the long-term impact of the current decision on subsequent decisions and rewards. The proposed FRONT method employs a sequence of exploratory and exploitative strategies to manage the intricacies of interference, ensuring robust parameter inference and regret minimization. Theoretically, we establish a tail bound for the online estimator and derive the asymptotic distribution of the parameters of interest under suitable conditions on the interference network. We further show that FRONT attains sublinear regret under two distinct definitions, capturing both the immediate and consequential impacts of decisions, and we establish these results with and without statistical inference. The effectiveness of FRONT is further demonstrated through extensive simulations and a real-world application to urban hotel profits.

在线学习干扰建模后悔最小化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。