融合奖励与对决反馈，提升强化学习中探索效率。
Fusing Reward and Dueling Feedback in Stochastic Bandits
- 同时利用绝对奖励和相对对决信息进行决策
- 新算法在多臂老虎机中实现理论最优后悔上界
- 适合需要高效探索的在线决策场景
本文研究了在随机多臂老虎机中融合绝对（奖励）与相对（对决）反馈的问题，每轮决策均可获取两种反馈。我们推导出一个后悔下界，证明高效算法对每个动作的后悔可控制在基于奖励和对决的后悔中较小者。为此提出两种融合方法：(1) 简单淘汰融合算法，通过共享候选动作集统一利用两类反馈；(2) 分解融合算法，根据反馈有效性动态选择探索动作，并在每轮随机分配一种反馈用于探索、另一种用于利用。淘汰融合因对决淘汰固有次优性，导致后悔中存在与臂数相关的次优乘性项；而分解融合在常见假设下，其后悔可达到与下界相差常数的最优水平。大量实验验证了算法的有效性与理论结果的正确性。
原文摘要 · Abstract (English)
This paper investigates the fusion of absolute (reward) and relative (dueling) feedback in stochastic bandits, where both feedback types are gathered in each decision round. We derive a regret lower bound, demonstrating that an efficient algorithm may incur only the smaller among the reward and dueling-based regret for each individual arm. We propose two fusion approaches: (1) a simple elimination fusion algorithm that leverages both feedback types to explore all arms and unifies collected information by sharing a common candidate arm set, and (2) a decomposition fusion algorithm that selects the more effective feedback to explore the corresponding arms and randomly assigns one feedback type for exploration and the other for exploitation in each round. The elimination fusion experiences a suboptimal multiplicative term of the number of arms in regret due to the intrinsic suboptimality of dueling elimination. In contrast, the decomposition fusion achieves regret matching the lower bound up to a constant under a common assumption. Extensive experiments confirm the efficacy of our algorithms and theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。