arXiv:2607.13402stat.MLcs.AI2026-07

为严格公平的强化学习设计新算法,首次给出公平性代价的精确下界。

Price of Fairness in Bandits: A Tight Minimax Characterization

论文配图:Price of Fairness in Bandits: A Tight Minimax Characterization
图 1 · 摘自论文原文
  • 用调和秩调度替代均匀探索,结合正均值锚点保障公平性。
  • 证明公平性代价最小为 Ω(σ√(k^max(1,q)/T)),且可被新算法逼近。
  • 适用于对早期公平性敏感的场景,如医疗试验、资源分配等。

在带通货膨胀问题中,标准的后悔最小化算法将探索视为分摊成本,可能导致临床试验等场景中早期参与者遭受不公平的预期待遇。近期工作通过广义 $p$-均值评估每轮期望收益,实现从功利主义($p=1$)到罗尔斯式公平($p\to-\infty$)的连续权衡。尽管 $p\ge0$ 的情形已有紧致上界,但严格公平性对应的 $q=-p>0$ 场景仍未解决,因负幂均值受最小收益主导。针对 $\sigma$-子高斯奖励且均值非负的情形,先前最优算法依赖均匀早期探索,其后悔为 $O(k^{(q+1)/2}/\sqrt{T})$,而唯一通用下界仅为经典 $Ω(σ\sqrt{k/T})$。因此无法判断 $k$ 的额外依赖是否源于公平性本质或探索方式。本文通过‘针中寻草’构造,证明了算法无关的下界 $Ω(σ\sqrt{k^{\max(1,q)}/T})$;当 $q>1$ 时,表明 $k^{q/2}$ 的惩罚是信息论上不可避免的。随后提出 extsf{UCB-HARE}(调和锚定秩探索),以反权重调和秩调度替代均匀探索,并引入经认证的正均值锚点。其后悔为 $\widetilde{O}(σ\sqrt{k^{\max(1,q)}/T})$,与下界仅差对数因子。合成实验验证 extsf{UCB-HARE} 显著优于均匀探索基线,且随着 $q$ 增大,优势更明显。

原文摘要 · Abstract (English)

In bandit problems, standard regret-minimizing algorithms treat exploration as an amortized cost, which can expose early participants to unfair ex-ante losses in settings such as clinical trials. Recent work addresses this by evaluating the sequence of per-round expected rewards through the generalized $p$-mean, interpolating between utilitarian welfare ($p=1$), Nash welfare ($p\to0$), and Rawlsian fairness ($p\to-\infty$). Although tight guarantees are known for $p\ge0$, the strictly fair regime $q=-p>0$ remains unresolved because negative-power means are dominated by the smallest per-round rewards. For $σ$-sub-Gaussian rewards with nonnegative means, the best prior algorithm relied on uniform early exploration and achieved regret $O(k^{(q+1)/2}/\sqrt{T})$, while the only general lower bound was the classical $Ω(σ\sqrt{k/T})$. Thus it was unclear whether the extra dependence on $k$ was intrinsic to strict fairness or an artifact of uniform exploration. We close this gap by identifying the exact polynomial price of strict fairness. Using a needle-in-haystack construction, we prove an algorithm-independent lower bound $Ω(σ\sqrt{k^{\max(1,q)}/T})$; for $q>1$, this shows that the penalty $k^{q/2}$ is information-theoretically unavoidable. We then introduce \textsf{UCB-HARE} (Harmonic Anchored Rank Exploration), which replaces uniform exploration with an inverse-weighted harmonic rank schedule protected by a certified positive-mean anchor. Its regret is $\widetilde{O}(σ\sqrt{k^{\max(1,q)}/T})$, matching the lower bound up to logarithmic factors. Experiments on synthetic instances confirm that \textsf{UCB-HARE} improves over uniform-exploration baselines, with gains increasing as $q$ grows.

公平性强化学习带通货膨胀

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。