多智能体在不确定环境下兼顾学习效率与公平性的新方法
Multi-Objective Multi-Agent Bandits: From Learning Efficiency to Fairness Optimization

- 设计双目标算法,分离效率与公平性优化机制
- 理论证明效率与公平性分别达到O(log T)和O(T^3/4)的收敛速度
- 适用于资源分配、协同决策等需要公平与高效并重的场景
我们研究随机奖励下的多目标多智能体多臂赌博机(MO-MA-MAB)问题,其中智能体观测异质奖励向量,并在时变图上通信。为实现高效学习,定义帕累托后悔(Pareto regret)并提出 extsc{Pareto UCB1 Gossip},其新颖探索半径将帕累托推断中的统计不确定性与共识误差分离。为建模公平性,基于偏好标量化奖励构建纳什社会福利目标,提出 extsc{Simulated NSW UCB Gossip},融合偏好奖励模拟、基于播送的效用估计与UCB式探索。理论上, extsc{Pareto UCB1 Gossip}实现 cal{O}(\log T)后悔与实例无关的 cal{O}(\sqrt{T})速率; extsc{Simulated NSW UCB Gossip}实现 cal{O}(T^{3/4})的实例无关后悔界。该分离揭示公平性约束对效率的代价:限制信息聚合,减缓收敛。实验表明,所提方法持续优于基线,在效率与公平性设置下性能提升约100%和50%。
原文摘要 · Abstract (English)
We study multi-objective multi-agent multi-armed bandits (MO-MA-MAB) under stochastic rewards, where agents observe heterogeneous reward vectors and communicate over time-varying graphs. We formulate this emerging problem setting to address \emph{efficient learning}, measured by Pareto regret, and incorporate \emph{fair learning} as an additional goal, captured via social welfare. To measure efficiency, we formulate Pareto regret and develop \textsc{Pareto UCB1 Gossip}, whose novel exploration radius explicitly separates statistical uncertainty in Pareto-based inference from consensus error. To express the fairness constraint, we formulate a Nash Social Welfare objective over preference-scalarized rewards and propose \textsc{Simulated NSW UCB Gossip}, which integrates preference-based reward simulation, gossip-based utility estimation, and UCB-style exploration. We prove that \textsc{Pareto UCB1 Gossip} achieves \(\mathcal{O}(\log T)\) regret and an instance-independent rate of \(\mathcal{O}(\sqrt{T})\), while \textsc{Simulated NSW UCB Gossip} achieves an instance-independent regret bound of \(\mathcal{O}(T^{3/4})\). This separation reveals the cost of imposing the fairness constraint to our efficiency objective: fairness limits information aggregation and slows convergence. Experiments show that our methods consistently outperform baselines, improving performance by approximately \(100\%\) and \(50\%\) in the efficiency and fairness settings, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。