提出新评价指标与算法，解决多目标强化学习中的平衡难题。
Stochastic Multi-Objective Multi-Armed Bandits: Regret Definition and Algorithm
- 设计全新后悔值度量，兼顾所有冲突目标的均衡表现
- 算法实现帕累托最优与高效帕累托最优臂的次线性后悔
- 适合需多目标权衡的在线决策场景，如推荐系统优化
多臂赌博机（MAB）广泛应用于需平衡探索与利用的在线优化任务。实际中常涉及多个相互冲突的目标，催生了多目标多臂赌博机（MO-MAB）。现有方法主要依赖文献\cite{drugan2013designing}提出的帕累托后悔度量，但该度量难以同时考虑所有帕累托最优臂。为此，本文提出一种新型且全面的后悔度量，确保在冲突目标间保持均衡性能。同时引入“高效帕累托最优臂”概念，专为在线优化设计。基于新度量，开发了两阶段MO-MAB算法，在帕累托最优臂和高效帕累托最优臂上均实现次线性后悔。
原文摘要 · Abstract (English)
Multi-armed bandit (MAB) problems are widely applied to online optimization tasks that require balancing exploration and exploitation. In practical scenarios, these tasks often involve multiple conflicting objectives, giving rise to multi-objective multi-armed bandits (MO-MAB). Existing MO-MAB approaches predominantly rely on the Pareto regret metric introduced in \cite{drugan2013designing}. However, this metric has notable limitations, particularly in accounting for all Pareto-optimal arms simultaneously. To address these challenges, we propose a novel and comprehensive regret metric that ensures balanced performance across conflicting objectives. Additionally, we introduce the concept of \textit{Efficient Pareto-Optimal} arms, which are specifically designed for online optimization. Based on our new metric, we develop a two-phase MO-MAB algorithm that achieves sublinear regret for both Pareto-optimal and efficient Pareto-optimal arms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。