从大动作空间中选出代表性动作，降低冗余。
Representative Action Selection for Large Action Space Bandit Families
- 随机采样任务求解，收集最优动作构建代表集。
- 在动作相关性强时，可大幅缩减动作空间且性能接近全集。
- 无需先验相关性知识，适合多任务强化学习场景。
我们研究在共享大动作空间的带宽族中选择子集的问题。在许多自然场景中，虽然动作名义上数量庞大，但动作间高度相关：在不同环境中多数动作产生相似回报，维持完整集合是低效的。目标是探究是否能（以及如何）选取一个更小的代表性动作集，使其表现几乎与完整动作空间相当。主要贡献是一种出人意料的简单算法：反复随机采样一个带宽实例，求解并收集其最优动作。当存在此类相关性时，该算法可显著减少动作空间，且无需预先知晓相关结构。我们为算法提供了理论性能保证，并通过与组合带宽、元学习带宽和缩放算法的实证比较，展示了其实际有效性。
原文摘要 · Abstract (English)
We study the problem of selecting a subset from a large action space shared by a family of bandits. In many natural situations, while the nominal set of actions is large, actions are highly correlated: many yield similar rewards across environments, making it wasteful to maintain the full set. Our aim is to understand whether it is possible -- and how -- to select a smaller set of representative actions that performs nearly as well as the full action space. Our main contribution is a surprisingly simple algorithm: repeatedly sample a bandit instance at random, solve it, and collect the optimal action. This algorithm can significantly reduce the action space when such correlations are present, without the need to know a-priori the correlation structure. We provide theoretical guarantees on the performance of the algorithm and demonstrate its practical effectiveness through empirical comparisons with Combinatorial Bandit, Meta Learning Bandit and Zooming baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。