提出抗奖励污染的鲁棒汤普森采样算法，提升实际应用中的可靠性。
Robust Thompson Sampling Algorithms Against Reward Poisoning Attacks
- 用伪后验替代真实后验，抵御攻击者篡改奖励信号
- 在任意攻击策略下仍保持近似最优累积损失
- 适用于已知或未知攻击预算的场景，适合安全敏感领域
汤普森采样是在线序列决策问题中广泛使用的学习算法，具有丰富的现实应用。然而，现有方法假设接收到的奖励未被污染，这在存在恶意奖励污染的现实场景中不成立。为提高汤普森采样的可靠性，本文致力于使其对攻击具备鲁棒性。主要挑战在于：由于奖励已被篡改，代理无法再计算真实奖励的后验分布。为此，本文提出基于伪后验的鲁棒算法，该伪后验更难被攻击者操纵。针对经典的随机和上下文线性老虎机场景，分别设计了在攻击者预算已知与未知情况下的算法。理论上证明，所提算法在任意攻击策略下均能保证近似最优的累积遗憾。
原文摘要 · Abstract (English)
Thompson sampling is one of the most popular learning algorithms for online sequential decision-making problems and has rich real-world applications. However, current Thompson sampling algorithms are limited by the assumption that the rewards received are uncorrupted, which may not be true in real-world applications where adversarial reward poisoning exists. To make Thompson sampling more reliable, we want to make it robust against adversarial reward poisoning. The main challenge is that one can no longer compute the actual posteriors for the true reward, as the agent can only observe the rewards after corruption. In this work, we solve this problem by computing pseudo-posteriors that are less likely to be manipulated by the attack. We propose robust algorithms based on Thompson sampling for the popular stochastic and contextual linear bandit settings in both cases where the agent is aware or unaware of the budget of the attacker. We theoretically show that our algorithms guarantee near-optimal regret under any attack strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。