提出不学习的策略，在高风险场景中避免不可逆错误。
Learning When Not to Learn: Risk-Sensitive Abstention in Bandits with Unbounded Rewards
- 设计可放弃决策的上下文老虎机模型，仅在安全区域行动
- 理论证明实现亚线性后悔，保障高风险环境下的安全探索
- 适合医疗、金融等容错率极低的部署场景
在高风险人工智能应用中，一次错误操作可能导致不可挽回的损失。然而，几乎所有顺序决策理论都假设所有错误均可恢复（如通过奖励有界）。激进探索的标准老虎机算法在此类场景下可能引发不可逆损害。已有工作通过引入导师来规避严重错误，但导师未必可用。本文将无导师情况下的无界奖励学习形式化为一个双动作上下文老虎机模型：每轮代理观察输入后选择放弃（固定0奖励）或执行预设任务策略。执行会带来上界有限但可任意负的奖励，且奖励在输入上满足Lipschitz连续性。我们提出一种基于谨慎性的算法：仅在现有证据无法证明有害时才进行决策。在输入独立同分布条件下，理论证明该算法具有亚线性后悔，验证了谨慎探索在高风险环境中部署学习代理的有效性。
原文摘要 · Abstract (English)
In high-stakes AI applications, even a single action can cause irreparable damage. However, nearly all of sequential decision-making theory assumes that all errors are recoverable (e.g., by bounding rewards). Standard bandit algorithms that explore aggressively may cause irreparable damage when this assumption fails. Some prior work avoids irreparable errors by asking for help from a mentor, but a mentor may not always be available. In this work, we formalize a model of learning with unbounded rewards without a mentor as a two-action contextual bandit with an abstain option: at each round the agent observes an input and chooses either to abstain (always 0 reward) or to commit (execute a preexisting task policy). Committing yields rewards that are upper-bounded but can be arbitrarily negative, and the commit reward is assumed Lipschitz in the input. We propose a caution-based algorithm that learns when not to learn: it chooses a trusted region and commits only where the available evidence does not already certify harm. Under these conditions and i.i.d. inputs, we establish sublinear regret guarantees, theoretically demonstrating the effectiveness of cautious exploration for deploying learning agents safely in high-stakes environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。