arXiv:2508.05441cs.LGcs.AI2025-08被引 5

提出两种新方法,让MCTS在高风险场景下更安全可靠。

Tail-Risk-Safe Monte Carlo Tree Search under PAC-Level Guarantees

  • 引入CVaR机制,精准控制最坏情况下的损失风险
  • 通过Wasserstein不确定性集降低样本不足带来的风险估计偏差
  • 在多种模拟环境中验证了更强的鲁棒性与收益稳定性

传统蒙特卡洛树搜索(MCTS)仅关注期望回报,忽视高风险极端后果。本文提出两种新方法:首先,CVaR-MCTS将条件风险价值(CVaR)嵌入MCTS,通过参数α实现对‘最差(1−α)%场景’下预期损失的显式尾部风险控制;其次,为解决小样本导致的尾部风险估计偏差,提出基于一阶Wasserstein模糊集$\mathcal{P}_{\varepsilon_{s}}(s,a)$的W-MCTS,其半径为$\varepsilon_{s}$。理论证明了两种方法在概率保证(PAC)下的尾部安全性及相应遗憾界。在多个模拟环境中的评估表明,所提方法显著优于现有基线,在保持更高奖励与稳定性的同时,有效实现鲁棒尾部风险保障。

原文摘要 · Abstract (English)

Making decisions with respect to just the expected returns in Monte Carlo Tree Search (MCTS) cannot account for the potential range of high-risk, adverse outcomes associated with a decision. To this end, safety-aware MCTS often consider some constrained variants -- by introducing some form of mean risk measures or hard cost thresholds. These approaches fail to provide rigorous tail-safety guarantees with respect to extreme or high-risk outcomes (denoted as tail-risk), potentially resulting in serious consequence in high-stake scenarios. This paper addresses the problem by developing two novel solutions. We first propose CVaR-MCTS, which embeds a coherent tail risk measure, Conditional Value-at-Risk (CVaR), into MCTS. Our CVaR-MCTS with parameter $α$ achieves explicit tail-risk control over the expected loss in the "worst $(1-α)\%$ scenarios." Second, we further address the estimation bias of tail-risk due to limited samples. We propose Wasserstein-MCTS (or W-MCTS) by introducing a first-order Wasserstein ambiguity set $\mathcal{P}_{\varepsilon_{s}}(s,a)$ with radius $\varepsilon_{s}$ to characterize the uncertainty in tail-risk estimates. We prove PAC tail-safety guarantees for both CVaR-MCTS and W-MCTS and establish their regret. Evaluations on diverse simulated environments demonstrate that our proposed methods outperform existing baselines, effectively achieving robust tail-risk guarantees with improved rewards and stability.

MCTS风险控制安全决策

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。