发现决策任务中存在行为一致但内部机制不同的多策略现象。
Translating the Rashomon Effect to Sequential Decision-Making Tasks
- 用形式化验证对比策略的完整概率行为,识别行为相同但结构不同的策略
- 实验表明该现象真实存在,且集合策略对分布偏移更鲁棒
- 基于该集合可生成高效且可验证的宽松策略,降低计算成本
Rashomon效应描述了在相同数据上训练出的多个模型虽产生相同预测,却依赖不同特征的现象。该效应在分类任务中已被广泛研究,但在序列决策任务中尚未涉及。本文将该效应引入序列决策,定义为:多个策略在环境中表现出完全一致的行为(访问相同状态、选择相同动作),但内部结构(如特征归因)存在差异。与分类任务中直接比较预测不同,序列决策中的随机转移导致同一策略在单条轨迹上可能成功或失败,因此我们采用形式化验证方法,构建并比较各策略的完整概率行为。实验表明,该效应在序列决策中确实存在。进一步发现,由Rashomon集合构建的集成策略比单个策略更抗分布偏移;同时,基于该集合推导出的宽松策略,在保持最优性能的前提下显著降低了验证计算开销。
原文摘要 · Abstract (English)
The Rashomon effect describes the phenomenon where multiple models trained on the same data produce identical predictions while differing in which features they rely on internally. This effect has been studied extensively in classification tasks, but not in sequential decision-making, where an agent learns a policy to achieve an objective by taking actions in an environment. In this paper, we translate the Rashomon effect to sequential decision-making. We define it as multiple policies that exhibit identical behavior, visiting the same states and selecting the same actions, while differing in their internal structure, such as feature attributions. Verifying identical behavior in sequential decision-making differs from classification. In classification, predictions can be directly compared to ground-truth labels. In sequential decision-making with stochastic transitions, the same policy may succeed or fail on any single trajectory due to randomness. We address this using formal verification methods that construct and compare the complete probabilistic behavior of each policy in the environment. Our experiments demonstrate that the Rashomon effect exists in sequential decision-making. We further show that ensembles constructed from the Rashomon set exhibit greater robustness to distribution shifts than individual policies. Additionally, permissive policies derived from the Rashomon set reduce computational requirements for verification while maintaining optimal performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。