用图结构建模不确定信念，让小问题学的策略直接用在大问题上。
GammaZero: Learning To Guide POMDP Belief Space Search With Graph Representations
- 把信念状态转为带不确定性的图，统一表示不同规模问题
- 在小问题上学策略，大问题上零样本泛化，性能接近基准方法
- 适合需要跨规模推理的强化学习任务，尤其对复杂环境规划有帮助
我们提出一种面向部分可观测马尔可夫决策过程（POMDP）的不确定性感知图表示框架，用于学习引导规划。与依赖特定领域或问题规模的神经网络架构的现有方法不同，GammaZero采用统一的图基信念表示，实现同一领域内不同规模问题的泛化。核心思路是将信念状态系统性地转化为不确定性感知图，使得在小型问题上学到的结构模式可迁移至更大实例。我们使用带有解码器架构的图神经网络，从计算上可处理的小规模问题的专家示范中学习价值函数和策略，随后将这些学得的启发式方法应用于更大规模问题的蒙特卡洛树搜索。在标准POMDP基准测试上的实验结果表明，当训练与测试问题规模相同时，GammaZero性能与BetaZero相当；而在未见的大规模问题上（2-6倍于训练规模），实现了零样本泛化。
原文摘要 · Abstract (English)
We introduce an uncertainty-aware graph representation framework for learning to guide planning in Partially Observable Markov Decision Processes (POMDPs). Unlike existing approaches that require domain or problem size specific neural architectures, GammaZero leverages a unified graph-based belief representation that enables generalization across problem sizes within a domain. Our key insight is that belief states can be systematically transformed into uncertainty-aware graphs where structural patterns learned on small problems transfer to larger instances. We employ a graph neural network with a decoder architecture to learn value functions and policies from expert demonstrations on computationally tractable problems, then apply these learned heuristics to guide Monte Carlo tree search on larger problems. Experimental results on standard POMDP benchmarks demonstrate that GammaZero achieves comparable performance to BetaZero when trained and tested on the same-sized problems, while enabling zero-shot generalization to problems 2-6x larger than those seen during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。