用大模型引导生成简洁概率程序，提升部分可观测决策模型学习效率
LLM-Guided Probabilistic Program Induction for POMDP Model Estimation
- 以大模型为先验生成候选概率程序，结合实测数据反馈优化
- 在多个玩具问题与真实机器人搜索任务中表现优于传统方法
- 适合需高效建模不确定决策的机器人与强化学习研究者
部分可观测马尔可夫决策过程（POMDP）用于建模不确定性下的决策。尽管已有多种近似求解方法，本文聚焦于学习此类模型本身。特别关注一类子集：模型各组件（观测函数、奖励函数、转移函数、初始状态分布）可表示为短概率程序的低复杂度概率图模型。我们提出利用大语言模型（LLM）作为先验，生成候选概率程序，并基于经验分布测试与反馈进行调整。实验涵盖经典玩具级POMDP问题、模拟MiniGrid环境，以及两个涉及部分可观测的真实移动机器人搜索任务。结果表明，使用LLM引导构建低复杂度POMDP模型，相比表格化POMDP学习、行为克隆或直接大模型规划更具优势。
原文摘要 · Abstract (English)
Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty. While there are many approaches to approximately solving POMDPs, we aim to address the problem of learning such models. In particular, we are interested in a subclass of POMDPs wherein the components of the model, including the observation function, reward function, transition function, and initial state distribution function, can be modeled as low-complexity probabilistic graphical models in the form of a short probabilistic program. Our strategy to learn these programs uses an LLM as a prior, generating candidate probabilistic programs that are then tested against the empirical distribution and adjusted through feedback. We experiment on a number of classical toy POMDP problems, simulated MiniGrid domains, and two real mobile-base robotics search domains involving partial observability. Our results show that using an LLM to guide in the construction of a low-complexity POMDP model can be more effective than tabular POMDP learning, behavior cloning, or direct LLM planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。