用自由能最小化改进蒙特卡洛树搜索，让智能体更聪明地探索与决策。
Boosting MCTS with Free Energy Minimization
- 将自由能最小化融入MCTS，结合奖励与信息增益优化动作选择
- 在连续控制任务中优于单独使用CEM或随机回溯的MCTS
- 适合需要高效探索与不确定感知的强化学习场景
主动推断基于自由能原理，为智能体在不确定环境中平衡探索与目标导向行为提供了有力视角。本文提出一种新的规划框架，将蒙特卡洛树搜索（MCTS）与主动推断目标结合，系统性降低认知不确定性的同时追求外部奖励。核心思想是：MCTS本身具备高效搜索能力，可通过融合期望奖励与信息增益，自然扩展以实现自由能最小化。具体而言，根节点采用交叉熵方法（CEM）优化动作提案，树扩展则结合奖励建模与内在探索奖励。该协同机制使规划器在保持价值与不确定性估计一致性的同时，不牺牲计算可行性。实验在多种连续控制任务上验证了该规划器性能优于独立使用CEM或随机回溯的MCTS。
原文摘要 · Abstract (English)
Active Inference, grounded in the Free Energy Principle, provides a powerful lens for understanding how agents balance exploration and goal-directed behavior in uncertain environments. Here, we propose a new planning framework, that integrates Monte Carlo Tree Search (MCTS) with active inference objectives to systematically reduce epistemic uncertainty while pursuing extrinsic rewards. Our key insight is that MCTS already renowned for its search efficiency can be naturally extended to incorporate free energy minimization by blending expected rewards with information gain. Concretely, the Cross-Entropy Method (CEM) is used to optimize action proposals at the root node, while tree expansions leverage reward modeling alongside intrinsic exploration bonuses. This synergy allows our planner to maintain coherent estimates of value and uncertainty throughout planning, without sacrificing computational tractability. Empirically, we benchmark our planner on a diverse set of continuous control tasks, where it demonstrates performance gains over both standalone CEM and MCTS with random rollouts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。