用自我反思的蒙特卡洛树搜索提升AI自动机器学习的决策质量
I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search
- 通过反思父节点和兄弟节点结果来迭代优化搜索路径
- 相比开源AutoML代理，任务性能提升4%绝对值
- 适合研究智能体自动机器学习与强化学习融合的学者
大型语言模型（LLMs）在自动化机器学习任务方面展现出巨大潜力。然而，现有基于LLM的智能体常面临思维多样性低、代码生成质量差的问题。尽管已有研究引入蒙特卡洛树搜索（MCTS）以缓解此问题，但节点生成的思维质量与多样性仍受限，且节点选择依赖的标量反馈机制不够有效。本文提出一种新型方法——自我反思蒙特卡洛树搜索（I-MCTS），通过迭代式反思过程，细致分析父节点与兄弟节点的解决方案与结果，持续优化搜索树中的节点，从而提升整体决策能力。此外，我们引入基于大语言模型（LLM）的价值评估模型，在进行完整计算回放前直接评估每个节点解的质量。采用混合奖励机制，实现从LLM估算得分到实际性能得分的平滑过渡，使高质量节点更早被探索。在多个机器学习任务上，该方法相较强基线开源AutoML智能体实现了4%的绝对性能提升，验证了其在增强智能体式AutoML系统中的有效性。代码已公开于 https://github.com/jokieleung/I-MCTS。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have shown remarkable potential in automating machine learning tasks. However, existing LLM-based agents often struggle with low-diversity and suboptimal code generation. While recent work has introduced Monte Carlo Tree Search (MCTS) to address these issues, limitations persist in the quality and diversity of thoughts generated, as well as in the scalar value feedback mechanisms used for node selection. In this study, we introduce Introspective Monte Carlo Tree Search (I-MCTS), a novel approach that iteratively expands tree nodes through an introspective process that meticulously analyzes solutions and results from parent and sibling nodes. This facilitates a continuous refinement of the node in the search tree, thereby enhancing the overall decision-making process. Furthermore, we integrate a Large Language Model (LLM)-based value model to facilitate direct evaluation of each node's solution prior to conducting comprehensive computational rollouts. A hybrid rewarding mechanism is implemented to seamlessly transition the Q-value from LLM-estimated scores to actual performance scores. This allows higher-quality nodes to be traversed earlier. Applied to the various ML tasks, our approach demonstrates a 4% absolute improvement in performance compared to the strong open-source AutoML agents, showcasing its effectiveness in enhancing agentic AutoML systems. Resource available at https://github.com/jokieleung/I-MCTS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。