arXiv:2512.09897cs.AIcs.CL2025-12

用一次LLM生成目标,让小模型高效完成文本环境中的长程规划。

SCOPE: Language Models as One-Time Teacher for Hierarchical Planning in Text Environments

  • 只在初始化时调用一次LLM生成子目标,训练中不再依赖大模型
  • 在TextCraft上成功率达0.56,推理时间从164.4秒降至3.0秒
  • 适合追求高效部署的文本任务规划,尤其适用于资源受限场景

在复杂文本环境中进行长程规划面临动作空间开放、观测模糊和反馈稀疏等挑战。尽管大语言模型(LLMs)蕴含丰富的世界知识,可辅助智能体进行高层推理,但现有方法普遍依赖训练与推理时反复调用LLM,导致计算开销大且难以高效部署。同时,这些方法通常固定使用预训练的未调整LLM,无法适应目标任务。为此,我们提出SCOPE(Subgoal-COnditioned Pretraining for Efficient planning),一种仅在初始化阶段利用LLM生成子目标来预训练轻量级学生模型的一次性分层规划器。不同于以往通过持续提示生成子目标进行知识蒸馏的方法,本方法直接从示例轨迹中提取子目标,避免了重复查询,显著提升效率,但牺牲部分可解释性和子目标最优性。尽管如此,在TextCraft环境上的实验表明,即使子目标非最优,仍能为文本规划任务提供强有力的层级目标分解起点。相比基线方法ADaPT(成功率为0.52),我们的方法达到0.56的成功率,推理时间由164.4秒减少至3.0秒。

原文摘要 · Abstract (English)

Long-term planning in complex, text-based environments presents significant challenges due to open-ended action spaces, ambiguous observations, and sparse feedback. Recent research suggests that large language models (LLMs) encode rich semantic knowledge about the world, which can be valuable for guiding agents in high-level reasoning and planning across both embodied and purely textual settings. However, existing approaches often depend heavily on querying LLMs during training and inference, making them computationally expensive and difficult to deploy efficiently. In addition, these methods typically employ a pretrained, unaltered LLM whose parameters remain fixed throughout training, providing no opportunity for adaptation to the target task. To address these limitations, we introduce SCOPE (Subgoal-COnditioned Pretraining for Efficient planning), a one-shot hierarchical planner that leverages LLM-generated subgoals only at initialization to pretrain a lightweight student model. Unlike prior approaches that distill LLM knowledge by repeatedly prompting the model to adaptively generate subgoals during training, our method derives subgoals directly from example trajectories. This design removes the need for repeated LLM queries, significantly improving efficiency, though at the cost of reduced explainability and potentially suboptimal subgoals. Despite their suboptimality, our results on the TextCraft environment show that LLM-generated subgoals can still serve as a strong starting point for hierarchical goal decomposition in text-based planning tasks. Compared to the LLM-based hierarchical agent ADaPT (Prasad et al., 2024), which achieves a 0.52 success rate, our method reaches 0.56 and reduces inference time from 164.4 seconds to just 3.0 seconds.

文本规划大模型蒸馏高效推理分层规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。