用知识框架提示,让大模型在规划任务上准确率从31.5%提升至97.3%
Knowledge Model Prompting Increases LLM Performance on Planning Tasks
- 引入认知科学中的TMK框架,明确任务、方法与知识的因果关系
- 在PlanBench的积木世界任务中,准确率从31.5%提升至97.3%
- 适合研究大模型推理机制或需结构化提示的规划任务应用
大型语言模型(LLM)在推理与规划任务中表现欠佳。尽管已有如思维链(CoT)等提示技术,但其推理能力仍受质疑。本文借鉴认知与教育科学中的任务-方法-知识(TMK)框架,探索其是否能超越教育场景,在提升语言模型推理能力方面发挥作用。该框架能显式表达行动的因果、目的与层级结构,且不同于HTN和BDI等层次化框架,其不仅说明‘做什么’和‘怎么做’,还解释‘为什么做’。实验基于PlanBench基准测试,在积木世界(Blocksworld)领域评估,检验TMK提示能否帮助模型将复杂规划问题分解为可管理子任务。结果显示,采用TMK提示后,模型在抽象符号任务(PlanBench中的随机版积木世界)上的准确率从原先的31.5%提升至97.3%,表明其可能弥合语义近似与符号操作之间的差距。研究发现,TMK不仅作为上下文,更是一种引导模型脱离默认语言模式、转向形式化代码执行路径的机制。
原文摘要 · Abstract (English)
Large Language Models (LLM) can struggle with reasoning ability and planning tasks. Many prompting techniques have been developed to assist with LLM reasoning, notably Chain-of-Thought (CoT); however, these techniques, too, have come under scrutiny as LLMs' ability to reason at all has come into question. Borrowing from the domain of cognitive and educational science, this paper investigates whether the Task-Method-Knowledge (TMK) framework can improve LLM reasoning capabilities beyond its previously demonstrated success in educational applications. The TMK framework's unique ability to capture causal, teleological, and hierarchical reasoning structures, combined with its explicit task decomposition mechanisms, makes it particularly well-suited for addressing language model reasoning deficiencies, and unlike other hierarchical frameworks such as HTN and BDI, TMK provides explicit representations of not just what to do and how to do it, but also why actions are taken. The study evaluates TMK by experimenting on the PlanBench benchmark, focusing on the Blocksworld domain to test for reasoning and planning capabilities, examining whether TMK-structured prompting can help language models better decompose complex planning problems into manageable sub-tasks. Results also highlight significant performance inversion in reasoning models. TMK prompting enables the reasoning model to achieve up to an accuracy of 97.3\% on opaque, symbolic tasks (Random versions of Blocksworld in PlanBench) where it previously failed (31.5\%), suggesting the potential to bridge the gap between semantic approximation and symbolic manipulation. Our findings suggest that TMK functions not merely as context, but also as a mechanism that steers reasoning models away from their default linguistic modes to engage formal, code-execution pathways in the context of the experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。