将大模型的复杂决策能力蒸馏到小模型,实现高效部署。
Embodied CoT Distillation From LLM To Off-the-shelf Agents
- 将大模型决策拆解为推理与规划双策略,分层蒸馏。
- 在ALFRED基准上超越主流方法,小模型表现接近大模型。
- 适合资源受限设备上的智能体部署,如移动端或嵌入式系统。
针对在资源受限的现成设备上运行决策系统时,如何有效利用大语言模型(LLMs)完成复杂具身任务的问题,本文提出DeDer框架,将LLM的具身推理能力分解并蒸馏至轻量级语言模型(sLM)驱动的策略中。DeDer将决策过程重构为推理策略与规划策略的层次结构:推理策略通过具身上下文学习和自验证生成的数据进行蒸馏,生成有效推理过程;规划策略则基于推理结果生成高效优化计划。该框架支持在现成设备上部署双策略的sLM。为进一步提升具身任务中的中间推理质量,引入具身知识图谱;为实现单次推理生成多条推理路径,采用对比提示注意力模型。在ALFRED基准上的实验表明,DeDer优于当前主流的语言规划与蒸馏方法,验证了其在sLM驱动的具身策略中的适用性与高效性。
原文摘要 · Abstract (English)
We address the challenge of utilizing large language models (LLMs) for complex embodied tasks, in the environment where decision-making systems operate timely on capacity-limited, off-the-shelf devices. We present DeDer, a framework for decomposing and distilling the embodied reasoning capabilities from LLMs to efficient, small language model (sLM)-based policies. In DeDer, the decision-making process of LLM-based strategies is restructured into a hierarchy with a reasoning-policy and planning-policy. The reasoning-policy is distilled from the data that is generated through the embodied in-context learning and self-verification of an LLM, so it can produce effective rationales. The planning-policy, guided by the rationales, can render optimized plans efficiently. In turn, DeDer allows for adopting sLMs for both policies, deployed on off-the-shelf devices. Furthermore, to enhance the quality of intermediate rationales, specific to embodied tasks, we devise the embodied knowledge graph, and to generate multiple rationales timely through a single inference, we also use the contrastively prompted attention model. Our experiments with the ALFRED benchmark demonstrate that DeDer surpasses leading language planning and distillation approaches, indicating the applicability and efficiency of sLM-based embodied policies derived through DeDer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。