让小模型精准模仿大模型的推理与行动流程。
Structured Agent Distillation for Large Language Model
- 按推理和行动阶段分段训练，分别优化对应行为。
- 在多个任务上压缩模型体积，性能损失小于5%。
- 适合需要高效部署智能代理的场景。
大型语言模型（LLMs）通过交织推理与动作表现出强大的决策能力，如ReAct框架所示。然而，其高推理成本和大模型尺寸限制了实际部署。本文提出结构化智能体蒸馏（Structured Agent Distillation），将基于LLM的智能体压缩为更小的学生模型，同时保持推理准确性和动作一致性。不同于传统的逐标记蒸馏，该方法将智能体轨迹划分为[REASON]和[ACT]段，并对每段应用特定损失函数，以对齐教师模型的行为。这种结构感知的监督使小型代理能更好地复现教师的决策过程。在ALFWorld、HotPotQA-ReAct和WebShop上的实验表明，该方法持续优于基于标记和模仿学习的基线，实现显著压缩且性能下降极小。缩放与消融实验进一步验证了段级对齐对高效可部署智能体的重要性。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit strong capabilities as decision-making agents by interleaving reasoning and actions, as seen in ReAct-style frameworks. Yet, their practical deployment is constrained by high inference costs and large model sizes. We propose Structured Agent Distillation, a framework that compresses large LLM-based agents into smaller student models while preserving both reasoning fidelity and action consistency. Unlike standard token-level distillation, our method segments trajectories into [REASON] and [ACT] spans, applying segment-specific losses to align each component with the teacher's behavior. This structure-aware supervision enables compact agents to better replicate the teacher's decision process. Experiments on ALFWorld, HotPotQA-ReAct, and WebShop show that our approach consistently outperforms token-level and imitation learning baselines, achieving significant compression with minimal performance drop. Scaling and ablation results further highlight the importance of span-level alignment for efficient and deployable agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。