将大模型推理蒸馏为可解释策略树，提升人机协作效率与透明度。
Distilling LLM Reasoning into an Interpretable Policy Tree for Human-AI Collaboration
- 通过蒸馏大模型推理生成可执行的策略树，实现闭环优化
- 在Overcooked-AI中奖励提升35.4%，LLM调用减少77.7%
- 适合关注可解释性、低延迟人机协作的研究者
构建高效可靠的辅助人类的策略对人机协作至关重要。现有方法主要分为两类：多数前期工作依赖多智能体强化学习（MARL）学习黑盒策略，导致可解释性差且存在安全风险；近期方法在每个决策步骤调用大语言模型（LLM），造成响应慢和推理成本高。我们提出协作策略树（Co-pi-tree），一种闭环方法，通过构建由伙伴行为预测树和代理动作选择树组成的可执行策略树，将大模型推理过程蒸馏为策略树代码。该方法通过与伙伴交互评估策略，获取反馈，并使用自然语言总结反馈以改进问题分支。在Overcooked-AI上的实验表明，Co-pi-tree相比基线平均奖励提升35.4%，同时减少77.7%的LLM查询次数，测试时延降低97.1%。
原文摘要 · Abstract (English)
Constructing efficient and reliable policies to assist humans is indispensable for human-AI collaboration. Existing methods mainly follow two lines of work. Most prior work relies on multi-agent reinforcement learning (MARL) to learn black-box policies, which limits interpretability and raises safety concerns. Recent methods query large language models (LLMs) at each decision step, causing slow responses and high inference costs. We propose Collaboration Policy Tree (Co-pi-tree), a closed-loop method that learns an executable policy tree consisting of a partner-behavior prediction tree and an agent-action selection tree. Co-pi-tree constructs a policy by distilling LLM reasoning into policy tree code. It then evaluates the policy through partner interaction, obtains feedback, and uses natural language to summarize the interaction feedback to improve problematic branches. Experiments in Overcooked-AI show that Co-pi-tree improves average reward by 35.4% over the baseline average, while reducing the number of LLM queries by 77.7% and test-time latency by 97.1%. Project page: https://beiwenzhang.github.io/Co-pi-tree/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。