arXiv:2506.19846cs.AI2025-06被引 8

构建首个电商场景的分层多智能体基准,实现高效联合训练

HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistants

  • 设计分层结构,用动态分组优化多智能体协同策略
  • 在22.8K数据上使小模型性能媲美大模型,平均提升6%
  • 适合做电商助手、多智能体系统研发的研究者参考

基于大语言模型的分层多智能体系统已成为垂直领域(如电商)构建AI助手的主流范式,其中主智能体协调多个专业子智能体。尽管应用广泛,真实场景下的训练与评估基准仍稀缺,跨功能智能体的联合优化仍具挑战。为此,我们提出首个面向电商场景的分层多智能体基准HiMA-Ecom,包含22.8K条实例,涵盖智能体级监督微调样本及系统级输入输出对,支持联合多智能体强化学习。在此基础上,提出联合训练方法HiMA-R1,采用基于初始轨迹的蒙特卡洛采样缓解指数级联合动作空间问题,并根据奖励方差选择信息量高的智能体组进行高效更新。此外,设计自适应记忆演化机制,将GRPO奖励作为无成本监督信号,消除重复推理,加速收敛。在HiMA-Ecom上的实验表明,基于3B/7B开源模型的方法性能可媲美更大的DeepSeek-R1模型,且平均超越DeepSeek-V3达6%。

原文摘要 · Abstract (English)

Hierarchical multi-agent systems based on large language models (LLMs) have become a common paradigm for building AI assistants in vertical domains such as e-commerce, where a master agent coordinates multiple specialized sub-agents. Despite their practical importance, realistic benchmarks for training and evaluating such systems remain scarce, and joint optimization across functionally distinct agents is still challenging. To address this gap, we introduce HiMA-Ecom, the first hierarchical multi-agent benchmark tailored for e-commerce scenarios. HiMA-Ecom contains 22.8K instances, including agent-specific supervised fine-tuning samples with memory and system-level input-output pairs for joint multi-agent reinforcement learning. Building upon it, a joint training method named HiMA-R1 is proposed. It presents Variance-Reduction Group Relative Policy Optimization (VR-GRPO), which employs initial trajectory-based Monte Carlo sampling to mitigate the exponential joint action space and selects informative agent groups for efficient updates based on reward variance. Furthermore, an adaptive memory evolution mechanism that repurposes GRPO rewards as cost-free supervisory signals is designed to eliminate repetitive reasoning and accelerate convergence. Experiments on HiMA-Ecom demonstrate that our method, built upon smaller 3B/7B open-source models, achieves performance comparable to that of larger LLMs, such as DeepSeek-R1, and surpasses DeepSeek-V3 by an average of 6\%.

多智能体电商助手联合训练强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。