构建中文购物模拟环境，评估并优化大模型导购能力
ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants
- 创建大规模中文购物仿真环境，支持多轮对话与个性化搜索
- 顶尖模型全成功率不足40%，暴露出深度搜索与选择短板
- 结合SFT与RL训练可显著提升性能，适合电商AI研究者参考
基于大语言模型（LLM）的智能导购系统在电商中日益普及。为实现精准、个性化的商品搜索，代理需理解用户偏好、开展多轮对话，并从高度相似的商品中准确筛选。然而，现有研究缺乏统一的仿真环境，且仅关注评估而无训练支持。本文提出ShopSimulator，一个大规模、高挑战性的中文购物仿真环境。利用该环境，我们在多种场景下评估了多个LLM，发现即使表现最佳的模型，全成功率也低于40%。错误分析显示，代理在长轨迹中存在深度搜索与商品选择困难，难以平衡个性化线索使用，且用户交互效率低。进一步的训练探索表明，结合监督微调（SFT）与强化学习（RL）能显著提升性能。代码与数据将开源于https://github.com/ShopAgent-Team/ShopSimulator。
原文摘要 · Abstract (English)
Large language model (LLM)-based agents are increasingly deployed in e-commerce shopping. To perform thorough, user-tailored product searches, agents should interpret personal preferences, engage in multi-turn dialogues, and ultimately retrieve and discriminate among highly similar products. However, existing research has yet to provide a unified simulation environment that consistently captures all of these aspects, and always focuses solely on evaluation benchmarks without training support. In this paper, we introduce ShopSimulator, a large-scale and challenging Chinese shopping environment. Leveraging ShopSimulator, we evaluate LLMs across diverse scenarios, finding that even the best-performing models achieve less than 40% full-success rate. Error analysis reveals that agents struggle with deep search and product selection in long trajectories, fail to balance the use of personalization cues, and to effectively engage with users. Further training exploration provides practical guidance for overcoming these weaknesses, with the combination of supervised fine-tuning (SFT) and reinforcement learning (RL) yielding significant performance improvements. Code and data will be released at https://github.com/ShopAgent-Team/ShopSimulator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。