一个轻量模型跨环境高效选动作，省去逐环境训练
Cross-Environment Neural Reranking for Sample-Efficient Action Selection in Text-Based Agents
- 用多环境联合训练让小模型学会跨域动作选择
- 仅用9.2%数据微调就恢复93%性能，数据多样性更重要
- 适合追求低成本部署的文本智能体开发者
大型语言模型代理在文本基准上表现优异但推理成本高昂，促使使用紧凑的神经重排序器进行动作选择。本文探究单一轻量模型是否能在多个异构环境中实现动作选择，从而避免针对每个环境单独维护模型。在 ALFWorld、WebShop 和 ScienceWorld 上联合训练 DeBERTa-v3（184M-434M 参数），采用少数类样本增强策略，发现重新平衡的双环境联合训练相比单环境 ALFWorld 性能提升 +0.412，同时保持与专门训练的 WebShop 模型相当的表现（+0.214 vs. +0.249）。三环境联合训练在4个种子下平均获得 +0.551 ± 0.024 的综合净增益，各环境性能接近专用模型，并展现出正向跨域迁移能力。跨环境适配极具样本效率：仅需目标领域9.2%的数据微调即可恢复93%全数据性能；扩大模型规模收益有限，表明数据多样性是核心驱动力。结合环境感知的 LoRA 路由与 PCGrad 方法，在最佳种子（seed 42）达到 +0.611，其他种子分别为 +0.554 和 +0.559，但存在较大波动（种子123降至+0.263，4种子均值+0.497±0.158），显示该方向潜力大但尚不稳定。干净数据划分与数据重平衡是联合训练的关键。论文将公开包含51,580条训练实例（41,740条原始唯一状态，经少数类增强）的三环境基准数据集及所有模型检查点。
原文摘要 · Abstract (English)
Large language model agents achieve strong performance on text-based benchmarks but incur prohibitive inference costs, motivating the use of compact neural rerankers for action selection. We investigate whether a single lightweight model can perform action selection across multiple diverse environments, a capability that would eliminate per-environment model maintenance. Training DeBERTa-v3 (184M-434M parameters) jointly on ALFWorld, WebShop, and ScienceWorld with minority-class upsampling, we find that rebalanced two-environment joint training substantially improves over single-environment ALFWorld performance (net gain +0.412) while maintaining competitive WebShop performance (+0.214 vs. +0.249 single-environment). Three-environment training yields a mean combined net gain of +0.551 +/- 0.024 across 4 seeds, with per-environment results approaching specialized single-environment models while providing positive cross-domain transfer. Cross-environment adaptation is highly sample-efficient: fine-tuning on only 9.2% of target-domain data recovers 93% of full-data performance, and scaling model capacity yields limited benefits, indicating data diversity is the primary driver. Environment-aware LoRA adapter routing with PCGrad achieves a best-seed result of +0.611 (seed 42), with seeds 456 and 789 at +0.554 and +0.559, but exhibits high variance due to seed 123 collapsing to +0.263 (4-seed mean +0.497 +/- 0.158), representing a promising but currently unstable direction. Joint training with clean splits and data rebalancing is a key ingredient. We will release our three-environment benchmark of 51,580 training instances (41,740 raw unique states with minority-class upsampling) and all model checkpoints upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。