轻量级微调框架,让多智能体感知模型快速适应新场景。
CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning
- 用适配器和提示词分层实现高效参数微调
- 仅需不到1%可训练参数即超越现有方法
- 适合算力受限的实时部署场景
多智能体协作感知通过交换互补信息有望显著提升感知性能,但训练鲁棒模型需覆盖所有可能协作场景的充足数据,实际部署成本过高,导致模型对新交通场景泛化能力差。现有域自适应方法虽在训练中引入部署数据缓解问题,但训练成本高,不适用于资源受限的智能体。本文提出基于参数高效微调的轻量级框架CoPEFT,实现低开销下快速适应新环境。CoPEFT设计协作适配器与智能体提示,分别进行宏观与微观层面的适应:协作适配器利用训练数据与少量部署数据中的内在知识,调整特征图以匹配新数据分布;智能体提示则注入环境细粒度上下文信息进一步增强适应能力。大量实验表明,CoPEFT仅使用少于1%的可训练参数,便优于现有方法,验证了其高效性与有效性。
原文摘要 · Abstract (English)
Multi-agent collaborative perception is expected to significantly improve perception performance by overcoming the limitations of single-agent perception through exchanging complementary information. However, training a robust collaborative perception model requires collecting sufficient training data that covers all possible collaboration scenarios, which is impractical due to intolerable deployment costs. Hence, the trained model is not robust against new traffic scenarios with inconsistent data distribution and fundamentally restricts its real-world applicability. Further, existing methods, such as domain adaptation, have mitigated this issue by exposing the deployment data during the training stage but incur a high training cost, which is infeasible for resource-constrained agents. In this paper, we propose a Parameter-Efficient Fine-Tuning-based lightweight framework, CoPEFT, for fast adapting a trained collaborative perception model to new deployment environments under low-cost conditions. CoPEFT develops a Collaboration Adapter and Agent Prompt to perform macro-level and micro-level adaptations separately. Specifically, the Collaboration Adapter utilizes the inherent knowledge from training data and limited deployment data to adapt the feature map to new data distribution. The Agent Prompt further enhances the Collaboration Adapter by inserting fine-grained contextual information about the environment. Extensive experiments demonstrate that our CoPEFT surpasses existing methods with less than 1\% trainable parameters, proving the effectiveness and efficiency of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。