让每个智能体在共享模型上做低秩微调，兼顾协作与个性。
Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning
- 在共享策略中加入低秩适配矩阵，实现个性化微调。
- 在SMAC和MAMuJoCo上性能超越或持平基线，内存开销更低。
- 适合需要高效多智能体协作的场景，如游戏、机器人群组。
多智能体强化学习（MARL）常依赖参数共享（PS）以实现高效扩展。然而，完全共享的策略会抑制智能体在异构环境中的个性发展，降低整体性能。我们提出低秩智能体特异性适配（LoRASA），将每个智能体的策略视为从共享主干微调出的特定任务。借鉴参数高效迁移方法，LoRASA 在共享策略的每一层后附加小型低秩适配矩阵，自然引入参数空间稀疏性，促进个体特化与可扩展性。我们在星战多智能体挑战（SMAC）和多智能体MuJoCo（MAMuJoCo）等难题上评估了该方法，并基于MAPPO和A2PO等主流算法进行实现。在多种任务中，LoRASA 在保持甚至超越现有基线性能的同时，显著降低了内存和计算开销。消融实验验证了适配器秩、位置和时机的灵活性与效率。结果表明，LoRASA 有潜力成为多智能体策略参数化的全新范式：在共享基础之上，通过低秩微调实现个体特化。
原文摘要 · Abstract (English)
Multi-agent reinforcement learning (MARL) often relies on \emph{parameter sharing (PS)} to scale efficiently. However, purely shared policies can stifle each agent's unique specialization, reducing overall performance in heterogeneous environments. We propose \textbf{Low-Rank Agent-Specific Adaptation (LoRASA)}, a novel approach that treats each agent's policy as a specialized ``task'' fine-tuned from a shared backbone. Drawing inspiration from parameter-efficient transfer methods, LoRASA appends small, low-rank adaptation matrices to each layer of the shared policy, naturally inducing \emph{parameter-space sparsity} that promotes both specialization and scalability. We evaluate LoRASA on challenging benchmarks including the StarCraft Multi-Agent Challenge (SMAC) and Multi-Agent MuJoCo (MAMuJoCo), implementing it atop widely used algorithms such as MAPPO and A2PO. Across diverse tasks, LoRASA matches or outperforms existing baselines \emph{while reducing memory and computational overhead}. Ablation studies on adapter rank, placement, and timing validate the method's flexibility and efficiency. Our results suggest LoRASA's potential to establish a new norm for MARL policy parameterization: combining a shared foundation for coordination with low-rank agent-specific refinements for individual specialization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。