arXiv:2410.17382cs.LGcs.MA2024-10被引 2

多智能体协作优化,在控制成本前提下降低集体错误

Cooperative Multi-Agent Constrained Stochastic Linear Bandits

  • 通过分布式共识算法让智能体共享平均收益与成本信息
  • 理论证明其累计误差随时间增长速率低于 $\mathcal{O}\left(\frac{1}{\sqrt{N}}\right)$
  • 适合需要协同决策且有成本约束的物联网或机器人系统

本文研究一种协作式多智能体随机线性带宽问题,网络中 $N$ 个智能体通过局部通信,在保持期望成本低于给定阈值 $τ$ 的前提下,最小化集体累积后悔值。每个智能体面对独立的线性带宽问题,拥有各自的奖励与成本参数(本地参数),目标是找到对应于这些参数平均值(全局参数)的最优行动。每轮随机选择一个智能体执行动作,所有智能体同时观测各自收益和成本。提出安全分布式上置信界算法 extit{MA-OPLB},利用加速共识机制,使智能体通过邻居间通信估计全网平均收益与成本。理论证明其 $T$ 轮后悔值以高概率满足 $\mathcal{O}\left(\frac{d}{τ-c_0}\frac{\log(NT)^2}{\sqrt{N}}\sqrt{\frac{T}{\log(1/|λ_2|)}}\right)$,其中 $λ_2$ 为通信矩阵第二大的特征值绝对值,$τ - c_0$ 为可行动作的成本余量。实验验证了算法在不同网络结构下的表现。

原文摘要 · Abstract (English)

In this study, we explore a collaborative multi-agent stochastic linear bandit setting involving a network of $N$ agents that communicate locally to minimize their collective regret while keeping their expected cost under a specified threshold $τ$. Each agent encounters a distinct linear bandit problem characterized by its own reward and cost parameters, i.e., local parameters. The goal of the agents is to determine the best overall action corresponding to the average of these parameters, or so-called global parameters. In each round, an agent is randomly chosen to select an action based on its current knowledge of the system. This chosen action is then executed by all agents, then they observe their individual rewards and costs. We propose a safe distributed upper confidence bound algorithm, so called \textit{MA-OPLB}, and establish a high probability bound on its $T$-round regret. MA-OPLB utilizes an accelerated consensus method, where agents can compute an estimate of the average rewards and costs across the network by communicating the proper information with their neighbors. We show that our regret bound is of order $ \mathcal{O}\left(\frac{d}{τ-c_0}\frac{\log(NT)^2}{\sqrt{N}}\sqrt{\frac{T}{\log(1/|λ_2|)}}\right)$, where $λ_2$ is the second largest (in absolute value) eigenvalue of the communication matrix, and $τ-c_0$ is the known cost gap of a feasible action. We also experimentally show the performance of our proposed algorithm in different network structures.

多智能体带宽问题分布式优化成本约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。