用最小通信量实现远程动作控制，让执行者自主生成动作。
Remote Action Generation: Remote Control with Minimal Communication
- 控制器发送极少信息，执行者基于重要性采样本地生成动作。
- 相比直接传动作，通信量平均减少12倍（连续空间达50倍）。
- 适合低带宽场景，尤其适用于大动作空间的远程控制任务。
我们研究远程控制问题：一个或多个执行者在无法直接获取奖励的情况下，通过通信受限的信道受控于控制器。控制器从观测到的奖励中学习最优策略，并向执行者传递动作指导，这一过程在大或连续动作空间下通信开销巨大。为实现高效通信，我们提出一种新型框架——远程生成（Remote Generation），控制器不再传输完整动作指令，而是发送最少信息，使执行者能基于控制器不断演化的目标策略进行采样生成动作。该过程通过重要性采样实现。同时，执行者将接收到的指导作为监督信号，学习控制器策略，从而提升本地采样能力，逐步降低未来通信需求。所提出的引导式远程动作采样策略（GRASP）在所有实验中实现平均12倍的数据压缩（连续动作空间达50倍），相较奖励传输减少41倍通信量。
原文摘要 · Abstract (English)
We address the challenge of remote control where one or more actors, lacking direct reward access, are steered by a controller over a communication-constrained channel. The controller learns an optimal policy from observed rewards and communicates action guidance to the actors, which becomes demanding for large or continuous action spaces. To achieve rate-efficient communication throughout this interactive learning and control process, we introduce a novel framework leveraging remote generation. Instead of transmitting full action specifications, the controller sends minimal information, enabling the actors to locally generate actions by sampling from the controller's evolving target policy. This guided sampling is facilitated by an importance sampling approach. Concurrently, the actors use the received guidance as supervised learning data to learn the controller's policy. This actor-side learning improves their local sampling capabilities, progressively reducing future communication needs. Our solution, Guided Remote Action Sampling Policy (GRASP), demonstrates significant communication reduction, achieving an average 12-fold data reduction across all experiments (50-fold for continuous action spaces) compared to direct action transmission, and a 41-fold reduction compared to reward transmission.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。