让强化学习控制器在低功耗硬件上用极少比特运行，还更抗干扰。
Learning Quantized Continuous Controllers for Integer Hardware
- 训练时就考虑硬件量化，自动选2~3比特的轻量模型
- 在五个任务上性能接近全精度，推理延迟微秒级，功耗微焦耳级
- 量化后反而更抗输入噪声，适合嵌入式设备部署
将连续控制强化学习策略部署到嵌入式硬件需满足严格的延迟与功耗要求。小型FPGA可实现此目标，但必须避免高成本的浮点计算流水线。本文研究针对整数推理的量化感知训练（QAT），提出一个从学习到硬件的自动化流程，能自动选择低比特策略并在Artix-7 FPGA上合成。在五个MuJoCo任务中，所获策略性能与全精度（FP32）相当，权重和内部激活值仅需3位甚至2位，前提是输入精度合理设定。在目标硬件上，这些策略实现微秒级推理延迟，每动作功耗仅为微焦耳级别，优于量化基准。此外，量化策略相比浮点基线展现出更强的输入噪声鲁棒性。
原文摘要 · Abstract (English)
Deploying continuous-control reinforcement learning policies on embedded hardware requires meeting tight latency and power budgets. Small FPGAs can deliver these, but only if costly floating-point pipelines are avoided. We study quantization-aware training (QAT) of policies for integer inference and we present a learning-to-hardware pipeline that automatically selects low-bit policies and synthesizes them to an Artix-7 FPGA. Across five MuJoCo tasks, we obtain policy networks that are competitive with full precision (FP32) policies but require as few as 3 or even only 2 bits per weight, and per internal activation value, as long as input precision is chosen carefully. On the target hardware, the selected policies achieve inference latencies on the order of microseconds and consume microjoules per action, favorably comparing to a quantized reference. Last, we observe that the quantized policies exhibit increased input noise robustness compared to the floating-point baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。