提出可自适应的度量方法，提升视觉强化学习中状态表示的准确性。
PAMD: Structured Adaptive Distances for Bisimulation Representations in Visual Reinforcement Learning
- 用配对条件的马氏距离替代固定度量，动态调整相似性计算。
- 在视觉MuJoCo任务上显著提升多种双模拟算法的最终性能。
- 无需修改主框架，可直接插入现有方法中使用。
许多视觉强化学习算法通过匹配潜在空间距离与由奖励和转移相似性诱导的行为距离来学习表示。实践中，潜在距离的选择会显著影响性能：使用固定全局范数（如ℓ_p范数或人工设计度量）可能过于僵化，无法捕捉行为距离；而无约束的成对距离又可能导致退化解，使度量损失下降但表示质量未提升。为此，我们提出PAMD：配对自适应马氏距离，参数化一个正定、配对条件的度量，用于衡量潜在状态相似性。PAMD是现有双模拟方法的即插即用组件，提供比固定度量更灵活且结构化的替代方案。我们在视觉MuJoCo连续控制任务上进行实验验证，结果表明，多个近期双模拟基强化学习算法在引入该距离后，最终性能得到显著提升。
原文摘要 · Abstract (English)
Many visual reinforcement learning (RL) algorithms learn representations by matching latent distances to a behavioral distance induced by reward and transition similarity. In practice, the choice of the latent distance can strongly affect performance: using a fixed, pre-specified global norms (e.g., $\ell_p$ norms or other hand-designed metrics) may be overly restrictive to capture the behavioral distance. In contrast, unconstrained pairwise distances may admit degenerate solutions that drive the metric loss down without improving the representation. To address this gap, we introduce **PAMD: Pairwise Adaptive Mahalanobis Distance**, which parameterizes a positive-definite, pair-conditioned metric for measuring latent state similarity. PAMD is a simple plug-in for existing bisimulation-based methods, offering a more expressive yet structured alternative to fixed, pre-specified latent distances. We empirically validate our method on visual MuJoCo continuous-control tasks, where final performance of several recent bisimulation-based RL algorithms is substantially improved when equipped with the distance we propose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。