让目标表示随当前状态动态调整，提升强化学习导航性能
DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention
- 用多尺度门控交叉注意力将静态目标编码转为状态相关表示
- 在OGBench上显著提升导航任务表现，关键依赖门控残差结构
- 适合需精准状态感知的导航类任务，不适用于所有强化学习场景
目标条件强化学习依赖于目标的编码方式。对比、度量、时序距离和信息论编码虽目标不同，但均忽略当前状态。这种状态无关的嵌入无法标记目标中仍需执行的部分，策略需反向解码两个编码器来恢复线索。本文提出DAGR，通过多尺度门控交叉注意力，将任意后期融合编码器的静态嵌入优化为状态相关表示。近恒等门控残差保留基础表征，差异感知目标交叉注意力则利用逐标记的状态-目标差异图调节注意力权重。在OGBench上，DAGR显著提升导航性能；消融实验表明收益主要来自门控残差，而非命名方法的差异偏差。在操控与谜题任务中，其表现与基线持平或略低。DAGR是结构化精炼，非普适性提升。
原文摘要 · Abstract (English)
Goal-conditioned reinforcement learning hinges on how the goal is encoded. Contrastive, metric, temporal-distance, and information-theoretic encoders differ in objective. They still share one trait. None of them sees the current state. Such a state-independent embedding cannot mark which part of the goal still needs action. The policy must then recover that cue by inverting both encoders. We propose DAGR. It refines the static embedding of any late-fusion encoder into a state-conditioned one through multi-scale gated cross-attention. A near-identity gated residual preserves the base representation. Difference-aware Goal Cross-Attention then biases the attention scores using a per-token state-goal discrepancy map. On OGBench, DAGR improves navigation. Our ablations trace the gain to the gated residual, not to the difference bias that names the method. On manipulation and puzzle tasks it matches or falls below the base. DAGR is a structured refinement, not a universal improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。