让视觉与语言理解在模型中交替进行，提升机器人任务表现
Adapting Interleaved Encoders with PPO for Language-Guided Reinforcement Learning in BabyAI
- 用交替的感知-决策结构替代传统分离设计
- 在BabyAI任务中实现更稳定奖励和更强语义对齐
- 适合研究多模态智能体与自主决策系统的人参考
深度强化学习代理在需要同时理解视觉与语言的任务中常表现不佳。传统架构通常将感知（如基于CNN的视觉编码器）与决策（策略网络）分开，这种分离效率低下，因为策略失败无法直接帮助感知模块学习关键信息。为此，我们实现了Mao等（2023）提出的感知-决策交错变压器（PDiT）架构，该模型在单一Transformer中交替排列感知与决策层，使决策反馈能动态优化感知特征。此外，我们引入受CLIP启发的对比损失，对齐文本任务嵌入与视觉场景特征。在BabyAI GoToLocal环境中评估PDiT编码器，结果表明该方法相比标准PPO基线展现出更稳定的奖励和更强的对齐能力。结果表明，交错式Transformer编码器是构建更集成自主智能体的有前景方向。
原文摘要 · Abstract (English)
Deep reinforcement learning agents often struggle when tasks require understanding both vision and language. Conventional architectures typically isolate perception (for example, CNN-based visual encoders) from decision-making (policy networks). This separation can be inefficient, since the policy's failures do not directly help the perception module learn what is important. To address this, we implement the Perception-Decision Interleaving Transformer (PDiT) architecture introduced by Mao et al. (2023), a model that alternates between perception and decision layers within a single transformer. This interleaving allows feedback from decision-making to refine perceptual features dynamically. In addition, we integrate a contrastive loss inspired by CLIP to align textual mission embeddings with visual scene features. We evaluate the PDiT encoders on the BabyAI GoToLocal environment and find that the approach achieves more stable rewards and stronger alignment compared to a standard PPO baseline. The results suggest that interleaved transformer encoders are a promising direction for developing more integrated autonomous agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。