对比两种强化学习网络在跨环境迁移中的表现，发现结构差异影响迁移效果。
A Controlled Study of Double DQN and Dueling DQN Under Cross-Environment Transfer
- 固定参数下用同一策略迁移表示，比较DDQN与Dueling DQN的迁移能力。
- DDQN迁移后表现稳定，接近从零训练的基线；而Dueling DQN出现负迁移，性能下降。
- 结果表明模型结构偏好直接影响迁移鲁棒性，适合研究迁移稳定性的读者。
深度强化学习中的迁移学习常被期望提升稳定性并降低训练成本，但在显著领域偏移下也可能失败。本文开展受控实验，研究双深度Q网络（DDQN）与斗士型深度Q网络（Dueling DQN）的架构差异对跨环境迁移行为的影响。以CartPole为源任务，LunarLander为结构迥异的目标任务，在相同超参数与训练条件下，采用固定层表示迁移协议，并以从头训练的基线代理作为参照。实验结果表明，DDQN在该设置下始终避免负迁移，其在目标环境中的学习动态与基线性能相当；而Dueling DQN在相同条件下持续表现出负迁移，表现为奖励下降和优化不稳定。多随机种子的统计分析证实了迁移条件下显著的性能差距。研究结果表明，在所考察的迁移协议下，架构归纳偏置与基于价值的深度强化学习在跨环境迁移中的鲁棒性密切相关。
原文摘要 · Abstract (English)
Transfer learning in deep reinforcement learning is often motivated by improved stability and reduced training cost, but it can also fail under substantial domain shift. This paper presents a controlled empirical study examining how architectural differences between Double Deep Q-Networks (DDQN) and Dueling DQN influence transfer behavior across environments. Using CartPole as a source task and LunarLander as a structurally distinct target task, we evaluate a fixed layer-wise representation transfer protocol under identical hyperparameters and training conditions, with baseline agents trained from scratch used to contextualize transfer effects. Empirical results show that DDQN consistently avoids negative transfer under the examined setup and maintains learning dynamics comparable to baseline performance in the target environment. In contrast, Dueling DQN consistently exhibits negative transfer under identical conditions, characterized by degraded rewards and unstable optimization behavior. Statistical analysis across multiple random seeds confirms a significant performance gap under transfer. These findings suggest that architectural inductive bias is strongly associated with robustness to cross-environment transfer in value-based deep reinforcement learning under the examined transfer protocol.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。