通过局部动态差异筛选源数据，提升目标域离线强化学习性能
Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning
- 按局部动态差异聚类源与目标数据，识别可复用的相似区域
- 在多种动态迁移场景下，性能优于当前最优方法
- 适合处理状态空间中存在局部差异的离线强化学习任务
离线强化学习中的跨动态域适应旨在仅用少量目标域数据和大量异动态源数据训练策略。现有方法或全局处理动态不匹配，或逐样本过滤，易忽略局部相似性或计算开销大。本文提出局部动态感知域自适应（LoDADA），对源与目标数据的转移进行聚类，并通过域判别估计各簇级动态差异：保留差异小的簇中源数据，剔除差异大的簇。该策略实现细粒度且可扩展的数据选择，避免粗略全局假设和高成本逐样本筛选。理论分析与多环境实验表明，LoDADA在多样全局与局部动态偏移下均显著优于现有先进方法，更有效利用局部分布差异。
原文摘要 · Abstract (English)
Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically address dynamics mismatch either globally over the state space or via pointwise data filtering; these approaches can miss localized cross-domain similarities or incur high computational cost. We propose Localized Dynamics-Aware Domain Adaptation (LoDADA), which exploits localized dynamics mismatch to better reuse source data. LoDADA clusters transitions from source and target datasets and estimates cluster-level dynamics discrepancy via domain discrimination. Source transitions from clusters with small discrepancy are retained, while those from clusters with large discrepancy are filtered out. This yields a fine-grained and scalable data selection strategy that avoids overly coarse global assumptions and expensive per-sample filtering. We provide theoretical insights and extensive experiments across environments with diverse global and local dynamics shifts. Results show that LoDADA consistently outperforms state-of-the-art off-dynamics offline RL methods by better leveraging localized distribution mismatch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。