从海量行为数据中精选少量样本，就能达到接近全量训练的性能。
Offline Behavioral Data Selection
- 按阶段逐步筛选早期数据，结合动作价值与状态密度双重排序
- 仅用10%数据即达到95%以上全量训练效果
- 适合大规模离线策略学习中的高效数据筛选场景
行为克隆是利用专家示范进行离线策略学习的常用方法。然而，大规模离线行为数据集在下游任务中导致计算开销巨大。本文发现离线行为数据存在显著的数据饱和现象：在小部分数据上训练即可迅速达到性能饱和。我们归因于策略性能与测试损失之间弱相关性，揭示了通过数据筛选大幅提升效率的空间。为此，提出一种简单而有效的方法——分步双重排序（SDR），从大规模离线行为数据集中提取紧凑且信息丰富的子集。SDR基于两个核心原则：(1) 分步裁剪，优先选择早期阶段数据；(2) 双重排序，同时考虑高动作价值排名和低状态密度排名。在D4RL基准上的大量实验与消融研究证明，SDR显著提升了离线行为数据的选择效果。
原文摘要 · Abstract (English)
Behavioral cloning is a widely adopted approach for offline policy learning from expert demonstrations. However, the large scale of offline behavioral datasets often results in computationally intensive training when used in downstream tasks. In this paper, we uncover the striking data saturation in offline behavioral data: policy performance rapidly saturates when trained on a small fraction of the dataset. We attribute this effect to the weak alignment between policy performance and test loss, revealing substantial room for improvement through data selection. To this end, we propose a simple yet effective method, Stepwise Dual Ranking (SDR), which extracts a compact yet informative subset from large-scale offline behavioral datasets. SDR is build on two key principles: (1) stepwise clip, which prioritizes early-stage data; and (2) dual ranking, which selects samples with both high action-value rank and low state-density rank. Extensive experiments and ablation studies on D4RL benchmarks demonstrate that SDR significantly enhances data selection for offline behavioral data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。