arXiv:2606.10229cs.ROcs.LG2026-06被引 1

发现演示数据筛选指标与最终策略性能无关,关键要选对指标。

What Demonstration Curation Metrics Do to Your Policy

  • 用真实缺陷数据测试筛选指标,发现高检测率未必带来好策略。
  • 最佳筛选方法使策略成功率90.0%,接近理想水平93.3%。
  • 需控制视频长度干扰,否则指标准确率会被严重夸大。

我们研究了用于检测缺陷训练片段的演示筛选指标,是否能提升下游行为克隆策略的性能。在具有可控结构缺陷(携带阶段提前释放夹爪)的接触密集型LIBERO抓取放置基准上,发现这两者显著解耦:检测能力最高(AUROC 0.804)的指标反而生成最差策略(任务成功率为13.3%),而一个检测能力较低(AUROC 0.638)的指标却使策略达到90.0%成功率,接近基于真实清洁数据训练的基准(93.3%)。进一步分析表明,七种评估指标中有五种依赖于片段长度作为缺陷标签的简单代理,这一混杂因素导致报告的AUROC接近完美值,一旦控制长度,其性能即大幅下降。所有条件下,污染基线仅3.3%成功,而表现最好的两种筛选方法将结果提升至接近93.3%的理论上限。研究强调,筛选方法应以生成策略的性能为评价标准,且任何筛选基准必须控制片段长度后才能报告检测精度。我们公开了测试平台、所有指标实现和评估流程。

原文摘要 · Abstract (English)

We study whether demonstration-curation metrics that detect defective training episodes also improve the downstream behavior-cloning policy that trains on the curated data. On a contact-rich LIBERO pick-and-place benchmark with a controlled structural defect (early gripper release during the carry phase), we find that the two quantities are sharply decoupled. The metric with the highest defect-detection AUROC (0.804) produces the worst curated policy (13.3% task success), while a metric with a substantially lower AUROC (0.638) produces a policy that nearly matches the oracle trained on ground-truth clean data (90.0% vs. 93.3%). We further show that five of the seven metrics we evaluate exploit episode length as a trivial proxy for the defect label, a confound that inflates reported AUROCs to near-perfect values and disappears once episode length is controlled. Across all conditions, the contaminated baseline succeeds on only 3.3% of rollouts, and the two best curation methods close this to within 3 percentage points of the 93.3% oracle ceiling. Our results argue that curation methods should be evaluated by the policy they produce, not the defects they flag, and that any curation benchmark must control for episode length before reporting detection accuracy. We release the testbed, all metric implementations, and the evaluation pipeline.

强化学习数据筛选行为克隆基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。