用测试数据反向筛选训练集,让模型更贴合真实部署环境。
Test-Time Coverage: Test-Conditioned Data Curation for Deployment-Aware Learning

- 基于测试数据构建任务知识图谱,指导训练数据选择
- 在自动驾驶场景中提升部署相关覆盖度与性能表现
- 无需推理时调整,适合跨城市等新场景快速适配
实际部署的AI系统通常从广泛的数据池中训练,需针对部署测试分布进行数据筛选。但传统方法仅评估训练侧指标,未直接优化部署匹配。本文提出TTCov(Test-Time Coverage),一种基于测试信息的训练数据筛选方法,在训练前利用测试端信息而非推理时更新权重。TTCov将部署条件下的数据筛选分解为覆盖度与分布两个部分:通过大语言模型生成描述部署关键概念的原子命题(APs)构建任务图谱(Atlas),并从无标签部署样本中提取未匹配的APs进行扩展;再以匹配的APs及其频率构建知识图谱(K-Atlas),作为数据筛选的目标分布。最终选取一个预算内的训练集,使其部署相关AP分布逼近该目标。在自动驾驶任务中,TTCov保持推理路径不变,所选数据具备更高的部署相关覆盖度、更接近目标分布,并显著优于现有数据筛选基线,且支持跨城市场景无缝扩展。
原文摘要 · Abstract (English)
Deployed AI systems are often trained from broad candidate data pools, necessitating data curation towards the deployment test distribution. However, standard data curation methods score training-side criteria rather than directly optimizing deployment match. We introduce TTCov (Test-Time Coverage), a data-level test-conditioned curation method that uses test-side information before training instead of updating model weights at inference. TTCov decomposes deployment-conditioned curation into coverage and distribution. To represent coverage, it builds a task Atlas, a collection of LLM-based atomic propositions (APs) describing deployment-relevant concepts, seeded from open task knowledge and expanded with unmatched APs extracted from unlabeled deployment samples. To represent distribution, it instantiates the matched deployment APs with their frequencies, yielding a Knowledge Atlas (K-Atlas) that operationalizes the deployment distribution as a curation target. TTCov then selects a budgeted training set whose deployment APs distribution approximates this target. We apply TTCov towards autonomous driving (AD), keeping adaptation off the inference path while selecting data with greater deployment-relevant coverage, closer K-Atlas matching, and stronger downstream end-to-end driving performance than data-curation baselines, including seamless adaptability to novel domains via city-to-city expansion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。