将数据采集从离线转为车端实时决策,提升数据质量与效率。
From Big Data to Fast Data: Towards High-Quality Datasets for Machine Learning Applications from Closed-Loop Data Collection
- 在车载端实时判断是否记录数据,实现上下文感知的数据选择。
- 生成的数据集覆盖关键场景更全,信息密度更高,冗余数据减少。
- 适合自动驾驶等依赖高质量数据的现代机器学习系统开发。
随着视觉-语言及多模态语言模型能力提升,智能汽车系统工程对数据质量与相关性的要求日益提高。传统大数据方法侧重大规模数据采集与离线处理,智能数据方法虽改进了数据选择策略,但仍依赖集中式、离线后处理。本文提出面向汽车工程的快速数据(Fast Data)概念,将数据筛选与记录直接部署于车辆作为数据源。通过在车端实现基于上下文的实时决策,判断是否及记录何种数据,使数据采集与数据质量目标形成闭环对齐。该方法显著提升数据的相关性与关键场景覆盖率,增强信息密度,同时减少无关数据及其成本。该方案为现代机器学习算法驱动的数据采集策略提供了结构化基础,支持高效数据获取,助力汽车系统工程中可扩展、低成本的机器学习开发流程。
原文摘要 · Abstract (English)
The increasing capabilities of machine learning models, such as vision-language and multimodal language models, are placing growing demands on data in automotive systems engineering, making the quality and relevance of collected data enablers for the development and validation of such systems. Traditional Big Data approaches focus on large-scale data collection and offline processing, while Smart Data approaches improve data selection strategies but still rely on centralized and offline post-processing. This paper introduces the concept of Fast Data for automotive systems engineering. The approach shifts data selection and recording onto the vehicle as the data source. By enabling real-time, context-aware decisions on whether and which data should be recorded, data collection can be directly aligned with data quality objectives and collection strategies within a closed-loop. This results in datasets with higher relevance, improved coverage of critical scenarios, and increased information density, while at the same time reducing irrelevant data and associated costs. The proposed approach provides a structured foundation for designing data collection strategies that are aligned with the needs of modern machine learning algorithms. It supports efficient data acquisition and contributes to scalable and cost-effective ML development processes in automotive systems engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。