arXiv:2412.18296cs.LGcs.AI2024-12被引 6

研究数据污染对模型影响,发现30%数据决定性能,噪声比缺失更致命。

Navigating Data Corruption in Machine Learning: Balancing Quality, Quantity, and Imputation Strategies

  • 通过实验分析数据污染与模型表现的衰减关系,提出指数型退化模型。
  • 噪声数据比缺失数据危害大,且在强化学习中引发严重训练不稳定。
  • 识别出插补策略的优劣势区域,建议按任务敏感性选择数据处理方式。

数据污染(包括缺失与噪声数据)在真实世界机器学习中构成重大挑战。本研究通过两个实验设置——自然语言处理的监督学习(NLP-SL)和交通信号优化的深度强化学习(Signal-RL),探究数据污染对模型性能的影响,并评估数据插补与数据量扩充策略的有效性。结果表明,模型性能随数据污染程度呈指数衰减趋势。相比缺失数据,噪声数据危害更大,尤其在序列决策任务如Signal-RL中导致严重性能下降与训练不稳。插补策略存在权衡:虽能恢复缺失信息,但可能引入新噪声;其有效性取决于插补准确率与污染比例。研究识别出插补优势热图中的‘有利角落’与‘不利边缘’,并依据任务决策边界将任务分类为‘噪声敏感’或‘噪声不敏感’。此外,扩大数据集可缓解污染影响,但边际收益随污染加剧而递减。经验法则显示:约30%的数据决定性能,其余70%影响甚微。这些发现为数据预处理、插补策略及数据采集提供可操作指导。

原文摘要 · Abstract (English)

Data corruption, including missing and noisy data, poses significant challenges in real-world machine learning. This study investigates the effects of data corruption on model performance and explores strategies to mitigate these effects through two experimental setups: supervised learning with NLP tasks (NLP-SL) and deep reinforcement learning for traffic signal optimization (Signal-RL). We analyze the relationship between data corruption levels and model performance, evaluate the effectiveness of data imputation methods, and assess the utility of enlarging datasets to address data corruption. Our results show that model performance under data corruption follows a diminishing return curve, modeled by the exponential function. Missing data, while detrimental, is less harmful than noisy data, which causes severe performance degradation and training instability, particularly in sequential decision-making tasks like Signal-RL. Imputation strategies involve a trade-off: they recover missing information but may introduce noise. Their effectiveness depends on imputation accuracy and corruption ratio. We identify distinct regions in the imputation advantage heatmap, including an "imputation advantageous corner" and an "imputation disadvantageous edge" and classify tasks as "noise-sensitive" or "noise-insensitive" based on their decision boundaries. Furthermore, we find that increasing dataset size mitigates but cannot fully overcome the effects of data corruption. The marginal utility of additional data diminishes as corruption increases. An empirical rule emerges: approximately 30% of the data is critical for determining performance, while the remaining 70% has minimal impact. These findings provide actionable insights into data preprocessing, imputation strategies, and data collection practices, guiding the development of robust machine learning systems in noisy environments.

数据污染模型鲁棒性插补策略强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。