通过分析模型训练过程中的损失轨迹,自动发现视频标注中的错误。
Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- 用每帧在不同训练阶段的平均损失变化作为学习能力指纹。
- 在EgoPER和Cholec80数据集上准确识别出误标与时间顺序错乱的片段。
- 无需真实错误标签,适用于多种视频任务的数据清洗。
高质量视频数据集是动作识别、阶段检测和事件分割等任务训练鲁棒模型的基础。然而,许多真实场景下的视频数据存在标注错误,如误标(段落被赋予错误类别)和错序(时间顺序不正确),尤其在阶段标注任务中,时间一致性至关重要。本文提出一种新型、模型无关的方法,通过分析累积样本损失(Cumulative Sample Loss, CSL)来检测标注错误——即模型在训练各阶段保存的检查点上对每帧计算损失的平均值。每帧的损失轨迹反映了其学习难易程度:误标或错序帧通常表现为持续高损失或异常波动,而正确标注帧则早期快速收敛至低损失。通过训练视频分割模型并保存每个训练周期的权重,再评估测试视频中各帧的损失,将长期高损耗帧标记为潜在错误。该方法不依赖标注错误的真实标签,具备跨数据集泛化能力。在EgoPER和Cholec80上的实验表明,该方法能有效识别细微的标注不一致,如误标和帧序错乱,为视频数据集审计与提升训练可靠性提供有力工具。
原文摘要 · Abstract (English)
High-quality video datasets are foundational for training robust models in tasks like action recognition, phase detection, and event segmentation. However, many real-world video datasets suffer from annotation errors such as *mislabeling*, where segments are assigned incorrect class labels, and *disordering*, where the temporal sequence does not follow the correct progression. These errors are particularly harmful in phase-annotated tasks, where temporal consistency is critical. We propose a novel, model-agnostic method for detecting annotation errors by analyzing the Cumulative Sample Loss (CSL)--defined as the average loss a frame incurs when passing through model checkpoints saved across training epochs. This per-frame loss trajectory acts as a dynamic fingerprint of frame-level learnability. Mislabeled or disordered frames tend to show consistently high or irregular loss patterns, as they remain difficult for the model to learn throughout training, while correctly labeled frames typically converge to low loss early. To compute CSL, we train a video segmentation model and store its weights at each epoch. These checkpoints are then used to evaluate the loss of each frame in a test video. Frames with persistently high CSL are flagged as likely candidates for annotation errors, including mislabeling or temporal misalignment. Our method does not require ground truth on annotation errors and is generalizable across datasets. Experiments on EgoPER and Cholec80 demonstrate strong detection performance, effectively identifying subtle inconsistencies such as mislabeling and frame disordering. The proposed approach provides a powerful tool for dataset auditing and improving training reliability in video-based machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。