简单模型在流量分类中表现惊人，因数据冗余导致性能虚高。
When Simple Model Just Works: Is Network Traffic Classification in Crisis?
- 用包序列元数据的k-NN基准表现优于复杂模型
- 12个数据集超50%样本重复，训练测试集重叠严重
- 提醒研究者重新设计任务与评估方式
机器学习用于网络流量分类已有二十多年。早期多采用浅层模型，2010年代后转向复杂神经网络，常报告近乎完美的准确率。然而近期发现，仅使用包序列元数据（大小、时间、方向）的简单k-NN基线，可达到甚至超过复杂方法的表现。本文进一步在12个数据集和15项分类任务上评估该基线，并探究其优异表现的原因。分析显示，多数数据集包含超过50%的重复样本（相同包序列），由于常见划分方式，这些重复流同时出现在训练和测试集中。这种冗余导致模型性能被高估，且当相同流具有不同标签时，理论最大准确率下降。鉴于其独特性，我们指出传统机器学习范式（如源自NLP或计算机视觉）可能不适用于流量分类。最后，我们提出新的任务设定与评估方向，以应对挑战并推动领域回归合理研究路径。
原文摘要 · Abstract (English)
Machine learning has been applied to network traffic classification (TC) for over two decades. While early efforts used shallow models, the latter 2010s saw a shift toward complex neural networks, often reporting near-perfect accuracy. However, it was recently revealed that a simple k-NN baseline using packet sequences metadata (sizes, times, and directions) can be on par or even outperform more complex methods. In this paper, we investigate this phenomenon further and evaluate this baseline across 12 datasets and 15 TC tasks, and investigate why it performs so well. Our analysis shows that most datasets contain over 50% redundant samples (identical packet sequences), which frequently appear in both training and test sets due to common splitting practices. This redundancy can lead to overestimated model performance and reduce the theoretical maximum accuracy when identical flows have conflicting labels. Given its distinct characteristics, we further argue that standard machine learning practices adapted from domains like NLP or computer vision may be ill-suited for TC. Finally, we propose new directions for task formulation and evaluation to address these challenges and help realign the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。