用低秩张量分解与深度展开实现高效、可解释的网络流量异常检测。
Adaptive Anomaly Detection in Network Flows with Low-Rank Tensor Decompositions and Deep Unrolling
- 将正常流量建模为低秩张量,异常为稀疏项,结合正则化优化提升鲁棒性。
- 在真实与合成数据上仅用少量训练数据即达高检测率,优于现有方法。
- 支持在线自适应,适合对可解释性与泛化性要求高的工业级网络监控场景。
异常检测(AD)被广泛认为是保障未来通信系统弹性的重要组成部分。尽管深度学习在AD任务中表现出色,但其在关键系统中的应用受限于训练数据效率、领域适应性和可解释性问题。本文针对不完整测量下的网络流量异常检测,提出一种基于鲁棒张量分解与深度展开的技术方案。首先设计一种新型块交替凸逼近算法,基于带正则化的模型拟合目标,将正常流量建模为低秩张量,异常视为稀疏成分,并引入目标函数增强以降低计算开销。进一步通过深度展开构建新型神经网络架构,将正则化参数设为可学习权重。受贝叶斯方法启发,扩展模型以实现对每流、每时间步统计特性的在线适应,在保持低参数量的同时维持问题的置换等变性。为优化网络权重以提升检测性能,采用基于接收机工作特征曲线下面积近似值的同伦优化方法。大量实验表明,所提深度网络架构具有高训练数据效率,显著优于基准方法,并能无缝适应不同网络拓扑。
原文摘要 · Abstract (English)
Anomaly detection (AD) is increasingly recognized as a key component for ensuring the resilience of future communication systems. While deep learning has shown state-of-the-art AD performance, its application in critical systems is hindered by concerns regarding training data efficiency, domain adaptation and interpretability. This work considers AD in network flows using incomplete measurements, leveraging a robust tensor decomposition approach and deep unrolling techniques to address these challenges. We first propose a novel block-successive convex approximation algorithm based on a regularized model-fitting objective where the normal flows are modeled as low-rank tensors and anomalies as sparse. An augmentation of the objective is introduced to decrease the computational cost. We apply deep unrolling to derive a novel deep network architecture based on our proposed algorithm, treating the regularization parameters as learnable weights. Inspired by Bayesian approaches, we extend the model architecture to perform online adaptation to per-flow and per-time-step statistics, improving AD performance while maintaining a low parameter count and preserving the problem's permutation equivariances. To optimize the deep network weights for detection performance, we employ a homotopy optimization approach based on an efficient approximation of the area under the receiver operating characteristic curve. Extensive experiments on synthetic and real-world data demonstrate that our proposed deep network architecture exhibits a high training data efficiency, outperforms reference methods, and adapts seamlessly to varying network topologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。