用轻量级张量压缩实时检测动态图中异常边,准确率高且速度快。
Adaptive-GraphSketch: Real-Time Edge Anomaly Detection via Multi-Layer Tensor Sketching and Temporal Decay
- 通过多层张量压缩与保守更新计数,高效追踪边频次模式。
- 在真实数据集上最高提升15.6%的异常检测准确率,2000万边处理<3.4秒。
- 适合需要快速响应的大规模在线图数据监控场景。
动态图中的异常检测对识别网络安全、电力系统等实际系统中的恶意行为和异常活动至关重要。然而,现有方法在可扩展性、概率可解释性以及对变化流量模式的适应性方面存在不足。本文提出 ADAPTIVE-GRAPHSKETCH,一种轻量且可扩展的实时异常检测框架,用于流式边数据。该方法结合时间多张量压缩与保守更新计数最小映射(CMS-CU),在有限内存下紧凑追踪边频率模式,同时缓解哈希冲突问题;引入贝叶斯推断进行概率异常评分,并采用指数加权移动平均(EWMA)实现随突发强度自适应调整的阈值。在四个真实入侵检测数据集上的实验表明,该方法优于 ANOEDGE-G/L、MIDAS-R、F-FADE 等前沿基线,在 CIC-IDS2018 上最高提升6.5% AUC,CIC-DDoS2019 上达15.6%,仅使用10个哈希函数即可在3.4秒内处理2000万条边,证明其在大规模流式图上的实用性与高效性。
原文摘要 · Abstract (English)
Anomaly detection in dynamic graphs is essential for identifying malicious activities, fraud, and unexpected behaviors in real-world systems such as cybersecurity and power grids. However, existing approaches struggle with scalability, probabilistic interpretability, and adaptability to evolving traffic patterns. In this paper, we propose ADAPTIVE-GRAPHSKETCH, a lightweight and scalable framework for real-time anomaly detection in streaming edge data. Our method integrates temporal multi-tensor sketching with Count-Min Sketch using Conservative Update (CMS-CU) to compactly track edge frequency patterns with bounded memory, while mitigating hash collision issues. We incorporate Bayesian inference for probabilistic anomaly scoring and apply Exponentially Weighted Moving Average (EWMA) for adaptive thresholding tuned to burst intensity. Extensive experiments on four real-world intrusion detection datasets demonstrate that ADAPTIVE-GRAPHSKETCH outperforms state-of-the-art baselines such as ANOEDGE-G/L, MIDAS-R, and F-FADE, achieving up to 6.5% AUC gain on CIC-IDS2018 and up to 15.6% on CIC-DDoS2019, while processing 20 million edges in under 3.4 seconds using only 10 hash functions. Our results show that ADAPTIVE-GRAPHSKETCH is practical and effective for fast, accurate anomaly detection in large-scale streaming graphs. Keywords: Anomaly Detection, Streaming, Real-time, Dynamic Graphs, Edge Streams, Tensor Sketching
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。