用深度自编码器检测企业ETL数据流异常，准确率高且鲁棒性强。
Deep Learning Approach to Anomaly Detection in Enterprise ETL Processes with Autoencoders
- 通过自编码器压缩数据并重建，用误差判断异常
- 在多种条件下AUC、召回率等指标均表现优异
- 适合需要实时监控数据质量的企业系统
针对企业级ETL数据流中常见的延迟、缺失值、重复加载和突发异常等问题，提出基于深度自编码器的异常检测方法。首先分析多种异常类型，通过数据标准化与特征建模确保输入稳定性。采用编码器-解码器结构将高维数据压缩为潜在表示并重构，以重构误差衡量异常程度。在潜在空间引入正则化约束，增强特征稀疏性与分布学习能力，提升复杂数据流中的鲁棒性。在不同超参数设置、环境变化和数据特性下的系统性分析表明，该方法在AUC、准确率（ACC）、精确率和召回率上均表现优异。结果表明，基于深度自编码器的检测机制能有效捕捉企业级ETL数据流中的潜在分布模式，精准识别多种异常，为企业的数据处理与智能分析提供可靠支持。
原文摘要 · Abstract (English)
An anomaly detection method based on deep autoencoders is proposed to address anomalies that often occur in enterprise-level ETL data streams. The study first analyzes multiple types of anomalies in ETL processes, including delays, missing values, duplicate loading, and sudden abnormal changes, and applies data standardization and feature modeling to ensure stable and usable inputs. In the method design, the encoder-decoder structure compresses high-dimensional inputs into latent representations and reconstructs them, while reconstruction error is used to measure anomaly levels. Regularization constraints are introduced in the latent space to enhance feature sparsity and distribution learning, thereby improving robustness in complex data streams. Systematic analyses under different hyperparameter settings, environmental changes, and data characteristics show that the proposed method achieves superior performance in AUC, ACC, Precision, and Recall. The results demonstrate that the deep autoencoder-based detection mechanism can effectively capture latent distribution patterns in enterprise-level ETL data streams and accurately identify diverse anomalies, providing reliable support for enterprise data processing and intelligent analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。