轻量级Transformer在设备端故障检测中表现不俗，但需权衡模型大小与推理速度。
Lightweight Transformer Models for On-Device Fault Detection: A Benchmark Study on Resource-Constrained Deployment
- 对比传统机器学习与轻量Transformer，评估其在设备端部署效果
- TinyBERT-4L模型仅55MB、18ms延迟，适合资源受限场景
- 自适应推理框架可将97.9%预测快速处理，兼顾准确率与效率
设备端故障检测实现无云实时诊断，但在资源受限硬件上部署机器学习模型需在精度、延迟和模型大小间权衡。本文对比传统机器学习方法（随机森林、XGBoost、SVM、逻辑回归）与轻量级Transformer架构（DistilBERT、TinyBERT-6L、TinyBERT-4L、MobileBERT）在三个公开数据集（NASA C-MAPSS涡轮机退化、SECOM半导体制造、UCI AI4I 2020预测性维护）上的二分类故障检测性能。评估指标包括分类性能（F1-score、AUC）、模型大小与CPU推理延迟，并进一步分析INT8动态量化及两级自适应推理流程。结果显示，在数据区分度高的情况下（如C-MAPSS），轻量Transformer达到87.8% F1，但模型大小为传统方法的100倍，延迟高达9000倍；其中TinyBERT-4L最具部署优势，模型大小55MB，CPU延迟18ms。INT8量化使模型缩小25%，同时保持86.9% F1。自适应推理流程将97.9%预测通过量化预筛模型处理，仅2.1%交由大型专家模型，平均延迟降至19.5ms，F1达87.6%。在严重不平衡数据集（SECOM、UCI-PM）上，两类方法均表现不佳，凸显当前方法在极端类别不平衡下的根本局限。所有代码已开源。
原文摘要 · Abstract (English)
On-device fault detection enables real-time diagnostics without cloud dependency, but deploying machine learning models on resource-constrained hardware demands careful tradeoffs between accuracy, latency, and model size. We present a benchmark comparing traditional ML methods (Random Forest, XGBoost, SVM, Logistic Regression) against lightweight transformer architectures (DistilBERT, TinyBERT-6L, TinyBERT-4L, MobileBERT) for binary fault detection across three public datasets: NASA C-MAPSS turbofan degradation, SECOM semiconductor manufacturing, and UCI AI4I 2020 predictive maintenance. We evaluate classification performance (F1-score, AUC), model size, and CPU inference latency, and further assess INT8 dynamic quantization and a two-stage adaptive inference pipeline. Our results reveal that on well-separated sensor data (C-MAPSS), lightweight transformers match traditional ML at 87.8% F1 but at 100x the model size and 9000x the latency. TinyBERT-4L emerges as the most deployment-friendly transformer at 55 MB and 18 ms CPU latency. INT8 quantization reduces size by 25% while preserving 86.9% F1. Our adaptive pipeline, routing 97.9% of predictions through a quantized triage model and only 2.1% to a larger expert, achieves 87.6% F1 at 19.5 ms average latency. On severely imbalanced datasets (SECOM, UCI-PM), both traditional and transformer methods struggle significantly, highlighting fundamental limitations of current approaches for extreme class imbalance in fault detection. All code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。