对比生成与判别LSTM模型在量化后的鲁棒性,发现校准数据不平衡会显著影响生成模型性能。
Post-Training Quantization of Generative and Discriminative LSTM Text Classifiers: A Study of Calibration, Class Balance, and Robustness
- 比较生成与判别LSTM在不同位宽下的量化效果,使用Brevitas库进行后训练量化
- 发现生成模型对量化位数、校准数据和输入噪声更敏感,低比特下性能下降明显
- 揭示校准数据类别不平衡导致权重调整不足,适合边缘部署的模型选型参考
文本分类在工业监控、健康诊断和智能助手等边缘计算场景中至关重要,需兼顾低延迟与高精度。生成式分类器对分布外和噪声数据具有更强鲁棒性,但部署于边缘设备时受限于计算与内存。后训练量化(PTQ)可在不重新训练的情况下降低模型大小与计算成本,适合边缘部署。本文使用Brevitas库,对基于LSTM的生成与判别型文本分类器进行全面比较研究,评估其在多种位宽下的表现,并在正常与含噪输入条件下测试鲁棒性。结果表明,判别模型保持稳健,而生成模型对位宽、校准数据及推理噪声更敏感。研究分析了校准数据中的类别不平衡问题,比较均匀与非均匀样本分布对权重调整和激活分布的影响。通过非参数假设检验统计量发现,在低比特下,使用类别不平衡校准数据会导致生成式LSTM权重适应不足,进而性能下降。该研究强调了校准数据在PTQ中的关键作用,为生成模型在噪声环境下的边缘部署提供依据。
原文摘要 · Abstract (English)
Text classification plays a pivotal role in edge computing applications like industrial monitoring, health diagnostics, and smart assistants, where low latency and high accuracy are both key requirements. Generative classifiers, in particular, have been shown to exhibit robustness to out-of-distribution and noisy data, which is an extremely critical consideration for deployment in such real-time edge environments. However, deploying such models on edge devices faces computational and memory constraints. Post Training Quantization (PTQ) reduces model size and compute costs without retraining, making it ideal for edge deployment. In this work, we present a comprehensive comparative study of generative and discriminative Long Short Term Memory (LSTM)-based text classification models with PTQ using the Brevitas quantization library. We evaluate both types of classifier models across multiple bitwidths and assess their robustness under regular and noisy input conditions. We find that while discriminative classifiers remain robust, generative ones are more sensitive to bitwidth, calibration data used during PTQ, and input noise during quantized inference. We study the influence of class imbalance in calibration data for both types of classifiers, comparing scenarios with evenly and unevenly distributed class samples including their effect on weight adjustments and activation profiles during PTQ. Using test statistics derived from nonparametric hypothesis testing, we identify that using class imbalanced data during calibration introduces insufficient weight adaptation at lower bitwidths for generative LSTM classifiers, thereby leading to degraded performance. This study underscores the role of calibration data in PTQ and when generative classifiers succeed or fail under noise, aiding deployment in edge environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。