通过损失曲面分析，提升量化模型在科学传感中的可靠性。
Loss Landscape Analysis for Reliable Quantized ML Models for Scientific Sensing
- 用损失曲面分析量化模型的鲁棒性，无需反复训练。
- 发现平缓曲面与抗输入/权重扰动能力强相关。
- 适合需要高可靠性的科学传感系统设计者。
本文提出一种对机器学习（ML）模型损失曲面进行经验分析的方法，应用于两个需量化部署的科学传感模型。这些模型受实验条件噪声和扰动影响，而现有研究尚未充分探讨量化精度与正则化技术对鲁棒性的影响。本方法通过损失曲面分析，揭示了性能、效率与鲁棒性之间的权衡关系，发现平缓的损失曲面与抗输入及权重扰动能力显著相关，并观察到若干非显而易见的现象。该方法可事先系统探索此类权衡，无需训练多个模型，从而提升开发效率。研究还强调将鲁棒性纳入多目标优化的重要性，助力构建更可靠、自适应的科学传感系统。
原文摘要 · Abstract (English)
In this paper, we propose a method to perform empirical analysis of the loss landscape of machine learning (ML) models. The method is applied to two ML models for scientific sensing, which necessitates quantization to be deployed and are subject to noise and perturbations due to experimental conditions. Our method allows assessing the robustness of ML models to such effects as a function of quantization precision and under different regularization techniques -- two crucial concerns that remained underexplored so far. By investigating the interplay between performance, efficiency, and robustness by means of loss landscape analysis, we both established a strong correlation between gently-shaped landscapes and robustness to input and weight perturbations and observed other intriguing and non-obvious phenomena. Our method allows a systematic exploration of such trade-offs a priori, i.e., without training and testing multiple models, leading to more efficient development workflows. This work also highlights the importance of incorporating robustness into the Pareto optimization of ML models, enabling more reliable and adaptive scientific sensing systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。