从预训练网络中提取线性模型，提升分类精度与稳定性
Extraction of linearized models from pre-trained networks via knowledge distillation
- 结合科波曼算子与知识蒸馏，将非线性网络转为线性模型
- 在MNIST和Fashion-MNIST上准确率优于传统最小二乘法
- 适合硬件部署需求，尤其适用于光子芯片等线性计算场景
近年来，光子集成电路与光学器件的发展推动了面向线性运算的机器学习架构研究。因此，探索在简单非线性预处理后仅依赖线性运算的机器学习方法具有重要意义。本文提出一种框架，通过融合科波曼算子理论与知识蒸馏，从预训练神经网络中提取用于分类任务的线性化模型。在MNIST与Fashion-MNIST数据集上的数值实验表明，所提模型在分类准确率和数值稳定性方面均持续优于传统的基于最小二乘法的科波曼近似。
原文摘要 · Abstract (English)
Recent developments in hardware, such as photonic integrated circuits and optical devices, are driving demand for research on constructing machine learning architectures tailored for linear operations. Hence, it is valuable to explore methods for constructing learning machines with only linear operations after simple nonlinear preprocessing. In this study, we propose a framework to extract a linearized model from a pre-trained neural network for classification tasks by integrating Koopman operator theory with knowledge distillation. Numerical demonstrations on the MNIST and the Fashion-MNIST datasets reveal that the proposed model consistently outperforms the conventional least-squares-based Koopman approximation in both classification accuracy and numerical stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。