将非神经网络的机器学习流水线转化为神经网络，实现统一推理与优化。
Neural Network Conversion of Machine Learning Pipelines
- 用神经网络作为学生模型，模仿随机森林等非神经网络流水线的输出。
- 在100个OpenML任务中，多数情况下学生网络能复现教师模型性能。
- 可用随机森林辅助选择最优神经网络超参数，提升转化效果。
迁移学习与知识蒸馏近年来在深度学习领域备受关注。其中，学生-教师学习方法已被证明可生成性能接近复杂教师网络的“小型”学生神经网络。本文探讨该方法的延伸：将非神经网络的机器学习流水线作为教师，将其知识迁移到神经网络学生模型中，从而实现多任务流水线组件的联合优化及单一统一推理引擎。特别地，我们研究了用神经网络替代随机森林分类器的可行性。在100个曾以随机森林表现优异的OpenML任务上，测试了多种神经网络结构。结果表明，在合理选择超参数的情况下，多数任务中学生网络可成功复现教师模型性能。同时，我们还探索了利用随机森林协助筛选最优神经网络超参数的方法。
原文摘要 · Abstract (English)
Transfer learning and knowledge distillation has recently gained a lot of attention in the deep learning community. One transfer approach, the student-teacher learning, has been shown to successfully create ``small'' student neural networks that mimic the performance of a much bigger and more complex ``teacher'' networks. In this paper, we investigate an extension to this approach and transfer from a non-neural-based machine learning pipeline as teacher to a neural network (NN) student, which would allow for joint optimization of the various pipeline components and a single unified inference engine for multiple ML tasks. In particular, we explore replacing the random forest classifier by transfer learning to a student NN. We experimented with various NN topologies on 100 OpenML tasks in which random forest has been one of the best solutions. Our results show that for the majority of the tasks, the student NN can indeed mimic the teacher if one can select the right NN hyper-parameters. We also investigated the use of random forest for selecting the right NN hyper-parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。