让随机森林和神经网络互相学习，提升大数据场景下的模型效率与可解释性。
Cross-Paradigm Knowledge Distillation: A Comprehensive Study of Bidirectional Transfer Between Random Forests and Deep Neural Networks for Big Data Applications
- 双向知识蒸馏：随机森林与神经网络互为教师学生，实现跨范式知识迁移。
- 多教师集成蒸馏效果最优，分类准确率达98.13%，回归任务R²达92.6%。
- 适合需要可解释性与高效部署的工业级大模型应用，尤其资源受限环境。
大数据的指数级增长加剧了对高效且可解释机器学习模型的需求，要求模型能处理多样化数据特征并保持计算效率。知识蒸馏传统上集中于神经网络之间的迁移，跨范式知识转移仍处于空白。本文首次系统研究随机森林（RF）与深度神经网络（DNN）之间的双向知识蒸馏，填补集成学习与模型压缩在大数据应用中的关键空白。提出新方法包括渐进式多阶段蒸馏、来自多样树模型的多教师集成蒸馏，以及不确定性感知的跨范式迁移机制。在6个不同数据集上进行144组实验，涵盖分类与回归任务，结果表明双向RF-DL蒸馏在性能上具有竞争力，同时兼具树模型的可解释性与神经网络的表达能力。多教师集成蒸馏表现持续优于传统方法，其中NN-COMPACT在分类任务中达到98.13%准确率，NN-WIDE在回归任务中取得92.6%的R²得分。所提框架提升了大数据环境中的部署灵活性，支持根据计算约束与可解释性需求动态选择最优模型。本工作确立了跨范式知识迁移的新方向，对可解释人工智能与资源受限系统中的可扩展模型部署具有重要意义。
原文摘要 · Abstract (English)
The exponential growth of big data has intensified the need for efficient and interpretable machine learning models that can handle diverse data characteristics while maintaining computational efficiency. Knowledge distillation has primarily focused on neural network-to-neural network transfer, leaving cross-paradigm knowledge transfer largely unexplored. This paper presents the first comprehensive study of bidirectional knowledge distillation between Random Forests (RF) and Deep Neural Networks (DNN), addressing critical gaps in ensemble learning and model compression for big data applications. We propose novel methodologies including progressive multi-stage distillation, multi-teacher ensemble distillation from diverse tree models, and uncertainty-aware cross-paradigm transfer mechanisms. Through 144 comprehensive experiments across 6 diverse datasets encompassing classification and regression tasks, we demonstrate that bidirectional RF-DL distillation achieves competitive performance while providing complementary benefits: interpretability from tree models and expressiveness from neural networks. Our results show that multi-teacher ensemble distillation consistently outperforms traditional approaches, with NN-COMPACT achieving 98.13% classification accuracy and NN-WIDE reaching 92.6% R^2 score in regression tasks. The proposed framework enables deployment flexibility in big data environments, allowing optimal model selection based on computational constraints and interpretability requirements. This work establishes a new research direction in cross-paradigm knowledge transfer with significant implications for interpretable AI and scalable model deployment in resource-constrained big data systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。