用知识蒸馏让神经PDE求解器更快更准，实现实时推理。
KD-PINN: Knowledge-Distilled PINNs for ultra-low-latency real-time neural PDE solvers
- 用KL散度连续优化，把大模型知识迁移到小模型。
- 推理速度提升4.8到6.9倍，平均延迟仅5.3毫秒。
- 适合需要实时响应的科学计算场景，如流体仿真。
本文提出知识蒸馏物理信息神经网络（KD-PINN），通过持续优化Kullback-Leibler散度，将高容量教师模型的预测精度迁移至轻量学生模型。为验证框架对多种动力学和维度的通用性，其在代表性偏微分方程（PDEs）上进行了评估。在所有基准测试中，学生模型实现4.8倍（纳维-斯托克斯方程）至6.9倍（伯格斯方程）的推理加速，同时保持高精度；经适当调优后，精度提升约1%。蒸馏过程还展现出正则化效果。在CPU上平均推理延迟仅为5.3毫秒，进入亚10毫秒的超低延迟实时范畴。本研究进一步分析了知识蒸馏如何降低PINNs推理延迟，助力开发高精度、超低延迟的神经PDE求解器。
原文摘要 · Abstract (English)
This work introduces Knowledge-Distilled Physics-Informed Neural Networks (KD-PINN), a framework that transfers the predictive accuracy of a high-capacity teacher model to a compact student through a continuous adaptation of the Kullback-Leibler divergence. In order to confirm its generality for various dynamics and dimensionalities, the framework is evaluated on a representative set of partial differential equations (PDEs). Across the considered benchmarks, the student model achieves inference speedups ranging from x4.8 (Navier-Stokes) to x6.9 (Burgers), while preserving accuracy. Accuracy is improved by on the order of 1% when the model is properly tuned. The distillation process also revealed a regularizing effect. With an average inference latency of 5.3 ms on CPU, the distilled models enter the ultra-low-latency real-time regime defined by sub-10 ms performance. Finally, this study examines how knowledge distillation reduces inference latency in PINNs, to contribute to the development of accurate ultra-low-latency neural PDE solvers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。