用大模型蒸馏出轻量交通流预测模型，省数据少资源
FlowDistill: Scalable Traffic Flow Prediction via Distillation from LLMs
- 用LLM做教师，蒸馏知识给小型MLP学生模型
- 仅需少量数据即可超越现有模型，内存和延迟更低
- 适合数据少、算力弱的现实城市部署场景
精准的交通流预测对优化城市出行至关重要，但受复杂时空依赖性和高质量数据稀缺影响，在许多城市仍具挑战。尽管基于图的深度模型预测能力强，却常伴随高计算开销和大量训练数据需求，难以在资源受限或数据稀疏环境下部署。本文提出FlowDistill，一种基于大语言模型（LLMs）知识蒸馏的轻量级、可扩展交通预测框架。在师生架构中，微调后的LLM通过信息瓶颈原理与教师约束回归损失的结合，指导一个紧凑的多层感知机（MLP）学生模型，确保其仅保留关键且可迁移的知识。显式编码空间与时间相关性以提升模型在不同城市环境中的泛化能力。尽管结构简单，FlowDistill在预测精度上持续优于当前最优模型，同时显著减少训练数据需求，并实现更低内存占用与推理延迟，凸显其高效性与实际可部署性。
原文摘要 · Abstract (English)
Accurate traffic flow prediction is vital for optimizing urban mobility, yet it remains difficult in many cities due to complex spatio-temporal dependencies and limited high-quality data. While deep graph-based models demonstrate strong predictive power, their performance often comes at the cost of high computational overhead and substantial training data requirements, making them impractical for deployment in resource-constrained or data-scarce environments. We propose the FlowDistill, a lightweight and scalable traffic prediction framework based on knowledge distillation from large language models (LLMs). In this teacher-student setup, a fine-tuned LLM guides a compact multi-layer perceptron (MLP) student model using a novel combination of the information bottleneck principle and teacher-bounded regression loss, ensuring the distilled model retains only essential and transferable knowledge. Spatial and temporal correlations are explicitly encoded to enhance the model's generalization across diverse urban settings. Despite its simplicity, FlowDistill consistently outperforms state-of-the-art models in prediction accuracy while requiring significantly less training data, and achieving lower memory usage and inference latency, highlighting its efficiency and suitability for real-world, scalable deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。