用分层推理减少表格模型流式处理的通信开销。
Streaming Hierarchical Inference with Tabular Foundation Models
- 边缘端用图检索近邻，本地处理确定性预测。
- 不确定样本才传到云端，降低90%以上通信量。
- 可调阈值与策略，适合实时数据流场景。
表格基础模型（TFMs）通过上下文学习展现出强大预测能力，但在高吞吐数据流部署中面临通信开销和延迟挑战。我们提出HINT框架，结合边缘检索与云端TFM推理。基于图的近似最近邻记忆在滑动窗口上维护，提供本地预测与置信度估计，使高置信度样本本地处理，低置信度实例连同检索上下文选择性上传至云端TFM。该框架支持可调的卸载阈值与邻居检索策略，用于平衡预测性能与通信成本。实验表明，HINT能持续找到理想的权衡点。
原文摘要 · Abstract (English)
Tabular Foundation Models (TFMs) have recently demonstrated strong predictive performance through in-context learning, but their deployment in high-throughput data streams remains challenging due to communication overhead and latency. We propose \textit{HINT}, a hierarchical inference framework that combines edge-based retrieval with cloud-based TFM inference. A graph-based approximate nearest neighbor memory maintained over a sliding window provides local predictions and uncertainty estimates, allowing confident samples to be processed locally while uncertain instances are selectively offloaded, together with their retrieved context, to a cloud-hosted TFM. The framework exposes an offloading threshold and a neighborhood retrieval policy that can be varied to balance predictive performance and communication cost. Experiments show \textit{HINT} consistently identifies favorable trade-offs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。