arXiv:2507.00082cs.LGcs.AI2025-07被引 7

用联邦学习优化边缘语言模型的通信策略,大幅减少云端调用次数。

Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission

  • 通过联邦学习动态调整低置信度触发云端模型的阈值。
  • 在新闻分类任务中将云端请求减少95%以上,精度几乎不变。
  • 支持设备间共享相似词结果,适合资源受限的边缘计算场景。

混合语言模型(HLM)结合边缘端小型语言模型(SLM)的低延迟与中心服务器大型语言模型(LLM)的高精度优势。不同于传统端到端推理,HLM仅在本地SLM预测不确定性高时才调用LLM,从而降低延迟和通信开销。然而,在带宽受限环境下,仍存在频繁调用LLM导致的通信负担。为此,本文提出FedHLM框架,将不确定性感知推理与联邦学习(FL)融合。其核心创新在于通过联邦学习协同优化决定是否调用LLM的词元级不确定性阈值,避免使用静态或人工设定的阈值。同时,利用基于嵌入的词元表示实现设备间点对点(P2P)复用,使客户端可直接采用语义相似同伴的预测结果,无需调用LLM。此外,引入分层模型聚合机制:边缘服务器通过客户端更新优化本地路由策略,跨集群协调统一全局决策边界,有效捕捉重复出现的不确定性模式,减少冗余查询。大规模新闻分类实验表明,FedHLM在保持精度几乎不变的前提下,将LLM传输次数降低超过95%,适用于可扩展、高效的边缘人工智能应用。

原文摘要 · Abstract (English)

Hybrid Language Models (HLMs) combine the low-latency efficiency of Small Language Models (SLMs) on edge devices with the high accuracy of Large Language Models (LLMs) on centralized servers. Unlike traditional end-to-end LLM inference, HLMs reduce latency and communication by invoking LLMs only when local SLM predictions are uncertain, i.e., when token-level confidence is low or entropy is high. However, ambiguous or low-confidence predictions still require frequent offloading to the LLM, leading to significant communication overhead in bandwidth-constrained settings. To address this, we propose FedHLM, a communication-efficient HLM framework that integrates uncertainty-aware inference with Federated Learning (FL). FedHLM's key innovation lies in collaboratively learning token-level uncertainty thresholds that govern when LLM assistance is needed. Rather than using static or manually tuned thresholds, FedHLM employs FL to optimize these thresholds in a privacy-preserving, distributed manner. Additionally, it leverages embedding-based token representations for Peer-to-Peer (P2P) resolution, enabling clients to reuse tokens inferred by semantically similar peers without engaging the LLM. We further introduce hierarchical model aggregation: edge servers refine local routing policies through client updates, while cross-cluster coordination aligns global decision boundaries. This layered design captures recurring uncertainty patterns, reducing redundant LLM queries. Experiments on large-scale news classification tasks show that FedHLM reduces LLM transmissions by over 95 percent with negligible accuracy loss, making it well-suited for scalable and efficient edge-AI applications.

联邦学习边缘计算语言模型通信效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。