压缩中间特征，让边缘设备高效微调大模型。
TSFLora: Token-Compressed Split Fine-Tuning for Wireless Edge Networks
- 只传关键特征，用注意力选点+合并+量化压缩数据
- 通信量减少6.8倍，内存节省41%，准确率不降
- 适合资源受限的无线边缘设备部署大模型
将大模型适配到个性化边缘数据面临挑战，因无线设备内存、算力和上行带宽有限。联邦微调虽保护数据隐私，但需设备加载完整模型；分片学习可降低设备内存占用，却带来大量激活数据传输。本文提出TSFLora，一种面向无线边缘网络的令牌压缩分片微调框架，通过注意力引导的令牌选择、令牌合并、低比特激活量化与基于LoRA的适配，在分片联邦训练中实现高效通信。核心思想是在传输前压缩中间令牌序列，从而在不修改冻结主干的前提下，同时减少上行流量与服务器端计算开销。在ViT模型上对CIFAR-10、CIFAR-100和TinyImageNet的实验表明，TSFLora实现了最高6.8倍的通信量缩减和41%的内存节省，且保持了竞争力的准确率。
原文摘要 · Abstract (English)
Adapting large AI models (LAMs) to personalized edge data is challenging because wireless devices have limited memory, computation, and uplink capacity. Federated fine-tuning preserves data privacy but still requires each device to host the full model, while split learning reduces device memory at the cost of heavy activation transmission. This paper proposes TSFLora, a token-compressed split fine-tuning framework for communication-efficient LAM adaptation at the edge. TSFLora combines attention-guided token selection, token merging, low-bit activation quantization, and LoRA-based adaptation within a split federated training pipeline. The key idea is to compress the intermediate token sequence before transmission so that the system reduces both uplink traffic and server-side processing without changing the frozen backbone. Experiments on ViT models over CIFAR-10, CIFAR-100, and TinyImageNet show that TSFLora achieves up to \textbf{6.8$\times$} communication reduction and \textbf{41\%} memory saving while maintaining competitive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。