TL++让分布式系统在不暴露数据的情况下，高效训练模型并接近集中式精度。
TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems
- 通过虚拟批量构建跨节点梯度，恢复集中训练行为。
- 在CIFAR-10上准确率达91.41%，比基线高超12个百分点。
- 支持激活值加密，适合隐私敏感的分布式机器学习场景。
分布式智能系统需在不集中原始数据的前提下跨数据孤岛训练。联邦学习虽保留数据本地性，但在异构分区下表现不佳，且需频繁交换完整模型；分割学习通过切层激活降低通信量，但通常无法还原集中式小批量梯度行为，且可能以明文暴露激活值与梯度。本文提出TL++，一种双模式遍历学习框架，在显式同步假设下构建跨节点虚拟批量，恢复集中式小批量梯度行为。基础模式交换切层激活与梯度而非完整模型；安全模式将每层激活与梯度在协调者与非共谋辅助方间进行秘密共享，防止任一服务器窥探明文张量。该保护限于半诚实两服务器环境，标签与损失相关输出仍对协调者可见。轻量级安全路径中，精确计算要求线性或仿射服务路径，非线性操作需非线性MPC或近似。我们形式化了TL++，分析通信与计算成本，并在CIFAR-10和BioGPT/PubMedQA上对比联邦学习与分割学习基线，使用全微调与LoRA。CIFAR-10上，TL++基础模式切1与精确安全模式切3分别达到91.41%（标准差0.19）与90.93%（标准差0.17），超过最强非TL++基线12个百分点以上。基础模式切1相比完整模型同步，每步通信减少13.1倍。PubMedQA结果亦显示优势。总体而言，TL++逼近集中训练性能，同时降低通信量并提供激活级秘密共享。
原文摘要 · Abstract (English)
Distributed intelligent systems increasingly need to train across data silos without centralizing raw data. Federated learning keeps data local but can suffer under heterogeneous partitions and requires repeated full-model exchange. Split learning reduces communication through cut-layer activations, but standard protocols generally do not recover centralized mini-batch gradient behavior and may expose activations and gradients in plaintext. We present TL++, a two-mode traversal-learning framework that constructs virtual batches across nodes to recover centralized mini-batch gradient behavior under explicit synchronization assumptions. Base mode exchanges cut-layer activations and gradients rather than full models. Secure mode secret-shares each cut-layer activation and gradient between an orchestrator and a non-colluding helper, preventing either server from observing plaintext cut-layer tensors. This protection is limited to a semi-honest two-server setting; labels and loss-related outputs remain visible to the orchestrator. In the lightweight secure path evaluated here, exactness requires a linear or affine server path, while nonlinear operations require nonlinear MPC or approximation. We formalize TL++, analyze communication and computation costs, and evaluate it against federated and split-learning baselines on CIFAR-10 and BioGPT/PubMedQA using full fine-tuning and LoRA. On CIFAR-10, TL++ base cut 1 and exact secure cut 3 achieve accuracies of 91.41% (SD 0.19) and 90.93% (SD 0.17), respectively, exceeding the strongest measured non-TL++ baseline by more than 12 percentage points. TL++ base cut 1 also reduces per-step communication by 13.1-fold relative to full-model synchronization. PubMedQA results similarly favor TL++. Overall, TL++ approaches centralized-training performance while reducing communication and providing activation-level secret sharing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。