让分布式AI通信保持语义对齐，不丢性能还省内存
Zero-Forget Preservation of Semantic Communication Alignment in Distributed AI Networks
- 用稀疏参数修改替代全量更新，避免破坏语义对齐
- 实验显示性能几乎无损，部分场景更优，额外内存<1%
- 适合需要长期维护通信对齐的分布式AI系统
未来通信网络将连接海量分布式人工智能。利用AI对之间的对齐先验知识，可将高维数据传输转化为高度压缩的语义通信（SC）。然而，为适应本地数据分布和用户偏好，AI通常会进行领域自适应，从根本上破坏了语义通信对齐。本文提出零遗忘领域自适应（ZFDA）框架以维持语义通信对齐。为防止自适应改变模型核心参数，设计了稀疏加性修改（SAM），可高效存储并随时关闭以恢复对齐。通过将SAM解耦为连续变量与二值掩码，并采用基于得分的优化方法处理掩码，实现高效优化。在图像传输的语义通信系统上的实验表明，该框架在几乎不损失自适应性能的前提下完美保留了语义通信对齐，甚至在某些情况下表现更优，额外内存开销低于1%。
原文摘要 · Abstract (English)
Future communication networks are expected to connect massive distributed artificial intelligence (AI). Exploiting aligned priori knowledge of AI pairs, it is promising to convert high-dimensional data transmission into highly-compressed semantic communications (SC). However, to accommodate the local data distribution and user preferences, AIs generally adapt to different domains, which fundamentally distorts the SC alignment. In this paper, we propose a zero-forget domain adaptation (ZFDA) framework to preserve SC alignment. To prevent the DA from changing substantial neural parameters of AI, we design sparse additive modifications (SAM) to the parameters, which can be efficiently stored and switched-off to restore the SC alignment. To optimize the SAM, we decouple it into tractable continuous variables and a binary mask, and then handle the binary mask by a score-based optimization. Experimental evaluations on a SC system for image transmissions validate that the proposed framework perfectly preserves the SC alignment with almost no loss of DA performance, even improved in some cases, at a cost of less than 1% of additional memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。