提出隐私保护的持续学习新方法，解决多方数据协作中的信息泄露问题。
Canonicalized Stable-List Replay for Private Federated Continual Learning over Language-Model Embeddings
- 用公共锚句生成签名，对齐各方无序的私有重放列表
- 在ε=4下比最强基线提升3.9–5.6分，任务性能显著改善
- 适合需要隐私保护的持续学习场景，如医疗、金融等敏感领域
联邦持续学习（FCL）使分布式客户端在不共享原始文本的情况下，通过语言模型头适应不断演化的自然语言任务。在用户级差分隐私（DP）约束下，基于重放的持续学习面临结构性障碍：客户端只能发布包含少量噪声候选重放摘要的小型无序列表。本文提出规范化的稳定列表重放（CSLR），客户端在共享的句子嵌入空间中私有生成候选重放分布，服务器则利用公共锚句诱导的签名进行对齐。锚句提供聚合可识别性，而非额外重放数据。我们证明，在可观测锚句签名间隔条件下，$O(\log(N/η)/p)$ 个锚句可在概率至少 $1-η$ 下区分 $N$ 个候选列表元素；并给出无锚句情形下非可识别性的限定结果。在五个种子下的持续分类、命名实体识别与对话基准测试中，CSLR 在报告的重放发布预算下，于 ε=4 时相较最强非-CSLR DP基线，最终平均任务指标提升 3.9–5.6 分，同时优于匈牙利匹配与最优传输匹配器。正式隐私保证涵盖重放发布；端到端私有训练还需与私有优化器组合以保障任务头更新的隐私。
原文摘要 · Abstract (English)
Federated continual learning (FCL) lets distributed clients adapt language-model heads to evolving NLP tasks without sharing raw text. Under user-level differential privacy (DP), replay-based continual learning faces a structural obstacle: clients can release only small noisy lists of candidate replay summaries, and those lists are unordered across clients. We introduce Canonicalized Stable-List Replay (CSLR), where clients privately produce candidate replay distributions over a shared sentence-embedding space and the server aligns them using signatures induced by public anchor sentences. The anchors provide identifiability for aggregation rather than additional replay data. We prove that, under an observable anchor-signature margin, $O(\log(N/η)/p)$ anchors distinguish $N$ candidate list elements with probability at least $1-η$, and we give a scoped anchorless non-identifiability result for unordered-label oracle models. Across five seeds on continual classification, NER, and dialogue benchmarks, CSLR improves the final average task metric by 3.9--5.6 points over the strongest non-CSLR DP baseline at $\eps=4$ under the reported replay-release budget, while also outperforming Hungarian and optimal-transport matchers. The formal privacy guarantee covers replay release; end-to-end private training additionally requires composition with a private optimizer for task-head updates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。