用数据蒸馏生成小量合成样本，实现长期无监督适应
Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation
- 部署前用数据蒸馏生成源数据的压缩表示
- 在线适应时通过语义匹配锚点保持性能稳定
- 适合需要长期更新且不能保存原始数据的场景
持续测试时适应（CTTA）旨在无标签数据条件下应对目标域演化，维持模型性能。但实际部署常因隐私或许可限制无法保留源数据，纯无源方法在长期分布偏移下易不稳定，出现自训练误差累积与灾难性遗忘。我们提出DO-ALL（Distill Once, Adapt Life-Long），一种即插即用框架，通过数据蒸馏（DD）以紧凑、隐私友好的形式重访源信息。部署前，DO-ALL执行数据蒸馏生成少量合成锚点，总结源分布。适应阶段，每个目标样本与最语义对齐的锚点匹配，提供稳定参考，支持源重播、表征对齐与流形平滑正则化。可无缝集成至现有CTTA算法，在CIFAR100-C、ImageNet-C和CCC基准上持续提升长期鲁棒性，验证了利用DD实现无原始数据存储的稳定连续适应的潜力。代码已公开于https://github.com/blue-531/DOALL。
原文摘要 · Abstract (English)
Continual Test-Time Adaptation (CTTA) aims to maintain model performance under evolving target domains by adapting online without labeled data. However, practical deployments often cannot retain the source dataset due to privacy or licensing constraints, and purely source-free CTTA methods tend to become unstable under long-term distribution shift, suffering from compounding self-training errors and catastrophic forgetting. We introduce DO-ALL (Distill Once, Adapt Life-Long), a plug-and-play framework that revisits source information in a compact and privacy-conscious form via Dataset Distillation (DD). Before deployment, DO-ALL performs DD to produce a small set of synthetic distilled anchors that summarize the source distribution. During adaptation, each target sample is matched with its most semantically aligned anchor, which provides a stable reference for various CTTA via source replay, representation alignment, and manifold-smoothing regularization. DO-ALL can be seamlessly integrated into existing CTTA algorithms, consistently improving long-term robustness across CIFAR100-C, ImageNet-C, and the CCC benchmark. This demonstrates the potential of leveraging DD to enable stable and continuous adaptation without retaining raw source data. The code is available at https://github.com/blue-531/DOALL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。