SPECTRA通过子空间重放缓解少样本音频分类遗忘问题
SPECTRA: Subspace-Preserving Embedding Calibration, Transport, and Replay for Fully Few-Shot Class-Incremental Audio Classification

- 用轻量适配器校准通用特征,提升任务相关性
- 在三个基准上平均准确率提升,遗忘率显著降低
- 适合需要持续学习新音频类别的场景
全少样本类别增量音频分类(FFCAC)要求仅用每会话少量标注样本识别新声音类别,同时避免遗忘旧类别且无需大规模基础数据集。现有方法通常冻结预训练音频-语言编码器并使用点原型分类,但因通用特征表示导致性能随会话显著下降。我们提出SPECTRA,基于冻结编码器构建框架,包含三项组件:(i) 轻量可训练适配器,用于校准通用嵌入以适应任务;(ii) 子空间特征重放,一种无示例的防遗忘机制,通过从旧类别存储特征的低秩子空间采样来重放旧类;(iii) 测试时的归纳最优传输原型精修。核心发现是子空间重放能有效减少遗忘,优于同方差的高斯重放。在三个FFCAC基准(NSynth-100、FSC-89、LS-100)上,SPECTRA在平均准确率和遗忘抑制方面均超越当前最优方法,消融实验统计验证了各组件有效性。
原文摘要 · Abstract (English)
Fully few-shot class-incremental audio classification (FFCAC) requires recognizing new sound classes from only a handful of labeled examples per session, without forgetting previously learned classes and without any large base dataset. Existing methods typically freeze a pre-trained audio--language encoder and classify with point prototypes, but they suffer from significant performance degradation throughout the sessions due to generic feature representations. We propose SPECTRA, a framework built on a frozen encoder which adds three components. (i) a lightweight trainable adapter that calibrates the generic embeddings to the task; (ii) subspace feature replay, an exemplar-free anti-forgetting scheme that replays old classes by sampling from the low-rank subspace of their stored features; and (iii) a transductive optimal-transport refinement of prototypes at test time. Our central finding is that the subspace structure of the replay diminishes forgetting and outperforms naive Gaussian replay of equal variance. On three FFCAC benchmarks (NSynth-100, FSC-89, LS-100), SPECTRA improves average accuracy and reduces forgetting over current state-of-the-art methods, and our ablations statistically validate each component.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。