通过文本预测噪声下音频嵌入偏移，提升零样本音视频分类鲁棒性。
Drift-Augmented Scoring: Text-Derived Noise Robustness for Zero-Shot Audio-Language Classification
- 基于文本预测噪声导致的嵌入偏移，为每类添加微调得分
- 在0 dB信噪比下，准确率提升2.60~5.75点，mAP提升1.50~1.74点
- 仅需一次计算，推理时无梯度、无批量，适合部署
对比音频-文本模型（如CLAP）可实现零样本音频分类：通过匹配音频嵌入与文本提示嵌入来打标签，无需标注音频。但在声学噪声下，准确率和mAP在标准基准上下降12-30个百分点（0 dB SNR时）。本文提出漂移增强评分（DAS），一种针对每类的小型奖励项，添加到余弦相似度分数中。该奖励基于文本单独推导，预先计算并缓存，推理时每类仅增加一次内积运算，无需梯度、无需测试时批量。在LAION CLAP主干网络上，我们在UrbanSound8K和完整FSD50K测试集上，对齐了Acevedo等人同期方法的四种变体进行对比，将每段音频混合城市环境噪声，在不同信噪比下评估。DAS在所有测试条件下均表现更优：UrbanSound8K上准确率提升2.60至5.75个百分点，FSD50K上mAP提升1.50至1.74个百分点。
原文摘要 · Abstract (English)
Contrastive audio-language models such as CLAP enable zero-shot audio classification: a sound is labelled by matching its embedding to text prompt embeddings, with no labelled audio. This matching breaks down under acoustic noise, where accuracy and mAP fall by 12-30 percentage points at 0 dB SNR on standard benchmarks. We propose Drift Augmented Scoring (DAS), a small per-class bonus added to the cosine score. The bonus rewards a class when the noisy audio embedding drifts in the direction that the class's noise-conditioned text prompts predict. It is derived from text alone, computed once and cached, and adds a single inner product per class at inference, with no gradients and no test-time batch. On a LAION CLAP backbone, we compare DAS against the four variants of Acevedo et al.'s concurrent method on UrbanSound8K and the full FSD50K eval set, mixing each clip with urban acoustic scene noise across a range of SNRs. DAS improves the metric on every test condition: by +2.60 to +5.75 accuracy points on UrbanSound8K and +1.50 to +1.74 mAP points on FSD50K.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。