用双编码器和对比学习实现语言驱动的音频检索，性能领先。
AISTAT lab system for DCASE2025 Task6: Language-based audio retrieval
- 双编码器分别处理音视频，通过对比学习对齐模态表征。
- 引入大模型增强数据，集成聚类辅助训练，提升检索精度。
- 适合关注多模态检索与数据增强技术的研究者。
本文介绍了AISTAT团队在DCASE2025 Task 6语言驱动音频检索任务中的参赛系统。该系统采用双编码器架构，分别编码音频与文本模态，并通过对比学习对齐其表示。借鉴往年挑战赛方法，我们实施了知识蒸馏策略，并利用大语言模型（LLMs）进行有效数据增强，包括回译与LLM mix。此外，引入聚类以构建辅助分类任务，用于进一步微调模型。最佳单系统在Clotho开发测试集上达到mAP@16为46.62，四系统集成后达48.83。
原文摘要 · Abstract (English)
This report presents the AISTAT team's submission to the language-based audio retrieval task in DCASE 2025 Task 6. Our proposed system employs dual encoder architecture, where audio and text modalities are encoded separately, and their representations are aligned using contrastive learning. Drawing inspiration from methodologies of the previous year's challenge, we implemented a distillation approach and leveraged large language models (LLMs) for effective data augmentation techniques, including back-translation and LLM mix. Additionally, we incorporated clustering to introduce an auxiliary classification task for further finetuning. Our best single system achieved a mAP@16 of 46.62, while an ensemble of four systems reached a mAP@16 of 48.83 on the Clotho development test split.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。