优化音频模型文本编码器特征空间，提升少样本音频识别性能
PALM: Few-Shot Prompt Learning for Audio Language Models
- 在文本编码器特征空间中优化提示，而非输入层
- 11个音频数据集上表现优于或相当基线方法
- 计算开销更低，适合资源受限场景
音频-语言模型（ALMs）近期在零样本音频识别任务中取得显著进展，通过将音频波形特征与特定类别的文本提示特征对齐，受到视觉-语言模型（VLMs）的启发。由于零样本性能对人工设计的文本提示敏感，许多提示学习技术被应用于VLMs。本文探索这些方法在ALMs中的有效性，提出一种新方法——音频语言模型中的提示学习（PALM），该方法优化文本编码器分支的特征空间。与现有在输入空间操作的方法不同，本方法训练效率更高。我们在11个音频识别数据集上验证了该方法的有效性，涵盖多种语音处理任务，并在少样本学习设置下与三个基线方法进行对比。结果表明，该方法在性能上达到或超过其他方法，同时计算成本更低。代码已公开于 https://asif-hanif.github.io/palm/
原文摘要 · Abstract (English)
Audio-Language Models (ALMs) have recently achieved remarkable success in zero-shot audio recognition tasks, which match features of audio waveforms with class-specific text prompt features, inspired by advancements in Vision-Language Models (VLMs). Given the sensitivity of zero-shot performance to the choice of hand-crafted text prompts, many prompt learning techniques have been developed for VLMs. We explore the efficacy of these approaches in ALMs and propose a novel method, Prompt Learning in Audio Language Models (PALM), which optimizes the feature space of the text encoder branch. Unlike existing methods that work in the input space, our approach results in greater training efficiency. We demonstrate the effectiveness of our approach on 11 audio recognition datasets, encompassing a variety of speech-processing tasks, and compare the results with three baselines in a few-shot learning setup. Our method is either on par with or outperforms other approaches while being computationally less demanding. Code is available at https://asif-hanif.github.io/palm/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。