AIBA让文本生成音乐模型的注意力可被量化分析，看清它关注了哪些音色频段。
AIBA: Attention-based Instrument Band Alignment for Text-to-Audio Diffusion
- 在推理时插入注意力记录，不改动模型权重
- 将注意力分布投影到梅尔频谱网格，与音频能量对齐
- 用三个可解释指标评估注意力与乐器频段的真实匹配度
我们提出AIBA（Attention-In-Band Alignment），一种轻量级、无需训练的管线，用于量化文本到音频扩散模型在时频平面上的注意力位置。AIBA（i）在推理阶段接入交叉注意力以记录注意力概率，无需修改模型权重；（ii）将其投影至固定大小的梅尔网格，可直接与音频能量进行比较；（iii）通过可解释指标（时频交并比/平均精度、频率轮廓相关性、指向游戏）评估注意力与乐器频段真值的一致性。在基于AudioLDM2主干网络的Slakh2100数据集上，AIBA揭示出稳定的乐器依赖趋势（如贝斯偏好低频段），并实现高精度与中等召回率。
原文摘要 · Abstract (English)
We present AIBA (Attention-In-Band Alignment), a lightweight, training-free pipeline to quantify where text-to-audio diffusion models attend on the time-frequency (T-F) plane. AIBA (i) hooks cross-attention at inference to record attention probabilities without modifying weights; (ii) projects them to fixed-size mel grids that are directly comparable to audio energy; and (iii) scores agreement with instrument-band ground truth via interpretable metrics (T-F IoU/AP, frequency-profile correlation, and a pointing game). On Slakh2100 with an AudioLDM2 backbone, AIBA reveals consistent instrument-dependent trends (e.g., bass favoring low bands) and achieves high precision with moderate recall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。