用音色结构匹配技术,精准检索吉他效果插件配置。
TimberAgent: Gram-Guided Retrieval for Executable Music Effect Control
- 基于Wav2Vec2特征的格拉姆矩阵构建音色感知表示
- 在1063个预设中实现最低参数误差,优于对比方法
- 适合音乐制作中需要快速调参的创作者使用
数字音频工作站虽提供丰富的效果链,但用户意图与底层信号处理参数之间仍存在语义鸿沟。本文研究以检索为基础的音频效果控制,输出为可编辑的插件配置而非最终波形。重点提出纹理共振检索(TRR),通过投影后Wav2Vec2中间层激活的格拉姆矩阵构建音频表示,保留纹理相关的共激活结构。在包含1063个候选预设和204个查询的吉他效果基准上进行评估,采用防止训练-测试泄露的交叉验证协议(Protocol-A)。对比CLAP及内部基线(Wav2Vec-RAG、Text-RAG、FeatureNN-RAG),使用基于物理DSP参数范围的归一化指标。消融实验验证了投影维度、层选择与投影类型的关键作用。近重复敏感性分析表明结果对知识库中平凡匹配不敏感。TRR在所有方法中取得最低归一化参数误差。26名参与者参与多刺激听觉测试,提供补充感知证据。结果支持纹理感知检索在可编辑音频效果控制中的有效性,而更广泛的个性化与真实音频鲁棒性主张尚无充分验证。
原文摘要 · Abstract (English)
Digital audio workstations expose rich effect chains, yet a semantic gap remains between perceptual user intent and low-level signal-processing parameters. We study retrieval-grounded audio effect control, where the output is an editable plugin configuration rather than a finalized waveform. Our focus is Texture Resonance Retrieval (TRR), an audio representation built from Gram matrices of projected mid-level Wav2Vec2 activations. This design preserves texture-relevant co-activation structure. We evaluate TRR on a guitar-effects benchmark with 1,063 candidate presets and 204 queries. The evaluation follows Protocol-A, a cross-validation scheme that prevents train-test leakage. We compare TRR against CLAP and internal retrieval baselines (Wav2Vec-RAG, Text-RAG, FeatureNN-RAG), using min-max normalized metrics grounded in physical DSP parameter ranges. Ablation studies validate TRR's core design choices: projection dimensionality, layer selection, and projection type. A near-duplicate sensitivity analysis confirms that results are robust to trivial knowledge-base matches. TRR achieves the lowest normalized parameter error among evaluated methods. A multiple-stimulus listening study with 26 participants provides complementary perceptual evidence. We interpret these results as benchmark evidence that texture-aware retrieval is useful for editable audio effect control, while broader personalization and real-audio robustness claims remain outside the verified evidence presented here.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。