用可解释方向控制音乐生成，不重训练也能精准调整音符和和弦。
Steering Autoregressive Music Generation with Recursive Feature Machines
- 通过分析模型内部梯度，找到对应音乐属性的概念方向。
- 生成目标音符准确率从0.23提升至0.82，提示匹配度仅下降0.02。
- 支持多属性同时控制，适合需要精细调节的音乐创作场景。
可控音乐生成仍具挑战性，现有方法常需重训练或引入听觉伪影。本文提出MusicRFM框架，将递归特征机器（RFMs）应用于冻结的预训练音乐模型，直接调控其内部激活以实现细粒度、可解释的控制。RFMs通过分析模型内部梯度，生成对应音符、和弦等音乐属性的可解释“概念方向”。首先训练轻量级RFM探测器在MusicGen隐状态中发现这些方向；推理时，将方向注入模型以实时引导生成，无需每步优化。我们提出动态时间调度及多属性同步控制机制。实验表明，目标音符生成准确率从0.23提升至0.82，文本提示遵循度仅下降约0.02，有效平衡了控制力与生成质量。代码已开源，鼓励进一步探索RFM在音乐领域的应用。
原文摘要 · Abstract (English)
Controllable music generation remains a significant challenge, with existing methods often requiring model retraining or introducing audible artifacts. We introduce MusicRFM, a framework that adapts Recursive Feature Machines (RFMs) to enable fine-grained, interpretable control over frozen, pre-trained music models by directly steering their internal activations. RFMs analyze a model's internal gradients to produce interpretable "concept directions", or specific axes in the activation space that correspond to musical attributes like notes or chords. We first train lightweight RFM probes to discover these directions within MusicGen's hidden states; then, during inference, we inject them back into the model to guide the generation process in real-time without per-step optimization. We present advanced mechanisms for this control, including dynamic, time-varying schedules and methods for the simultaneous enforcement of multiple musical properties. Our method successfully navigates the trade-off between control and generation quality: we can increase the accuracy of generating a target musical note from 0.23 to 0.82, while text prompt adherence remains within approximately 0.02 of the unsteered baseline, demonstrating effective control with minimal impact on prompt fidelity. We release code to encourage further exploration on RFMs in the music domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。