提出线性复杂度的音乐结构位置编码，提升符号音乐生成效率
F-StrIPE: Fast Structure-Informed Positional Encoding for Symbolic Music Generation
- 用随机特征近似技术实现线性复杂度的位置编码
- 在旋律和声任务中显著提升生成质量与速度
- 适合需要高效处理长音乐序列的研究者
尽管音乐仍是生成模型如Transformer的挑战领域，近期进展得益于引入音乐相关的先验知识。一种方法是将音乐结构信息融入位置编码（PE）模块。然而，Transformer在序列长度上具有二次计算开销。本文提出F-StrIPE，一种结构感知的位置编码方案，实现线性复杂度。基于现有的随机特征核近似技术，我们证明F-StrIPE是随机位置编码（SPE）的推广。通过符号音乐的旋律和声任务，验证了F-StrIPE的实证优势。
原文摘要 · Abstract (English)
While music remains a challenging domain for generative models like Transformers, recent progress has been made by exploiting suitable musically-informed priors. One technique to leverage information about musical structure in Transformers is inserting such knowledge into the positional encoding (PE) module. However, Transformers carry a quadratic cost in sequence length. In this paper, we propose F-StrIPE, a structure-informed PE scheme that works in linear complexity. Using existing kernel approximation techniques based on random features, we show that F-StrIPE is a generalization of Stochastic Positional Encoding (SPE). We illustrate the empirical merits of F-StrIPE using melody harmonization for symbolic music.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。