将音频直接转为吉他英雄游戏谱，自动识别音符位置与节奏
audio2chart: End to End Audio Transcription into playable Guitar Hero charts
- 把生成乐谱看作序列预测任务，用离散时间步对齐音频
- 加入音频条件后，音符预测准确率显著提升
- 代码和预训练模型开源，支持可复现研究
本文提出 audio2chart，一个从原始音频端到端生成 Guitar Hero 风格乐谱的框架。该任务被建模为序列预测问题，模型在离散时间步上生成与音频对齐的乐谱标记。无条件基线模型表现良好，而引入音频条件后，在各项基于准确率的指标上均实现稳定提升。结果表明，音频条件对自动乐谱生成中的音符预测具有可行性和有效性。完整的训练与推理代码已公开于 GitHub，支持神经乐谱生成的研究可复现性。多个预训练模型也已发布于 Hugging Face。
原文摘要 · Abstract (English)
This work introduces audio2chart, a framework for the automatic generation of Guitar Hero style charts directly from raw audio. The task is formalized as a sequence prediction problem, where models are trained to generate discrete chart tokens aligned with the audio on discrete time steps. An unconditional baseline demonstrates strong predictive performance, while the addition of audio conditioning yields consistent improvements across accuracy based metrics. This work demonstrates that incorporating audio conditioning is both feasible and effective for improving note prediction in automatic chart generation. The complete codebase for training and inference is publicly available on GitHub supporting reproducible research on neural chart generation. A family of pretrained models is released on Hugging Face.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。