让钢琴转录延迟低于30毫秒，突破实时音乐应用瓶颈
Exploring System Adaptations For Minimum Latency Real-Time Piano Transcription
- 去掉非因果处理，共享计算并调整模型大小以降低延迟
- 在MAESTRO数据集上实现<30ms延迟，但准确率略有下降
- 适合开发低延迟实时音乐系统的研究者参考
神经网络设计进步和大规模标注数据集的出现推动了钢琴转录技术的显著提升。现有方法要么适用于离线场景（无计算限制），要么在线转录延迟为128-320毫秒，但多数实时音乐应用要求延迟低于30毫秒。本文研究当前最先进的在线转录模型如何适配实时钢琴转录。具体包括：移除所有非因果处理，通过核心组件间共享计算、调整模型规模来减少计算量；探索不同预/后处理策略与标签编码方案对实时转录的适用性。在MAESTRO数据集上的评估显示，严格因果处理导致转录准确率下降，且预处理延迟与预测准确率之间存在权衡。我们公开了该系统作为基线，支持研究人员向最小延迟实时转录方向推进。
原文摘要 · Abstract (English)
Advances in neural network design and the availability of large-scale labeled datasets have driven major improvements in piano transcription. Existing approaches target either offline applications, with no restrictions on computational demands, or online transcription, with delays of 128-320 ms. However, most real-time musical applications require latencies below 30 ms. In this work, we investigate whether and how the current state-of-the-art online transcription model can be adapted for real-time piano transcription. Specifically, we eliminate all non-causal processing, and reduce computational load through shared computations across core model components and variations in model size. Additionally, we explore different pre- and postprocessing strategies, and related label encoding schemes, and discuss their suitability for real-time transcription. Evaluating the adaptions on the MAESTRO dataset, we find a drop in transcription accuracy due to strictly causal processing as well as a tradeoff between the preprocessing latency and prediction accuracy. We release our system as a baseline to support researchers in designing models towards minimum latency real-time transcription.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。