为实时音频应用优化神经网络推理，解决延迟问题。
ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications
- 将推理任务从音频回调中解耦到静态线程池，避免实时中断。
- 状态无关模型用ONNX Runtime最快，状态相关模型用LibTorch最优。
- 发现初始推理延迟更高，尤其在出现实时违规时更明显。
当前众多神经网络推理工具无法满足实时音频应用需求。为此，我们提出anira，一个高效的跨平台推理库。anira支持ONNX Runtime、LibTorch和TensorFlow Lite作为后端，以兼容多种神经网络架构与框架。各推理引擎均存在实时违规现象，anira通过将推理任务从音频回调解耦至静态线程池予以缓解。库内集成延迟管理与全面基准测试功能，保障信号连续性。对三种用于音频效果模拟的神经网络架构在不同配置下进行基准测试，并采用统计建模分析性能影响因素。结果表明：对于无状态模型，ONNX Runtime运行时间最短；对于有状态模型，LibTorch表现最佳。此外，某些模型-引擎组合的初始推理耗时更长，且实时违规频率更高。
原文摘要 · Abstract (English)
Numerous tools for neural network inference are currently available, yet many do not meet the requirements of real-time audio applications. In response, we introduce anira, an efficient cross-platform library. To ensure compatibility with a broad range of neural network architectures and frameworks, anira supports ONNX Runtime, LibTorch, and TensorFlow Lite as backends. Each inference engine exhibits real-time violations, which anira mitigates by decoupling the inference from the audio callback to a static thread pool. The library incorporates built-in latency management and extensive benchmarking capabilities, both crucial to ensure a continuous signal flow. Three different neural network architectures for audio effect emulation are then subjected to benchmarking across various configurations. Statistical modeling is employed to identify the influence of various factors on performance. The findings indicate that for stateless models, ONNX Runtime exhibits the lowest runtimes. For stateful models, LibTorch demonstrates the fastest performance. Our results also indicate that for certain model-engine combinations, the initial inferences take longer, particularly when these inferences exhibit a higher incidence of real-time violations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。