为AI加速器设计专用基准测试，更真实反映深度学习负载性能。
Introducing Milabench: Benchmarking Accelerators for AI
- 基于867篇论文和1000+研究员调研，定制26个核心基准
- 涵盖NVIDIA、AMD、Intel多款GPU，实测性能差异显著
- 开源可复现，适合硬件选型与科研评估
深度学习等AI工作负载正为高性能计算系统引入新型使用模式，现有标准HPC基准无法全面覆盖。作为全球最大的深度学习学术研究中心之一，Mila发现其超1000名研究人员的需求难以被通用基准满足，因此开发了专用基准套件Milabench。该套件的设计基于对867篇文献的广泛调研及对本中心研究者的问卷调查，最终选定26个核心基准用于采购评估，另设16个可选基准供深入分析。本文详细阐述设计方法、套件结构，并提供了NVIDIA、AMD、Intel GPU的性能评测结果。Milabench已开源，可通过github.com/mila-iqia/milabench获取。
原文摘要 · Abstract (English)
AI workloads, particularly those driven by deep learning, are introducing novel usage patterns to high-performance computing (HPC) systems that are not comprehensively captured by standard HPC benchmarks. As one of the largest academic research centers dedicated to deep learning, Mila identified the need to develop a custom benchmarking suite to address the diverse requirements of its community, which consists of over 1,000 researchers. This report introduces Milabench, the resulting benchmarking suite. Its design was informed by an extensive literature review encompassing 867 papers, as well as surveys conducted with Mila researchers. This rigorous process led to the selection of 26 primary benchmarks tailored for procurement evaluations, alongside 16 optional benchmarks for in-depth analysis. We detail the design methodology, the structure of the benchmarking suite, and provide performance evaluations using GPUs from NVIDIA, AMD, and Intel. The Milabench suite is open source and can be accessed at github.com/mila-iqia/milabench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。