为RISC-V微内核加速生成式AI，提升模型推理效率。

Accelerating GenAI Workloads by Enabling RISC-V Microkernel Support in IREE

• 将MLIR算子降级至RISC-V64的mmt4d指令集
• 针对RISC-V优化微内核，性能超越上游IREE与Llama.cpp
• 适合关注边缘端GenAI部署的开发者

本项目在基于MLIR的机器学习编译器与运行时IREE中实现了对RISC-V微内核的支持。方法上，首先在IREE的优化流水线中启用MLIR linalg方言中的收缩操作向RISC-V64目标的linalg.mmt4d操作降级，随后开发了针对RISC-V的优化微内核。性能对比测试基于Llama-3.2-1B-Instruct模型，结果表明该方案相较于上游IREE和Llama.cpp均实现显著加速。

原文摘要 · Abstract (English)

This project enables RISC-V microkernel support in IREE, an MLIR-based machine learning compiler and runtime. The approach begins by enabling the lowering of MLIR linalg dialect contraction ops to linalg.mmt4d op for the RISC-V64 target within the IREE pass pipeline, followed by the development of optimized microkernels for RISC-V. The performance gains are compared with upstream IREE and Llama.cpp for the Llama-3.2-1B-Instruct model.

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。