低功耗设备上实现多语言翻译,靠极致量化与硬件协同设计。
Bhasha-Rupantarika: Algorithm-Hardware Co-design approach for Multilingual Neural Machine Translation
- 通过算法-硬件协同设计,支持亚字节精度(如FP4)推理。
- 模型缩小4.1倍,推理速度提升4.2倍,吞吐达66词/秒(提升4.8倍)。
- 适合资源受限的物联网场景,尤其支持印度本地语言实时翻译。
本文提出Bhasha-Rupantarika,一种面向资源受限环境的轻量级多语言神经机器翻译系统,采用算法-硬件协同设计。该方法探索在子字节精度(FP8、INT8、INT4、FP4)下的模型部署,实验表明使用FP4可使模型大小减少4.1倍,推理速度提升4.2倍,吞吐量提高至66 tokens/s(相比基准提升4.8倍)。结果验证了超低精度量化对基于FPGA加速器的物联网设备实现实时部署的重要性。评估涵盖印地语与国际语言间的双向翻译,展现了在低资源语言场景下的适应性。FPGA部署中,查找表(LUTs)减少1.96倍,触发器(FFs)减少1.65倍,吞吐量相较OPU提升2.2倍,较HPTA提升4.6倍。整体证明该方案具备可部署的多语言AI系统潜力。代码与数据集已公开,支持复现与后续研究。
原文摘要 · Abstract (English)
This paper introduces Bhasha-Rupantarika, a light and efficient multilingual translation system tailored through algorithm-hardware codesign for resource-limited settings. The method investigates model deployment at sub-octet precision levels (FP8, INT8, INT4, and FP4), with experimental results indicating a 4.1x reduction in model size (FP4) and a 4.2x speedup in inference speed, which correlates with an increased throughput of 66 tokens/s (improvement by 4.8x). This underscores the importance of ultra-low precision quantization for real-time deployment in IoT devices using FPGA accelerators, achieving performance on par with expectations. Our evaluation covers bidirectional translation between Indian and international languages, showcasing its adaptability in low-resource linguistic contexts. The FPGA deployment demonstrated a 1.96x reduction in LUTs and a 1.65x decrease in FFs, resulting in a 2.2x enhancement in throughput compared to OPU and a 4.6x enhancement compared to HPTA. Overall, the evaluation provides a viable solution based on quantisation-aware translation along with hardware efficiency suitable for deployable multilingual AI systems. The entire codes [https://github.com/mukullokhande99/Bhasha-Rupantarika/] and dataset for reproducibility are publicly available, facilitating rapid integration and further development by researchers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。