混合本地与云端模型，高效处理马来语资源受限场景。
Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara
- 本地用34M小模型，云端用1.3B大模型协同工作。
- 34M模型用一半预训练数据达更高准确率。
- 适合低算力设备上部署马来语智能系统。
在计算和数据资源有限的环境下，高资源语言模型往往难以满足马来语等低资源语言的需求。本文提出一种个人智能系统，融合本地设备端的SLiM-34M模型与服务器端的MANYAK-1.3B模型，实现高效协同。SLiM-34M专为低内存与低功耗设计，而MANYAK-1.3B支持可扩展的高性能任务处理。该系统在机器翻译、问答及IndoMMLU翻译评估中表现优异，其中SLiM-34M在使用仅一半预训练令牌的情况下仍显著提升准确率。本研究挑战了“构建有效模型必须依赖大规模算力”的普遍认知，推动了马来语领域资源高效模型的发展。
原文摘要 · Abstract (English)
In contexts with limited computational and data resources, high-resource language models often prove inadequate, particularly when addressing the specific needs of Malay languages. This paper introduces a Personal Intelligence System designed to efficiently integrate both on-device and server-based models. The system incorporates SLiM-34M for on-device processing, optimized for low memory and power usage, and MANYAK-1.3B for server-based tasks, allowing for scalable, high-performance language processing. The models achieve significant results across various tasks, such as machine translation, question-answering, and translate IndoMMLU. Particularly noteworthy is SLiM-34M's ability to achieve a high improvement in accuracy compared to other LLMs while using 2 times fewer pre-training tokens. This work challenges the prevailing assumption that large-scale computational resources are necessary to build effective language models, contributing to the development of resource-efficient models for the Malay language with the unique orchestration between SLiM-34M and MANYAK-1.3B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。