10亿参数小模型经优化后,在电商意图识别上达到99%准确率,性能媲美大模型。
Performance Trade-offs of Optimizing Small Language Models for E-Commerce
- 用QLoRA微调生成的合成数据,提升小模型在电商场景的表现。
- 10亿参数模型达99%准确率,媲美大模型GPT-4.1,但成本大幅降低。
- 根据硬件选适配格式:CPU用GGUF可提速18倍,显存降超90%。
大型语言模型在自然语言理解与生成任务中表现卓越,但其在电商等专用任务中的部署常受高计算成本、延迟和运营开销限制。本文探讨了小型开源模型作为资源高效替代方案的可行性。我们提出一种针对多语言电商意图识别优化的10亿参数Llama 3.2模型的方法:基于合成用户查询数据集,采用量化低秩适配(QLoRA)进行微调,并进一步应用后训练量化技术,生成适配GPU(GPTQ)与CPU(GGUF)的版本。结果表明,该专用10亿参数模型达到99%准确率,与更大规模的GPT-4.1模型性能相当。详细分析揭示关键硬件依赖性权衡:4位GPTQ在旧版显卡(NVIDIA T4)上虽减少41%显存占用,却因反量化开销导致推理速度下降82%;而GGUF格式在CPU上实现最高18倍的吞吐加速及超过90%的内存消耗降低。结论表明,经过恰当优化的小型开放权重模型不仅是可行选择,更适用于特定领域,以极低算力代价达成前沿性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) offer state-of-the-art performance in natural language understanding and generation tasks. However, the deployment of leading commercial models for specialized tasks, such as e-commerce, is often hindered by high computational costs, latency, and operational expenses. This paper investigates the viability of smaller, open-weight models as a resource-efficient alternative. We present a methodology for optimizing a one-billion-parameter Llama 3.2 model for multilingual e-commerce intent recognition. The model was fine-tuned using Quantized Low-Rank Adaptation (QLoRA) on a synthetically generated dataset designed to mimic real-world user queries. Subsequently, we applied post-training quantization techniques, creating GPU-optimized (GPTQ) and CPU-optimized (GGUF) versions. Our results demonstrate that the specialized 1B model achieves 99% accuracy, matching the performance of the significantly larger GPT-4.1 model. A detailed performance analysis revealed critical, hardware-dependent trade-offs: while 4-bit GPTQ reduced VRAM usage by 41%, it paradoxically slowed inference by 82% on an older GPU architecture (NVIDIA T4) due to dequantization overhead. Conversely, GGUF formats on a CPU achieved a speedup of up to 18x in inference throughput and a reduction of over 90% in RAM consumption compared to the FP16 baseline. We conclude that small, properly optimized open-weight models are not just a viable but a more suitable alternative for domain-specific applications, offering state-of-the-art accuracy at a fraction of the computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。