分析大模型在纯CPU下的内存访问特征，寻找优化推理速度的突破口。
Memory Access Characterization of Large Language Models in CPU Environment and its Potential Impacts
- 通过修改缓存架构提升大模型在无加速器环境下的推理效率。
- 实验发现不同缓存配置对性能影响显著，存在可优化的内存访问模式。
- 适合关注低资源部署、模型推理优化的研究者和开发者。
随着机器学习算法日益重要，对其访问的需求也持续增长。在能源、安全或成本受限的环境中，缺乏加速器使得运行大型模型的推理变得不可行。为提高这些模型的可用性，本文旨在通过改进缓存架构，提升大模型在纯CPU环境下的推理速度。为此，我们使用Llama.cpp和QWEN模型进行了两项实验：测试多种缓存配置下的性能表现，并输出内存占用的追踪数据。基于这些实验，我们研究了内存访问模式与性能特性，识别出潜在的优化方向。
原文摘要 · Abstract (English)
As machine learning algorithms are shown to be an increasingly valuable tool, the demand for their access has grown accordingly. Oftentimes, it is infeasible to run inference with larger models without an accelerator, which may be unavailable in environments that have constraints such as energy consumption, security, or cost. To increase the availability of these models, we aim to improve the LLM inference speed on a CPU-only environment by modifying the cache architecture. To determine what improvements could be made, we conducted two experiments using Llama.cpp and the QWEN model: running various cache configurations and evaluating their performance, and outputting a trace of the memory footprint. Using these experiments, we investigate the memory access patterns and performance characteristics to identify potential optimizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。