ELANA是一款轻量级工具，可分析大模型在不同硬件上的延迟与功耗。
ELANA: A Simple Energy and Latency Analyzer for LLMs
- 基于命令行的轻量级分析工具，支持多平台部署
- 精准测量预填充、生成及端到端延迟，兼容主流Hugging Face模型
- 适合研究高效模型或小规模验证，支持能效日志记录
大语言模型（LLMs）在从移动边缘设备到云GPU集群的各类硬件平台上部署时，延迟和功耗是主要瓶颈。基准测试对于优化模型部署效率及下一代模型研发至关重要。为此，我们开源了一款轻量级性能分析工具——ELANA，用于评估LLMs在多GPU和边缘GPU平台上的模型大小、键值缓存大小、预填充延迟（首次输出时间，TTFT）、生成延迟（每输出一个词的时间，TPOT）以及端到端延迟（末次输出时间，TTLT）。ELANA支持Hugging Face上所有公开模型，提供简单命令行界面，并可选记录能耗数据。它完全兼容主流Hugging Face API，易于定制或适配压缩模型与低比特模型，特别适合高效大模型研究或小规模概念验证。工具已开源：https://github.com/enyac-group/Elana。
原文摘要 · Abstract (English)
The latency and power consumption of large language models (LLMs) are major constraints when serving them across a wide spectrum of hardware platforms, from mobile edge devices to cloud GPU clusters. Benchmarking is crucial for optimizing efficiency in both model deployment and next-generation model development. To address this need, we open-source a simple profiling tool, \textbf{ELANA}, for evaluating LLMs. ELANA is designed as a lightweight, academic-friendly profiler for analyzing model size, key-value (KV) cache size, prefilling latency (Time-to-first-token, TTFT), generation latency (Time-per-output-token, TPOT), and end-to-end latency (Time-to-last-token, TTLT) of LLMs on both multi-GPU and edge GPU platforms. It supports all publicly available models on Hugging Face and offers a simple command-line interface, along with optional energy consumption logging. Moreover, ELANA is fully compatible with popular Hugging Face APIs and can be easily customized or adapted to compressed or low bit-width models, making it ideal for research on efficient LLMs or for small-scale proof-of-concept studies. We release the ELANA profiling tool at: https://github.com/enyac-group/Elana.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。