arXiv:2602.10377cs.LGcs.CL2026-02被引 3

为设备端大模型设计提供硬件协同优化的量化方法,显著提升性能效率。

Hardware Co-Design Scaling Laws via Roofline Modelling for On-Device LLMs

  • 基于架构超参数建模训练损失,结合屋顶模型分析推理延迟。
  • 在Jetson Orin上验证1942种架构,实现精度与延迟的直接对应关系。
  • 可将模型选型时间从月缩短至天,适合边缘AI部署场景。

视觉-语言-动作模型(VLAs)已成为物理人工智能的关键范式,广泛应用于自动驾驶、机器人和智能空间中。在资源受限的设备端部署场景下,选择合适的大型语言模型(LLM)骨干网络是一项关键挑战:模型需在准确性与严格的推理延迟及硬件效率约束之间取得平衡。这使得软硬件协同设计成为设备端LLM部署的核心需求,每种硬件平台都需要定制化的架构解决方案。本文提出一种硬件协同设计定律,联合刻画模型准确率与推理性能。具体而言,将训练损失显式建模为架构超参数的函数,并通过屋顶模型(roofline modelling)表征推理延迟。我们在NVIDIA Jetson Orin上对1,942个候选架构进行实证评估,训练了170个选定模型,每个模型均以100亿个词元进行训练,拟合出架构与训练损失之间的缩放定律。通过将该缩放定律与延迟建模耦合,建立了准确率与延迟的直接对应关系,并识别出硬件协同设计的帕累托前沿。进一步地,我们将架构搜索形式化为精度与性能的联合优化问题,推导出在工业硬件与应用预算下的可行设计区域。本方法将架构选择时间从数月缩短至数日。在与Qwen2.5-0.5B相同延迟下,所设计架构在WikiText-2数据集上困惑度降低19.42%。据我们所知,这是首个系统且可操作的设备端LLM硬件协同设计缩放定律框架。代码与相关检查点将公开发布。

原文摘要 · Abstract (English)

Vision-Language-Action Models (VLAs) have emerged as a key paradigm of Physical AI and are increasingly deployed in autonomous vehicles, robots, and smart spaces. In these resource-constrained on-device settings, selecting an appropriate large language model (LLM) backbone is a critical challenge: models must balance accuracy with strict inference latency and hardware efficiency constraints. This makes hardware-software co-design a game-changing requirement for on-device LLM deployment, where each hardware platform demands a tailored architectural solution. We propose a hardware co-design law that jointly captures model accuracy and inference performance. Specifically, we model training loss as an explicit function of architectural hyperparameters and characterise inference latency via roofline modelling. We empirically evaluate 1,942 candidate architectures on NVIDIA Jetson Orin, training 170 selected models for 10B tokens each to fit a scaling law relating architecture to training loss. By coupling this scaling law with latency modelling, we establish a direct accuracy-latency correspondence and identify the Pareto frontier for hardware co-designed LLMs. We further formulate architecture search as a joint optimisation over precision and performance, deriving feasible design regions under industrial hardware and application budgets. Our approach reduces architecture selection from months to days. At the same latency as Qwen2.5-0.5B on the target hardware, our co-designed architecture achieves 19.42% lower perplexity on WikiText-2. To our knowledge, this is the first principled and operational framework for hardware co-design scaling laws in on-device LLM deployment. We will make the code and related checkpoints publicly available.

大模型部署硬件协同边缘计算模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。