提出CARB框架，精准预测CNN推理能耗、延迟和内存占用，加速部署筛选。
CARB: A Characterization-Guided Framework for CNN Inference Cost Prediction and Deployment Screening
- 基于硬件观测数据构建分层集成模型，联合预测三类资源开销。
- 在两种显卡上实现预测精度R2≈0.99，高负载下延迟与能耗差异达3倍。
- 可秒级筛除90%候选模型，适合大规模神经网络设计优化场景。
在资源受限的GPU平台上部署卷积神经网络时，准确预估推理阶段的能耗、延迟和峰值内存需求日益重要。现有方法多依赖浮点运算量（FLOPs）、延迟测量或单设备性能分析作为能耗代理，忽视了模型架构与硬件负载之间的非线性相互作用。本研究对13,419种CNN配置在RTX 5090和RTX 3080两块显卡上进行了基于GPU遥测的工作负载特征分析，发现能耗、延迟和内存表现出根本不同的扩展特性：高计算负载下，能耗与延迟差异可达3倍；而跨平台迁移能力不同——能耗和延迟需针对具体平台建模，内存则可在两平台间良好迁移。基于这些发现，我们提出CARB框架，采用级联混合集成方法，联合预测三类指标，预测精度达R² ~ 0.99；并设计两阶段部署筛选流程，可在数秒内剔除超过90%的候选方案，将庞大设计空间压缩为经真实硬件验证的帕累托优先短名单。
原文摘要 · Abstract (English)
Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resource-constrained GPU platforms. Existing approaches rely on FLOPs, latency measurements, or single-device profiling as energy proxies, overlooking the non-linear interactions between architectural design and hardware load. We present a workload characterization study of 13 419 CNN configurations on two GPU platforms (RTX 5090 and RTX 3080) under GPU telemetry, revealing that energy, latency, and memory exhibit fundamentally distinct scaling behaviors: energy and latency diverge by 3x under high computational demand, and cross-GPU transferability differs by target--energy and latency require platform-specific models while memory transfers well across the two tested platforms. Building on these characterization findings, we develop CARB, a cascade-blended ensemble that jointly predicts all three targets with R2 ~0.99, and a two-stage deployment screening workflow that eliminates over 90% of candidates in seconds, reducing large design spaces to a Pareto-prioritized shortlist validated against real hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。