GaLe让小设备运行大模型，速度提升65%、内存减少90%
GaLe: memory-efficient Global Approximate and Local Exact features
- 将特征图分作局部精确与全局近似两部分，兼顾细节与长程依赖
- 在ImageNet上性能接近全精度，比切片法快65%、内存降90%
- 适用于分类、检测、生成任务，适合资源受限设备部署
嵌入式设备通常缺乏GPU算力，现有推理方法要么计算开销高（基于切片），要么准确率低（近似方法）。我们提出GaLe，一种无需微调即可在资源受限设备上部署预训练模型的内存高效技术。GaLe将特征图分为局部精确（Le）表示以保留细节，以及全局近似（Ga）表示以保持长程依赖。不同于标准切片方式，GaLe支持混合卷积-变压器模型中的全局操作与注意力机制。在ImageNet上验证表明，该方法性能接近精确推理，相比基于切片的推理，在Cortex-M33上实现最高65%的速度提升和90%的内存占用降低。我们进一步展示了GaLe在分类、检测与生成任务中的通用性，凸显其作为资源高效架构设计基础的潜力。
原文摘要 · Abstract (English)
Embedded devices typically lack the resources of GPU-equipped machines, and existing inference methods suffer from either high computational overhead (patch-based) or accuracy loss (approximation-based). We propose GaLe, a memory-efficient technique that enables the deployment of pretrained networks on constrained devices without retraining. GaLe partitions feature maps into two components: a local exact (Le) representation that preserves fine details and a global approximate (Ga) representation that retains long-range dependencies. Unlike standard tiling, GaLe supports global operations and attention mechanisms found in hybrid CNN-transformer models. Validated on ImageNet, our method matches exact-inference performance while achieving up to 65% speedup and 90% RAM reduction on a Cortex-M33 compared to patch-based inference. We further demonstrate GaLe's versatility across classification, detection, and generation tasks, highlighting its potential as a foundation for resource-efficient architecture design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。