用统一量化蒸馏让一个模型支持多个图像生成任务,适合移动端部署。
Quantization with Unified Adaptive Distillation to enable multi-LoRA based one-for-all Generative Vision Models on edge
- 把LoRA权重当输入而非固化模型,实现运行时动态切换任务
- 内存占用降低6倍,延迟减少4倍,保持高质量图像输出
- 专为手机NPU设计,适配多种芯片,适合边缘设备部署
生成式人工智能(GenAI)如图像编辑、物体移除和提示引导的图像转换正越来越多地融入移动应用。然而,由于大型视觉模型(LVMs)对内存和计算资源要求高,在资源受限的设备上部署仍具挑战性。尽管低秩适配器(LoRAs)实现了参数高效的任务适配,现有移动端部署方案通常为每个LoRA单独编译模型二进制文件并复制基础模型,导致存储冗余和运行开销增加。本文提出一种统一框架,通过单一共享模型实现多任务生成式视觉推理。核心思想是将LoRA权重作为运行时输入而非嵌入模型图中,支持无需重新编译即可动态切换任务。为支持设备端高效执行,我们引入QUAD(统一自适应蒸馏量化),一种量化感知训练策略,使多个LoRA适配器在统一量化配置下对齐。我们在兼容移动端NPU的轻量级运行时栈上实现该系统,并在多个芯片组上评估。实验表明,在保持多任务高视觉质量的前提下,内存占用减少最高达6倍,延迟降低最高达4倍。
原文摘要 · Abstract (English)
Generative Artificial Intelligence (GenAI) features such as image editing, object removal, and prompt-guided image transformation are increasingly integrated into mobile applications. However, deploying Large Vision Models (LVMs) for such tasks on resource-constrained devices remains challenging due to their high memory and compute requirements. While Low-Rank Adapters (LoRAs) enable parameter-efficient task adaptation, existing Mobile deployment pipelines typically compile separate model binaries for each LoRA + a copy of the foundation model, resulting in redundant storage and increased runtime overhead. In this work, we present a unified framework for enabling multi-task GenAI inference on edge devices using a single shared model. Our key idea is to treat LoRA weights as runtime inputs rather than embedding them into the compiled model graph, allowing dynamic task switching at runtime without recompilation. Then, to support efficient on-device execution, we introduce QUAD (Quantization with Unified Adaptive Distillation), a quantizationaware training strategy that aligns multiple LoRA adapters under a shared quantization profile. We implement the proposed system with a lightweight runtime stack compatible with mobile NPUs and evaluate it across multiple chipsets. Experimental results demonstrate up to 6x and 4x reduction in memory footprint and latency improvements, respectively, while maintaining high visual quality across multiple GenAI tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。