4-bit量化让大模型在助盲设备上高效运行,误差更小、内存降七成。
RPIQ: Residual-Projected Multi-Collaboration Closed-Loop and Single Instance Quantization for Visually Impaired Assistance
- 用闭环补偿+单实例校准,解决量化中块间误差累积问题。
- 4比特压缩后内存减少60%-75%,性能接近全精度模型。
- 适合部署在资源受限的助盲智能设备,尤其文本理解与视觉问答场景。
视障用户在日常信息获取和实时环境感知方面面临巨大挑战,亟需具备精准识别能力的智能辅助系统。尽管大规模模型在感知与推理方面表现优异,但其在辅助设备上的实际部署受制于过高的内存消耗和推理成本。现有量化方法常忽略块间误差累积,导致模型稳定性下降。为此,本文提出一种新型量化框架——残差投影多协作闭环与单实例量化(RPIQ),其量化过程采用基于单实例校准与高斯-赛德尔迭代量化的多协作闭环补偿机制。在OPT、Qwen、LLaMA等语言模型及CogVLM2等视觉语言模型上的实验表明,RPIQ可将模型压缩至4比特表示,同时显著降低峰值内存消耗(相比原全精度模型约减少60%-75%)。该方法在多种语言与视觉任务中保持接近全精度模型的性能,在复杂场景下的文本理解与视觉问答等关键应用中展现出优异的识别与推理能力。研究验证了RPIQ在真实辅助系统部署中的有效性,提升了大规模模型的计算效率与可靠性,使模型能为视障用户提供准确、快速的信息支持。
原文摘要 · Abstract (English)
Visually impaired users face significant challenges in daily information access and real-time environmental perception, and there is an urgent need for intelligent assistive systems with accurate recognition capabilities. Although large-scale models provide effective solutions for perception and reasoning, their practical deployment on assistive devices is severely constrained by excessive memory consumption and high inference costs. Moreover, existing quantization strategies often ignore inter-block error accumulation, leading to degraded model stability. To address these challenges, this study proposes a novel quantization framework -- Residual-Projected Multi-Collaboration Closed-Loop and Single Instance Quantization(RPIQ), whose quantization process adopts a multi-collaborative closed-loop compensation scheme based on Single Instance Calibration and Gauss-Seidel Iterative Quantization. Experiments on various types of large-scale models, including language models such as OPT, Qwen, and LLaMA, as well as vision-language models such as CogVLM2, demonstrate that RPIQ can compress models to 4-bit representation while significantly reducing peak memory consumption (approximately 60%-75% reduction compared to original full-precision models). The method maintains performance highly close to full-precision models across multiple language and visual tasks, and exhibits excellent recognition and reasoning capabilities in key applications such as text understanding and visual question answering in complex scenarios. While verifying the effectiveness of RPIQ for deployment in real assistive systems, this study also advances the computational efficiency and reliability of large models, enabling them to provide visually impaired users with the required information accurately and rapidly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。