用超复数乘法层逐步替换大模型模块，实现高效量化部署
Beyond Real Weights: Hypercomplex Representations for Stable Quantization
- 用超复数乘法层替代密集前馈网络，渐进式压缩模型
- 参数量与计算量大幅下降，性能几乎无损
- 适合追求高效多模态推理的部署场景
多模态语言模型（MLLMs）需大量参数以对齐高维视觉特征与语言表示，导致计算开销大、部署困难。本文提出一种渐进重参数化策略，通过逐步将密集前馈网络块替换为紧凑的参数化超复数乘法（PHM）层来压缩模型。结合残差插值调度及轻量级重建与知识蒸馏损失，确保PHM模块在训练中继承原密集层的功能行为。该方法显著降低参数量与浮点运算量（FLOPs），同时保持强多模态对齐能力，实现更快推理且不降低输出质量。我们在多个视觉语言模型（VLMs）上评估该方法，结果表明其性能接近基线模型，但模型尺寸与推理延迟大幅减少。渐进式PHM替换为更高效的多模态推理提供了兼容现有架构的路径，并可与低比特量化技术互补。
原文摘要 · Abstract (English)
Multimodal language models (MLLMs) require large parameter capacity to align high-dimensional visual features with linguistic representations, making them computationally heavy and difficult to deploy efficiently. We introduce a progressive reparameterization strategy that compresses these models by gradually replacing dense feed-forward network blocks with compact Parameterized Hypercomplex Multiplication (PHM) layers. A residual interpolation schedule, together with lightweight reconstruction and knowledge distillation losses, ensures that the PHM modules inherit the functional behavior of their dense counterparts during training. This transition yields substantial parameter and FLOP reductions while preserving strong multimodal alignment, enabling faster inference without degrading output quality. We evaluate the approach on multiple vision-language models (VLMs). Our method maintains performance comparable to the base models while delivering significant reductions in model size and inference latency. Progressive PHM substitution thus offers an architecture-compatible path toward more efficient multimodal reasoning and complements existing low-bit quantization techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。