统一大模型特征编码,让不同模型特征能高效压缩和通用传输。
DT-UFC: Universal Large Model Feature Coding via Peaky-to-Balanced Distribution Transformation
- 通过学习型分布转换,将尖峰特征变为均衡分布。
- 在3个大模型上实现压缩效率与跨模型泛化显著提升。
- 无需修改编码器,适合多模型通用部署场景。
如同视觉数据传输中的图像编码,特征编码对大模型的分布式部署至关重要,可大幅降低传输与存储负担。然而,以往研究多聚焦于特定任务或模型,缺乏对跨多种大模型的通用特征编码系统性探索。本文首次系统研究大模型的通用特征编码问题,核心挑战在于不同模型提取的特征具有本质差异且分布不兼容:例如,DINOv2的特征呈高度集中尖峰分布,而Stable Diffusion 3(SD3)的特征则更分散均匀。这种分布异质性严重制约压缩效率与跨模型泛化能力。为此,我们提出一种可学习的尖峰到均衡分布转换方法,将高度偏斜的特征分布重塑至统一的均衡目标空间。该转换非均匀、数据驱动且即插即用,可在不修改下游编码器的前提下有效对齐异构分布。在此基础上,基于均衡目标分布训练的通用编码器可有效适配来自不同模型与任务的特征。我们在代表性的三个大模型(LLaMA3、DINOv2、SD3)上验证了该方法,在多个任务与模态下均取得显著优于特定任务基线的压缩效率与跨模型泛化性能。所有源码已公开于 https://github.com/chansongoal/DT-UFC。
原文摘要 · Abstract (English)
Like image coding in visual data transmission, feature coding is essential for the distributed deployment of large models by significantly reducing transmission and storage burden. However, prior studies have mostly targeted task- or model-specific scenarios, leaving the challenge of universal feature coding across diverse large models largely unexplored. In this paper, we present the first systematic study on universal feature coding for large models. The key challenge lies in the inherently diverse and distributionally incompatible nature of features extracted from different models. For example, features from DINOv2 exhibit highly peaky, concentrated distributions, while those from Stable Diffusion 3 (SD3) are more dispersed and uniform. This distributional heterogeneity severely hampers both compression efficiency and cross-model generalization. To address this, we propose a learned peaky-to-balanced distribution transformation, which reshapes highly skewed feature distributions into a common, balanced target space. This transformation is non-uniform, data-driven, and plug-and-play, enabling effective alignment of heterogeneous distributions without modifying downstream codecs. With this alignment, a universal codec trained on the balanced target distribution can effectively generalize to features from different models and tasks. We validate our approach on three representative large models (LLaMA3, DINOv2, and SD3) across multiple tasks and modalities. Extensive experiments show that our method achieves notable improvements in both compression efficiency and cross-model generalization over task-specific baselines. All source code has been made available at https://github.com/chansongoal/DT-UFC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。