为大模型中间特征编码建立首个系统化评测框架。
Towards Large Model Feature Coding

- 构建跨任务、多场景的特征数据集LaMoFCBench,覆盖16种典型设置。
- 发现现有通用编码器与大模型异构特征严重不匹配,性能下降超40%。
- 适合研究大模型分布式推理、特征压缩及系统优化的开发者参考。
大模型在感知与生成任务中表现卓越,但部署受限于计算、内存和隐私需求。分片执行通过跨设备划分计算缓解压力,却带来中间特征的大量传输与存储。传统针对CNN的特征编码方法通常处理同质的空间激活图,而现代大模型产生具有不同统计分布和压缩容忍度的异构特征,如多层级/多模态表示和自回归上下文缓存。这些特性要求将大模型特征编码(LaMoFC)作为基础系统组件,并建立系统性评估框架。本文提出首个全面的LaMoFC基准与评估框架:首先构建涵盖4类任务、16种场景的特征数据集LaMoFCBench,集成主流架构与多种分片计算设置;其次根据实际应用设定代表性分片点,提取中间特征,建立统一可复现的比较流程;最后对主流通用编码器进行基准测试,揭示现有编码范式与大模型特征异构性的深层错配。结果表明,LaMoFC需根本性突破现有范式,而LaMoFCBench为推动这一转变提供共享实证基础。数据与代码将开源于https://github.com/lartpang/LaMoFCBench。
原文摘要 · Abstract (English)
Large models have delivered remarkable performance across a wide range of perception and generation tasks, yet practical deployment is increasingly constrained by computational and memory budgets, as well as privacy requirements. Split execution alleviates these constraints by partitioning computation across devices, but it inevitably introduces intensive transmission and storage of intermediate features. Unlike conventional feature coding for CNNs that typically targets homogeneous spatial activation maps, modern large models generate heterogeneous features with varying statistical distributions and compression tolerances, e.g., multi-level/multi-modal representations and autoregressive context caches. These characteristics necessitate treating large model feature coding (LaMoFC) as a fundamental system component and call for a systematic evaluation framework. In this paper, we present a comprehensive benchmark and evaluation framework for LaMoFC. We first build the feature dataset LaMoFCBench, covering diverse task requirements across 4 categories and 16 scenarios while integrating widelyadopted architectures and various split-computing settings. We then specify representative split points according to practical application scenarios to extract intermediate features, establishing a unified pipeline for fair and reproducible comparisons. Finally, we benchmark mainstream universal feature codecs, exposing the profound misalignment between existing coding paradigms and the heterogeneous nature of large model features. These findings reveal that LaMoFC demands a fundamental departure from existing paradigms, and LaMoFCBench provides the shared empirical foundation to drive this transition. The data and code will be available at https://github.com/lartpang/LaMoFCBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。