首个压缩特征质量评估基准,解决高效传输中语义失真难量化问题。
Compressed Feature Quality Assessment: Dataset and Baselines
- 构建首个压缩特征质量评估数据集,含300个原始特征和12000个压缩特征。
- 验证传统指标在捕捉语义退化方面能力有限,需更优评估方法。
- 适合研究模型压缩、特征传输与高效部署的科研人员参考。
大型模型在资源受限环境中的广泛应用凸显了中间特征表示高效传输的需求。在此背景下,特征编码将特征压缩为紧凑比特流,成为特征传输、存储与重用的关键环节。然而,压缩过程不可避免引入难以用传统度量量化语义退化。为此,本文提出压缩特征质量评估(CFQA)研究问题,旨在衡量压缩特征的语义保真度。为推动该领域研究,我们构建首个基准数据集,包含300个原始特征和12000个由三个视觉任务与四种特征编码器生成的压缩特征,并提供任务特定性能下降作为真实语义失真标签。系统评估了三种常用度量:均方误差(MSE)、余弦相似度与中心核对齐(CKA)在捕捉语义退化方面的表现。结果表明,所提数据集具有代表性,同时凸显了现有度量的局限性,亟需更精细的语义失真度量方法。本工作通过建立基础基准,为社区探索CFQA提供了关键资源。相关数据集与代码已开源:https://github.com/chansongoal/Compressed-Feature-Quality-Assessment。
原文摘要 · Abstract (English)
The widespread deployment of large models in resource-constrained environments has underscored the need for efficient transmission of intermediate feature representations. In this context, feature coding, which compresses features into compact bitstreams, becomes a critical component for scenarios involving feature transmission, storage, and reuse. However, this compression process inevitably introduces semantic degradation that is difficult to quantify with traditional metrics. To address this, we formalize the research problem of Compressed Feature Quality Assessment (CFQA), aiming to evaluate the semantic fidelity of compressed features. To advance CFQA research, we propose the first benchmark dataset, comprising 300 original features and 12000 compressed features derived from three vision tasks and four feature codecs. Task-specific performance degradation is provided as true semantic distortion for evaluating CFQA metrics. We systematically assess three widely used metrics -- MSE, cosine similarity, and Centered Kernel Alignment (CKA) -- in terms of their ability to capture semantic degradation. Our findings demonstrate the representativeness of the proposed dataset while underscoring the need for more sophisticated metrics capable of measuring semantic distortion in compressed features. This work advances the field by establishing a foundational benchmark and providing a critical resource for the community to explore CFQA. To foster further research, we release the dataset and all associated source code at https://github.com/chansongoal/Compressed-Feature-Quality-Assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。