用视觉语言模型挖掘图像隐含语义，提升数据蒸馏效果
EDITS: Enhancing Dataset Distillation with Implicit Textual Semantics
- 通过视觉语言模型生成外部文本，融合图像特征构建语义先验
- 利用大语言模型生成精准提示，构建图文原型，提升样本代表性
- 基于扩散模型的双原型引导机制，合成高质量小型数据集
数据蒸馏旨在从大规模原始数据中合成紧凑数据集，实现高效学习并保持优异模型性能。然而，传统方法主要捕捉低层视觉特征，忽略了图像中固有的高层语义与结构信息。本文提出EDITS框架，利用图像数据中的隐含文本语义实现更优蒸馏。首先，通过视觉语言模型（VLM）生成外部文本，经全局语义查询模块与图像特征融合，形成先验聚类缓冲区；随后，局部语义感知模块从中选取代表性样本，构建图像与文本原型，后者通过精心设计的提示引导大语言模型（LLM）生成；最终，采用双原型引导策略，借助扩散模型生成最终合成数据集。大量实验验证了该方法的有效性。源代码见：https://github.com/einsteinxia/EDITS。
原文摘要 · Abstract (English)
Dataset distillation aims to synthesize a compact dataset from the original large-scale one, enabling highly efficient learning while preserving competitive model performance. However, traditional techniques primarily capture low-level visual features, neglecting the high-level semantic and structural information inherent in images. In this paper, we propose EDITS, a novel framework that exploits the implicit textual semantics within the image data to achieve enhanced distillation. First, external texts generated by a Vision Language Model (VLM) are fused with image features through a Global Semantic Query module, forming the prior clustered buffer. Local Semantic Awareness then selects representative samples from the buffer to construct image and text prototypes, with the latter produced by guiding a Large Language Model (LLM) with meticulously crafted prompt. Ultimately, Dual Prototype Guidance strategy generates the final synthetic dataset through a diffusion model. Extensive experiments confirm the effectiveness of our method.Source code is available in: https://github.com/einsteinxia/EDITS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。