用图文信息提升小数据集的代表性，兼顾性能与隐私。
Leveraging Multi-Modal Information to Enhance Dataset Distillation
- 融合图像与文本信息，通过图文匹配增强数据表征。
- 引入目标中心掩码，减少背景干扰，提升关键特征学习。
- 适合需要隐私保护的视觉任务，如医疗图像分析。
数据集蒸馏旨在生成一个小型且高度代表性的合成数据集，以保留大规模真实数据集的核心信息。除了降低存储和计算成本外，该方法还为计算机视觉中的隐私保护提供了新路径，避免存储或共享敏感的真实图像。现有方法仅关注视觉表征优化，忽视了多模态信息潜力。本文提出一种多模态数据集蒸馏框架，包含两个关键改进：基于描述的监督与目标中心掩码。为利用文本信息，引入两种策略：描述拼接（将描述嵌入与视觉特征融合进行分类）和描述匹配（通过基于描述的损失强制真实与合成数据间的语义对齐）。为提升数据效用并减少冗余背景噪声，采用分割掩码隔离目标对象，并引入两种新损失：掩码特征对齐与掩码梯度匹配，均旨在促进目标中心学习。大量实验表明，该方法在保持下游性能的同时，通过最小化真实数据暴露，增强了隐私保护能力。
原文摘要 · Abstract (English)
Dataset distillation aims to create a small and highly representative synthetic dataset that preserves the essential information of a larger real dataset. Beyond reducing storage and computational costs, related approaches offer a promising avenue for privacy preservation in computer vision by eliminating the need to store or share sensitive real-world images. Existing methods focus solely on optimizing visual representations, overlooking the potential of multi-modal information. In this work, we propose a multi-modal dataset distillation framework that incorporates two key enhancements: caption-guided supervision and object-centric masking. To leverage textual information, we introduce two strategies: caption concatenation, which fuses caption embeddings with visual features during classification, and caption matching, which enforces semantic alignment between real and synthetic data through a caption-based loss. To improve data utility and reduce unnecessary background noise, we employ segmentation masks to isolate target objects and introduce two novel losses: masked feature alignment and masked gradient matching, both aimed at promoting object-centric learning. Extensive evaluations demonstrate that our approach improves downstream performance while promoting privacy protection by minimizing exposure to real data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。