通过追踪训练轨迹提升合成数据质量，显著改善下游任务性能。
TGDD: Trajectory Guided Dataset Distillation with Balanced Distribution
- 基于训练轨迹动态对齐特征分布，增强合成数据表达力
- 在高分辨率数据集上实现5.0%的准确率提升，超越现有方法
- 无需额外优化开销，适合高效部署的模型压缩场景
数据蒸馏可将大规模数据集压缩为紧凑的合成数据以降低存储与计算成本。现有基于分布匹配的方法虽高效，但常忽略训练过程中特征表示的演变，限制了合成数据的表达能力并削弱下游性能。为此，我们提出轨迹引导的数据蒸馏（TGDD），将分布匹配重构为沿模型训练轨迹的动态对齐过程。在每个训练阶段，TGDD通过对齐合成数据与原始数据的特征分布来捕捉语义演化，并引入分布约束正则化以减少类别重叠。该设计使合成数据同时保持语义多样性与代表性，显著提升下游任务表现。无需额外优化开销，TGDD在十组数据集上实现最优性能，尤其在高分辨率基准上取得5.0%的准确率提升。
原文摘要 · Abstract (English)
Dataset distillation compresses large datasets into compact synthetic ones to reduce storage and computational costs. Among various approaches, distribution matching (DM)-based methods have attracted attention for their high efficiency. However, they often overlook the evolution of feature representations during training, which limits the expressiveness of synthetic data and weakens downstream performance. To address this issue, we propose Trajectory Guided Dataset Distillation (TGDD), which reformulates distribution matching as a dynamic alignment process along the model's training trajectory. At each training stage, TGDD captures evolving semantics by aligning the feature distribution between the synthetic and original dataset. Meanwhile, it introduces a distribution constraint regularization to reduce class overlap. This design helps synthetic data preserve both semantic diversity and representativeness, improving performance in downstream tasks. Without additional optimization overhead, TGDD achieves a favorable balance between performance and efficiency. Experiments on ten datasets demonstrate that TGDD achieves state-of-the-art performance, notably a 5.0% accuracy gain on high-resolution benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。