用更结构化的轨迹替代随机优化路径,提升医疗数据压缩效率
Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation
- 用二次贝塞尔曲线替代梯度下降路径,生成更可控的监督信号
- 在5个临床数据集上表现优于传统方法,尤其在小样本和低预算下优势明显
- 揭示了轨迹匹配的几何瓶颈,为高效数据压缩提供新思路,适合医疗AI研究者
数据压缩通过构建紧凑的合成数据集保留大规模真实数据的训练价值,推动模型高效开发,并支持医疗等受控领域的下游研究。轨迹匹配(TM)是一种广泛应用的压缩方法,通过模拟真实数据训练时模型参数的变化来监督合成数据,但其监督信号的结构尚不清晰。本文从几何角度分析发现,固定合成数据集只能再现有限范围的参数变化;当监督信号频谱过宽时,会形成可表示性瓶颈。为此,我们提出贝塞尔轨迹匹配(BTM),以初始与终态之间的二次贝塞尔曲线替代SGD轨迹。该方法优化路径上的平均损失,将宽频的SGD监督替换为结构更优、秩更低的信号,且显著降低轨迹存储开销。在五个临床数据集上的实验表明,BTM始终达到或超过标准轨迹匹配性能,尤其在低发病率和低合成预算场景中提升最大。结果表明,有效轨迹匹配的关键在于结构化监督信号,而非复现随机优化路径。
原文摘要 · Abstract (English)
Dataset condensation constructs compact synthetic datasets that retain the training utility of large real-world datasets, enabling efficient model development and potentially supporting downstream research in governed domains such as healthcare. Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data, yet the structure of this supervision signal remains poorly understood. In this paper, we provide a geometric characterisation of trajectory matching, showing that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes. When the resulting supervision signal is spectrally broad, this creates a conditional representability bottleneck. Motivated by this mismatch, we propose Bezier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bezier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset, and they substantially reduce trajectory storage. Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. These results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。