用物理振动模式提取数据核心特征,让模型训练更稳定高效
DRIFT: Data Reduction via Informative Feature Transformation- Generalization Begins Before Deep Learning starts
- 通过板振动模态构建低维特征基,提前提炼数据关键模式
- MNIST仅需约50特征,CIFAR100少于100特征即达良好分类精度
- 对批量大小、网络结构等不敏感,适合鲁棒性要求高的场景
现代深度学习架构在数据进入网络后表现优异,但真正的瓶颈在于输入准备:如何获得最小、显著且反映数据本质模式的输入。本文提出DRIFT(基于信息特征变换的数据压缩),受物理系统振动分析启发,用于在训练前识别并提取输入数据中最共振的模式。与传统模型在信号与噪声中学习不同,DRIFT模仿物理感知,强化信息特征,剔除无关成分。结果得到更紧凑可解释的表示,提升训练稳定性和泛化性能。图像被投影到由板状物空间振动模态构成的低维基上,形成具有物理意义的特征集。该方法使神经网络输入维度大幅降低——MNIST仅需约50特征,CIFAR100少于100特征,仍能实现竞争力分类准确率。在MNIST和CIFAR100上的大量实验表明,相比标准像素模型和PCA,DRIFT在训练稳定性、抗过拟合能力和泛化鲁棒性方面均表现更优。尤为突出的是,其对批量大小、网络结构、图像分辨率变化几乎无敏感性,凸显其作为高效稳健数据表征策略的潜力。本工作将焦点从架构设计转向输入优化,强调物理驱动数据变换对提升深度学习性能的关键作用。
原文摘要 · Abstract (English)
Modern deep learning architectures excel at optimization, but only after the data has entered the network. The true bottleneck lies in preparing the right input: minimal, salient, and structured in a way that reflects the essential patterns of the data. We propose DRIFT (Data Reduction via Informative Feature Transformation), a novel preprocessing technique inspired by vibrational analysis in physical systems, to identify and extract the most resonant modes of input data prior to training. Unlike traditional models that attempt to learn amidst both signal and noise, DRIFT mimics physics perception by emphasizing informative features while discarding irrelevant elements. The result is a more compact and interpretable representation that enhances training stability and generalization performance. In DRIFT, images are projected onto a low-dimensional basis formed by spatial vibration mode shapes of plates, offering a physically grounded feature set. This enables neural networks to operate with drastically fewer input dimensions (~ 50 features on MNIST and less than 100 on CIFAR100) while achieving competitive classification accuracy. Extensive experiments across MNIST and CIFAR100 demonstrate DRIFT's superiority over standard pixel-based models and PCA in terms of training stability, resistance to overfitting, and generalization robustness. Notably, DRIFT displays minimal sensitivity to changes in batch size, network architecture, and image resolution, further establishing it as a resilient and efficient data representation strategy. This work shifts the focus from architecture engineering to input curation and underscores the power of physics-driven data transformations in advancing deep learning performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。