通过融合数据内与数据间特征，提升小样本下的模型精度与效率
Deep-and-Wide Learning: Enhancing Data-Driven Inference via Synergistic Learning of Inter- and Intra-Data Representations
- 构建双通道网络，同时学习数据内部与跨数据特征
- 在少量数据下精度显著超越现有深度网络，计算效率提升数量级
- 适用于小样本科学建模，尤其适合资源受限场景
深度学习的进展正重塑科学与工程领域。其成功主要源于从输入数据中提取高维（HD）特征并据此做出推断的能力。然而，当前深度神经网络（DNN）模型面临数据与计算资源需求巨大的挑战。本文提出一种新型学习范式——深宽学习（DWL），系统捕捉单个输入数据内的特征（内数据特征）以及跨数据间的特征（间数据特征）。我们设计了双交互通道网络（D-Net），基于贝叶斯框架实现低维（LD）间数据特征提取，并与传统高维表示协同作用，显著提升计算效率与推断性能。该方法已应用于多个学科领域的分类与回归任务。结果表明，DWL在有限训练数据下显著优于现有最优DNN模型，且计算效率提升达数量级。这一策略深刻改变了数据驱动学习范式，对新兴的大规模基础模型发展具有重要启示。
原文摘要 · Abstract (English)
Advancements in deep learning are revolutionizing science and engineering. The immense success of deep learning is largely due to its ability to extract essential high-dimensional (HD) features from input data and make inference decisions based on this information. However, current deep neural network (DNN) models face several challenges, such as the requirements of extensive amounts of data and computational resources. Here, we introduce a new learning scheme, referred to as deep-and-wide learning (DWL), to systematically capture features not only within individual input data (intra-data features) but also across the data (inter-data features). Furthermore, we propose a dual-interactive-channel network (D-Net) to realize the DWL, which leverages our Bayesian formulation of low-dimensional (LD) inter-data feature extraction and its synergistic interaction with the conventional HD representation of the dataset, for substantially enhanced computational efficiency and inference. The proposed technique has been applied to data across various disciplines for both classification and regression tasks. Our results demonstrate that DWL surpasses state-of-the-art DNNs in accuracy by a substantial margin with limited training data and improves the computational efficiency by order(s) of magnitude. The proposed DWL strategy dramatically alters the data-driven learning techniques, including emerging large foundation models, and sheds significant insights into the evolving field of AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。