用极简提示模块让通用模型快速适配新任务,兼顾高效与泛化。
Convolutional Networks as Extremely Small Foundation Models: Visual Prompting and Theoretical Perspective
- 设计极简提示模块,基于学习理论优化结构以提升泛化能力。
- 在DAVIS数据集上实现无训练实时分割,性能媲美端到端模型。
- 适合追求轻量、快速部署的视觉任务开发者使用。
相较于针对特定任务训练的深度神经网络,基于通用数据集(如ImageNet分类)预训练的基础深度网络,得益于大规模数据、更简单的网络结构和更易训练的方法而表现出优势。本文设计了一种提示模块,实现对通用深度网络的少样本适应。受学习理论驱动,推导出尽可能简单的提示模块,因其在相同训练误差下具有更好的泛化性。通过视频对象分割的案例研究,提出一种具体提示模块——半参数深度森林(SDForest),结合相关滤波、随机森林、图像引导滤波等非参数方法,与ImageNet预训练深度网络融合。从学习理论视角看,这些模型的VC维或复杂度显著更低,因而更易泛化;实证表明,该简单集成模型在训练误差上可达到与端到端训练深度网络相当的效果。此外,本文还提出一种新的视频对象分割场景下的泛化分析方法,使泛化界更紧致。实践中,SDForest计算开销极低,在CPU上即可实现实时运行。在DAVIS2016和DAVIS2017数据集上,无需任何训练或微调即取得与纯深度学习方法竞争的性能。
原文摘要 · Abstract (English)
Comparing to deep neural networks trained for specific tasks, those foundational deep networks trained on generic datasets such as ImageNet classification, benefits from larger-scale datasets, simpler network structure and easier training techniques. In this paper, we design a prompting module which performs few-shot adaptation of generic deep networks to new tasks. Driven by learning theory, we derive prompting modules that are as simple as possible, as they generalize better under the same training error. We use a case study on video object segmentation to experiment. We give a concrete prompting module, the Semi-parametric Deep Forest (SDForest) that combines several nonparametric methods such as correlation filter, random forest, image-guided filter, with a deep network trained for ImageNet classification task. From a learning-theoretical point of view, all these models are of significantly smaller VC dimension or complexity so tend to generalize better, as long as the empirical studies show that the training error of this simple ensemble can achieve comparable results from a end-to-end trained deep network. We also propose a novel methods of analyzing the generalization under the setting of video object segmentation to make the bound tighter. In practice, SDForest has extremely low computation cost and achieves real-time even on CPU. We test on video object segmentation tasks and achieve competitive performance at DAVIS2016 and DAVIS2017 with purely deep learning approaches, without any training or fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。