小数据也能练出好模型：设计领域无需依赖大样本预训练。
Rethinking Pretraining for Specialized Design Data: Evidence from the JONES-19 Cultural Design Dataset
- 用小规模高质量设计数据从零训练，效果接近大模型预训练。
- 重复局部采样（多裁剪）可显著提升小数据模型性能。
- 适合关注设计领域、数据量有限的研究者参考。
设计与建筑档案以图形形式记录了专家人类知识，为缺乏典型计算机视觉基准的设计驱动机器学习挑战提供了关键测试平台。基于源自《装饰语法》（伦敦，1857年）的小规模图像数据集JONES-19，我们评估了卷积神经网络（CNNs）在两种训练策略下的判别性能：(a) 在ImageNet上进行预训练以获取通用视觉常识；(b) 直接在JONES-19设计数据上从零训练。结果表明，尽管通用先验知识能提升判别性能，但通过重复局部采样（多裁剪）从零训练同样能有效恢复这些增益。对于高度结构化的设计数据，局部设计驱动表示已足以提供学习基础，挑战了对大规模通用预训练的依赖。这表明，在特定设计领域中，精心构建的小规模高质量数据集，若能捕捉经验与形式设计原则，可能比盲目追求大规模数据收集更有效、更具信息量。
原文摘要 · Abstract (English)
Design and architectural archives encode expert human knowledge in graphical formats, providing a critical testbed for design-inspired Machine Learning (ML) challenges absent with typical computer vision benchmarks. Building on JONES-19, a small-size image dataset based on The Grammar of Ornament (London, 1857), we evaluate the discriminative performance of Convolutional Neural Networks (CNNs) in two model training strategies: (a) ImageNet pretraining for domain-general "visual common sense," and (b) learning from scratch on the design data in JONES-19. We find that while domain-general priors improve discriminative performance, learning from scratch augmented with repeated local sampling (multi-crop) effectively recovers these gains. For highly structured design data, local design-driven representations provide sufficient foundation for learning, challenging a reliance on massive general-purpose pretraining. These findings suggest that in specialized design domains, careful curation of smaller high-quality datasets that capture empirical and formal design principles may prove more effective and informative on the nature of a particular design domain than prioritizing large-scale data collection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。