让自回归图像模型保持原生长宽比，提升视觉理解能力
NARAIM: Native Aspect Ratio Autoregressive Image Models
- 模型在原始长宽比图像上训练，不裁剪也不拉伸
- 下游分类任务准确率显著优于方形图像训练模型
- 适合需要精确空间信息的视觉任务如医学影像分析
尽管视觉变换器能解决多种计算机视觉任务，但尚未有预训练方法展现出与语言模型相同的扩展规律。自回归模型虽具潜力，但通常在被裁剪或转为正方形的图像上训练，导致输入信息失真或丢失。为此，我们提出NARAIM，一种基于自回归目标、使用原生长宽比图像进行预训练的视觉模型。通过保留原生长宽比，我们维持了原始空间上下文，从而增强模型对视觉信息的解析能力。实验表明，保持长宽比可显著提升下游分类任务性能。
原文摘要 · Abstract (English)
While vision transformers are able to solve a wide variety of computer vision tasks, no pre-training method has yet demonstrated the same scaling laws as observed in language models. Autoregressive models show promising results, but are commonly trained on images that are cropped or transformed into square images, which distorts or destroys information present in the input. To overcome this limitation, we propose NARAIM, a vision model pre-trained with an autoregressive objective that uses images in their native aspect ratio. By maintaining the native aspect ratio, we preserve the original spatial context, thereby enhancing the model's ability to interpret visual information. In our experiments, we show that maintaining the aspect ratio improves performance on a downstream classification task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。