研究视觉Transformer在图像上进行上下文学习的能力，探索其对复杂函数的泛化潜力。
In Context Learning with Vision Transformers: Case Study
- 用图像数据测试Vision Transformer的上下文学习能力
- 发现模型能学习卷积神经网络等复杂函数
- 为理解大模型图像推理机制提供新视角
大型Transformer模型已证明具备上下文学习能力。通过在提示中使用示例和查询，它们能够完成少样本、单样本或零样本学习任务，输出对应答案。现有研究表明，这些模型可在随机数据上学习线性函数和小型两层神经网络等通用函数（Garg et al, 2023）。本文旨在将该能力扩展至图像空间，分析Vision Transformer在图像数据上对更复杂函数（如卷积神经网络及其他方法）的上下文学习能力。
原文摘要 · Abstract (English)
Large transformer models have been shown to be capable of performing in-context learning. By using examples in a prompt as well as a query, they are capable of performing tasks such as few-shot, one-shot, or zero-shot learning to output the corresponding answer to this query. One area of interest to us is that these transformer models have been shown to be capable of learning the general class of certain functions, such as linear functions and small 2-layer neural networks, on random data (Garg et al, 2023). We aim to extend this to the image space to analyze their capability to in-context learn more complex functions on the image space, such as convolutional neural networks and other methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。