用更少数据和更低成本训练出高性能视觉模型,让顶尖视觉模型人人可用。
TuringViT: Making SOTA Vision Transformers Accessible to All

- 采用线性注意力与动态分辨率预训练,提升高分辨率建模效率。
- 仅用10%数据超越主流开源视觉模型,下游任务性能更强。
- 适合需要快速定制、高效部署的工业级视觉系统开发者。
当前视觉语言模型常采用现成的ViT作为视觉编码器,但不同下游任务对延迟、时序建模和集成需求各异,亟需定制化SOTA级ViT。然而,训练此类模型通常需海量图文数据,且标准softmax注意力在高分辨率或动态分辨率预训练中代价高昂,往往被迫低分辨率预训练再后处理适应。TuringViT通过三项设计解决:Turing线性注意力(TLA)实现高效序列建模,VISTA-Curation构建高质量图文视频训练数据,原生支持动态分辨率预训练,从一开始就兼容灵活输入并可无缝迁移至下游VLM。结果表明,TuringViT仅用10%数据即超越领先开源ViT基线,下游VLM性能更优,且在高分辨率输入下延迟表现显著提升。规模定律分析显示,其性能随数据量持续提升,远未饱和。其快速适应性、硬件友好设计与高效部署能力已使其成为小鹏智能系统中的统一视觉基础。总体而言,TuringViT提供可复现的训练管道,大幅降低社区训练、定制与部署顶级ViT的成本,推动高性能视觉模型真正面向所有人开放。
原文摘要 · Abstract (English)
Modern VLMs and VLA systems commonly adopt off-the-shelf ViTs such as SigLIP2 as visual encoders, but diverse downstream requirements in latency, temporal modeling, and VLM integration often call for customized SOTA-level ViTs. Training such encoders remains beyond the reach of much of the community, as it requires massive image-text data, while standard softmax attention makes high-resolution or dynamic-resolution pretraining prohibitively costly and often forces low-resolution pretraining followed by post-hoc adaptation. TuringViT addresses these challenges with three key designs: Turing Linear Attention (TLA) for efficient sequence modeling, VISTA-Curation to construct supervision-rich image-video training data, and native dynamic-resolution pretraining that supports flexible inputs from the start and transfers seamlessly to downstream VLMs. As a result, TuringViT outperforms leading open-source ViT baselines with only 10% of the data, achieves stronger downstream VLM performance, and delivers substantially better latency scaling on high-resolution inputs. Our scaling-law analysis further shows that TuringViT continues to improve predictably with curated data scale, far from saturation. Its fast adaptation, hardware-friendly design, and efficient deployment have made it a unified visual foundation across XPeng's AI systems. More broadly, TuringViT provides a reproducible pipeline that dramatically lowers the cost for the community to train, customize, and deploy SOTA-level ViTs, moving toward making such Vision Transformers accessible to all.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。