通过解耦全局与局部处理，实现高效视觉模型。
Efficiency Follows Global-Local Decoupling
- 双分支结构分离全局上下文与局部细节处理。
- 计算复杂度低于二次方，比全全局注意力更省算力。
- 适合对效率和精度有平衡需求的视觉任务。
现代视觉模型需在保留局部细节的同时捕捉图像级上下文，且计算成本可控。本文重新审视这一权衡，提出一个简单原则：解耦全局推理与局部表征的作用。为此，我们引入ConvNeur——一种双分支架构，其中轻量级神经记忆分支在少量紧凑标记上聚合全局上下文，而保持局部性分支提取精细结构。一个可学习门控机制使全局线索调节局部特征，而不混淆其目标。该分离实现了图像尺寸的次二次方级缩放，保留了局部处理的归纳偏置，并降低了相对于全全局注意力的开销。在标准分类、检测和分割基准上，ConvNeur在相似或更低计算量下达到或超越同类方法，且在相似预算下展现出更优的准确率-延迟权衡。这些结果支持‘效率源于全局-局部解耦’的观点。
原文摘要 · Abstract (English)
Modern vision models must capture image-level context without sacrificing local detail while remaining computationally affordable. We revisit this tradeoff and advance a simple principle: decouple the roles of global reasoning and local representation. To operationalize this principle, we introduce ConvNeur, a two-branch architecture in which a lightweight neural memory branch aggregates global context on a compact set of tokens, and a locality-preserving branch extracts fine structure. A learned gate lets global cues modulate local features without entangling their objectives. This separation yields subquadratic scaling with image size, retains inductive priors associated with local processing, and reduces overhead relative to fully global attention. On standard classification, detection, and segmentation benchmarks, ConvNeur matches or surpasses comparable alternatives at similar or lower compute and offers favorable accuracy versus latency trade-offs at similar budgets. These results support the view that efficiency follows global-local decoupling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。