用基础模型在测试时自动调整图像视角，提升视觉系统鲁棒性。
Test-Time Canonicalization by Foundation Models for Robust Perception
- 测试时通过优化选择最典型视角的图像，无需重训练或改架构。
- 在多种旋转、光照变化下，对CLIP和SAM等模型显著提升鲁棒性。
- 适合需要高适应性的实际部署场景，如自动驾驶、机器人视觉。
真实世界中的感知需应对多变的观察条件。现有方法常依赖特定架构或预定义数据增强，适应性受限。受人类视觉中心理旋转启发，我们提出FOCAL，一种测试时鲁棒性框架，将输入转换为最典型的视图。推理时，FOCAL探索一组变换后的图像，并选择在基础模型先验下似然最高的图像。该测试时优化提升了鲁棒性，且无需重新训练或修改架构。应用于CLIP和SAM等模型，在2D与3D旋转、对比度与光照变化、昼夜交替等多种变换下均显著增强性能。我们还探讨了其在主动视觉中的潜在应用。通过将不变性重定义为测试时优化问题，FOCAL提供了一种通用且可扩展的鲁棒性解决方案。代码已公开：https://github.com/sutkarsh/focal。
原文摘要 · Abstract (English)
Perception in the real world requires robustness to diverse viewing conditions. Existing approaches often rely on specialized architectures or training with predefined data augmentations, limiting adaptability. Taking inspiration from mental rotation in human vision, we propose FOCAL, a test-time robustness framework that transforms the input into the most typical view. At inference time, FOCAL explores a set of transformed images and chooses the one with the highest likelihood under foundation model priors. This test-time optimization boosts robustness while requiring no retraining or architectural changes. Applied to models like CLIP and SAM, it significantly boosts robustness across a wide range of transformations, including 2D and 3D rotations, contrast and lighting shifts, and day-night changes. We also explore potential applications in active vision. By reframing invariance as a test-time optimization problem, FOCAL offers a general and scalable approach to robustness. Our code is available at: https://github.com/sutkarsh/focal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。