用分布外检测思想提升视觉模型对变形输入的鲁棒性,无需重训练。
Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

- 将归一化问题转化为分布外检测,任选分布外评分函数
- 基于距离的评分与随机搜索组合效果最佳,跨多种数据集验证
- 引入门控机制,仅对偏离分布的输入进行变换,保护原精度
预训练视觉模型在面对旋转、缩放或剪切等仿射变换时容易误分类,尽管物体类别未变。现有方法通常通过构建等变结构或数据增强重训练来恢复鲁棒性,均需修改或重新训练模型。测试时归一化则保持分类器不变,通过将输入映射到接近训练分布的规范形式后再分类。现有归一化器依赖有限的基于逻辑值的能量评分和专用搜索流程,限制了评分函数与优化器的设计空间。本文将归一化重新定义为分布外(OOD)检测任务,使任意分布外评分均可作为变换空间中的最小化能量。我们在手写字符、草图、自然图像及3D点云等多种基准上系统评估了约20种分布外评分与9种搜索算法,发现基于距离的评分结合随机搜索与局部优化表现最优。由于对已对齐输入进行归一化可能损害准确率,我们引入门控机制,仅当分布外评分指示需要调整时才执行变换,从而在保持大部分分布内准确率的同时,显著提升对变换输入的鲁棒性。代码已公开于github.com/johschm/its。
原文摘要 · Abstract (English)
Pretrained vision models often misclassify inputs that are rotated, scaled, or sheared, even though these affine transformations leave the object class unchanged. Robustness is usually restored either by building equivariance into the architecture or by retraining with augmentation, both of which require changing or retraining the model. Test-time canonicalization instead leaves the classifier untouched. It undoes the transformation of each input, mapping it to a canonical form near the training distribution before classification. Existing canonicalizers, however, rely on a narrow set of logit-based energy scores and bespoke search procedures, leaving the design space of scoring functions and optimizers unexplored. We reframe canonicalization as out-of-distribution (OOD) detection, which lets any OOD score serve as the energy minimized over transformations. Across benchmarks ranging from handwritten characters and sketches to natural images and 3D point clouds, we systematically evaluate around twenty OOD scores and nine search algorithms, finding that distance-based scores paired with random search and local refinement perform best overall. Because canonicalizing an already-aligned input can hurt accuracy, we add a gated mechanism that transforms an input only when its OOD score indicates this is needed, preserving most in-distribution accuracy while retaining the robustness gains on transformed inputs. Code is available at github.com/johschm/its.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。