仅用两次前向传播实现模型实时自适应,适合边缘设备部署。
EVA-0: Test-Time Model Evolution with Only Two Forward Passes per Sample

- 零阶优化避免反向传播,仅需两次前向计算
- 在ImageNet-C上性能超越依赖反向传播的方法
- 适合资源受限场景,如量化模型和黑盒系统
测试时模型演化为已部署模型提供了从无标签测试数据中持续改进的潜力,但现有方法多依赖反向传播(BP),带来高昂内存开销,难以在边缘设备、量化模型、专用加速器或黑盒模型上部署。本文研究在严格双前向预算下的测试时演化,推动适应机制向高效真实部署演进。揭示了零阶测试时优化的三大障碍:易陷入捷径解、权重漂移失控、更新方向估计无效。为此提出EVA-0,一种极简零阶自适应框架:1)保持损失尺度不变以防止捷径解;2)设计锚点引导策略缓解权重漂移;3)采用样本级对称双向扰动进行更新方向估计与推理。EVA-0无需反向传播,每样本仅需两次前向传播完成推理与自适应。在ImageNet-C与ViT-Base上的实验表明,EVA-0优于基于BP的DeYO和无BP的FOA,在相同条件下比FOA快14倍。代码将公开。
原文摘要 · Abstract (English)
Test-time model evolution offers a promising way for deployed models to improve from unlabeled test-time experience, yet most existing methods depend on backpropagation (BP), which incurs substantial memory overhead and makes them difficult to deploy on edge devices, quantized models, specialized accelerators, or black-box models. In this work, we study test-time model evolution under a strict two-forward budget, a setting that pushes adaptation toward highly efficient real-world deployment. We reveal three key obstacles in zeroth-order test-time optimization: susceptibility to shortcut solutions, uncontrolled weight drift, and ineffective update direction estimation. To overcome them, we propose EVA-0, a minimal zeroth-order adaptation framework that: 1) keeps the loss scale-invariant to prevent shortcut solutions; 2) devises an anchor-guided optimization strategy to alleviate weight drift; 3) uses sample-wise symmetric two-sided perturbation for update direction estimation and inference. EVA-0 requires no BP and performs both inference and adaptation within only two forward passes per sample. Results on ImageNet-C & ViT-Base show that EVA-0 outperforms both BP-based DeYO and BP-free FOA, while achieving a 14x speed-up over FOA. Code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。