arXiv:2602.06136cs.LGcs.CV2026-02中稿 · ICML被引 1

提出时间约束下的TTA评估框架,揭示模型排名随时间压力变化的真相。

Tempora: Characterising the Time-Contingent Utility of Online Test-Time Adaptation

  • 构建三种时间场景下的评估指标,量化准确率与延迟的权衡。
  • 11种TTA方法在750+测试中排名不稳,无通用最优方案。
  • 适合关注实时推理性能的开发者和部署优化的研究者。

测试时自适应(TTA)能缓解模型在领域偏移下的性能下降,通过仅使用无标签数据实现实时泛化提升。然而,传统评估假设无限计算时间,忽视了准确率与延迟的权衡。随着机器学习广泛应用于对延迟敏感的场景,过晚的预测将失去意义。本文提出Tempora框架,包含模拟部署约束的时间场景、可操作的评估协议及时间相关的效用度量。我们设计三类度量:(1) 异步流中的离散效用(硬截止时间),(2) 交互式场景中的连续效用(价值随延迟衰减),(3) 预算受限部署的均摊效用。对11种TTA方法的750+次评估显示,不同时间压力下排名持续不稳定;最优方法依赖于具体偏移类型与时间约束,无全局最优解。Tempora首次实现跨多样化时间约束的系统评估,揭示排名变化机制,为实践者提供选型依据,为研究者指明可部署适配的目标。

原文摘要 · Abstract (English)

Test-time adaptation (TTA) offers a compelling remedy for machine learning (ML) models that degrade under domain shifts, improving generalisation on-the-fly with only unlabelled samples. This flexibility suits real deployments, yet conventional evaluations unrealistically assume unbounded processing time, overlooking the accuracy-latency trade-off. As ML increasingly underpins latency-sensitive and user-facing use-cases, temporal pressure constrains the viability of adaptable inference; predictions arriving too late to act on are futile. We introduce Tempora, a framework for evaluating TTA under this pressure. It consists of temporal scenarios that model deployment constraints, evaluation protocols that operationalise measurement, and time-contingent utility metrics that quantify the accuracy-latency trade-off. We instantiate the framework with three such metrics: (1) discrete utility for asynchronous streams with hard deadlines, (2) continuous utility for interactive settings where value decays with latency, and (3) amortised utility for budget-constrained deployments. By applying Tempora to 11 TTA methods, we find that rank instability persists across 750+ temporal evaluations spanning diverse datasets, models, and hardware platforms; i.e., conventional rankings do not predict rankings under temporal pressure. The highest-utility method varies with the shift and temporal pressure, with no clear winner. By enabling systematic evaluation across diverse temporal constraints for the first time, Tempora reveals when and why rankings change, offering practitioners a lens for method selection and researchers a target for deployable adaptation. Code: https://github.com/sudotensor/tempora.

测试时自适应延迟敏感评估框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。