无需预训练即可识别分布外图像的高效测试时训练框架
Asynchronous Perception Machine For Efficient Test-Time-Training
- 异步处理图像块，单次蒸馏即可学习语义特征
- 在无预训练情况下实现分布外检测，性能媲美现有方法
- 适用于图像聚类与感知-插值统一硬件实现
本文提出异步感知机（APM），一种计算高效的测试时训练（TTT）架构。APM 可任意顺序异步处理图像块，仍能保持网络对语义信息的感知能力。实验表明，APM 在无需数据集特定预训练、增强或预训练任务的情况下，即可识别分布外图像。其仅需一次测试样本表示蒸馏即可完成学习，并立即开始预测语义感知特征。此外，APM 能在单次前向传播中对2D图像数据集进行语义聚类，展现出超越TTT的应用潜力。该模型还首次为GLOM理论提供实证支持：输入感知是一个场。因此，APM推动了在共享连接主义硬件上同时实现插值与感知的可行性。代码已公开。
原文摘要 · Abstract (English)
In this work, we propose Asynchronous Perception Machine (APM), a computationally-efficient architecture for test-time-training (TTT). APM can process patches of an image one at a time in any order asymmetrically and still encode semantic-awareness in the net. We demonstrate APM's ability to recognize out-of-distribution images without dataset-specific pre-training, augmentation or any-pretext task. APM offers competitive performance over existing TTT approaches. To perform TTT, APM just distills test sample's representation once. APM possesses a unique property: it can learn using just this single representation and starts predicting semantically-aware features. APM demostrates potential applications beyond test-time-training: APM can scale up to a dataset of 2D images and yield semantic-clusterings in a single forward pass. APM also provides first empirical evidence towards validating GLOM's insight, i.e. input percept is a field. Therefore, APM helps us converge towards an implementation which can do both interpolation and perception on a shared-connectionist hardware. Our code is publicly available at this link: https://rajatmodi62.github.io/apm_project_page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。