用单次前向计算快速生成权重，实现高效测试时适应。
FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation
- 通过闭式公式将标签数据编译为快速权重，全程仅需前向传播。
- 适配时间减少90%以上，内存占用降低95%，性能媲美传统方法。
- 适合资源受限场景，尤其适用于需要快速适应的任务。
在微调预训练模型时，通常面临反向传播成本高与基于记忆或上下文学习推理开销大的权衡。本文提出 FAAST，一种仅依赖前向传播的关联式适配方法，可单次遍历将带标签样本解析为快速权重。通过消除对记忆或上下文的依赖，FAAST 实现恒定时间推理，并将任务适配与预训练表示解耦。在图像分类与语言建模基准上，FAAST 的性能达到或超过基于反向传播的适配方法，且适配时间降低超90%，内存占用减少达95%，同时在效率上可比肩基于记忆/上下文的方法。结果表明，FAAST 是一种高效、可扩展的监督任务适配方案，特别适用于资源受限模型。代码与模型已开源：https://github.com/baoguangsheng/faast。
原文摘要 · Abstract (English)
Adapting pretrained models typically involves a trade-off between the high training costs of backpropagation and the heavy inference overhead of memory-based or in-context learning. We propose FAAST, a forward-only associative adaptation method that analytically compiles labeled examples into fast weights in a single pass. By eliminating memory or context dependence, FAAST achieves constant-time inference and decouples task adaptation from pretrained representation. Across image classification and language modeling benchmarks, FAAST matches or exceeds backprop-based adaptation while reducing adaptation time by over 90% and is competitive to memory/context-based adaptation while saving memory usage by up to 95%. These results demonstrate FAAST as a highly efficient, scalable solution for supervised task adaptation, particularly for resource-constrained models. We release the code and models at https://github.com/baoguangsheng/faast.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。