用简单变换提升模型在分布外数据下的适应能力
Context is All You Need
- 通过加法与乘法变换调节神经特征，实现上下文自适应
- 在分类与生成任务中均显著提升域外泛化性能
- 轻量易集成，适合部署时快速增强模型鲁棒性
人工神经网络（ANN）在真实场景中常面临训练与测试数据分布不一致的问题。这一挑战是领域泛化（DG）和测试时自适应（TTA）的核心目标。现有方法通常复杂且资源消耗大。本文提出CONTXT（Contextual augmentation for Neural feature X Transforms），一种简洁直观的上下文自适应方法。它通过简单的加法与乘法特征变换来调节内部表示。在TTA设置下，CONTXT在判别任务（如ANN/CNN分类）和生成模型（如LLMs）中均取得稳定提升。该方法轻量、易集成，计算开销极小，可在不增加复杂度的前提下实现强域偏移鲁棒性。更广泛而言，CONTXT提供了一种无需重训练即可调控信息流与神经处理的紧凑方案。
原文摘要 · Abstract (English)
Artificial Neural Networks (ANNs) are increasingly deployed across diverse real-world settings, where they must operate under data distributions that differ from those seen during training. This challenge is central to Domain Generalization (DG), which trains models to generalize to unseen domains without target data, and Test-Time Adaptation (TTA), which improves robustness by adapting to unlabeled test data at deployment. Existing approaches to address these challenges are often complex, resource-intensive, and difficult to scale. We introduce CONTXT (Contextual augmentatiOn for Neural feaTure X Transforms), a simple and intuitive method for contextual adaptation. CONTXT modulates internal representations using simple additive and multiplicative feature transforms. Within a TTA setting, it yields consistent gains across discriminative tasks (e.g., ANN/CNN classification) and generative models (e.g., LLMs). The method is lightweight, easy to integrate, and incurs minimal overhead, enabling robust performance under domain shift without added complexity. More broadly, CONTXT provides a compact way to steer information flow and neural processing without retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。