通过上下文学习增强可信度，让模型更准预测新情况。
In-Context Learning Enhanced Credibility Transformer
- 用相似样本组成的上下文批量增强输入表示
- 在风险模式上提升准确率，支持新类别泛化
- 适合处理训练未见的新类型数据，如新车型号
本文在可信度Transformer基础上引入上下文学习机制，通过引入包含相似实例的上下文批次，增强模型对输入实例的CLS token表示。该方法使模型能利用额外上下文信息进行微调，从而提升预测性能。实验表明，该机制可有效适应相似风险模式，提高预测准确性。更重要的是，它使模型能够泛化到训练中未出现的新实例，例如汽车制造商新推出的车型（其类别特征在训练时未出现）。该方法显著提升了模型对未知类别的适应能力。
原文摘要 · Abstract (English)
The starting point of our network architecture is the Credibility Transformer which extends the classical Transformer architecture by a credibility mechanism to improve model learning and predictive performance. This Credibility Transformer learns credibilitized CLS tokens that serve as learned representations of the original input features. In this paper we present a new paradigm that augments this architecture by an in-context learning mechanism, i.e., we increase the information set by a context batch consisting of similar instances. This allows the model to enhance the CLS token representations of the instances by additional in-context information and fine-tuning. We empirically verify that this in-context learning enhances predictive accuracy by adapting to similar risk patterns. Moreover, this in-context learning also allows the model to generalize to new instances which, e.g., have feature levels in the categorical covariates that have not been present when the model was trained -- for a relevant example, think of a new vehicle model which has just been developed by a car manufacturer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。