利用邻近无标签数据一致性,提升大模型推理效果。
TestNUC: Enhancing Test-Time Computing Approaches and Scaling through Neighboring Unlabeled Data Consistency
- 通过分析输入样本及其邻近无标签数据的一致性改进预测。
- 在8个数据集上均优于标准提示和自一致性方法。
- 可无缝集成现有推理优化方法,适合实际部署场景。
测试时计算方法通过推理阶段增加计算资源,已被证明能有效提升大语言模型性能。本文提出一种线性可扩展的新方法TestNUC,通过利用邻近无标签数据的局部一致性来改进测试时预测:对一个输入实例的分类不仅基于模型对其自身预测,还考虑其邻近无标签实例的预测结果。我们在涵盖意图识别、主题挖掘、领域发现和情感检测等八种不同任务的数据集上评估TestNUC,结果表明其持续优于标准提示和自一致性等基线方法。此外,TestNUC可无缝集成至现有测试时计算方法中,显著提升其性能。分析显示,TestNUC在无标签数据增多时表现良好,且对不同嵌入模型具有鲁棒性,适用于真实场景。代码已开源:https://github.com/HenryPengZou/TestNUC。
原文摘要 · Abstract (English)
Test-time computing approaches, which leverage additional computational resources during inference, have been proven effective in enhancing large language model performance. This work introduces a novel, linearly scaling approach, TestNUC, that improves test-time predictions by leveraging the local consistency of neighboring unlabeled data-it classifies an input instance by considering not only the model's prediction on that instance but also on neighboring unlabeled instances. We evaluate TestNUC across eight diverse datasets, spanning intent classification, topic mining, domain discovery, and emotion detection, demonstrating its consistent superiority over baseline methods such as standard prompting and self-consistency. Furthermore, TestNUC can be seamlessly integrated with existing test-time computing approaches, substantially boosting their performance. Our analysis reveals that TestNUC scales effectively with increasing amounts of unlabeled data and performs robustly across different embedding models, making it practical for real-world applications. Our code is available at https://github.com/HenryPengZou/TestNUC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。