用大模型判断物体属性是否合理，提升开放世界零样本学习效果
Feasibility with Language Models for Open-World Compositional Zero-Shot Learning
- 通过大模型评估状态-物体组合的合理性，引入语义可行性判断
- 在三个基准上均显著提升开放世界零样本学习性能
- 适合研究零样本学习、大模型应用与常识推理的学者参考
人类能轻易判断某个属性（也称状态）对某物体是否合理，例如火可以热，但不能湿。在开放世界组合零样本学习中，当所有可能的状态-物体组合被视为未见类别时，零样本预测器表现较差。本文聚焦于利用外部辅助知识判断状态-物体组合的可行性。提出的语言模型可行性方法（FLM）是一种简单而有效的方法，借助大语言模型（LLMs）更好地理解状态与物体之间的语义关系。FLM通过向LLM查询给定组合的可行性并获取正向回答的输出logit值实现。为缓解大模型在罕见或完全不合理组合上的误导问题，我们发现上下文学习能力至关重要。实验表明，Vicuna和ChatGPT表现最佳，且FLM在所有三个基准上均持续提升开放世界组合零样本学习性能。
原文摘要 · Abstract (English)
Humans can easily tell if an attribute (also called state) is realistic, i.e., feasible, for an object, e.g. fire can be hot, but it cannot be wet. In Open-World Compositional Zero-Shot Learning, when all possible state-object combinations are considered as unseen classes, zero-shot predictors tend to perform poorly. Our work focuses on using external auxiliary knowledge to determine the feasibility of state-object combinations. Our Feasibility with Language Model (FLM) is a simple and effective approach that leverages Large Language Models (LLMs) to better comprehend the semantic relationships between states and objects. FLM involves querying an LLM about the feasibility of a given pair and retrieving the output logit for the positive answer. To mitigate potential misguidance of the LLM given that many of the state-object compositions are rare or completely infeasible, we observe that the in-context learning ability of LLMs is essential. We present an extensive study identifying Vicuna and ChatGPT as best performing, and we demonstrate that our FLM consistently improves OW-CZSL performance across all three benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。