用基础向量法检测图像状态变化,提升细粒度识别能力。
Basis Vector Metric: A Method for Robust Open-Ended State Change Detection
- 基于语言嵌入构建基础向量,判断图像状态变化。
- 在名词状态分类任务中表现优于多种对比方法。
- 适合需要细粒度图像理解的开放性场景研究者。
我们测试了一种新方法(简称BVM,基础向量法),通过语言嵌入来判断图像中的状态变化。实验使用包含约5.3万张图像的MIT-States数据集,该数据集包含225个名词和115个形容词,每个名词对应约9个形容词,共形成约1000个名词-形容词组合。第一阶段实验中,我们评估BVM在单独分类每个名词类别状态时的表现,并与余弦相似度、点积、产品量化、二值索引、朴素贝叶斯及自定义神经网络等方法对比。结果显示,BVM在名词状态分类上表现最佳。第二阶段实验中,我们测试BVM区分每个形容词之间差异的能力,并与MIT-States论文提出的逻辑回归模型进行比较。结果未显示BVM显著优于逻辑回归模型,但发现了改进方向,提示可通过方法调整进一步提升性能。
原文摘要 · Abstract (English)
We test a new method, which we will abbreviate using the acronym BVM (Basis Vectors Method), in its ability to judge the state changes in images through using language embeddings. We used the MIT-States dataset, containing about 53,000 images, to gather all of our data, which has 225 nouns and 115 adjectives, with each noun having about 9 different adjectives, forming approximately 1000 noun-adjective pairs. For our first experiment, we test our method's ability to determine the state of each noun class separately against other metrics for comparison. These metrics are cosine similarity, dot product, product quantization, binary index, Naive Bayes, and a custom neural network. Among these metrics, we found that our proposed BVM performs the best in classifying the states for each noun. We then perform a second experiment where we try using BVM to determine if it can differentiate adjectives from one another for each adjective separately. We compared the abilities of BVM to differentiate adjectives against the proposed method the MIT-States paper suggests: using a logistic regression model. In the end, we did not find conclusive evidence that our BVM metric could perform better than the logistic regression model at discerning adjectives. Yet, we were able to find evidence for possible improvements to our method; this leads to the chance of increasing our method's accuracy through certain changes in our methodologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。