用ImageBind融合图文生成联合嵌入，提升二手车信息检索效果
From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space
- 将图像与文本嵌入简单融合，生成跨模态联合表示
- 通过聚类分析验证联合嵌入具备语义区分能力
- 发现纯音频嵌入也能匹配相关商品，暗示新研究方向
本研究探讨ImageBind在在线汽车零部件发布信息中生成有意义的多模态融合嵌入的能力。我们提出一种简化的嵌入融合流程，旨在捕捉图像/文本对之间的重叠信息，最终将帖子语义整合为联合嵌入。将这些融合嵌入存入向量数据库后，我们进行降维处理，并通过聚类及分析靠近聚类中心的帖子，实证验证了联合嵌入的语义质量。此外，初步结果显示，ImageBind的零样本跨模态检索能力中，纯音频嵌入能与语义相似的市场列表相关联，表明未来研究的新路径。
原文摘要 · Abstract (English)
This study investigates ImageBind's ability to generate meaningful fused multimodal embeddings for online auto parts listings. We propose a simplistic embedding fusion workflow that aims to capture the overlapping information of image/text pairs, ultimately combining the semantics of a post into a joint embedding. After storing such fused embeddings in a vector database, we experiment with dimensionality reduction and provide empirical evidence to convey the semantic quality of the joint embeddings by clustering and examining the posts nearest to each cluster centroid. Additionally, our initial findings with ImageBind's emergent zero-shot cross-modal retrieval suggest that pure audio embeddings can correlate with semantically similar marketplace listings, indicating potential avenues for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。