仅用图片就能自动提取电商商品属性，省去手动输入描述的麻烦。
Visual Zero-Shot E-Commerce Product Attribute Value Extraction
- 基于CLIP框架，仅需图像输入，通过文本解码器生成属性值。
- 零样本场景下性能超越其他微调过的视觉语言模型。
- 结合OCR和提示式大模型，提升跨域属性识别准确率。
现有电商零样本商品属性值提取方法依赖单模态或双模态模型，要求卖家提供详细文本描述（如产品说明），但手动输入耗时且繁琐。为此，我们提出一种基于CLIP的跨模态零样本属性值生成框架ViOC-AG，仅需商品图像作为输入。ViOC-AG采用纯文本训练流程，使用冻结的CLIP文本编码器与定制化文本解码器联合训练，缓解模态差距与任务不匹配问题。在零样本推理阶段，冻结的CLIP图像编码器与训练好的文本解码器连接，生成商品属性；同时利用冻结的基于提示的LLM和OCR tokens对输出进行校正，以处理领域外属性值。实验表明，ViOC-AG在零样本属性值提取任务上显著优于其他微调过的视觉-语言模型。
原文摘要 · Abstract (English)
Existing zero-shot product attribute value (aspect) extraction approaches in e-Commerce industry rely on uni-modal or multi-modal models, where the sellers are asked to provide detailed textual inputs (product descriptions) for the products. However, manually providing (typing) the product descriptions is time-consuming and frustrating for the sellers. Thus, we propose a cross-modal zero-shot attribute value generation framework (ViOC-AG) based on CLIP, which only requires product images as the inputs. ViOC-AG follows a text-only training process, where a task-customized text decoder is trained with the frozen CLIP text encoder to alleviate the modality gap and task disconnection. During the zero-shot inference, product aspects are generated by the frozen CLIP image encoder connected with the trained task-customized text decoder. OCR tokens and outputs from a frozen prompt-based LLM correct the decoded outputs for out-of-domain attribute values. Experiments show that ViOC-AG significantly outperforms other fine-tuned vision-language models for zero-shot attribute value extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。