用检索方法解决电商商品属性值识别难题,支持海量数据高效部署。
TACLR: A Scalable and Efficient Retrieval-based Method for Industrial Product Attribute Value Identification
- 将属性值识别转为检索任务,结合分类树结构设计硬负样本
- 可处理数万属性、百万级数值,支持动态阈值推理
- 已在闲鱼平台日均处理百万商品,适合工业级应用
商品属性值识别(PAVI)是从商品信息中提取属性值的关键任务,对电商平台的搜索、推荐和商业分析至关重要。现有方法在隐含值推断、分布外值处理和输出归一化方面存在挑战。为此,本文提出基于检索的Taxonomy-Aware Contrastive Learning Retrieval(TACLR),首次将PAVI建模为信息检索问题:将商品描述与候选值编码为嵌入向量,通过相似度匹配进行检索。TACLR采用分类树感知的对比学习训练策略,引入分类树结构引导的硬负样本采样,并设计自适应推理机制,使用动态阈值提升泛化能力。该方法具备三大优势:(1)有效识别隐含值和分布外值,输出标准化结果;(2)可扩展至数千类别、十万级属性、百万级候选值;(3)支持高负载场景下的高效推理。在私有及公开数据集上的实验验证了其有效性与效率。同时,已在真实电商平台闲鱼成功部署,日均处理数百万商品,支持频繁更新的大规模属性分类体系。代码已开源,便于复现与后续研究。
原文摘要 · Abstract (English)
Product Attribute Value Identification (PAVI) involves identifying attribute values from product profiles, a key task for improving product search, recommendation, and business analytics on e-commerce platforms. However, existing PAVI methods face critical challenges, such as inferring implicit values, handling out-of-distribution (OOD) values, and producing normalized outputs. To address these limitations, we introduce Taxonomy-Aware Contrastive Learning Retrieval (TACLR), the first retrieval-based method for PAVI. TACLR formulates PAVI as an information retrieval task by encoding product profiles and candidate values into embeddings and retrieving values based on their similarity. It leverages contrastive training with taxonomy-aware hard negative sampling and employs adaptive inference with dynamic thresholds. TACLR offers three key advantages: (1) it effectively handles implicit and OOD values while producing normalized outputs; (2) it scales to thousands of categories, tens of thousands of attributes, and millions of values; and (3) it supports efficient inference for high-load industrial deployment. Extensive experiments on proprietary and public datasets validate the effectiveness and efficiency of TACLR. Further, it has been successfully deployed on the real-world e-commerce platform Xianyu, processing millions of product listings daily with frequently updated, large-scale attribute taxonomies. We release the code to facilitate reproducibility and future research at https://github.com/SuYindu/TACLR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。