arXiv:2507.19679cs.CVcs.AI2025-07ICCV被引 1

用少量标注数据+大量无标签商品图,高效训练小模型预测商品属性。

Efficient Learning for Product Attributes with Compact Multimodal Models

  • 通过低秩适配器与自洽性筛选,用无标签数据生成偏好对优化模型。
  • 仅用无标签数据训练,准确率超越全监督模型,且数据越多效果越好。
  • 适合资源有限但有海量无标签电商数据的场景,如中小企业应用。

电商平台基于图像的商品属性预测至关重要。传统视觉语言模型(VLM)的监督微调因标注成本高而面临规模挑战。本文研究针对小型VLM(2B-3B参数)的标签高效半监督微调策略,利用直接偏好优化(DPO)融合无标签商品信息。初始使用少量API标注数据,通过参数高效微调(PEFT)训练低秩适配器;对每个无标签样本生成多个推理-回答链,依据自洽性分为偏好与非偏好组;再以DPO损失更新适配器权重,并迭代推进。该方法在覆盖十二个电商垂直领域的数据集上,仅用无标签数据的DPO微调显著优于监督模型。实验表明,随着无标签数据增多,准确率持续提升,证明大规模无标签数据可有效驱动性能改进。

原文摘要 · Abstract (English)

Image-based product attribute prediction in e-commerce is a crucial task with numerous applications. The supervised fine-tuning of Vision Language Models (VLMs) faces significant scale challenges due to the cost of manual or API based annotation. In this paper, we investigate label-efficient semi-supervised fine-tuning strategies for compact VLMs (2B-3B parameters) that leverage unlabeled product listings through Direct Preference Optimization (DPO). Beginning with a small, API-based, annotated, and labeled set, we first employ PEFT to train low-rank adapter modules. To update the adapter weights with unlabeled data, we generate multiple reasoning-and-answer chains per unlabeled sample and segregate these chains into preferred and dispreferred based on self-consistency. We then fine-tune the model with DPO loss and use the updated model for the next iteration. By using PEFT fine-tuning with DPO, our method achieves efficient convergence with minimal compute overhead. On a dataset spanning twelve e-commerce verticals, DPO-based fine-tuning, which utilizes only unlabeled data, demonstrates a significant improvement over the supervised model. Moreover, experiments demonstrate that accuracy with DPO training improves with more unlabeled data, indicating that a large pool of unlabeled samples can be effectively leveraged to improve performance.

商品属性小模型半监督DPO

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。