arXiv:2510.21835cs.LGcs.AI2025-10

用一张图自动生成电商图文信息,更准且更快。

A Multimodal, Multitask System for Generating E Commerce Text Listings from Images

  • 多任务联合训练视觉编码器,同时预测颜色、价格等属性
  • 通过分层生成减少事实错误,幻觉率降44.5%
  • 适合需要高准确率电商内容生成的商家和平台

手动撰写吸引人的商品名称和描述对零售商而言既费时又低效。尽管生成式AI通过视觉语言模型(VLM)提供了自动化方案,但现有VLM易产生事实性“幻觉”。孤立的单任务模型不仅效率低,还难以捕捉特征间的依赖关系。为此,我们提出一个端到端多任务系统,仅凭一张图像生成事实可靠的文本列表。研究贡献包括两项:第一,采用多任务学习微调视觉编码器,单一视觉主干联合训练颜色、下摆、领型等属性预测及价格回归;第二,引入分层生成机制,将模型自身预测的属性嵌入提示词输入文本解码器,提升事实一致性。实验表明该架构优越:多任务方法在价格回归上比独立模型提升3.6% R²,在属性分类上提升6.6% F1得分;分层生成使幻觉率从12.7%降至7.1%,相对降低44.5%;同时将自回归文本生成延迟降低3.5倍,优于同规模直接视觉-语言模型。唯一不足是ROUGE-L得分比直接模型低3.5%。

原文摘要 · Abstract (English)

Manually generating catchy descriptions and names is labor intensive and a slow process for retailers. Although generative AI provides an automation solution in form of Vision to Language Models (VLM), the current VLMs are prone to factual "hallucinations". Siloed, single task models are not only inefficient but also fail to capture interdependent relationships between features. To address these challenges, we propose an end to end, multi task system that generates factually grounded textual listings from a single image. The contributions of this study are two proposals for the model architecture. First, application of multi task learning approach for fine tuning a vision encoder where a single vision backbone is jointly trained on attribute prediction such as color, hemline and neck style and price regression. Second, introduction of a hierarchical generation process where the model's own predicted attributes are embedded in a prompt and fed to the text decoder to improve factual consistency. The experiments demonstrate the superiority of this architecture. The multi tasking approach outperforms both the independent price regression, with a 3.6% better R2 Value and attribute classification, with a 6.6% improvement F1 score. Critically, the hierarchical generation process proves highly effective, slashing the factual hallucination rate from 12.7% to 7.1%, a 44.5% relative reduction, compared to a non hierarchical ablation. The hierarchical approach also reduces the latency of the autoregressive text generation process by a factor of 3.5 when compared to direct vision to language model of similar size. One minor caveat is that the model does perform 3.5% worse than direct vision-to-language model on ROUGE-L score.

多模态电商生成视觉语言事实一致

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。