arXiv:2511.19149cs.CVcs.AI2025-11被引 1

用检索增强生成时尚图文,让描述更准更自然。

From Pixels to Posts: Retrieval-Augmented Fashion Captioning and Hashtag Generation

  • 先检测衣服再查库找风格,用事实证据引导大模型写文案。
  • 属性覆盖率达80%,标签生成在50%阈值下全覆盖。
  • 适合想做精准时尚内容生成的开发者和设计师。

本文提出一种检索增强的自动时尚图文生成框架,结合多衣物检测、属性推理与大语言模型提示技术,旨在生成视觉真实、描述准确且风格有趣的时尚文本。该系统通过基于YOLO的检测器定位九类服饰,k-means聚类提取主色调,并利用CLIP-FAISS检索模块从结构化商品库中推断面料与性别属性。这些属性与检索到的风格样例共同构成事实证据包,用于引导大模型生成拟人化标题与上下文丰富的标签。以微调后的BLIP模型为基线进行对比,实验表明:YOLO检测器在九类服饰上[email protected]达0.71;该检索增强管道生成的描述具备更高属性一致性,标签生成在50%阈值下实现全覆盖,属性覆盖率均值达0.80;相比基线模型,其词汇重叠更高但泛化能力更强。结果验证了检索增强生成在事实准确性、抗幻觉及可扩展性方面的优势,展现出在多样化服装领域部署的巨大潜力。

原文摘要 · Abstract (English)

This paper introduces the retrieval-augmented framework for automatic fashion caption and hashtag generation, combining multi-garment detection, attribute reasoning, and Large Language Model (LLM) prompting. The system aims to produce visually grounded, descriptive, and stylistically interesting text for fashion imagery, overcoming the limitations of end-to-end captioners that have problems with attribute fidelity and domain generalization. The pipeline combines a YOLO-based detector for multi-garment localization, k-means clustering for dominant color extraction, and a CLIP-FAISS retrieval module for fabric and gender attribute inference based on a structured product index. These attributes, together with retrieved style examples, create a factual evidence pack that is used to guide an LLM to generate human-like captions and contextually rich hashtags. A fine-tuned BLIP model is used as a supervised baseline model for comparison. Experimental results show that the YOLO detector is able to obtain a mean Average Precision ([email protected]) of 0.71 for nine categories of garments. The RAG-LLM pipeline generates expressive attribute-aligned captions and achieves mean attribute coverage of 0.80 with full coverage at the 50% threshold in hashtag generation, whereas BLIP gives higher lexical overlap and lower generalization. The retrieval-augmented approach exhibits better factual grounding, less hallucination, and great potential for scalable deployment in various clothing domains. These results demonstrate the use of retrieval-augmented generation as an effective and interpretable paradigm for automated and visually grounded fashion content generation.

时尚生成检索增强图文生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。