用LoRA微调的Florence-2模型，从服装图自动提取结构化属性。
Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction
- 基于规则工程压缩标签，用LoRA微调生成结构化输出
- 类别准确率94.6%，材质准确率63.0%，风格F1达0.753
- 输出为合法JSON，适合推荐系统直接使用
我们提出Fashion Florence，一个在iMaterialist Fashion数据集（228个标签）上通过规则工程简化为6类、16色、19风格的紧凑标签体系后，用LoRA（r=16, alpha=32）微调的Florence-2视觉语言模型。在3,688个样本上训练3轮，对461张测试图像，该模型实现94.6%类别准确率和63.0%材质准确率，显著优于GPT-4o-mini（89.3%/43.3%）与Gemini 2.5 Flash（87.4%）。风格标签F1达0.753，高于Gemini（0.612）与GPT-4o-mini（0.398）。99.8%输出为有效JSON，仅需0.77B参数，在单卡上零额外推理开销。模型已部署于Hugging Face Space，并集成至开源穿搭推荐系统Loom。
原文摘要 · Abstract (English)
We present Fashion Florence, a Florence-2 vision-language model fine-tuned with LoRA to extract structured fashion attributes from clothing images. Given a single photograph, the model generates a JSON object containing category, color, material, style tags, and occasion tags, structured output suitable for direct programmatic consumption by downstream recommendation and retrieval systems. Fine-tuning data is derived from the iMaterialist Fashion dataset (228 labels), where we collapse fine-grained annotations into a compact 6-category, 16-color, 19-style schema via rule-based label engineering. We apply LoRA (r=16, alpha=32) to all decoder linear layers, training for 3 epochs on 3,688 examples. On a held-out test set of 461 images, Fashion Florence achieves 94.6% category accuracy and 63.0% material accuracy, compared to 89.3% / 43.3% for GPT-4o-mini and 87.4% for Gemini 2.5 Flash. Fashion Florence produces valid JSON in 99.8% of outputs while running at 0.77B parameters on a single GPU at zero marginal inference cost. Style tag F1 reaches 0.753 vs. 0.612 (Gemini) and 0.398 (GPT-4o-mini). The model is deployed as a Hugging Face Space and integrated into Loom, an open-source outfit recommendation system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。