用图文双扩散模型生成可解释的时尚推荐结果
Dual-Diffusional Generative Fashion Recommendation

- 图文双分支扩散架构联合建模用户偏好
- 生成商品图与描述,支持视觉与语义双重对齐
- 轻量微调提升多样性,适合需要解释性的推荐场景
个性化生成式推荐系统在时尚推荐中展现出潜力。然而,现有方法主要依赖历史交互中的隐式视觉嵌入,常包含无关偏好信息,导致用户行为建模不足,且通常仅生成物品图像,可解释性有限。为此,我们提出 DualFashion——一种双扩散生成式时尚推荐架构,通过图像与文本双分支联合建模,利用结构化属性级标题和视觉穿搭信息作为条件信号,精准捕捉用户行为。该架构同时生成时尚单品图像与文本描述,确保视觉一致性并提供明确语义解释。此外,引入文本增强微调策略,在不增加大量计算成本的前提下提升生成多样性并实现有效跨模态知识迁移。在 iFashion 与 Polyvore-U 数据集上的广泛实验表明,DualFashion 在行为建模、可解释性与效率方面均优于当前最优方法。代码与模型权重已公开于 https://github.com/LinkMingzhe/DualFashion。
原文摘要 · Abstract (English)
Personalized generative recommender systems have emerged as a promising solution for fashion recommendation. However, existing methods primarily rely on implicit visual embeddings from historical interactions, which often contain preference-irrelevant information and result in insufficient user behavior modeling. Moreover, these models typically generate only item images, providing limited interpretability. To address these limitations, we propose DualFashion, a Dual-Diffusional Generative Fashion Recommendation Architecture that jointly models image and text modalities for personalized and explainable recommendation. DualFashion adopts a dual-diffusion Transformer with image and text branches, where structured attribute-level captions and visual outfit information are jointly used as conditioning signals to model user behavior. The proposed architecture produces both fashion item images and textual descriptions, ensuring visual compatibility while providing explicit semantic interpretability. Furthermore, we introduce a text-augmented fine-tuning strategy that enhances generation diversity and enables effective cross-modal knowledge transfer without incurring heavy computational costs. Extensive experiments on iFashion and Polyvore-U across Personalized Fill-in-the-Blank and Generative Outfit Recommendation tasks demonstrate that DualFashion achieves strong performance in behavior modeling, interpretability, and efficiency compared to state-of-the-art methods. Our code and model checkpoints are available at https://github.com/LinkMingzhe/DualFashion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。