arXiv:2608.12570cs.CVcs.IR2026-08

将服装检索的多属性解耦为可独立控制的四个槽位,提升精准度。

Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval

论文配图:Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval
图 1 · 摘自论文原文
  • 设计多模态槽编码器,将文本图像嵌入分解为颜色、图案等四类可调控属性槽。
  • 在H&M数据集上,颜色属性召回率提升至0.889,绝对增益达0.568。
  • 学习到的门控机制可解释,适合需要精细属性控制的场景。

服装检索常需同时满足类别、颜色、图案和人口统计等多重属性。现有方法将这些信号混合于单一向量中,难以在检索时实现属性控制。许多语义ID方法提供离散物品编码,但通常优化为物品级或残差地址,未暴露命名且可独立控制的属性槽。本文提出MM-slotgate,一种多模态槽编码器,将Fashion-CLIP的文本与图像嵌入分解为四个命名属性槽。每个槽学习独立的图文门控机制:视觉相关属性(如颜色、图案)更依赖图像证据,而分类属性(如类别、人口统计)则更依赖文本。在H&M数据集上,结合槽相似性与槽置信度得分,MM-slotgate实现0.7566的宏约束满足率@10,优于等权融合(0.7142)与fCLIP纯文本检索(0.4755)。颜色性能从0.321提升至0.889,绝对提升0.568,因颜色门控赋予图像证据57.4%权重。门控机制无需模态监督即可解释:颜色偏向图像,类别偏向文本,图案与人口统计居中。槽编码仍具可控性:线性探测显示泄漏低于标签相关基线,量化槽码支持定向干预,颜色控制提升达15.3倍。结果表明,有类型、属性条件化的多模态槽优于全局嵌入或不透明的物品级语义ID。

原文摘要 · Abstract (English)

Fashion retrieval often requires satisfying multiple attributes at once, such as category, color, pattern, and demographic. Monolithic embeddings mix these signals into a single vector, making attribute-specific control difficult at retrieval time. Many existing semantic-ID methods provide discrete item codes, but these codes are typically optimized as item-level or residual addresses and do not expose named, independently controllable attribute slots. We introduce MM-slotgate, a multimodal slot encoder that factorizes Fashion-CLIP text and image embeddings into four named attribute slots. Each slot learns its own text-image gate, so visually grounded attributes such as color and pattern can rely more on image evidence, while taxonomy-oriented attributes such as category and demographic can remain more text-driven. On H&M, using a combined slot-similarity and slot-logit retrieval score, MM-slotgate achieves 0.7566 macro ConstraintSatisfied@10, outperforming equal-weight multimodal fusion (0.7142) and fCLIP text-only retrieval (0.4755). The largest gain is on color, which improves from 0.321 to 0.889 (+0.568 absolute), as the learned color gate assigns 57.4% weight to image evidence. The learned gates are interpretable without modality supervision: color is image-leaning, category is text-leaning, and pattern and demographic lie near the middle. The resulting slots also remain controllable: linear probes show no measured excess leakage beyond the label-correlation baseline, and quantized slot codes support targeted intervention, including a 15.3x lift for color. These results suggest that controllable fashion retrieval benefits from typed, attribute-conditioned multimodal slots rather than either a single global embedding or opaque item-level semantic IDs.

服装检索多模态属性控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。