用图像识别方法处理商品描述,让推荐系统更准更稳。
When Text-as-Vision Meets Semantic IDs in Generative Recommendation: An Empirical Study
- 将文本描述转为图像,用视觉模型提取语义标识
- 在四个数据集上表现优于传统文本编码器
- 对压缩鲁棒性强,适合实际部署
语义ID学习是生成式推荐模型的关键接口,通过预训练文本编码器将物品映射为基于附加信息的离散标识。然而,这些编码器主要针对规范自然语言优化,在真实推荐数据中,物品描述常为符号化、属性导向内容,包含数字、单位和缩写,易被切分为碎片化标记,削弱语义连贯性并扭曲属性间关系。此外,多模态推荐中依赖标准文本编码器会引入额外障碍:文本与图像嵌入存在几何结构不匹配,导致跨模态融合效果差且不稳定。本文重新审视语义ID表示设计,将文本视为视觉信号。通过系统实证研究基于OCR的文本表示——将物品描述渲染为图像后,使用视觉型OCR模型编码。在四个数据集和两种生成式主干网络上的实验表明,基于OCR的文本表示在单模态和多模态设置下均稳定达到或超越标准文本嵌入性能。进一步发现,该方法在极端空间分辨率压缩下仍保持鲁棒性,体现出强健的实用性和高效性。
原文摘要 · Abstract (English)
Semantic ID learning is a key interface in Generative Recommendation (GR) models, mapping items to discrete identifiers grounded in side information, most commonly via a pretrained text encoder. However, these text encoders are primarily optimized for well-formed natural language. In real-world recommendation data, item descriptions are often symbolic and attribute-centric, containing numerals, units, and abbreviations. These text encoders can break these signals into fragmented tokens, weakening semantic coherence and distorting relationships among attributes. Worse still, when moving to multimodal GR, relying on standard text encoders introduces an additional obstacle: text and image embeddings often exhibit mismatched geometric structures, making cross-modal fusion less effective and less stable. In this paper, we revisit representation design for Semantic ID learning by treating text as a visual signal. We conduct a systematic empirical study of OCR-based text representations, obtained by rendering item descriptions into images and encoding them with vision-based OCR models. Experiments across four datasets and two generative backbones show that OCR-text consistently matches or surpasses standard text embeddings for Semantic ID learning in both unimodal and multimodal settings. Furthermore, we find that OCR-based Semantic IDs remain robust under extreme spatial-resolution compression, indicating strong robustness and efficiency in practical deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。