用合成数据解决搜索冷启动问题,让AI生成真实用户查询和标签。
CASTLE: Contrastive and Seed-Guided Training for Cold-Start Natural Language Search

- 结合结构提示与种子查询,防止生成的查询重复坍缩。
- 通过预订会话构建对比对,零误标率生成相关性标签。
- 适合需要长尾查询生成的推荐系统研发者使用。
部署自然语言搜索系统面临冷启动挑战:缺乏真实用户查询学习语言模式,也无相关性标签训练排序模型。本文提出CASTLE(Contrastive And Seed-guided Training for natural Language sEarch),一个基于大模型的框架,从结构化房源数据生成合成查询与相关性标签,支撑Airbnb全生命周期的自然语言搜索。首先,通过结构引导提示与用户研究种子查询,结合模板、少样本及属性锚定提示,并引入显式多样性机制,生成真实感强的查询,避免查询坍缩。其次,通过预订会话构建对比房源对,构造相关性标签,实现近零误报,无需依赖LLM判断。第三,结构化输入设计灵活,可融合客评、照片描述等丰富信号,生成反映主观偏好的长尾查询(如“有壁炉的温馨小屋”),超越仅依赖属性表达的范围。相比InPars、Promptagator和纯对比基线,CASTLE在查询分布上达到KL=1.01,较最佳基线(9.33)提升9.2倍,属性类型KL散度最低(0.08),优于问卷种子查询(0.09)。200个样本的人工评估显示标注一致性达91%-93%。已投入生产,每日生成合成数据用于嵌入检索与排序评估。合成数据不仅用于冷启动,还可覆盖有机流量中稀疏的尾部查询,并自然扩展至多轮对话搜索。
原文摘要 · Abstract (English)
Deploying natural language search systems presents a critical cold-start challenge: no real user queries to learn linguistic patterns, and no relevance labels to train ranking models. We present CASTLE (Contrastive And Seed-guided Training for natural Language sEarch), an LLM-based framework for generating synthetic queries and relevance labels from structured catalog data, powering Airbnb's natural language search across its full lifecycle. CASTLE makes three contributions. First, we generate realistic queries by combining structure-guided prompting with seed queries from user research, using template, few-shot, and attribute-grounded prompt variants together with explicit variety mechanisms to prevent query collapse. Second, we produce relevance labels by construction via contrastive listing pairs derived from booking sessions, achieving near-zero false positives without LLM judgment. Third, CASTLE's structured input design is flexible: incorporating richer signals such as guest reviews and photo captions alongside listing attributes enables generation of niche, long-tail queries that reflect subjective user preferences (e.g., "cozy cabin with fireplace") beyond what catalog attributes alone can express. Compared against InPars-style, Promptagator, and contrastive-only baselines, CASTLE achieves KL 1.01 vs. real users -- a 9.2x improvement over the best baseline (9.33) -- and the lowest attribute-type KL divergence (0.08), outperforming even survey seed queries (0.09). A human evaluation on 200 sampled triplets confirms label quality: annotators agree with CASTLE labels at 91-93%. We deploy production pipelines generating synthetic examples daily for embedding-based retrieval and ranking evaluation. Synthetic data remains valuable beyond cold-start: it targets tail queries underrepresented in organic traffic and extends naturally to multi-turn conversational search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。