测试对话式搜索优化效果,发现多数方法无效还可能降权。
C-SEO Bench: Does Conversational SEO Work?
- 设计多任务多领域竞合评估框架,模拟真实竞争场景。
- 多数C-SEO方法无效甚至降低文档排名,传统SEO更有效。
- 随着采用者增多,整体收益下降,呈现零和竞争特征。
大型语言模型正将搜索引擎转变为对话式搜索引擎(CSE),推动搜索优化向对话式搜索优化(C-SEO)演进。现有C-SEO方法多在有限领域测试,缺乏跨领域有效性验证;且通常仅在单个文档采用的场景下评估,无法反映多参与者竞争的真实情况。本文提出C-SEO Bench,首个涵盖多任务、多领域及多参与者的评估基准,包含两个搜索任务(问答与商品推荐)和三个领域。我们引入新评估协议,支持不同采用率下的对比实验。实验结果表明,多数当前C-SEO方法不仅无效,反而常导致文档排名下降,与预期相反。相比之下,旨在提升源文档在大模型上下文中的排名的传统SEO策略显著更有效。同时,随着采用者数量增加,整体收益递减,揭示该问题具有拥堵性与零和特性。代码与数据已开源:https://github.com/parameterlab/c-seo-bench 及 https://huggingface.co/datasets/parameterlab/c-seo-bench。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are transforming search engines into Conversational Search Engines (CSE). Consequently, Search Engine Optimization (SEO) is being shifted into Conversational Search Engine Optimization (C-SEO). We are beginning to see dedicated C-SEO methods for modifying web documents to increase their visibility in CSE responses. However, they are often tested only for a limited breadth of application domains; we do not know whether certain C-SEO methods would be effective for a broad range of domains. Moreover, existing evaluations consider only a single-actor scenario where only one web document adopts a C-SEO method; in reality, multiple players are likely to competitively adopt the cutting-edge C-SEO techniques, drawing an analogy from the dynamics we have seen in SEO. We present C-SEO Bench, the first benchmark designed to evaluate C-SEO methods across multiple tasks, domains, and number of actors. We consider two search tasks, question answering and product recommendation, with three domains each. We also formalize a new evaluation protocol with varying adoption rates among involved actors. Our experiments reveal that most current C-SEO methods are not only largely ineffective but also frequently have a negative impact on document ranking, which is opposite to what is expected. Instead, traditional SEO strategies, those aiming to improve the ranking of the source in the LLM context, are significantly more effective. We also observe that as we increase the number of C-SEO adopters, the overall gains decrease, depicting a congested and zero-sum nature of the problem. Our code and data are available at https://github.com/parameterlab/c-seo-bench and https://huggingface.co/datasets/parameterlab/c-seo-bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。