用大模型模拟文档作者竞争排名，可生成可复现的评测数据。
CSP: A Simulator For Multi-Agent Ranking Competitions
- 构建基于大模型的多智能体排名竞赛仿真器，支持灵活配置。
- 生成多个真实感强的数据集，验证了大模型在排名竞争中的行为模式。
- 适合研究搜索引擎优化、大模型内容生成与排名机制的研究者使用。
在排名竞赛中，文档作者通过修改内容以争取更高排名。以往研究主要聚焦于受控环境下的人类参与者（如学生）。随着生成式AI，特别是大语言模型（LLMs）的兴起，引入了新范式：将LLM作为文档作者。该方法克服了人力竞赛的可扩展性限制，并反映了网络上大模型生成内容日益增长的角色。本文提出一个高度可配置的排名竞赛仿真器，利用LLM作为文档作者，并包含分析工具以研究生成数据集。我们通过生成多个数据集并进行广泛分析，展示了其能力。代码与数据集已公开，供研究使用。
原文摘要 · Abstract (English)
In ranking competitions, document authors compete for the highest rankings by modifying their content in response to past rankings. Previous studies focused on human participants, primarily students, in controlled settings. The rise of generative AI, particularly Large Language Models (LLMs), introduces a new paradigm: using LLMs as document authors. This approach addresses scalability constraints in human-based competitions and reflects the growing role of LLM-generated content on the web-a prime example of ranking competition. We introduce a highly configurable ranking competition simulator that leverages LLMs as document authors. It includes analytical tools to examine the resulting datasets. We demonstrate its capabilities by generating multiple datasets and conducting an extensive analysis. Our code and datasets are publicly available for research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。