构建可复现的预测市场回测框架，评估交易智能体表现。
PredictionMarketBench: A SWE-bench-Style Framework for Backtesting Trading Agents on Prediction Markets
- 基于历史订单簿数据，实现确定性事件驱动回放。
- 手续费与结算风险导致新手策略表现低于基准。
- 支持传统策略与大模型工具调用，适合量化研究者。
预测市场为交易智能体提供了天然的测试环境：合约具有二元收益，价格可解释为概率，实际表现高度依赖市场微观结构、手续费和结算风险。我们提出 PredictionMarketBench，一个类 SWE-bench 的基准框架，通过确定性、事件驱动方式回放历史限价单簿与交易数据，评估算法及基于大模型的交易智能体。该框架标准化了三方面：(i) 从原始交易所流中构建任务周期（订单簿、交易、生命周期、结算）；(ii) 支持做市商/吃单方语义与费用建模的执行真实模拟器；(iii) 提供工具化智能体接口，支持经典策略与工具调用型大模型智能体，并生成可复现轨迹。我们发布了四个基于 Kalshi 的任务周期，涵盖加密货币、天气与体育赛事。基线结果表明，因交易成本与结算损失，简单策略可能表现不佳；而考虑费用的算法策略在高波动场景中仍具竞争力。
原文摘要 · Abstract (English)
Prediction markets offer a natural testbed for trading agents: contracts have binary payoffs, prices can be interpreted as probabilities, and realized performance depends critically on market microstructure, fees, and settlement risk. We introduce PredictionMarketBench, a SWE-bench-style benchmark for evaluating algorithmic and LLM-based trading agents on prediction markets via deterministic, event-driven replay of historical limit-order-book and trade data. PredictionMarketBench standardizes (i) episode construction from raw exchange streams (orderbooks, trades, lifecycle, settlement), (ii) an execution-realistic simulator with maker/taker semantics and fee modeling, and (iii) a tool-based agent interface that supports both classical strategies and tool-calling LLM agents with reproducible trajectories. We release four Kalshi-based episodes spanning cryptocurrency, weather, and sports. Baseline results show that naive trading agents can underperform due to transaction costs and settlement losses, while fee-aware algorithmic strategies remain competitive in volatile episodes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。