用强化学习优化比特币抛售，比传统方法更省成本。
RL-Exec: Impact-Aware Reinforcement Learning for Opportunistic Optimal Liquidation, Outperforms TWAP and a Book-Liquidity VWAP on BTC-USD Replays
- 基于PPO算法训练代理，考虑订单簿深度和市场瞬时影响。
- 120分钟内比基准方案低23个基点，时间越长优势越明显。
- 适合量化交易员研究智能执行策略，尤其关注成本控制。
本文研究在固定期限内对BTC-USD限价订单簿进行机会式最优平仓。提出RL-Exec，一个使用PPO算法的智能体，在历史回放数据上训练，数据包含内生性瞬时冲击（恢复性）、部分成交、做市/做市商费用及延迟。策略观察前20层订单簿特征与微观结构指标，在仅卖出的持仓约束下达成剩余目标。评估采用严格的时间分割（训练：2020年1月；测试：2020年2月）和每日协议：每天运行十次独立起始时间并汇总为单日得分，避免伪重复。对比对象为TWAP和一种基于对手方订单簿流动性（前20层）的类VWAP基准，均在同一时间戳与成本下执行。统计检验采用单边威尔科克森符号秩检验，结合贝尼哈明-霍赫伯格错误发现率校正与自助法置信区间。在2020年2月测试集上，RL-Exec显著优于两个基线，差距随执行周期延长而扩大：30分钟时领先2-3基点，60分钟达7-8基点，120分钟达23基点。代码开源：github.com/Giafferri/RL-Exec
原文摘要 · Abstract (English)
We study opportunistic optimal liquidation over fixed deadlines on BTC-USD limit-order books (LOB). We present RL-Exec, a PPO agent trained on historical replays augmented with endogenous transient impact (resilience), partial fills, maker/taker fees, and latency. The policy observes depth-20 LOB features plus microstructure indicators and acts under a sell-only inventory constraint to reach a residual target. Evaluation follows a strict time split (train: Jan-2020; test: Feb-2020) and a per-day protocol: for each test day we run ten independent start times and aggregate to a single daily score, avoiding pseudo-replication. We compare the agent to (i) TWAP and (ii) a VWAP-like baseline allocating using opposite-side order-book liquidity (top-20 levels), both executed on identical timestamps and costs. Statistical inference uses one-sided Wilcoxon signed-rank tests on daily RL-baseline differences with Benjamini-Hochberg FDR correction and bootstrap confidence intervals. On the Feb-2020 test set, RL-Exec significantly outperforms both baselines and the gap increases with the execution horizon (+2-3 bps at 30 min, +7-8 bps at 60 min, +23 bps at 120 min). Code: github.com/Giafferri/RL-Exec
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。