用椭圆不确定性集提升金融强化学习的鲁棒性，解决交易影响带来的性能下降问题。
Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets
- 引入椭圆不确定性集建模市场影响的方向性特征。
- 在单资产和多资产交易中实现更高夏普比率，且随交易量增加仍保持稳定。
- 适合研究金融交易中的强化学习鲁棒性与实际部署应用。
在金融应用中，强化学习（RL）代理通常基于历史数据训练，其行为不改变价格。但在实际部署时，代理在实时市场中交易会引发资产价格变动，即市场影响。这种训练与部署环境的不一致会导致性能显著下降。传统鲁棒强化学习通过在不确定集上优化最差情况表现来应对模型误设，但通常依赖对称结构，难以捕捉市场影响的方向性。为此，本文提出一类新的椭圆不确定性集，建立了隐式与显式闭式解，实现高效可计算的最坏情况鲁棒策略评估。在单资产与多资产交易任务上的实验表明，该方法在提高夏普比率的同时，在交易量增加时仍保持鲁棒性，为金融市场的强化学习提供了更真实、可扩展的解决方案。
原文摘要 · Abstract (English)
In financial applications, reinforcement learning (RL) agents are commonly trained on historical data, where their actions do not influence prices. However, during deployment, these agents trade in live markets where their own transactions can shift asset prices, a phenomenon known as market impact. This mismatch between training and deployment environments can significantly degrade performance. Traditional robust RL approaches address this model misspecification by optimizing the worst-case performance over a set of uncertainties, but typically rely on symmetric structures that fail to capture the directional nature of market impact. To address this issue, we develop a novel class of elliptic uncertainty sets. We establish both implicit and explicit closed-form solutions for the worst-case uncertainty under these sets, enabling efficient and tractable robust policy evaluation. Experiments on single-asset and multi-asset trading tasks demonstrate that our method achieves superior Sharpe ratio and remains robust under increasing trade volumes, offering a more faithful and scalable approach to RL in financial markets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。