对比不同归一化方法对强化学习投资组合优化的影响
Comparing Normalization Methods for Portfolio Optimization with Reinforcement Learning
- 测试两种主流归一化方法在三种市场的表现
- 发现状态归一化会降低代理性能，尤其在非加密货币市场
- 适合关注金融RL中数据预处理影响的研究者
近年来，强化学习在机器人、游戏、自然语言处理和金融等领域取得显著成果。在金融领域，该方法被用于投资组合优化，即通过智能体持续调整资产配置以最大化收益。已有大量研究提出新的仿真环境、神经网络架构和训练算法。其中，一种领域特定的策略梯度算法因轻量、快速且优于其他方法而备受关注。然而，近期研究表明，该算法在非加密货币市场中常出现结果不一致、表现不佳的情况。可能原因在于常用的状态归一化方法会丢失资产真实价值的关键信息。本文通过在IBOVESPA、NYSE和加密货币三个市场评估两种最广泛使用的归一化方法，并与训练前标准归一化进行对比，结果表明：在此特定领域，状态归一化确实会损害代理性能。
原文摘要 · Abstract (English)
Recently, reinforcement learning has achieved remarkable results in various domains, including robotics, games, natural language processing, and finance. In the financial domain, this approach has been applied to tasks such as portfolio optimization, where an agent continuously adjusts the allocation of assets within a financial portfolio to maximize profit. Numerous studies have introduced new simulation environments, neural network architectures, and training algorithms for this purpose. Among these, a domain-specific policy gradient algorithm has gained significant attention in the research community for being lightweight, fast, and for outperforming other approaches. However, recent studies have shown that this algorithm can yield inconsistent results and underperform, especially when the portfolio does not consist of cryptocurrencies. One possible explanation for this issue is that the commonly used state normalization method may cause the agent to lose critical information about the true value of the assets being traded. This paper explores this hypothesis by evaluating two of the most widely used normalization methods across three different markets (IBOVESPA, NYSE, and cryptocurrencies) and comparing them with the standard practice of normalizing data before training. The results indicate that, in this specific domain, the state normalization can indeed degrade the agent's performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。