分层强化学习框架,让选股与执行更智能高效
Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Execution
- 分两层决策:先选涨跌方向,再按风险调整仓位
- 夏普比率提升至1.24,换手率降至0.090
- 适合关注文本信号与风险控制的量化交易研究者
自动化股票交易需在风险、换手率和交易成本约束下,将嘈杂的市场与新闻信号转化为可执行的投资组合决策。我们提出分层强化交易者(HRT),一种用于多资产股票市场的文本感知投资组合管理双层强化学习框架。HRT将交易分解为两个协同决策:低维稀疏的高层控制器(HLC)从紧凑的市场与文本衍生信号中选择个股的增持、减持或持有方向;低层控制器(LLC)则在换手率、回撤和文本风险惩罚下,将这些方向转化为可行的权重调整。该分解避免了完整联合动作空间枚举,使选择与执行更易解释。我们在一个固定89只纳斯达克股票的公开新闻基准上评估HRT,训练期为2013–2018,验证期为2019,测试期为2020–2023;测试时间受限于同一时间戳清洁文本协议下的公开数据可用性。相比市场代理、同宇宙投资组合、纯阿尔法、平坦强化学习及分层消融基线,HRT在学习型收益-风险-成本权衡上表现最优。完整模型将夏普比率从1.06提升至1.24,日均换手率由0.112降至0.090,并在交易成本压力下保持稳健。结果表明,将稀疏方向选择与风险感知执行分离,是有效融合市场预测与文本风险信号的可行路径。
原文摘要 · Abstract (English)
Automated equity trading requires converting noisy market and news signals into executable portfolio decisions under risk, turnover, and transaction costs. We propose Hierarchical Reinforced Trader (HRT), a bi-level reinforcement learning framework for text-aware portfolio management in multi-asset equity markets. HRT separates trading into two coordinated decisions: a factorized sparse High-Level Controller (HLC) selects asset-level increase, reduce, or hold directions from compact market and text-derived signals, while a risk-aware Low-Level Controller (LLC) converts these directions into feasible portfolio weight adjustments under turnover, drawdown, and text-risk penalties. This decomposition avoids enumerating the full joint action space and makes selection and execution easier to inspect. We evaluate HRT on an open stock-news benchmark with a fixed 89-stock Nasdaq universe, using 2013--2018 for training, 2019 for validation, and 2020--2023 for final out-of-sample testing; the test horizon is restricted to 2020--2023 due to public benchmark data availability under the same timestamp-clean text-aware protocol. Across market-proxy, same-universe portfolio, alpha-only, flat-RL, and hierarchical ablation baselines, HRT delivers the strongest learning-based return--risk--cost trade-off. The full model improves Sharpe from 1.06 for HRT-Base to 1.24, reduces daily turnover from 0.112 to 0.090, and remains robust under transaction-cost stress. These results suggest that separating sparse directional selection from risk-aware execution is an effective way to incorporate market forecasts and text-derived risk signals into portfolio management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。