arXiv:2606.08450cs.AI2026-06被引 1

用大模型提升金融强化学习的状态与奖励设计,让策略更稳更准。

GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning

论文配图:GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning
图 1 · 摘自论文原文
  • 用因子引导生成状态特征,融合金融知识增强输入信号
  • 基于风险规则生成辅助奖励,显著改善学习信号质量
  • 通过策略回放诊断自动优化接口,适合追求稳健收益的研究者

金融资产组合交易天然可建模为强化学习问题,智能体需在市场动态变化中持续调仓以平衡收益、风险与交易成本。然而在非平稳市场中,原始的OHLCV状态与短期回报奖励常导致学习信号不足,我们提出GIFT框架,利用大语言模型注入金融知识来改进状态与奖励设计,同时约束生成的开放性。GIFT不直接做交易决策,而是通过因子引导的状态增强生成状态特征,基于风险规则的奖励重塑生成辅助奖励,并通过策略回放诊断进行候选接口的迭代修正。经修正后固定接口用于评估,测试阶段不再调用大模型或更新接口。跨多种市场环境和投资场景的滚动窗口实验表明,GIFT显著提升了学习信号质量,并在事后风险调整收益上优于基线方法。代码与数据见:https://github.com/KAG778/GIFT。

原文摘要 · Abstract (English)

Financial portfolio trading is naturally formulated as a reinforcement learning problem, where an agent sequentially rebalances assets under changing market conditions to balance return, risk, and transaction costs. Yet in non-stationary markets, raw OHLCV states and short-horizon return rewards often provide an under-specified learning interface, motivating large language models as a way to inject financial knowledge into state and reward design while constraining open-ended generation. To this end, we propose GIFT, an LLM-guided framework for state-reward interface design in PPO-based financial reinforcement learning. Rather than using the LLM to make trading decisions, GIFT uses Factor-guided State Enhancement to generate state features from financial-factor primitives, Risk-rule-guided Reward Shaping to generate auxiliary rewards from portfolio-risk rules, and Diagnostic-guided Refinement to revise candidate interfaces using PPO rollout diagnostics. After refinement, GIFT fixes the selected state-reward interface before evaluation, with no further LLM queries or interface updates at test time. Comprehensive rolling-window experiments across diverse market regimes and portfolio scenarios demonstrate that GIFT improves learning-signal quality and out-of-sample risk-adjusted portfolio performance over baselines. Code and data are available at: https://github.com/KAG778/GIFT .

强化学习金融交易大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。