arXiv:2605.15412cs.CEcs.AI2026-05被引 4

用强化学习让大模型自主进化选股因子,避免反馈堆积问题。

From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery

论文配图:From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery
图 1 · 摘自论文原文
  • 将历史反馈转为模型参数更新,避免提示词膨胀
  • 在三个真实市场数据集上提升因子质量与多样性
  • 适合量化交易研究者和金融AI开发者使用

现代量化交易越来越依赖系统化模型从大规模金融数据中提取预测信号,其中选股因子发现是关键环节。近期基于大语言模型的方法虽能自动生成因子,但多依赖提示词级的生成-评估-反馈循环进行迭代优化。随着循环延长,历史候选因子与反馈不断累积,导致上下文爆炸、推理成本上升、信息稀释及反馈漂移。此外,这些方法常依赖超大模型,其稳定生成偏好易造成表达结构相似、候选因子冗余、搜索停滞。为此,我们提出 extsc{QuantEvolver},一种基于强化微调的自演化选股因子发现框架。该框架不将反馈堆叠于提示词中,而是将可执行量化评估转化为策略更新,使矿工模型通过参数学习内化历史优化经验。具体而言, extsc{QuantEvolver} 构建高质量种子因子,构建多样化的种子-时间窗训练任务,生成可执行的因子DSL表达式,通过制度回测评估,并以多样性和互补性奖励优化矿工模型。训练过程中,高质量因子持续积累至已挖掘因子数据库,构成最终发现的因子库。在三个真实市场基准上的实验表明, extsc{QuantEvolver} 在各项任务中均持续优于现有基于大模型的因子发现基线,产出更高品质且更具互补性的因子池。

原文摘要 · Abstract (English)

Modern quantitative trading increasingly relies on systematic models to extract predictive signals from large-scale financial data, where alpha factor discovery plays a central role in transforming market observations into tradable signals. Recent LLM-based methods have shown promise in automating factor generation, but most of them still rely on prompt-level generation--evaluation--feedback loops for iterative optimization. As the loop becomes longer, repeatedly appended historical candidates and feedback can cause context explosion, increase inference cost, dilute useful information, and introduce feedback drift. Moreover, these methods often depend on very large LLMs whose stable generation preferences may lead to structurally similar expressions, redundant candidates, and search stagnation. To address these limitations, we propose \textsc{QuantEvolver}, a self-evolving alpha factor discovery framework based on reinforcement fine-tuning. Instead of accumulating feedback in the prompt, \textsc{QuantEvolver} converts executable quantitative evaluation into policy updates, enabling a Miner LLM to internalize historical optimization experience through parameter learning. Specifically, \textsc{QuantEvolver} constructs high-quality seed factors, builds diverse seed--time-window training tasks, generates executable Factor DSL expressions, evaluates them through Regime Backtest, and optimizes the Miner LLM with Diversity-Complementarity Reward. During training, high-quality factors are continuously accumulated in a Mined Factor Database, which serves as the final discovered factor library. Extensive experiments on three realistic market benchmarks demonstrate the effectiveness of \textsc{QuantEvolver}, which consistently improves the primary evaluation metric of each task over existing LLM-based alpha factor discovery baselines, produces higher-quality and more complementary factor pools.

量化交易大模型强化学习因子发现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。