用新闻情绪增强强化学习，实现比特币和特斯拉的主动交易决策。
CLaC@FinMMEval 2026 Task 3: Sentiment-Augmented Deep Reinforcement Learning for Active Trading -- An Alpha-Reward Approach
- 结合新闻情感与技术指标，构建基于收益差的奖励机制。
- 深度确定性策略梯度（DDPG）在测试集表现最优，特斯拉收益超买持有38个百分点。
- 验证到测试集存在显著泛化差距，提醒投资者警惕牛市策略在熊市失效。
本文针对CLEF 2026 FinMMEval任务3，提出一个基于深度强化学习的主动交易系统，需对比特币（BTC）和特斯拉（TSLA）每日做出多空或持币决策，输入为新闻与历史市场数据。将问题建模为离散动作马尔可夫决策过程，对比了四种算法：策略梯度（PG）、近端策略优化（PPO）、深度Q网络（DQL）与深度确定性策略梯度（DDPG）。代理使用技术指标、周期性日历编码及由LLaMA 3.2 1B生成的每日新闻情感分数。为减少过拟合并使训练目标与超越买入持有策略对齐，引入基于超额市场回报的alpha奖励，并随机化训练期起点。通过Ray Tune进行180次超参数搜索，采用早停与基于验证夏普比率的模型选择。在测试集上，DDPG表现最佳；因DQL在验证集夏普比最高，被选为线上部署模型。对于TSLA，DDPG与DQL累计收益率分别为54.96%和52.62%，远超买入持有的16.45%；对于BTC，DDPG获得1.58%正收益，而买入持有为-34.27%。结果揭示了验证集到测试集间显著的泛化差距，表明牛市选取的策略难以适应熊市环境。
原文摘要 · Abstract (English)
This paper presents our system for Task 3 of the CLEF 2026 FinMMEval Lab, which requires daily long, flat, or short trading decisions for Bitcoin (BTC) and Tesla (TSLA) using news and historical market data. We formulate the problem as a discrete-action Markov Decision Process and compare four deep reinforcement learning algorithms: Policy Gradient (PG), Proximal Policy Optimization (PPO), Deep Q-Learning (DQL), and Deep Deterministic Policy Gradient (DDPG). The agents use technical indicators, cyclical calendar encodings, and daily news sentiment scores produced by LLaMA 3.2 1B. To reduce overfitting and align training with the objective of outperforming buy-and-hold, we introduce an alpha reward based on excess market return and randomize episode start dates. Hyperparameters are optimized with Ray Tune over 180 trials per algorithm-asset pair, with early stopping and model selection based on validation Sharpe ratio. On the CLEF Task 3 test set, DDPG achieves the strongest overall performance. DQL was selected a priori for the live endpoint because it obtained the highest validation Sharpe ratio, with selection performed without access to the test period. For TSLA, DDPG and DQL achieve cumulative returns of 54.96% and 52.62%, respectively, compared with 16.45% for buy-and-hold. For BTC, DDPG achieves a positive return of 1.58% while buy-and-hold declines by -34.27%. The results also reveal a substantial validation-to-test generalization gap, highlighting the difficulty of transferring policies selected in bull-market conditions to a bear-market regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。