用三阶段强化学习实现税优个性投资组合管理,兼顾多目标与用户行为。
A Three-Phase Foundation Model for Tax-Aware Personalized Portfolio Management
- 分三阶段:先无股票标识编码,再多目标专家混合,最后用少量参数个性化
- 支持6种投资目标并行优化,税收减免和长期收益可同时达成
- 通过真实交易行为推断用户目标,无需问卷,适合个人投顾系统
我们提出一种三阶段深度强化学习系统,用于个性化投资组合管理,解决以往金融强化学习的三大缺陷:1)股票代码锁定问题;2)单一目标限制;3)静态用户建模。第一阶段通过自监督学习在多资产语料上预训练一个不依赖股票标识的跨资产编码器,并融合基于T5的时间序列基础模型Chronos,通过可学习门控机制进行特征融合。该编码器可泛化至任意公开交易资产,仅需50维可观测元数据向量,无需对新股票重新训练。第二阶段在PPO框架下微调基于专家混合(MoE)的策略-价值网络,采用条件奖励机制,在每轮采样中同时服务六种投资目标:短期阿尔法、短期收益、长期收益、资本保全、税务亏损收割、仅长期收益。每个目标由专用专家头处理(动量型、增长型、防御型、税优型),由可学习意图路由器根据当前目标和市场状态动态组合专家,避免跨目标梯度冲突。第三阶段引入轻量级个性化层,通过76参数的LoRA模块在推理时针对个体用户微调,基于真实券商交易历史推断投资目标,而非问卷调查;自然语言意图解析器可将自由文本目标转化为结构化参数。
原文摘要 · Abstract (English)
We present a three-phase deep reinforcement learning system for personalized portfolio management that addresses three limitations shared by all prior financial RL work: 1) ticker lock-in, 2) monolithic objectives , and 3) static user models. Phase 1 pretrains a ticker-identity-free cross asset encoder via self-supervised learning on a multi-asset corpus, augmented by a frozen parallel branch using Chronos, a T5-based time series foundation model, fused via a learned gating mechanism. To our knowledge, this is the first application of a time series foundation model to portfolio management RL. The encoder generalizes to any publicly traded asset via a 50-dimensional observable metadata vector that requires no retraining for new tickers. Phase 2 fine-tunes a MoE (Mixture of Experts) portfolio actor critic with PPO under an objective-conditioned reward that simultaneously serves six distinct investment goals sampled per episode: short-term alpha, short-term gain, long-term gain, capital preservation, tax-loss harvesting, and long-term-gains-only. A MoE architecture assigns each objective to a specialized expert head (momentum, growth, defensive, tax-aware), and a learned intent router blends experts based on the active objective and current market regime, which eliminates cross-objective gradient conflict. Phase 3 adds a lightweight personalization layer further adapted at inference time to each individual via a 76-parameter LoRA module fine-tuned on real brokerage transaction history, inferring investment objectives from revealed trading behavior rather than questionnaires. A natural language intent parser converts free-form goals directly into structured investment objective parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。