用量子电路缓解低信噪比金融强化学习中的偏差问题
Mitigating Bias in Low-SNR Financial Reinforcement Learning via Quantum Representations
- 在策略网络前加紧凑量子电路，约束特征传播路径
- 相较标准SAC提升66.89%累计收益，稳定性显著增强
- 适合金融量化交易、低信噪比强化学习研究者
金融市场是典型的低信噪比（SNR）环境，常导致SAC等离策略最大熵方法不稳定。具体表现为：噪声状态表示产生不可靠的Q值估计，且在线更新会放大误差，形成我们称为“金融熵陷阱”的失效模式。本文提出FPQC-SAC，一种高效且可直接替换的SAC变体，在演员和评论家网络前引入紧凑有界的参数化量子电路（PQC），从表征层面约束特征传播，而非过滤原始输入或在自举后正则化Q值。该方法有效降低极端市场波动对贝尔曼目标估计的影响，同时可训练的量子纠缠保持了跨资产交互的灵活性。在真实世界投资组合管理任务上的实证评估表明，FPQC-SAC 显著提升了样本外稳定性和累计收益，相比未约束的SAC实现66.89%的相对收益提升，并优于最佳连续控制深度强化学习基线约27%。开源代码见：https://github.com/ZeyuLIU-UST/FPQC-SAC-main。
原文摘要 · Abstract (English)
The financial market is a typical low signal-to-noise ratio (SNR) setting, which often destabilizes off-policy maximum-entropy methods like Soft Actor-Critic (SAC). Specifically, noisy state representations may produce unreliable Q-value estimates, and bootstrapping amplifies these errors, forming a failure mode we call the "Financial Entropy Trap". In this paper, we propose FPQC-SAC, an efficient and plug-and-play SAC variant that places a compact and bounded Parameterized Quantum Circuit (PQC) before the actor and critic networks to constrain feature propagation at the representation level, rather than filtering raw inputs or regularizing Q-values after bootstrapping. Notably, FPQC-SAC reduces the impact of extreme market fluctuations on Bellman target estimation, while trainable quantum entanglement preserves flexible cross-asset interactions. Empirical evaluations on real-world portfolio management tasks demonstrate that FPQC-SAC substantially enhances out-of-sample stability and cumulative returns by achieving a 66.89% relative gain in cumulative return over standard unconstrained SAC and outperforms the best continuous-control deep reinforcement learning baseline by approximately 27%. Open-source code is available at https://github.com/ZeyuLIU-UST/FPQC-SAC-main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。