用会员推理和模型分歧检测大模型金融信号中的记忆污染，提升预测可靠性。
MemGuard-Alpha: Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting via Membership Inference and Cross-Model Disagreement
- 融合五种会员推理方法与时间特征，生成记忆污染评分。
- 过滤后信号夏普比率提升49%，日均收益相差7倍。
- 适用于量化交易中实时清洗大模型生成信号的场景。
大语言模型（LLM）被广泛用于生成金融阿尔法信号，但越来越多证据表明，这些模型会从训练语料中记忆历史金融数据，产生看似准确却无法外推的虚假预测。这种记忆导致的前瞻偏差威胁量化策略的有效性。现有解决方案如重训练或输入匿名化成本过高或损失信息。本文提出MemGuard-Alpha，一个无需额外训练的信号级过滤框架，包含两个算法：(i) MemGuard复合评分（MCS），通过逻辑回归融合五种会员推理攻击方法与时间接近特征，实现柯恩d=18.57的污染分离效果（仅用MIA特征时d=0.39–1.37）；(ii) 跨模型记忆分歧（CMMD），利用不同模型训练截止日期差异，区分记忆信号与真实推理。在7个模型（124M–7B参数）、50只标普100股票、42,800条提示、5种MIA方法覆盖2019–2024年共5.5年的评估中，CMMD使夏普比率从2.76提升至4.11（+49%），清洁信号平均日收益达14.48基点，污染信号仅2.13基点（差距7倍）。显著交叉现象显示：样本内准确率随污染上升（40.8%→52.5%），而样本外准确率下降（47%→42%），直接证明记忆夸大表观性能却损害泛化能力。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to generate financial alpha signals, yet growing evidence shows that LLMs memorize historical financial data from their training corpora, producing spurious predictive accuracy that collapses out-of-sample. This memorization-induced look-ahead bias threatens the validity of LLM-based quantitative strategies. Prior remedies -- model retraining and input anonymization -- are either prohibitively expensive or introduce significant information loss. No existing method offers practical, zero-cost signal-level filtering for real-time trading. We introduce MemGuard-Alpha, a post-generation framework comprising two algorithms: (i) the MemGuard Composite Score (MCS), which combines five membership inference attack (MIA) methods with temporal proximity features via logistic regression, achieving Cohen's d = 18.57 for contamination separation (d = 0.39-1.37 using MIA features alone); and (ii) Cross-Model Memorization Disagreement (CMMD), which exploits variation in training cutoff dates across LLMs to separate memorized signals from genuine reasoning. Evaluated across seven LLMs (124M-7B parameters), 50 S&P 100 stocks, 42,800 prompts, and five MIA methods over 5.5 years (2019-2024), CMMD achieves a Sharpe ratio of 4.11 versus 2.76 for unfiltered signals (49% improvement). Clean signals produce 14.48 bps average daily return versus 2.13 bps for tainted signals (7x difference). A striking crossover pattern emerges: in-sample accuracy rises with contamination (40.8% to 52.5%) while out-of-sample accuracy falls (47% to 42%), providing direct evidence that memorization inflates apparent accuracy at the cost of generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。