arXiv:2608.23308cs.CRcs.AI2026-08

检验大模型生成交易策略的逻辑、代码与实绩是否一致。

FIDES: A Concordance Protocol for LLM-Generated Trading Strategies

  • 用三视图比对方法验证策略的说辞、代码和实际表现
  • 40个策略仅2个跑赢买入持有,多数自评严重高估
  • 换模型评估能改变结果,提示大模型策略可信度存疑

让大模型生成交易策略时,会同时输出自然语言解释、可执行代码和运行记录。这些输出是否一致却很少被验证。我们提出FIDES协议,将三者视为需对齐的三个视角而非单一交付物。通过单次调用返回语言策略与自包含函数(df),FIDES在沙箱中对8只美国流动ETF进行滞后一期的样本外回测,评估三项一致性差距:说与做、做与实、说与果。在四个模型及两阶段诱导设置下,共测试40个策略(2023-2024年样本外),发现三点显著结论:第一,一致性不预示盈利——仅2个策略跑赢买入持有,而简单SMA(50,200)规则优于所有模型平均夏普比率;第二,自我评估严重失准——40个中有32个声称超越买入持有,但仅1个实现;第三,将语言-代码判别器替换为另一模型后,超过一半策略的“说”与“做”出现反转。引入Close.shift(-1)使“做”与“实”的差距平均下降0.33,而我们的运行时未来信息探测器在清洁与注入代码上均未触发。我们强调FIDES是测量信度的协议,而非对市场表现的主张。

原文摘要 · Abstract (English)

An LLM asked for a trading strategy returns three artifacts at once: a natural-language rationale, an executable implementation, and once run, a track record. Whether these are the same object is rarely checked. We present FIDES, a measurement protocol that treats them as three views to be reconciled rather than one deliverable to be graded. Through dual delivery, a single model call returns both a natural-language strategy with an explicit claimed edge and a self-contained strategy(df) function. FIDES executes the code in a sandbox against a lag-one out-of-sample backtest and scores three concordance gaps: say to do, do to real, and say to result. On 8 liquid US ETFs across four models plus a two-stage elicitation arm, 40 strategies, 2023 to 2024 out-of-sample, three findings stand out. First, concordance does not predict profit: only 2 of 40 strategies beat buy-and-hold, and a plain sma(50,200) rule outperforms every model's mean Sharpe. Second, self-assessment is badly calibrated: 32 of 40 strategies claim to beat buy-and-hold and exactly one does. Third, swapping the language-code judge for a second model flips say to do on more than half of items. Injecting Close.shift(-1) drops do to real by 0.33 on average, while our runtime future-information probe fired on neither clean nor injected code. We frame FIDES as a protocol for measurement fidelity, not a claim about market performance.

大模型量化交易可靠性评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。