让大模型微调过程可验证,确保更新方式符合声明规范。
Fine-Tuning Integrity for Modern Neural Networks: Structured Drift Proofs via Norm, Rank, and Sparsity Certificates
- 用零知识证明技术验证模型更新是否在预设结构内
- 三种更新类型证明均与参数量无关,保持轻量
- 适合需要审计模型更新可信性的机构或开发者
微调是主流的大模型适配范式,但现有部署流程无法验证发布的模型是否按声明方式更新。我们提出细调完整性(FTI),一种可验证模型仅在指定更新类别内变化的密码学目标。构建了简短模型差异证明(SMDP),无需暴露权重即可验证参数漂移。框架支持三种基本更新类:范数有界、低秩和稀疏漂移,覆盖正则化训练、LoRA 和前缀微调等常见方法。所有情况下,证明大小与验证开销取决于更新结构而非参数总量。我们证明了每种构造的完备性、零知识性和简洁性,并建立 Ω(n) 下界,表明结构假设对简洁验证不可或缺。原型评估在合成基准和 GPT-2 微调上显示,证明保持紧凑,验证高效且可扩展至实际规模。
原文摘要 · Abstract (English)
Fine-tuning is the dominant paradigm for adapting large machine learning models, yet current deployment pipelines provide no way to verify how a released model was updated. In particular, a model provider or auditor cannot check whether a fine-tuned model adheres to a claimed update procedure without access to its parameters. We introduce \emph{fine-tuning integrity} (FTI), a cryptographic objective for verifying that a deployed model differs from a trusted base model only within a declared class of admissible updates. We construct \emph{succinct model difference proofs} (SMDPs), zero-knowledge protocols that certify structured parameter drift without revealing model weights. Our framework supports three fundamental update classes: norm-bounded, low-rank, and sparse drift, covering common fine-tuning methods such as regularized training, LoRA, and prefix tuning. In all cases, proof size and verification cost depend on the structure of the update rather than the number of parameters. We prove soundness, zero-knowledge, and succinctness for each construction, and establish a matching $Ω(n)$ lower bound showing that structural assumptions are necessary for succinct verification. A prototype evaluation on synthetic benchmarks and GPT-2 fine-tuning demonstrates that proofs remain compact and verification is efficient at realistic scales.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。