用零知识证明验证大模型训练数据来源,既保隐私又高效。
ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models
- 通过零知识证明绑定训练数据、模型参数与回答结果
- 80亿参数模型端到端开销低于3.3秒,证明生成与验证亚线性增长
- 适合医疗等对数据合规性要求高的敏感场景
随着大语言模型在敏感领域应用增多,如何在不泄露训练数据的前提下准确验证其计算来源成为关键挑战,尤其在医疗等监管严格领域。传统方法或计算成本过高,或会向验证方泄露未经授权的信息。为此,我们提出ZKPROV,一种新型密码学框架,使用户可验证模型响应确实基于权威认证的数据集训练而成。该框架确保数据内容与用户查询相关,同时不暴露数据本身或模型参数的敏感信息。ZKPROV通过将训练数据、模型参数和输出结果绑定,并为模型输出附加零知识证明,实现隐私与效率的平衡。实验表明,证明生成与验证具有亚线性扩展性,80亿参数模型端到端延迟低于3.3秒,具备实际部署可行性。我们还提供了形式化安全证明,证实该方法能保护数据机密性并保障可信的数据溯源。
原文摘要 · Abstract (English)
As large language models (LLMs) are used in sensitive fields, accurately verifying their computational provenance without disclosing their training datasets poses a significant challenge, particularly in regulated sectors such as healthcare, which have strict requirements for dataset use. Traditional approaches either incur substantial computational cost to fully verify the entire training process or leak unauthorized information to the verifier. Therefore, we introduce ZKPROV, a novel cryptographic framework allowing users to verify that the LLM's responses to their prompts are trained on datasets certified by the authorities that own them. Additionally, it ensures that the dataset's content is relevant to the users' queries without revealing sensitive information about the datasets or the model parameters. ZKPROV offers a unique balance between privacy and efficiency by binding training datasets, model parameters, and responses, while also attaching zero-knowledge proofs to the responses generated by the LLM to validate these claims. Our experimental results demonstrate sublinear scaling for generating and verifying these proofs, with end-to-end overhead under 3.3 seconds for models up to 8B parameters, presenting a practical solution for real-world applications. We also provide formal security guarantees, proving that our approach preserves dataset confidentiality while ensuring trustworthy dataset provenance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。