用零知识证明保障大模型微调安全,可验证且不泄露隐私。
VeriLoRA: Fine-Tuning Large Language Models with Verifiable Security via Zero-Knowledge Proofs
- 将低秩微调与零知识证明结合,实现全流程可验证。
- 在130亿参数的LLaMA上验证,支持前向、反向传播和参数更新的正确性证明。
- 适合高安全性要求的场景,如金融、医疗等敏感领域部署。
大语言模型微调对适配特定任务至关重要,但计算成本高且在不可信环境中的正确性与隐私问题仍存。尽管低秩微调(LoRA)显著降低资源消耗,但在零知识条件下确保微调过程的安全与可验证性仍是未解难题。为此,我们提出VeriLoRA,首个将LoRA微调与零知识证明(ZKPs)结合的框架,实现可证明的安全性和正确性。VeriLoRA采用查找论证、sumcheck协议和多项式承诺等先进密码技术,验证基于Transformer架构的算术与非算术操作。该框架提供从前向传播、反向传播到参数更新的端到端可验证性,同时保护模型参数与训练数据隐私。通过基于GPU的实现,VeriLoRA在开源LLM如LLaMA上实验验证,支持高达130亿参数规模。通过结合参数高效微调与零知识证明,VeriLoRA填补关键空白,使大模型可在敏感或不可信环境中安全可信部署。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) is crucial for adapting them to specific tasks, yet it remains computationally demanding and raises concerns about correctness and privacy, particularly in untrusted environments. Although parameter-efficient methods like Low-Rank Adaptation (LoRA) significantly reduce resource requirements, ensuring the security and verifiability of fine-tuning under zero-knowledge constraints remains an unresolved challenge. To address this, we introduce VeriLoRA, the first framework to integrate LoRA fine-tuning with zero-knowledge proofs (ZKPs), achieving provable security and correctness. VeriLoRA employs advanced cryptographic techniques -- such as lookup arguments, sumcheck protocols, and polynomial commitments -- to verify both arithmetic and non-arithmetic operations in Transformer-based architectures. The framework provides end-to-end verifiability for forward propagation, backward propagation, and parameter updates during LoRA fine-tuning, while safeguarding the privacy of model parameters and training data. Leveraging GPU-based implementations, VeriLoRA demonstrates practicality and efficiency through experimental validation on open-source LLMs like LLaMA, scaling up to 13 billion parameters. By combining parameter-efficient fine-tuning with ZKPs, VeriLoRA bridges a critical gap, enabling secure and trustworthy deployment of LLMs in sensitive or untrusted environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。