不同类型的代码隐私信息在大模型中泄露风险不同，且可被因果分析证实。
Understanding Privacy Risks in Code Models Through Training Dynamics: A Causal Approach
- 构建多类型隐私数据集，通过训练动态分析学习难易度
- 易学的IP地址泄露率高，难学的密钥密码泄露少
- 为代码模型设计针对性防护提供科学依据
用于代码的大语言模型（LLM4Code）虽显著提升开发效率，但依赖开源仓库中的大量个人身份信息（PII），引发隐私风险。已有研究显示商业模型会复现敏感信息，但大多将PII视为单一类别，忽略不同类型间的差异。本文探究不同类型的PII在被LLM4Code学习和泄露时是否存在差异，以及这种关系是否具有因果性。方法包括构建包含多样PII类型的语料库，对不同规模模型进行微调，基于真实PII数据计算训练动态，并建立结构化因果模型以估计学习能力对泄露的因果效应。结果表明，不同类型的PII泄露风险差异显著，与训练动态相关：易于学习的实例如IP地址更易泄露，而较难学习的密钥、密码泄露较少；模糊类型则表现混合。本工作首次提供因果证据，证明泄露风险依赖于类型，并为开发类型感知和学习能力感知的防御策略提供指导。
原文摘要 · Abstract (English)
Large language models for code (LLM4Code) have greatly improved developer productivity but also raise privacy concerns due to their reliance on open-source repositories containing abundant personally identifiable information (PII). Prior work shows that commercial models can reproduce sensitive PII, yet existing studies largely treat PII as a single category and overlook the heterogeneous risks among different types. We investigate whether distinct PII types vary in their likelihood of being learned and leaked by LLM4Code, and whether this relationship is causal. Our methodology includes building a dataset with diverse PII types, fine-tuning representative models of different scales, computing training dynamics on real PII data, and formulating a structural causal model to estimate the causal effect of learnability on leakage. Results show that leakage risks differ substantially across PII types and correlate with their training dynamics: easy-to-learn instances such as IP addresses exhibit higher leakage, while harder types such as keys and passwords leak less frequently. Ambiguous types show mixed behaviors. This work provides the first causal evidence that leakage risks are type-dependent and offers guidance for developing type-aware and learnability-aware defenses for LLM4Code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。