开发可让用户自查大模型关联信息的工具，发现普通人11项特征可被准确预测。
Human-Centred LLM Privacy Audits: Findings and Frictions
- 设计浏览器插件LMP2，支持用户自主检测模型对其姓名的关联信息
- GPT-4o对普通人的50项特征中，有11项预测准确率超60%
- 揭示隐私审计中的机制困境，适合关注模型透明性与个人数据控制的研究者
大型语言模型（LLMs）从海量训练语料和用户交互中学习统计关联，部署系统可能暴露或推断个体信息。然而公众缺乏实际手段检查模型对其姓名的关联内容。我们报告一项正在进行研究的中期发现，并提出LMP2——一款基于浏览器的自我审计工具。在两项用户研究（共458人）中，GPT-4o对普通人的50项特征中有11项预测准确率达60%以上；参与者虽未将所有输出视为隐私侵犯，但仍希望掌控模型生成的关联信息。为验证探测方法，我们在公众人物和虚构姓名上评估了八种LLM，观察到稳定姓名相关关联与模型默认输出之间存在清晰区分。研究还揭示生成式AI评估的深层危机：当输出具有概率性、依赖上下文且通过用户诱导产生时，模型-个体关联的界定模糊，评估依赖难以验证和比较的探针与指标。为推动可信赖、可行动的人类中心化大模型隐私审计，我们识别出九类摩擦，并提出未来工作建议。
原文摘要 · Abstract (English)
Large language models (LLMs) learn statistical associations from massive training corpora and user interactions, and deployed systems can surface or infer information about individuals. Yet people lack practical ways to inspect what a model associates with their name. We report interim findings from an ongoing study and introduce LMP2, a browser-based self-audit tool. In two user studies ($N_{total}{=}458$), GPT-4o predicts 11 of 50 features for everyday people with $\ge$60\% accuracy, and participants report wanting control over LLM-generated associations despite not considering all outputs privacy violations. To validate our probing method, we evaluate eight LLMs on public figures and non-existent names, observing clear separation between stable name-conditioned associations and model defaults. Our findings also contribute to exposing a broader generative AI evaluation crisis: when outputs are probabilistic, context-dependent, and user-mediated through elicitation, what model--individual associations even include is under-specified and operationalisation relies on crafting probes and metrics that are hard to validate or compare. To move towards reliable, actionable human-centred LLM privacy audits, we identify nine frictions that emerged in our study and offer recommendations for future work and the design of human-centred LLM privacy audits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。