测试大模型在家庭机器人中识别隐私需求的能力,发现其表现远不如人类。
Benchmarking LLM Privacy Recognition for Social Robot Decision Making
- 基于隐私完整性框架设计家庭场景测试用例
- 450人调研与10个大模型对比,人类与模型一致性低
- 尝试四种提示策略提升模型隐私判断,仍存明显不足
尽管机器人过去采用规则系统或概率模型进行人机交互,但大语言模型(LLMs)的快速发展为提升人机交互能力带来了新机遇。为实现这一潜力,机器人需收集音频、细粒度图像、视频及位置等数据,导致LLMs经常处理敏感个人信息,尤其是在家庭等私密环境中。鉴于效用与隐私风险之间的张力,评估当前LLMs对敏感数据的管理能力至关重要。本文旨在探索现成大模型在家庭机器人情境下的隐私意识程度。我们基于情境完整性(Contextual Integrity, CI)框架构建了一系列隐私相关场景,首先调查了450名用户对家庭机器人行为的隐私偏好,并分析其隐私取向如何影响选择;随后将相同场景与问题提供给10个前沿大模型进行测试,发现人类与模型间的一致性普遍较低。为进一步探究LLMs作为潜在隐私控制者的性能,我们实施了四种额外的提示策略并进行比较。研究讨论了所评模型的表现及其在人机交互中人工智能隐私意识的潜力与意义。
原文摘要 · Abstract (English)
While robots have previously utilized rule-based systems or probabilistic models for user interaction, the rapid evolution of large language models (LLMs) presents new opportunities to develop LLM-powered robots for enhanced human-robot interaction (HRI). To fully realize these capabilities, however, robots need to collect data such as audio, fine-grained images, video, and locations. As a result, LLMs often process sensitive personal information, particularly within private environments, such as homes. Given the tension between utility and privacy risks, evaluating how current LLMs manage sensitive data is critical. Specifically, we aim to explore the extent to which out-of-the-box LLMs are privacy-aware in the context of household robots. In this work, we present a set of privacy-relevant scenarios developed using the Contextual Integrity (CI) framework. We first surveyed users' privacy preferences regarding in-home robot behaviors and then examined how their privacy orientations affected their choices of these behaviors (N = 450). We then provided the same set of scenarios and questions to state-of-the-art LLMs (N = 10) and found that the agreement between humans and LLMs was generally low. To further investigate the capabilities of LLMs as potential privacy controllers, we implemented four additional prompting strategies and compared their results. We discuss the performance of the evaluated models as well as the implications and potential of AI privacy awareness in human-robot interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。