语音识别错误会引发机器人执行危险指令，威胁智能体安全。
When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI
- 模拟语音识别错误，测试其对机器人安全响应的影响
- 部分错误使危险指令更模糊，削弱模型拒绝能力
- 自动纠错虽可降低风险，但效果不稳定，适合关注安全的开发者
我们研究了用户输入中的自动语音识别（ASR）错误是否会导致具身人工智能（EAI）模型产生不安全输出。通过模拟ASR错误，并结合现有安全基准（SafeAgentBench和POEX），评估不同错误对具身AI安全的影响。结果发现，某些错误保留语义结构但增加有害歧义，另一些则削弱模型拒绝行为，导致不安全计划被生成并执行。我们还发现，在某些情况下，自动纠正ASR错误可降低风险，但并非总是有效。总体而言，ASR错误显著增加了具身AI的安全风险。
原文摘要 · Abstract (English)
We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that ASR errors can lead to harmful instructions being accepted and executed by EAI models, thereby reducing safety. We simulate ASR errors and combine them with existing safety benchmarks (SafeAgentBench and POEX) to evaluate how different errors affect embodied AI safety. We find that some of them preserve semantic structure but increase harmful ambiguity, while others weaken the model refusal behaviour and allow unsafe plans to be generated and executed. We show that in some cases automatic correction of ASR errors can reduce the risk, but this is not always effective. Overall, we show that ASR errors lead to significant safety risks for embodied AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。