实时检测语音威胁，90毫秒内完成转写与安全判断。
VoiceSHIELD-Small: Real-Time Malicious Speech Detection and Transcription
- 端到端模型同时完成语音转写与有害内容识别
- 947段音频测试准确率99.16%，误检率2.33%
- 轻量设计适配中端显卡，适合部署于实时语音系统
语音交互正日益成为人机交互的主流方式，但也带来了提示注入、社会工程和恶意语音指令等安全风险。传统方法依赖先转写再过滤，存在延迟且忽略音频特征。本文提出VoiceSHIELD-Small，一个基于OpenAI Whisper-small编码器的轻量级模型，在同一阶段实现语音转写与安全检测。通过添加均值池化层和简单分类头，该模型在中端GPU上仅需90-120毫秒即可完成音频分类，同时进行转写。在包含947段音频的平衡数据集上，模型达到99.16%准确率和0.9865的F1分数，默认设置下漏检率为2.33%。交叉验证显示性能稳定（F1标准差=0.0026）。论文还阐述了模型设计、训练数据、性能权衡及负责任使用指南。VoiceSHIELD已开源，采用MIT许可，旨在推动语音人工智能安全研究与应用。
原文摘要 · Abstract (English)
Voice interfaces are quickly becoming a common way for people to interact with AI systems. This also brings new security risks, such as prompt injection, social engineering, and harmful voice commands. Traditional security methods rely on converting speech to text and then filtering that text, which introduces delays and can ignore important audio cues. This paper introduces VoiceSHIELD-Small, a lightweight model that works in real time. It can transcribe speech and detect whether it is safe or harmful, all in one step. Built on OpenAI's Whisper-small encoder, VoiceSHIELD adds a mean-pooling layer and a simple classification head. It takes just 90-120 milliseconds to classify audio on mid-tier GPUs, while transcription happens at the same time. Tested on a balanced set of 947 audio clips, the model achieved 99.16 percent accuracy and an F1 score of 0.9865. At the default setting, it missed 2.33 percent of harmful inputs. Cross-validation showed consistent performance (F1 standard deviation = 0.0026). The paper also covers the model's design, training data, performance trade-offs, and responsible use guidelines. VoiceSHIELD is released under the MIT license to encourage further research and adoption in voice AI security.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。