用加密计算实现音频信号的私密处理,支持语音识别与分类。
Quantized Approximate Signal Processing (QASP): Towards Homomorphic Encryption for audio
- 基于同态加密与量化神经网络,安全计算四种时频表示。
- 约简误差率,实现隐私保护下的语音特征分析与分类准确率提升。
- 提供参数选择启发式方法,适合关注音频隐私的研究者使用。
音频和语音数据在语音识别、说话人识别及心理健康监测等机器学习应用中日益普及,但音频设备被动采集引发严重隐私问题。全同态加密(FHE)可通过加密数据计算保护用户隐私,但此前在音频处理中难以安全计算时频表示这一关键步骤。本文提出一个完全安全的处理流程,利用FHE与量化神经网络操作,实现短时傅里叶变换(STFT)、梅尔滤波组、梅尔频率倒谱系数(MFCCs)和伽玛通滤波器四种基础时频表示的加密计算,并支持音频描述符与卷积神经网络(CNN)分类器的私密运算。此外,我们设计了近似STFT算法,降低计算开销与比特占用,适用于统计与机器学习分析。在VocalSet和OxVoc数据集上的实验表明,该方法实现了完全私密的音频标记统计分析与声乐练习分类;近似STFT显著降低错误率,优于传统FHE-STFT实现。还实现了基于原始音频的性别与声乐练习分类的完整私密化。最后,我们提出实用的参数选择启发式方案,使量化近似信号处理更易为研究者与实践者采用。
原文摘要 · Abstract (English)
Audio and speech data are increasingly used in machine learning applications such as speech recognition, speaker identification, and mental health monitoring. However, the passive collection of this data by audio listening devices raises significant privacy concerns. Fully homomorphic encryption (FHE) offers a promising solution by enabling computations on encrypted data and preserving user privacy. Despite its potential, prior attempts to apply FHE to audio processing have faced challenges, particularly in securely computing time frequency representations, a critical step in many audio tasks. Here, we addressed this gap by introducing a fully secure pipeline that computes, with FHE and quantized neural network operations, four fundamental time-frequency representations: Short-Time Fourier Transform (STFT), Mel filterbanks, Mel-frequency cepstral coefficients (MFCCs), and gammatone filters. Our methods also support the private computation of audio descriptors and convolutional neural network (CNN) classifiers. Besides, we proposed approximate STFT algorithms that lighten computation and bit use for statistical and machine learning analyses. We ran experiments on the VocalSet and OxVoc datasets demonstrating the fully private computation of our approach. We showed significant performance improvements with STFT approximation in private statistical analysis of audio markers, and for vocal exercise classification with CNNs. Our results reveal that our approximations substantially reduce error rates compared to conventional STFT implementations in FHE. We also demonstrated a fully private classification based on the raw audio for gender and vocal exercise classification. Finally, we provided a practical heuristic for parameter selection, making quantized approximate signal processing accessible to researchers and practitioners aiming to protect sensitive audio data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。