用大模型修复键盘音频中的错误,让嘈杂环境下的窃听攻击更有效。
Making Acoustic Side-Channel Attacks on Noisy Keyboards Viable with LLM-Assisted Spectrograms' "Typo" Correction
- 用视觉变换器捕捉长时间上下文信息,提升识别准确率
- 引入大语言模型纠正误识别,噪声环境下性能显著提升
- 轻量级微调模型即可达到接近GPT-4o的效果,适合实际部署
麦克风在设备中的广泛集成增加了声学侧信道攻击(ASCA)的机会,因为这些设备可捕获可能泄露敏感信息的击键音频。然而,当前最先进的ASCA模型(如卷积神经网络和混合模型CoAtNet)在真实嘈杂条件下仍表现有限。解决此问题需依赖:(i) 模型具备更强能力,从长序列中推断上下文信息,使初始嘈杂输入的单词与后续非噪声输入的同一单词匹配;或 (ii) 利用上下文修正误识别信息——因用户不会随意打字,而是选择最符合语境的词。本文首次证明两种策略均有效且互补,可使ASCA在现实中可行。我们发现现有方法未利用先进Transformer架构的优势,提出:(i) 视觉变换器(VTs)适用于捕捉长期上下文信息;(ii) 基于Transformer的大语言模型(LLMs)可用于修正模型误判。因此,我们提出首个融合VTs与LLMs的ASCA方法。实验表明,VTs在击键分类任务上超越原有CNN基准,达到SOTA;LLMs(如GPT-4o)能有效缓解真实噪声影响。在自然句子测试中,(i) 引入LLMs可显著提升纠错性能;(ii) 使用仅67倍小于GPT-4o的轻量级微调模型即可获得相当效果。
原文摘要 · Abstract (English)
The large integration of microphones into devices increases the opportunities for Acoustic Side-Channel Attacks (ASCAs), as these can be used to capture keystrokes' audio signals that might reveal sensitive information. However, the current State-Of-The-Art (SOTA) models for ASCAs, including Convolutional Neural Networks (CNNs) and hybrid models, such as CoAtNet, still exhibit limited robustness under realistic noisy conditions. Solving this problem requires either: (i) an increased model's capacity to infer contextual information from longer sequences, allowing the model to learn that an initially noisily typed word is the same as a futurely collected non-noisy word, or (ii) an approach to fix misidentified information from the contexts, as one does not type random words, but the ones that best fit the conversation context. In this paper, we demonstrate that both strategies are viable and complementary solutions for making ASCAs practical. We observed that no existing solution leverages advanced transformer architectures' power for these tasks and propose that: (i) Visual Transformers (VTs) are the candidate solutions for capturing long-term contextual information and (ii) transformer-powered Large Language Models (LLMs) are the candidate solutions to fix the ``typos'' (mispredictions) the model might make. Thus, we here present the first-of-its-kind approach that integrates VTs and LLMs for ASCAs. We first show that VTs achieve SOTA performance in classifying keystrokes when compared to the previous CNN benchmark. Second, we demonstrate that LLMs can mitigate the impact of real-world noise. Evaluations on the natural sentences revealed that: (i) incorporating LLMs (e.g., GPT-4o) in our ASCA pipeline boosts the performance of error-correction tasks; and (ii) the comparable performance can be attained by a lightweight, fine-tuned smaller LLM (67 times smaller than GPT-4o), using...
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。