轻量级模型统一解决多种语音相关分类任务，兼顾隐私与效率
AnySimLite: A Lightweight Few-Shot Similarity Encoder for On-Device Speech-Adjacent Classification
- 融合词级与字级通道的轻量相似度编码器
- 少样本下性能达当前最优，模型体积不足基准的1/250
- 适合资源受限设备上的多任务语音分类应用
为降低智能手机等边缘设备上的隐私风险和推理延迟，轻量级本地模型对终端用户应用仍至关重要。许多应用涉及自然语言分类，但部署多个专用模型会带来内存负担。本文探讨：能否通过细粒度文本相似性建模，用单一轻量架构解决多个语音相关（SA）分类任务？我们提出 AnySimLite，一种结合词级与字符级通道的轻量相似度编码器，并配合数据集转换策略，在多个 SA 分类任务上评估。结果表明，其在少样本设置下持续实现最先进（SOTA）或接近 SOTA 性能，同时保持极低内存占用。最差情况下性能下降低于7%，模型尺寸不足 SOTA 基线 qLLaMA_LoRA-7B 的 1/250。
原文摘要 · Abstract (English)
To minimize privacy concerns and inference latency on edge devices like smartphones, lightweight on-device models remain important for end-user applications. Many of these applications involve natural language classification, but deploying multiple specialized models creates a memory footprint challenge. We investigate: Can a single lightweight architecture solve multiple Speech-Adjacent (SA) classification tasks through reduction to a nuanced text similarity formulation? We propose AnySimLite, a lightweight similarity encoder that combines word-level and character-level channels. Together with a dataset transformation strategy, we evaluate AnySimLite across multiple SA classification tasks and show that it consistently achieves state-of-the-art (SOTA) or SOTA-competitive performance in few-shot settings while maintaining a low memory footprint. Even in the worst case, the performance drop remains below 7% while using $<\frac{1}{250}^{\mathrm{th}}$ of the model size of the SOTA qLLaMA_LoRA-7B baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。