用GPT生成标签缓解数据不平衡，提升手机端表情包预测准确率
On-Device Emoji Classifier Trained with GPT-based Data Augmentation for a Mobile Keyboard
- 用GPT自动生成表情标签，扩充稀有类别的训练数据
- 在移动端部署MobileBert模型，内存与延迟均满足实时要求
- 结合用户历史偏好动态调整输出，提升冷门表情使用率
表情符号能提升智能手机用户通过移动键盘交流的体验。为基于输入文本预测表情符号，需兼顾设备端低内存与低延迟限制，同时解决表情数据集普遍存在的类别不平衡问题，并使预测结果适配用户常用表情。本文提出一种面向SwiftKey的设备端表情分类器，基于MobileBert设计，在保证合理内存占用和延迟的前提下实现高效推理。针对数据不平衡，利用GPT为每个表情类别生成一个或多个标签，将原始样本与生成语句合并后以对应表情标记，无需人工干预即可缓解类别不均衡。推理阶段，通过融合用户历史表情使用记录对输出进行插值，提升预测准确性。实验表明，该方法在部署于SwiftKey时显著提升了稀有表情的预测准确率及用户表情使用活跃度。
原文摘要 · Abstract (English)
Emojis improve communication quality among smart-phone users that use mobile keyboards to exchange text. To predict emojis for users based on input text, we should consider the on-device low memory and time constraints, ensure that the on-device emoji classifier covers a wide range of emoji classes even though the emoji dataset is typically imbalanced, and adapt the emoji classifier output to user favorites. This paper proposes an on-device emoji classifier based on MobileBert with reasonable memory and latency requirements for SwiftKey. To account for the data imbalance, we utilize the widely used GPT to generate one or more tags for each emoji class. For each emoji and corresponding tags, we merge the original set with GPT-generated sentences and label them with this emoji without human intervention to alleviate the data imbalance. At inference time, we interpolate the emoji output with the user history for emojis for better emoji classifications. Results show that the proposed on-device emoji classifier deployed for SwiftKey increases the accuracy performance of emoji prediction particularly on rare emojis and emoji engagement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。