用合成数据提升小模型在混合印地语英语对话中的表现
Sample-Efficient Language Model for Hinglish Conversational AI
- 用合成对话+真实数据混合训练，缓解语料稀缺问题
- 小模型经微调后在对话任务上表现媲美大模型
- 适合资源有限但需支持印地语英语混合对话的场景
本文提出一种面向混合印地语英语（Hinglish）对话系统的样本高效语言模型构建方法。由于拼写不统一、缺乏标准化及高质量对话数据稀少，该语言形式带来独特计算挑战。研究评估了Gemma3-4B和Qwen2.5-7B等多款预训练跨语言模型，并通过微调技术提升其在Hinglish对话任务上的表现。方法结合合成生成对话与现有Hinglish数据集，以应对数据匮乏。实验表明，在高质量代码混杂数据上适当微调的小参数模型，可在保持计算效率的同时实现具有竞争力的对话生成性能。
原文摘要 · Abstract (English)
This paper presents our process for developing a sample-efficient language model for a conversational Hinglish chatbot. Hinglish, a code-mixed language that combines Hindi and English, presents a unique computational challenge due to inconsistent spelling, lack of standardization, and limited quality of conversational data. This work evaluates multiple pre-trained cross-lingual language models, including Gemma3-4B and Qwen2.5-7B, and employs fine-tuning techniques to improve performance on Hinglish conversational tasks. The proposed approach integrates synthetically generated dialogues with insights from existing Hinglish datasets to address data scarcity. Experimental results demonstrate that models with fewer parameters, when appropriately fine-tuned on high-quality code-mixed data, can achieve competitive performance for Hinglish conversation generation while maintaining computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。