arXiv:2512.17092cs.CL2025-12

用合成与真实数据提升戒烟聊天机器人理解力,效果显著。

Data Augmentation Supporting a Conversational Agent Designed for Smoking Cessation Support Groups

  • 分两步增广数据:生成高质合成语料,爬取超万条真实帖子。
  • 模型F1得分提升32%,合成与真实数据效果相当。
  • 适合医疗对话系统、数据稀缺场景的研究者参考。

在线戒烟支持群虽经济便捷,但常面临用户参与度低和污名化问题。引入自动对话代理可及时回应,提升参与感。针对高质量数据不足的挑战,本文提出两级数据增广策略:先微调开源大模型识别低F1分数的用户意图,再用GPT通过提示工程生成合成数据,人工评估显示87%的合成帖子质量高。合成增广覆盖原帖43%,扩展量达140%。同时从相关社区爬取超10,000条真实帖子,其中73%经人工验证为高质量。所有新增数据均经人工审核确保质量与相关性。将扩充数据与原始数据结合,重新训练意图分类器,性能评估显示F1提升32%,验证了该方法的有效性。合成与真实数据增广带来相似改进。本研究提供了一个可复现的框架,适用于数据稀缺领域的对话系统优化。

原文摘要 · Abstract (English)

Online support groups for smoking cessation are economical and accessible, yet they often face challenges with low user engagement and stigma. The use of an automatic conversational agent would improve engagement by ensuring that all user comments receive a timely response.). We address the challenge of insufficient high-quality data by employing a two-level data augmentation strategy: synthetic data augmentation and real data augmentation. First, we fine-tuned an open source LLM to classify posts from our existing smoking cessation support groups and identify intents with low F1 (precision+recall) scores. Then, for these intents, we generate additional synthetic data using prompt engineering with the GPT model, with an average of 87\% of the generated synthetic posts deemed high quality by human annotators. Overall, the synthetic augmentation process resulted in 43\% of the original posts being selected for augmentation, followed by 140\% synthetic expansion of these posts. Additionally, we scraped more than 10,000 real posts from a related online support context, of which 73\% were validated as good quality by human annotators. Each synthetic or scraped post underwent rigorous validation involving human reviewers to ensure quality and relevance. The validated new data, combined with the original support group posts, formed an augmented dataset used to retrain the intent classifier. Performance evaluation of the retrained model demonstrated a 32\% improvement in F1, confirming the effectiveness of our data augmentation approach. Synthetic and real post augmentation led to similar performance improvements. This study provides a replicable framework for enhancing conversational agent performance in domains where data scarcity is a critical issue.

对话系统数据增广戒烟支持大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。