首个面向人文学科的高质量数据合成方法,提升大模型在开放领域表现。
HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs

- 以主题为中心构建14个学科体系,通过多轮过滤与角色化回译生成指令
- 产出23.7万条高质量指令数据,在16个基准上超越14个基线模型
- 适合需要增强人文社科理解能力的大模型开发者使用
高质量、多样化的数据对大语言模型至关重要,但人文学科与社会科学(HSS)数据稀缺且成本高。现有数据合成方法多聚焦封闭任务,难以应对HSS的开放性挑战。本文提出首个面向HSS的合成范式——HSS-Synth,涵盖14个主流学科领域。该方法包含三步:(1) 从网络语料中筛选并精炼种子文档,经人工评估验证;(2) 通过“需求+角色”设定,将种子文档回译为多样且忠实的指令,确保问答严格对齐;(3) 利用教师强制回答机制,在生成过程中注入种子文档,突破模型输出限制,减少幻觉,保持语调与完整性。最终生成23.7万条高质量指令微调样本,在16个基准上优于14个主流基线。基于Qwen3-8B-Base微调后达到新SOTA,接近官方版本,同时提升人类偏好与知识能力,无性能波动。实验验证其鲁棒性与可迁移性。代码已开源:https://github.com/pengr/HSS-Synth。
原文摘要 · Abstract (English)
High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly. Data synthesis is a viable alternative and succeeds on closed tasks, yet the humanities and social sciences (HSS) are overlooked, and their open-ended nature makes synthesis challenging. Moving beyond prior capability-centric, fragmented attempts, we adopt a subject-centric paradigm, define the first HSS domain system covering 14 mainstream fields, and introduce HSS-Synth, the first data synthesis pipeline for HSS. HSS-Synth comprises: (1) constructing seed documents from web corpora via multi-step filtering and text refinement evaluated by a judge; (2) specifying "requirements + persona" to backtranslate seed documents into diverse yet faithful instructions with a strict Q&A alignment check; and (3) breaking LLM response limits via teacher-forced Answering that feeds seed documents during response generation to anchor semantics, reduce hallucinations, and preserve tone and integrity. HSS-Synth yields 237k high-quality, diverse instruction-tuning samples that outperform 14 leading baselines on 16 benchmarks. The fine-tuned Qwen3-8B-Base sets a new SOTA and approaches the official Qwen3-8B, improving both human preference and knowledge capabilities without performance seesaws. Extensive experiments demonstrate HSS-Synth's robustness and transferability. Our code is publicly available at https://github.com/pengr/HSS-Synth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。