无领域训练数据下,跨域文本生成效果更优。
Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data

- 用结构保持的增强方法提升模型跨域泛化能力
- 17亿参数模型在5个基准上均优于微调和零样本
- 适合无标注数据的跨领域文本生成任务
结构化数据(如表格、知识图谱、图表和时间序列)可转换为文本,但多数现有工作局限于特定任务与数据集,依赖特定任务训练数据或大语言模型的零样本能力。本文研究在无领域训练文本与测试参考答案的情况下,跨域多任务文本生成问题,面对领域、生成目标和输入结构差异大的挑战。对比数据驱动的知识蒸馏(DDKD)与零样本推理及微调,在五个基准上验证:固定模型规模(17亿参数),DDKD始终优于微调与零样本;且小模型在两个领域超越更大规模微调模型,其余三个领域表现相当。我们构建了五倍扩展的QUINTD-5,发现仅放大真实目标域输入收益有限,而我们的结构增强策略更有效且成本更低。
原文摘要 · Abstract (English)
Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (D2T) generation has focused on specific tasks and datasets, relying either on task-specific training data or on the zero-shot capabilities of large language models. We study cross-domain D2T generation in a setting where neither in-domain training text nor test references are available, and where domains, generation goals, and input structures vary substantially. We compare data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduce structure-preserving augmentation via structural subsampling and perturbation. Experiments on five benchmarks show that, at constant model size (1.7B parameters), DDKD consistently outperforms both fine-tuning and zero-shot inference. Moreover, the resulting small models outperform a much larger finetuned model on two of the five domains, achieving comparable performance on the remaining three. We further construct QUINTD-5, a fivefold extension of QUINTD-1, and show that simply scaling real target-domain inputs yields only modest gains, whereas our augmentation strategy remains more effective and more cost-efficient for cross-domain distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。