arXiv:2511.08145cs.CL2025-11被引 3

小模型在梵语诗转散文任务上胜过大模型,证明通用模型不万能。

Still Not There: Can LLMs Outperform Smaller Task-Specific Seq2Seq Models on the Poetry-to-Prose Conversion Task?

  • 用专用编码器-解码器模型微调,比指令微调大模型更有效。
  • 专用模型在自动和人工评估中均显著领先,提升约15%以上。
  • 适合低资源语言研究者、古典文本处理专家参考。

大型语言模型(LLMs)常被视为跨自然语言任务的通用解决方案,尤其在英语领域。但这一假设是否适用于低资源、形态丰富的语言如梵语?我们通过对比指令微调和上下文提示的LLMs与小型专用编码器-解码器模型,在梵语诗歌转散文任务上进行了评估。该任务极具挑战性:梵语诗句具有自由词序与严格的韵律约束,其转换为规范散文(anvaya)需多步推理,包括复合词切分、依存关系解析与句法线性化。为此,我们对通用模型进行指令微调,并设计基于婆罗门语法与古典注释启发的上下文学习模板;同时,对ByT5-Sanskrit Seq2Seq模型进行全微调。实验表明,针对领域的微调使ByT5-Sanskrit显著优于所有指令驱动的LLM方法。人工评估结果强烈支持此结论,得分与肯德尔等级相关系数高度一致。此外,我们的提示策略为缺乏领域语料时提供无需微调的替代方案,且专用模型在跨域评估中表现稳健。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are increasingly treated as universal, general-purpose solutions across NLP tasks, particularly in English. But does this assumption hold for low-resource, morphologically rich languages such as Sanskrit? We address this question by comparing instruction-tuned and in-context-prompted LLMs with smaller task-specific encoder-decoder models on the Sanskrit poetry-to-prose conversion task. This task is intrinsically challenging: Sanskrit verse exhibits free word order combined with rigid metrical constraints, and its conversion to canonical prose (anvaya) requires multi-step reasoning involving compound segmentation, dependency resolution, and syntactic linearisation. This makes it an ideal testbed to evaluate whether LLMs can surpass specialised models. For LLMs, we apply instruction fine-tuning on general-purpose models and design in-context learning templates grounded in Paninian grammar and classical commentary heuristics. For task-specific modelling, we fully fine-tune a ByT5-Sanskrit Seq2Seq model. Our experiments show that domain-specific fine-tuning of ByT5-Sanskrit significantly outperforms all instruction-driven LLM approaches. Human evaluation strongly corroborates this result, with scores exhibiting high correlation with Kendall's Tau scores. Additionally, our prompting strategies provide an alternative to fine-tuning when domain-specific verse corpora are unavailable, and the task-specific Seq2Seq model demonstrates robust generalisation on out-of-domain evaluations.

梵语序列建模小模型提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。