arXiv:2508.15418cs.CLcs.AI2025-08被引 5

首个全流程开源的语音语言模型研究框架,解决可复现性难题

LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model

  • 构建从数据到评测的完整开源体系,支持端到端研究
  • 推出3.8亿参数模型,标准化得分0.72,超越同类模型
  • 适合语音与语言模型研究者快速搭建实验基准

大型语音-语言模型(LSLM)的发展因架构碎片化和透明度不足而受阻,导致研究难以系统比较与复现。与视觉-语言领域不同,该领域普遍存在只发布模型权重而无训练数据与配置的情况。为此,我们提出首个完全开源、端到端的大型语音-语言建模框架LLaSO,提供三大核心资源:(1) LLaSO-Align,包含1200万条语音-文本对齐数据;(2) LLaSO-Instruct,涵盖1350万条多任务指令微调数据;(3) LLaSO-Eval,支持标准化评估的可复现基准。为验证框架有效性,我们训练并发布了仅基于公开数据的3.8亿参数参考模型LLaSO-Base,其标准化得分为0.72,建立强且可复现的基线,优于现有可比模型。分析显示,更广的训练覆盖虽提升性能,但在未见任务尤其是纯音频场景中仍存在显著泛化差距。通过开放全部数据、基准与模型,LLaSO确立了统一的研究标准,推动社区协作发展。代码、数据集、预训练模型及结果已发布于https://github.com/EIT-NLP/LLaSO。

原文摘要 · Abstract (English)

The development of Large Speech-Language Models (LSLMs) has been slowed by fragmented architectures and a lack of transparency, hindering the systematic comparison and reproducibility of research. Unlike in the vision-language domain, the LSLM field suffers from the common practice of releasing model weights without their corresponding training data and configurations. To address these critical gaps, we introduce LLaSO, the first fully open, end-to-end framework for large-scale speech-language modeling. LLaSO provides the community with three essential resources: (1) LLaSO-Align, a 12M-instance speech-text alignment corpus; (2) LLaSO-Instruct, a 13.5M-instance multi-task instruction-tuning dataset; and (3) LLaSO-Eval, a reproducible benchmark for standardized evaluation. To validate our framework, we build and release LLaSO-Base, a 3.8B-parameter reference model trained exclusively on our public data. It achieves a normalized score of 0.72, establishing a strong, reproducible baseline that surpasses comparable models. Our analysis reveals that while broader training coverage enhances performance, significant generalization gaps persist on unseen tasks, particularly in pure audio scenarios. By releasing the complete stack of data, benchmarks, and models, LLaSO establishes a foundational open standard to unify research efforts and accelerate community-driven progress in LSLMs. We release the code, dataset, pretrained models, and results in https://github.com/EIT-NLP/LLaSO.

语音语言模型可复现研究开源框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。