针对医学通俗语言生成难题,提出高效可调的异构数据适配框架。
Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation
- 采用共享矩阵+独立矩阵的非对称LoRA结构,分离摘要与风格生成
- 引入语义不变约束,保持生成内容与原文语义一致
- 支持多风格切换,适合医疗科普、健康传播等场景
医学通俗语言生成(MLLG)对提升科学内容可及性至关重要。现有方法多采用低秩适配(LoRA)微调大模型,但面对多源异构数据时,标准LoRA难以兼顾语义保真度和多样化的通俗风格。本文提出Magical,一种面向异构数据的非对称LoRA架构:共享矩阵$A$用于抽象摘要,多个独立矩阵$B$实现多样风格生成;通过语义不变性约束缓解矩阵$A$的语义子空间偏移;引入推荐引导切换机制,外部接口控制模型在不同$B$矩阵间切换。在三个真实世界通俗语言生成数据集上的实验表明,Magical持续优于提示法、原始LoRA及其近期变体,同时减少31.66%可训练参数。代码已公开于https://github.com/tianlwang/Magical.git。
原文摘要 · Abstract (English)
Medical Lay Language Generation (MLLG) plays a vital role in improving the accessibility of complex scientific content for broader audiences. Recent literature to MLLG commonly employ parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) to fine-tuning large language models (LLMs) using paired expert-lay language datasets. However, LoRA struggles with the challenges posed by multi-source heterogeneous MLLG datasets. Specifically, through a series of exploratory experiments, we reveal that standard LoRA fail to meet the requirement for semantic fidelity and diverse lay-style generation in MLLG task. To address these limitations, we propose Magical, an asymmetric LoRA architecture tailored for MLLG under heterogeneous data scenarios. Magical employs a shared matrix $A$ for abstractive summarization, along with multiple isolated matrices $B$ for diverse lay-style generation. To preserve semantic fidelity during the lay language generation process, Magical introduces a Semantic Invariance Constraint to mitigate semantic subspace shifts on matrix $A$. Furthermore, to better adapt to diverse lay-style generation, Magical incorporates the Recommendation-guided Switch, an externally interface to prompt the LLM to switch between different matrices $B$. Experimental results on three real-world lay language generation datasets demonstrate that Magical consistently outperforms prompt-based methods, vanilla LoRA, and its recent variants, while also reducing trainable parameters by 31.66%. Our code is publicly available at https://github.com/tianlwang/Magical.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。