用两阶段微调让小模型精准回答临床问题并找对依据句子
QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment
- 分两步微调4位量化的小模型，先学医学知识再学答题风格
- 问答任务测试得分32.87，证据句匹配准确率达67.16
- 数据量太小是核心瓶颈，适合临床问答与信息抽取研究者
我们提出一个统一系统，同时解决ArchEHR-QA共享任务中的子任务3（答案生成）和子任务4（证据句对齐）。针对子任务3，采用两阶段量化低秩适配（QLoRA）对4位NF4量化后的Qwen3-4B进行微调：第一阶段在emrQA-MedSQuAD语料的30,000个样本上建立临床领域能力，第二阶段在20个标注开发集上学习特定输出风格。系统在官方test-2026数据集上总得分为32.87（BLEU=9.42，ROUGE-L=27.04，SARI=55.42，BERTScore=43.00，AlignScore=25.28，MEDCON=37.04）。针对子任务4，构建三种检索方法加权集成模型——带相对阈值的BM25、TF-IDF余弦相似度与微调交叉编码器——用于定位支持答案的病历句子，在100条测试集上实现微平均F1为67.16。实验表明，两个子任务均暴露同一根本挑战：20个标注训练样本不足以区分相关与无关临床句，提示数据增强是最具潜力的未来方向。
原文摘要 · Abstract (English)
We present a unified system addressing both Subtask 3 (answer generation) and Subtask 4 (evidence sentence alignment) of the ArchEHR-QA Shared Task. For Subtask 3, we apply two-stage Quantised Low-Rank Adaptation (QLoRA) to Qwen3-4B loaded in 4-bit NF4 quantisation: first on 30,000 samples from the emrQA-MedSQuAD corpus to establish clinical domain competence, then on the 20 annotated development cases to learn the task-specific output style. Our system achieves an overall score of 32.87 on the official test-2026 split (BLEU = 9.42, ROUGE-L = 27.04, SARI = 55.42, BERTScore = 43.00, AlignScore = 25.28, MEDCON = 37.04). For Subtask 4, we develop a weighted ensemble of three retrieval methods - BM25 with relative thresholding, TF-IDF cosine similarity, and a fine-tuned cross-encoder - to identify note sentences supporting a given gold answer, achieving a micro-F1 of 67.16 on the 100-case test set. Experiments reveal that both subtasks expose the same fundamental challenge: 20 annotated training cases are insufficient to distinguish relevant from irrelevant clinical sentences, pointing to data augmentation as the highest-leverage future direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。