arXiv:2606.29375cs.CL2026-06

根据临床信心与覆盖度动态调整医疗大模型的低秩适配预算。

TriageRA-CCF: Source-Side Clinical Confidence and Coverage Signals for Adaptive Rank Budgeting in Medical LLMs

论文配图:TriageRA-CCF: Source-Side Clinical Confidence and Coverage Signals for Adaptive Rank Budgeting in Medical LLMs
图 1 · 摘自论文原文
  • 基于源数据计算信心、覆盖度和反事实信号,指导低秩通道激活决策。
  • 在两个80亿参数模型上,平均准确率优于现有方法0.16~0.21点。
  • 适合需要高效精准医疗问答的场景,尤其关注资源受限部署。

医疗大语言模型通常采用固定低秩预算进行微调,但不同医学问题在置信度、临床覆盖范围及跨领域难度上差异显著。本文研究参数高效的自适应低秩预算分配:针对每个问题,适配器自主选择激活小、中、大三类LoRA秩通道。核心挑战在于,朴素的自适应路由机制易陷入不稳定选择或浪费资源而无法提升性能。为此提出TriageRA-CCF,一种仅依赖源训练数据计算的源端教师方案,融合三项信号:基础模型答案置信度、元数据单元临床覆盖度、反事实近失代理信号。这些信号联合监督一个直通式预算路由器(激活秩为{2,4,8}),并引入预算成本、熵与秩平衡正则化。在匹配的CMB-source训练协议下,TriageRA-CCF在Qwen3-8B与Llama3.1-8B上均取得优于LoRA、DoRA及MoELoRA基线的最佳平均准确率,相较最强外部基线分别提升+0.21与+0.16个百分点。组件消融实验表明,三类信号均提供有效预算监督,但其组合效果并非在所有主干模型上单调最优。

原文摘要 · Abstract (English)

Medical large language models are commonly adapted with a fixed low-rank budget, even though medical questions differ substantially in confidence, clinical coverage, and cross-domain difficulty. We study adaptive rank budgeting for parameter-efficient medical question answering: for each question, the adapter decides whether to activate a small, medium, or large subset of LoRA rank channels. The central challenge is that a naive adaptive budget router can collapse to unstable choices or spend capacity without improving shifted benchmarks. We propose TriageRA-CCF, a source-side teacher for adaptive rank-budgeted LoRA. It combines three signals computed only from source training data: base-model answer confidence, metadata-cell clinical coverage, and a counterfactual close-miss proxy. These signals supervise a straight-through budget router over active ranks {2,4,8}, together with budget-cost, entropy, and rank-balance regularization. Under a matched CMB-source training protocol, TriageRA-CCF achieves the best average accuracy among LoRA, DoRA, and MoELoRA baselines on both Qwen3-8B and Llama3.1-8B. The gains are modest and non-uniform across benchmarks: +0.21 average points over the strongest external baseline on Qwen3-8B and +0.16 on Llama3.1-8B. Component ablations show that confidence, coverage, and counterfactual signals all provide useful budget supervision, but their combination is not monotonically best on every backbone.

医疗LLM低秩适配自适应预算LoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。