用丢弃容忍度检测大模型输出是否离域，提升医疗等领域安全风险识别能力。
Polysemantic Dropout: Conformal OOD Detection for Specialized LLMs
- 基于多层丢弃容忍度构建新非符合度度量，结合置信区间理论保证误报率。
- 在医学专用模型上实现2%-37%的AUROC提升，显著优于基线方法。
- 适合对输出可靠性要求高的专业领域应用，如医疗、金融等关键系统。
我们提出一种针对专业化大语言模型的新型推理阶段域外（OOD）检测算法。尽管通过微调在域内任务上达到顶尖性能，专业化LLM在面对域外输入时仍可能产生错误或不可靠输出，危及关键应用场景。本方法采用归纳性合取异常检测（ICAD）框架，引入基于模型丢弃容忍度的新非符合度度量。受近期关于LLM多义性和冗余性的启发，我们假设域内输入比域外输入具有更高的丢弃容忍度。通过有效集成多层丢弃容忍度，提升检测性能，同时保持ICAD的理论误报率边界。在医疗专用模型上的实验表明，该方法在将域外数据点视为正例、域内测试数据点为负例的设定下，相较基线方法实现2%至37%的AUROC提升。
原文摘要 · Abstract (English)
We propose a novel inference-time out-of-domain (OOD) detection algorithm for specialized large language models (LLMs). Despite achieving state-of-the-art performance on in-domain tasks through fine-tuning, specialized LLMs remain vulnerable to incorrect or unreliable outputs when presented with OOD inputs, posing risks in critical applications. Our method leverages the Inductive Conformal Anomaly Detection (ICAD) framework, using a new non-conformity measure based on the model's dropout tolerance. Motivated by recent findings on polysemanticity and redundancy in LLMs, we hypothesize that in-domain inputs exhibit higher dropout tolerance than OOD inputs. We aggregate dropout tolerance across multiple layers via a valid ensemble approach, improving detection while maintaining theoretical false alarm bounds from ICAD. Experiments with medical-specialized LLMs show that our approach detects OOD inputs better than baseline methods, with AUROC improvements of $2\%$ to $37\%$ when treating OOD datapoints as positives and in-domain test datapoints as negatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。