用大模型自动提取语言特征,检测临床对话中的越狱行为。
Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extraction
- 用医学和通用领域BERT模型直接预测四大语言特征。
- 跨验证与保留集测试中,系统检测准确率表现优异。
- 适合医疗AI安全监控,可解释性强,易扩展至其他场景。
在临床训练的大规模语言模型中检测越狱尝试,需精准建模暗示不安全或偏离任务的语义偏差。先前在2-Sigma临床模拟平台的研究表明,人工标注的语言特征可支持越狱检测,但依赖人工标注限制了可扩展性和表达能力。本研究通过专家标注四个核心语言特征(专业性、医学相关性、伦理行为、上下文干扰),训练多个通用与医学领域的BERT基大模型,直接从文本中预测这些特征。选取每个维度最可靠的特征回归器作为第二层分类器的特征提取器,评估了树模型、线性模型、概率模型及集成方法等多种预测模型,以判断越狱可能性。在交叉验证和保留数据集上均取得良好性能,表明大模型衍生的语言特征是自动化越狱检测的有效基础。误差分析揭示了当前标注与特征表示的关键局限,提示未来改进方向:更丰富的标注方案、更细粒度的特征提取,以及捕捉对话过程中越狱风险演化的机制。本工作展示了在关键医疗对话系统中可扩展且可解释的越狱行为检测方法。
原文摘要 · Abstract (English)
Detecting jailbreak attempts in clinical training large language models (LLMs) requires accurate modeling of linguistic deviations that signal unsafe or off-task user behavior. Prior work on the 2-Sigma clinical simulation platform showed that manually annotated linguistic features could support jailbreak detection. However, reliance on manual annotation limited both scalability and expressiveness. In this study, we extend this framework by using experts' annotations of four core linguistic features (Professionalism, Medical Relevance, Ethical Behavior, and Contextual Distraction) and training multiple general-domain and medical-domain BERT-based LLM models to predict these features directly from text. The most reliable feature regressor for each dimension was selected and used as the feature extractor in a second layer of classifiers. We evaluate a suite of predictive models, including tree-based, linear, probabilistic, and ensemble methods, to determine jailbreak likelihood from the extracted features. Across cross-validation and held-out evaluations, the system achieves strong overall performance, indicating that LLM-derived linguistic features provide an effective basis for automated jailbreak detection. Error analysis further highlights key limitations in current annotations and feature representations, pointing toward future improvements such as richer annotation schemes, finer-grained feature extraction, and methods that capture the evolving risk of jailbreak behavior over the course of a dialogue. This work demonstrates a scalable and interpretable approach for detecting jailbreak behavior in safety-critical clinical dialogue systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。