仅修改0.19%参数，就能让医疗模型彻底遗忘特定病患信息。
Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning
- 只更新关键词嵌入和小分类头，冻结全部编码器层。
- 在MIMIC-IV上实现近完全遗忘（遗忘F1=0.0004）且保留性能（保留F1=0.4766）。
- 适合需要快速合规删除敏感数据的医疗AI系统部署场景。
机器遗忘对临床语言模型愈发重要，因隐私法规与机构政策要求在不从头训练的前提下移除已部署系统中的敏感信息。实际中，删除请求需在有效遗忘目标信息与保持模型实用性、最小化参数修改间取得平衡。本文提出稀疏词元嵌入遗忘（STEU），一种参数高效的类级行为遗忘方法：仅更新由PMI选择的词元嵌入及少量分类头，其余编码器层完全冻结。在MIMIC-IV、MIMIC-III和eICU数据集上，使用BioClinicalBERT、BERT-base和DistilBERT进行实验，STEU始终有效抑制目标类，同时大幅保留其他任务性能。在主要的MIMIC-IV设置中，仅修改0.19%模型参数，即实现近乎完全遗忘（遗忘F1=0.0004），同时保持竞争力的保留性能（保留平均F1=0.4766）。结果表明，通过稀疏嵌入编辑即可实现针对性行为遗忘，无需修改深层编码表示。
原文摘要 · Abstract (English)
Machine unlearning is increasingly important for clinical language models, where privacy regulations and institutional policies may require removing sensitive information from deployed systems without retraining from scratch. In practice, deletion requests must balance effective forgetting of targeted information with preservation of model utility and minimal parameter modification. We introduce Sparse Token Embedding Unlearning (STEU), a parameter-efficient method for behavioral class-level unlearning that updates only PMI-selected token embeddings together with a small classifier head while keeping all encoder layers frozen. Across experiments on MIMIC-IV, MIMIC-III, and eICU using BioClinicalBERT, BERT-base, and DistilBERT, STEU consistently suppresses the target class while largely preserving retained task performance. In the primary MIMIC-IV setting, STEU achieves near-complete forgetting (forget F1 = 0.0004) while maintaining competitive retained utility (retain avg F1 = 0.4766) after modifying only 0.19\% of model parameters. These results suggest that targeted behavioral unlearning can be achieved through sparse embedding edits without modifying deeper encoder representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。