用视觉语言模型预测保释风险，发现其对弱势群体误判率高且自信错误。
Judging by Appearances? Auditing and Intervening Vision-Language Models for Bail Prediction
- 通过RAG引入法律先例，再微调视觉语言模型提升判断力。
- 多模型测试显示，干预后保释预测准确率显著提升。
- 适合关注算法公平性与司法科技伦理的研究者阅读。
大型语言模型（LLMs）已广泛用于基于案件报告和犯罪记录的法律判决预测。随着大规模视觉语言模型（VLMs）的普及，如今可结合罪犯图像与文本信息进行判决预测。此类系统可能带来意外后果，甚至被恶意利用。本文审计了独立VLM在保释决策任务中的表现，发现其在多个交叉群体中表现不佳，且会以极高置信度错误拒绝应获保释者。我们设计了多种干预算法：首先通过RAG管道引入法律先例，再采用创新微调方案对VLM进行优化。实验表明，这些干预显著提升了保释预测性能。本工作为未来在真实司法场景部署前，设计更智能的VLM干预策略奠定了基础。
原文摘要 · Abstract (English)
Large language models (LLMs) have been extensively used for legal judgment prediction tasks based on case reports and crime history. However, with a surge in the availability of large vision language models (VLMs), legal judgment prediction systems can now be made to leverage the images of the criminals in addition to the textual case reports/crime history. Applications built in this way could lead to inadvertent consequences and be used with malicious intent. In this work, we run an audit to investigate the efficiency of standalone VLMs in the bail decision prediction task. We observe that the performance is poor across multiple intersectional groups and models \textit{wrongly deny bail to deserving individuals with very high confidence}. We design different intervention algorithms by first including legal precedents through a RAG pipeline and then fine-tuning the VLMs using innovative schemes. We demonstrate that these interventions substantially improve the performance of bail prediction. Our work paves the way for the design of smarter interventions on VLMs in the future, before they can be deployed for real-world legal judgment prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。