测试大模型在角色权限下的拒绝能力，提升系统安全性。
Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models
- 构建带角色权限的文本转SQL数据集，模拟真实数据库访问控制。
- 两阶段验证框架显著提高拒绝准确率，减少错误放行。
- 微调模型在安全与可用性间取得更好平衡，适合实际部署。
访问控制是安全计算的核心，但大语言模型常模糊角色边界，产生不受限响应。本文研究角色条件下的拒绝行为，评估模型在授权时回答、未授权时拒绝的能力。为此，我们扩展了Spider和BIRD文本转SQL数据集，引入真实的PostgreSQL基于角色的权限策略（表级与列级）。比较三种方案：(i) 零样本或少样本提示，(ii) 生成-验证两阶段流程，检查生成的SQL是否符合策略，(iii) 通过LoRA微调使模型直接学习权限感知。在多个模型家族中，显式验证（两阶段框架）提升了拒绝精度并降低了误放行。同时，微调模型在安全性与实用性之间实现更优平衡（考虑执行准确率）。更长、更复杂的策略会持续降低所有系统的可靠性。相关数据集与代码已开源。
原文摘要 · Abstract (English)
Access control is a cornerstone of secure computing, yet large language models often blur role boundaries by producing unrestricted responses. We study role-conditioned refusals, focusing on the LLM's ability to adhere to access control policies by answering when authorized and refusing when not. To evaluate this behavior, we created a novel dataset that extends the Spider and BIRD text-to-SQL datasets, both of which have been modified with realistic PostgreSQL role-based policies at the table and column levels. We compare three designs: (i) zero or few-shot prompting, (ii) a two-step generator-verifier pipeline that checks SQL against policy, and (iii) LoRA fine-tuned models that learn permission awareness directly. Across multiple model families, explicit verification (the two-step framework) improves refusal precision and lowers false permits. At the same time, fine-tuning achieves a stronger balance between safety and utility (i.e., when considering execution accuracy). Longer and more complex policies consistently reduce the reliability of all systems. We release RBAC-augmented datasets and code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。