用可交互树形结构高效发现大模型生成中的潜在有害内容。
InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation
- 将采样结果组织成可导航的树形结构，支持按需探索和过滤。
- 相比随机采样，覆盖相同有害响应只需最多5倍更少样本。
- 适合安全评估人员、模型训练团队快速识别潜在风险输出。
即使在评估中表现安全的大语言模型，在实际部署中仍可能生成有害响应。由于采样具有随机性，低概率的有害输出仍可能大规模暴露给用户。当前的人工评估流程通常为每个提示生成大量随机样本，并在静态表格中逐一审查，效率低下，且重复阅读近似前缀。为此，我们提出 InFerActive，一个交互式系统，将采样结果以可导航的可读短语树形式呈现，使评估者能够按需筛选、探索和扩展生成空间。InFerActive 采用广度优先采样这一新颖的树构建方法，在保持与随机采样相当的有害响应覆盖率的同时，最多可减少5.0倍的样本数量。两次受控用户研究（每组 N=12）表明，InFerActive 在评估效率和覆盖率上显著优于传统表格和基础树形基线。
原文摘要 · Abstract (English)
Even LLMs that appear safe during evaluation can still produce harmful responses in deployment. Because stochastic sampling yields different responses to the same prompt, low-probability harmful outputs can still reach users at scale. Common human evaluation workflows generate many random samples per prompt and review them in static spreadsheets. The practice scales poorly, forcing evaluators to repeatedly reread near-duplicate prefixes. To address this, we present InFerActive, an interactive system that visualizes sampling results as a navigable tree of readable phrases, allowing evaluators to filter, explore, and extend the generation space on demand. InFerActive utilizes breadth-first sampling, a novel tree construction procedure that matches the harmful-response coverage of random sampling while requiring up to 5.0x fewer samples. Two controlled user studies (N = 12 each) demonstrate that InFerActive significantly improves evaluation efficiency and coverage over both spreadsheet and basic tree baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。