交互式工具Amplio帮助发现文本数据盲区，提升模型安全测试多样性。
Exploring Empty Spaces: Human-in-the-Loop Data Augmentation
- 通过人机协作识别文本数据中的空白区域，系统化探索未知边缘场景。
- 18名专业红队成员使用后，生成了更高质量、多样且相关的模型安全测试提示。
- 适合安全评估、模型鲁棒性研究等需要人工参与的数据增强场景。
数据增强对提升机器学习模型的鲁棒性与安全性至关重要。然而，生成多样化的数据点以充分评估模型在边缘情况下的表现并缓解潜在风险，是一项耗时且依赖创造力的任务。本工作提出Amplio，一个交互式工具，帮助从业者在非结构化文本数据集中导航‘未知的未知’，通过系统识别待探索的空数据空间来提升数据多样性。Amplio包含三种人机协同的数据增强技术：基于概念的增广、插值增广和基于大语言模型的增广。在包含18名专业红队成员的用户研究中，我们验证了这些方法在生成高质量、多样化且相关性强的模型安全提示方面的有效性。结果表明，Amplio使红队成员能快速而富有创意地扩充数据，凸显了交互式增强流程的变革潜力。
原文摘要 · Abstract (English)
Data augmentation is crucial to make machine learning models more robust and safe. However, augmenting data can be challenging as it requires generating diverse data points to rigorously evaluate model behavior on edge cases and mitigate potential harms. Creating high-quality augmentations that cover these "unknown unknowns" is a time- and creativity-intensive task. In this work, we introduce Amplio, an interactive tool to help practitioners navigate "unknown unknowns" in unstructured text datasets and improve data diversity by systematically identifying empty data spaces to explore. Amplio includes three human-in-the-loop data augmentation techniques: Augment With Concepts, Augment by Interpolation, and Augment with Large Language Model. In a user study with 18 professional red teamers, we demonstrate the utility of our augmentation methods in helping generate high-quality, diverse, and relevant model safety prompts. We find that Amplio enabled red teamers to augment data quickly and creatively, highlighting the transformative potential of interactive augmentation workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。