SafeChat让聊天机器人更安全可靠，特别适合选举等高风险场景。
SafeChat: A Framework for Building Trustworthy Collaborative Assistants and a Case Study of its Usefulness
- 构建可追溯、有来源的聊天机器人框架，支持拒绝有害回答。
- 自动摘要长回复并评估可信度，提升用户理解与信任。
- 用CSV快速开发，无需深度AI知识，适合医疗、选举等领域。
协作式助手（如聊天机器人）是基于数据的任务辅助系统，支持自然交互完成任务。尽管在现代社会中具有关键作用，其可靠性与可信性仍受关注。当前基于大语言模型的聊天机器人（如ChatGPT、Gemini、DeepSeek）虽日益普及，但仍存在无法解释生成逻辑、可能输出不当内容、缺乏标准化可靠性测试、需深厚AI知识且开发周期长等问题，难以应用于选举、医疗等高敏感领域。为此，我们提出SafeChat框架，专注于信息检索类应用，具备三大特性：（a）安全性——通用领域设计，响应可溯源至权威来源（溯源性），并支持‘不回应’策略以规避危险回答；（b）可用性——自动提取式摘要长回复并标注来源，以及自动化可信度评估，可传达预期行为特征（如情感倾向）；（c）快速可扩展开发——采用CSV驱动流程、自动化测试，并兼容多种设备。我们基于开源平台Rasa实现SafeChat原型。案例研究展示了其在ElectionBot-SC中的应用，该机器人用于安全传播官方选举信息。SafeChat已在多个领域部署，验证了其潜力，代码已开源：https://github.com/ai4society/trustworthy-chatbot。
原文摘要 · Abstract (English)
Collaborative assistants, or chatbots, are data-driven decision support systems that enable natural interaction for task completion. While they can meet critical needs in modern society, concerns about their reliability and trustworthiness persist. In particular, Large Language Model (LLM)-based chatbots like ChatGPT, Gemini, and DeepSeek are becoming more accessible. However, such chatbots have limitations, including their inability to explain response generation, the risk of generating problematic content, the lack of standardized testing for reliability, and the need for deep AI expertise and extended development times. These issues make chatbots unsuitable for trust-sensitive applications like elections or healthcare. To address these concerns, we introduce SafeChat, a general architecture for building safe and trustworthy chatbots, with a focus on information retrieval use cases. Key features of SafeChat include: (a) safety, with a domain-agnostic design where responses are grounded and traceable to approved sources (provenance), and 'do-not-respond' strategies to prevent harmful answers; (b) usability, with automatic extractive summarization of long responses, traceable to their sources, and automated trust assessments to communicate expected chatbot behavior, such as sentiment; and (c) fast, scalable development, including a CSV-driven workflow, automated testing, and integration with various devices. We implemented SafeChat in an executable framework using the open-source chatbot platform Rasa. A case study demonstrates its application in building ElectionBot-SC, a chatbot designed to safely disseminate official election information. SafeChat is being used in many domains, validating its potential, and is available at: https://github.com/ai4society/trustworthy-chatbot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。