评测5大LLM在威胁情报增强中的易用性，助安全人员高效可信地使用。
Evaluating the Usability of LLMs in Threat Intelligence Enrichment
- 通过界面设计、错误处理等五方面评估主流LLM的易用性
- 发现各模型在学习曲线与工具集成上存在明显差异
- 适合安全从业者和AI工具设计者参考优化实践
大型语言模型（LLMs）有望通过自动化收集、预处理和分析威胁数据，显著提升威胁情报能力。然而，这些工具的可用性对安全专业人员的有效采用至关重要。尽管LLMs具备先进功能，但其可靠性、准确性以及生成错误信息的潜在风险仍令人担忧。本研究针对五款LLM——ChatGPT、Gemini、Cohere、Copilot和Meta AI，从用户界面设计、错误处理、学习曲线、性能表现及与现有工具的集成等方面，开展全面的可用性评估。采用启发式走查与用户研究方法，识别关键可用性问题，并提出可操作的改进建议。研究旨在弥合LLM功能与用户体验之间的差距，推动更高效、更准确的威胁情报实践，确保工具兼具友好性与可靠性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have the potential to significantly enhance threat intelligence by automating the collection, preprocessing, and analysis of threat data. However, the usability of these tools is critical to ensure their effective adoption by security professionals. Despite the advanced capabilities of LLMs, concerns about their reliability, accuracy, and potential for generating inaccurate information persist. This study conducts a comprehensive usability evaluation of five LLMs ChatGPT, Gemini, Cohere, Copilot, and Meta AI focusing on their user interface design, error handling, learning curve, performance, and integration with existing tools in threat intelligence enrichment. Utilizing a heuristic walkthrough and a user study methodology, we identify key usability issues and offer actionable recommendations for improvement. Our findings aim to bridge the gap between LLM functionality and user experience, thereby promoting more efficient and accurate threat intelligence practices by ensuring these tools are user-friendly and reliable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。