Home // International Journal On Advances in Security, volume 16, numbers 3 and 4, 2023 // View article

Evaluation of Threat Information Quality Provided by Twitter

Authors:
Ryu Saeki
Kazumasa Oida

Keywords: cyber security; Twitter; cyberthreat intelligence; threat information quality; ChatGPT.

Abstract:
Twitter (currently being re-branded as “X”) is widely used as a tool for collecting and disseminating information about the latest security incidents. However, the quality of threat information provided by Twitter (earliness, detailedness, and reliability) has not been studied sufficiently so far. Fresh information is required by both security experts and non-experts. Detailedness and reliability are also important criteria in two aspects. First, there are many accounts on Twitter, and anyone is free to post fake news. Second, true information that is not detailed is inconsequential to experts. This study compares the quality provided by Twitter and a security news site, which is expected to be very trustworthy. Because Emotet is having a serious impact in Japan, this study verified the earliness and detailedness by measuring when and how often words characterizing Emotet variants have appeared on Twitter and the news site in the past. Experiments revealed that Twitter alerted far earlier and more frequently about more diverse malware types, malicious attachment extensions, and malicious subject lines. Reliability was assessed based on two criteria: website reliability and text reliability. The news site was superior in terms of website reliability. In terms of text reliability, on the other hand, their difference was insignificant. The text reliability was derived from discrepancies between articles about the same security incidents, which were detected by humans and a state-of-the-art machine learning model. Overall, the quality of information on Twitter is higher than on the news site.

Pages: 106 to 115

Copyright: Copyright (c) to authors, 2023. Used with permission.

Publication date: December 30, 2023

Published in: journal

ISSN: 1942-2636