Synthesizing Social Media Data Using Information Morphing

Intelligence analysts are often faced with information overload. This plethora of information makes manual analysis of intelligence data infeasible. Social Network Analysis (SNA) software can be used to automatically process and analyze data gathered from popular social media websites, such as Twitter. SNA software is often tested with small-scale data sets manually constructed by analysts to coincide with typical military scenarios. However, for SNA software to be practical for real world intelligence analysis, such software must be scalable when tested with large-scale data sets for which manual construction is too costly. An information morphing algorithm (Info Morph) is presented in this paper. Info Morph can generate large-scale synthetic data sets which follow specific scenarios by morphing existing large-scale real data sets. Morphing transforms the large-scale real data sets into large-scale synthetic data sets by replacing entity references according to a substitution table. In this paper we tested Info Morph with two Twitter data sets: 1) 1, 007 tweets from the Kandahar province in Afghanistan gathered in 2013 and 2) 738, 717 tweets and 10, 000 news articles gathered about the Egypt Unrest in 2011. Testing SNA software with Twitter data is important because tweets contain many abbreviations and acronyms which make them more challenging to parse. The first data set was morphed to coincide with part of a scenario designed for the Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) On The Move (OTM) exercise in 2013. The second data set was morphed to the Ali Baba data set, which is a synthetic data set containing simulated text communications about a terrorist plot in London.