Empirical analysis of factors affecting malware URL detection

Many organizations, from antivirus companies to motivated volunteers, maintain blacklists of URLs suspected of distributing malware in order to protect users. Detection rates can vary widely, but it is not known why. We posit that much variation can be explained by differences in the type of malware and differences in the blacklists themselves. To that end, we conducted an empirical analysis of 722 malware URLs submitted to the Malware Domain List (MDL) over 6 months in 2012-2013. We ran each URL through VirusTotal, a tool that allowed us to check each URL against 38 different malware URL blacklists, within an hour from when they were first blacklisted by the MDL. We followed up on each for two weeks following. We then ran logisitic regressions and Cox proportional hazard models to identify factors affecting blacklist accuracy and speed. We find that URLs belonging to known exploit kits such as Blackhole and Styx were more likely to be blacklisted and blacklisted quicker. We also found that blacklists that are used to actively block URLs are more effective than those that do not, and furthermore that paid services are more effective than free ones.