Browserbite: Accurate Cross-Browser Testing via Machine Learning over Image Features

Cross-browser compatibility testing is a time consuming and monotonous task. In its most manual form, Web testers open Web pages one-by-one on multiple browser-platform combinations and visually compare the resulting page renderings. Automated cross-browser testing tools speed up this process by extracting screenshots and applying image processing techniques so as to highlight potential incompatibilities. However, these systems suffer from insufficient accuracy, primarily due to a large percentage of false positives. Improving accuracy in this context is challenging as the criteria for classifying a difference as an incompatibility are to some extent subjective. We present our experience building a cross-browser testing tool (Browser bite) based on image segmentation and differencing in conjunction with machine learning. An experimental evaluation involving a dataset of 140 pages, each rendered in 14 browser-system combinations, shows that the use of machine learning in this context leads to significant accuracy improvement, allowing us to attain an F-score of over 90%.