Benchmarking Short Sequence Mapping Tools

The development of next-generation sequencing instruments has lead to the generation of millions of short sequences in a single run. The process of aligning these reads to a reference genome is time consuming and demands the development of fast and accurate alignment tools. However, the current proposed tools make different compromises between the accuracy and the speed of mapping. Moreover, many important aspects are overlooked when comparing the performance of a newly developed tool to the state of the art. Therefore, there is a need for an objective evaluation method that covers the various aspects. In this work, we introduce a benchmarking suite to extensively analyze various tools with respect to the different comparison aspects and provide an objective comparison. In order to assess our work, we applied our benchmarking tests on seven well known mapping tools, namely, Bowtie, BWA, SOAP, MAQ, RMAP, GSNAP, and FANGS. Bowtie, BWA, SOAP, GSNAP, and FANGS are based on indexing the reference genome, whereas MAQ and RMAP are based on building hash tables for the reads. It is shown that the benchmarking tests reveal the strengths and weaknesses of each tool. In addition, the tests can be further applied to other tools. The results show that there is no clear winner. However, Bowtie maintained the best throughput for most of the tests while BWA performed better for longer read lengths.