BioSecure Signature Evaluation Campaign (ESRA´2011): evaluating systems on quality-based categories of skilled forgeries

In this paper, we present the main results of the BioSecure Signature Evaluation Campaign (ESRA\´2011). The objective of ESRA\´2011 is to evaluate through two different tasks the resistance of different online signature systems to skilled forgeries categorized automatically according to their quality. Task 1 aims at studying with only coordinate time functions the influence of acquisition conditions (digitizing tablet vs. PDA) on systems\´ performance. The two BioSecure Data Sets DS2 and DS3 make this possible, since they contain data from the same 382 people, acquired respectively on a digitizer and on a PDA. Task 2 then aims at assessing the contribution of the five time functions available on a digitizer (coordinates, pressure, pen inclination) on systems\´ resistance to different qualities of skilled forgeries. Results of the 13 systems involved in this competition are reported and analyzed for both tasks in this paper. We observe that the best system in terms of performance on forgeries of "bad" quality is not necessarily the most resistant to an increased quality of skilled forgeries. Also, we note that mobile conditions are still threatening independently of the quality of forgeries. Finally, when adding pen inclination time functions to pressure and coordinates, we find that the gap between systems in terms of performance is wider than when only pen coordinates and pressure are considered.