Evaluating Volunteers´ Contributions in a Citizen Science Project

Today, with freely available data coming from different satellites and instruments, efficient algorithms for image classification, and increased connectivity and computing power, together with international policy initiatives, such as the United Nations Framework Convention of Climate Change Reducing Emissions from Deforestation and Forest Degradation (REDD) program, more and more countries are starting to invest in their own national forest monitoring schemes. Still, tropical forests remain under threat worldwide. Recently a citizen science project that enables citizens around the globe to be involved in forest monitoring tasks has been proposed. Called Forest Watchers (www.forestwatchers.net), its main goal is to allow volunteers (many of them with no scientific training) around the globe, with their own smartphones, tablets and notebooks, to review satellite images of forested regions, and confirm whether automatic assignments of forested and deforested regions are correct. Inspected images are then sent to a central database where results are integrated to generate up-to-date deforestation maps. This approach offers a low-cost way to both strengthen the scientific infrastructure and engage members of the public in science. Here we describe the procedures developed within the scope of the Forest Watchers project to assess the tasks performed by the volunteers. These procedures have been evaluated with data of one of the project´s preliminary applications. Called BestTile, it asks volunteers to select among several images of the same area, which one has the least cloud cover. Results from more than 500 volunteers show that with simple statistical tests it is possible to attain a triple goal: to increase the overall efficiency of the data collecting tasks, by reducing the required number of volunteers per task, to identify malicious behavior and outliers, and to motivate volunteers, to continue their contributions.