Evaluating books finding tools on social media: A case study of aNobii

A user study of aNobii was conducted with an aim to exploring possible criteria for evaluating social navigational tools. A set of measures designed to capture various aspects of the benefits provided by the tools was proposed. To test the applicability of these measures, a within-subject experimental design was adopted where fifty regular aNobii users searched alternately with three book-finding tools: browsing “friends’ bookshelves”, “similar bookshelves”, and “books by known authors”. Other than the self-report user experience and search result measures, the “choice set” model was used as a novel framework for navigational effectiveness. Further analyses were conducted to explore whether three aspects of reader preference, “preference insight”, “preference diversity”, and “reading involvement” might influence the performance of the tools. Some major findings are as follows. While the author browsing function was shown to be most efficient, browsing friends’ bookshelves was shown to generate more interesting and informative browsing experiences. Three evaluative dimensions were derived from our study: search experience, search efficiency, and result quality. The disagreement of these measures shows a need for a multi-faceted evaluative framework for these exploration-based navigational tools. Furthermore, interaction effects on performance were found between users’ preference characteristics and tools. While users with high preference insight relied more heavily on author browsing to obtain more accurate results, highly involved readers tended percentage wise to examine and select more titles when browsing friends’ bookshelves.