Abstract :
Evaluation of research quality is becoming more important
in the field of library and information science, as in
other fields. This pilot study is a preliminary attempt to
address issues associated with determining the quality
of the published research in one area of library and information
science, specifically school librarianship. The
main aims were (1) to test the extent to which experienced
evaluators agreed in their rankings of published
research articles on the basis of quality and (2) to investigate
the approaches to evaluation used by the experienced
evaluators. A qualitative, naturalistic research
design was used. On the basis of a comprehensive literature
review, four approaches to evaluation were identified;
they were generally supported through an analysis
of the responses of the experienced evaluators. However,
although the majority of the evaluators agreed on
the article ranked lowest, basic statistical analyses
showed less agreement about the other articles. Although
subject knowledge (of the field of school librarianship
in this case) may have some influence on the
evaluations, cluster analysis suggests that there may be
differences in the value perceptions of the evaluators
that also carry weight. More research would be needed
to gain a better understanding of these value perceptions
and their relationship, if any, to the four approaches
to evaluation that were identified through the
literature.
