‘‘Smallball’’ evaluation: a prescription for studying community-based information interventions*

Objectives: This paper argues that focused evaluation studies of community-based informational interventions conducted over the life-cycle of the project (ʹʹsmallballʹʹ studies) are more informative and useful than randomized experiments conducted only at the projectʹs conclusion (ʹʹpowerballʹʹ studies). Method: Based on two contrasting strategies in baseball, smallball and powerball studies are compared and contrasted, emphasizing how the distinctive features of community-based interventions lend advantage to smallball approaches. Results: Smallball evaluations have several important advantages over powerball evaluations: before system development, they ensure that information resources address real community needs; during deployment, they ensure that the systems are suited to the capabilities of the users and to community constraints; and, after deployment, they enable as much as possible to be learned about the effects of the intervention in environments where randomized studies are usually impossible. Implications: Many in informatics see powerball studies as the only legitimate form of evaluation and so expect powerball studies to be done. These expectations should be revised in favor of smallball studies.