Electronic versus paper surveys: analysis of potential psychometric biases

The Internet and World Wide Web are increasingly being used for survey distribution and administration in both academic and practitioner research. Little systematic research exists on the efficacy of these survey administration media or their potential psychometric effects. This paper reports on a study of the potential biasing effects of online versus paper surveys. We consider issues specifically related to the information systems research context and introduce psychometric issues of general interest to those considering testing for response stability between online and paper survey administration. The study provides an assessment of the psychometric differences between paper-and-pencil and online survey administration for the well-known technology acceptance model (TAM) instrument. The results indicate that biasing effects can occur which significantly reduce the stability of an instrument across administration methods. We sound a cautionary note on the practice of placing any existing paper-and-pencil survey instrument on the Web without consideration of the biasing effects of Web-based survey administration.