Do We Preach What We Practice? A Survey of Methods in Political Science Journals and Curricula

Scholars have recently focused increasing attention on the balance among alternative methodological approaches in empirical research and publications in political science.1 Key issues include whether formal modeling is taking, or should take, a more prominent role in political science or in its subfields; whether graduate training in the three leading methods (formal modeling, statistics, and qualitative research) is adequate; and whether the American Political Science Review (APSR), in some sense the flagship journal of the field, is or should be methodologically representative of the field as a whole and substantively representative of the sub-fields. Thus far, however, the strong opinions expressed on all sides of these questions have been accompanied by very little systematic data on the basic issues involved: how much research is published using each method and how this is changing over time; what methodological courses are being taught or required; and what mix of methods and subfields is represented in APSR and other leading journals in the field. To address these issues, we undertook a survey of the methods used in empirical research in over 2,200 articles in 10 top journals in political science in the United States, and a companion survey of the methodological courses required and offered in graduate programs at the top 30 political science departments in the United States. Our results challenge widelyheld assumptions about the prevalence of alternative research methods. In particular, four key findings emerge from our data. First, formal modeling does not appear to be increasingly prevalent in published journal research. Among the top journals that publish work from different methodological approaches, formal modeling was more common in articles published in 1985 than those published in the last several years. In general, the proportion of articles using each of the three leading approaches has remained relatively stable since 1975. This is in sharp contrast to the “behavioral revolution” of the 1960s and early 1970s, during which the proportion of research using statistics and formal modeling rose sharply and the proportion using qualitative methods dropped markedly. Second, there is a disjuncture between the high proportion of research performed with qualitative methods, the relatively low proportion of courses offered in these methods, and the even lower proportion of departments that require them. There is arguably a similar but smaller gap in the number of departments that offer or require formal modeling.3 All of the top 30 departments surveyed offered courses in statistical methods, while only two-thirds of the departments offered courses in formal modeling and qualitative methods. Two-thirds of the departments required students to take statistics, while formal modeling and qualitative methods were required in only two departments. Third, qualitative or case study research in American Politics is in steep decline in most of the top journals. In the top seven multimethod journals, the proportion of articles presenting case studies in American Politics fell from 12% in 1975, to 7% in 1985, to 1% in 1999–2000. Fourth, the mix of articles in APSR has been highly unrepresentative of the substantive and methodological mix in other journals. In the late 1990s, APSR had almost twice the average proportion of articles using formal modeling and articles on American Politics than the top 10 journals, while it had about half the average of articles in International Relations and less than one-fourth the average of articles using case studies. Perhaps most striking, of the International Relations articles in APSR, only one in 20 used case studies, compared to more than four out of 10 of the International Relations articles among the top 10 journals. We detail these results below and conclude with recommendations for maintaining a productive balance of methods in departments, curricula, and journals, and for increasing multimethod collaboration among researchers.