Publication Bias in the Computer Science Education Research Literature

Abstract: Publication bias is the tendency for investigations with primarily nonstatis- tically significant findings to be withheld from the research record. Because publication bias has serious negative consequences for research and practice, we gathered informa- tion about the prevalence and predictors of publication bias in the computer science education literature. From an initial random sample of 352 recent computer science education articles, we reviewed the 38 empirical articles that used inferential statistical analyses. We found that (a) the proportion of articles reporting primarily statistically significant findings in computer science education was very similar to the proportion in medical research, (b) that an article’s having a female first author was a strong predictor of an article’s having primarily statistically significant results, and (c) that there was a tendency for authors to emphasize statistically significant findings and deemphasize nonstatistically significant findings. Neither whether an investigation was reported in a journal or conference proceeding nor whether the source of funding was disclosed were significant predictors of an article’s having statistically positive results.