Practical relevance of experiments in comprehensibility of requirements specifications

Recently, the Requirements Engineering (RE) community has become increasingly aware of the importance of carrying out industry-relevant research. Researchers and practitioners should be able to evaluate the relevance of their empirical research to increase the likely adoption of RE methods in software industry. It is in this perspective that we evaluate 24 experimental studies on comprehensibility of software requirements specifications to determine their practical value. To that end a checklist based on Kitchenham´s approach was operationalized from a practitioner´s perspective and an analysis with respect to the main factors that affecting on comprehensibility was carried out. Although 100% of the papers reviewed reported statistically significant results, and 96% of them take examples from a real-life project. 80% of the papers do not scale to real life, 54% of the papers do not specify the context in which the results are expected to be useful. We also found that there is a lack of underlying theory in the formulation of comprehensibility questions.