We need more coverage, stat! classroom experience with the software ICU

For empirical software engineering to reach its fullest potential, we must develop effective, experiential approaches to learning about it in a classroom setting. In this paper, we report on a case study involving a new approach to classroom-based empirical software engineering called the ldquoSoftware ICUrdquo. In this approach, students learn about nine empirical project ldquovital signsrdquo and use the Hackystat framework to put their projects into a virtual ldquointensive care unitrdquo where these vital signs can be assessed and monitored. We used both questionnaire and log data to gain insight into the strengths and weaknesses of this approach. Our evaluation provides both quantitative and qualitative evidence concerning the overhead of the system; the relative utility of different vital signs; the frequency of use; and the perceived appropriateness outside of the classroom setting. In addition to benefits, we found evidence of measurement dysfunction induced directly by the presence of the Software ICU. We compare these results to case studies we performed in 2003 and 2006 using the Hackystat framework but not the Software ICU. We use these findings to orient future research on empirical software engineering both inside and outside of the classroom.