Using activity traces to characterize programming behaviour beyond the lab

Systematically improving the efficiency of programmers requires understanding what activities occur during programming, which activities are inefficient and then assessing languages, tools and processes proposed to improve the situation. Conducting the experiments required to support a systematic approach is difficult for many reasons, including the lack of availability of experienced programmers and the common belief that individual programmer effectiveness varies greatly. In this paper, we investigate whether generic activity traces of how a programmer interacts with a development environment can help bridge between results gathered in the lab with how programming occurs in the field. We describe the kinds of information that can be gleaned from activity traces, consider whether positive indication of a behaviour seen in the lab translates to data collected from the field, and discuss challenges with gathering appropriate data and with using gathered data appropriately.