Lessons Learned from Building and Deploying a Code Review Analytics Platform

Tool-based code review is growing in popularity and has become a standard part of the development process at Mi-crosoft. Adoption of these tools makes it possible to mine data from code reviews and provide access to it. In this paper, we pre-sent an experience report for CodeFlow Analytics, a system that collects code review data, generates metrics from this data, and provides a number of ways for development teams to access the metrics and data. We discuss the design, design decisions and chal-lenges that we encountered when building CodeFlow Analytics. We contacted teams that used CodeFlow Analytics over the past two years and discuss what prompted them to use CodeFlow Ana-lytics, how they have used it, and what the impact has been. Fur-ther, we survey research that has been enabled by using the Code-Flow Analytics platform. We provide a series of lessons learned from this experience to help others embarking on a task of building an analytics platform in an enterprise setting.