Improved Metrics of the Impacts of Knowledge Management: Linking Proximal Outcomes to Critical Success Factors

In order to gain and maintain support for the KM system we need to employ several categories of metrics that have been in existence for some time. These include operational and behavioral indicators of use, such as number of users, level of utilization of the system, number of knowledge "nuggets" found or transferred, and the satisfaction of users of knowledge management systems. However, these metrics are of little interest or use to managers who are mostly concerned with the contributions of the system to the "bottom line" and other measures of organizational performance and success. Over the past several decades we have developed methodologies that track and relate immediate or proximal outputs and outcomes to the "downstream" or "up the tree" measures of most interest to managers. We have used two approaches: (1) Flow or stage models to display indicators and measures at several stages in an organizational process. We have applied this successfully in measuring the contributions of research and development (R&D) in various organizational settings. This approach can also be applied to measure the contributions of knowledge management systems to organizational success. (2) Criterion trees which display similar information in a vertical format, showing the relationships of criteria of success at adjacent levels. The critical success factors (CSFs) for the organization and the functional or operating units are linked at various levels to the direct or proximal outcomes and impacts from knowledge management systems. This paper describes our methodology and pilot studies designed for several federal technology organizations. The paper discusses benefits and limitations of the methodology.