Role and utility of metrics in data systems

Numerous systems have been developed and have been operating in recent years for processing, archiving and distributing data from remote sensing satellites and in situ observations and increasing the utility of data by providing data products and/or information services to users. Among such data systems, this paper considers those that NASA develops and/or utilizes in order to fulfill one of its strategic objectives: “Study Earth from space to advance scientific under- standing and meet societal needs.” Such data systems are of various sizes and meet a variety of requirements. Some data systems constitute NASA’s Core Capabilities, and provide the basic infrastructure for robust and reliable data capture, processing, archiving and distributing a set of data products to a large and diverse user community. The Earth Observing Data and Information System (EOSDIS) is one example of such a system. Another class of data systems, constituting Community Capabilities, provides specialized and innovative services to data users and/or research products offering new scientific insight. Such systems are generally supported by NASA through peer reviewed competition. This paper focuses on the community capabilities including projects under the Research, Education and Applications Solutions Network (REASoN). Well-established goals and metrics measuring progress relative to those goals are essential to assess effectiveness of any systems, programs or organizations. Detailed metrics are needed by system managers to observe performance and plan improvements. Metrics of interest to program managers tend to be oriented towards the utility of the systems in meeting strategic goals, and sizes of communities served. The metrics must be tailored to the types of community capabilities. For example, systems providing data and those providing services need to be assessed using different metrics. Metrics aggregated across systems are of interest in measuring program effec- tiveness supporting NASA’a science, applied science, and education objectives.