Automated data quality assurance for marine observations

The ocean monitoring community requires high quality data that is Data Management and Communications (DMAC)-compliant for both near real time and climate data records. The authors describe a flexible and cost effective automated data quality assurance (ADQA) system that can be used to assess the quality of marine observations and provide quality controlled data to a wide variety of end users. For example, if a researcher needs data sets from different sources for a modeling project, ADQA provides a means of characterizing the relative quality of these input sets. The ADQA system has been implemented by integrating a set of data quality algorithms based upon National Data Buoy Center (NDBC) Technical Document 03-02 "Handbook of Automated Data Quality Control Checks and Procedures of the National Data Buoy Center" into a processing system that was created using the CALIPSO science data processing framework. The CALIPSO framework is actually a library of reusable components that provide the core non-science functionality of any science data processing system. This design separates the science processing components from the basic, reusable system infrastructure and allows for the addition or removal of algorithms with relative ease. The generic infrastructure of the framework includes substantial core functionality that is common to many science data management applications and is easily configurable to work with a wide variety of marine instruments and science data sets. The basic architecture includes the following subsystems: Input, Control, Data Store and Output. The system is a modular design that simplifies the integration of additional data quality assurance (DQA) processing components.