Validation requirements for diffuse reflectance soil characterization models with a case study of VNIR soil C prediction in Montana

There has been growing interest in the use of diffuse reflectance as a quick, inexpensive tool for soil characterization. Some studies, using techniques like Partial Least Squares (PLS) regression of 1st derivative spectra have reported predictive accuracies for soil Organic C (OC) and Inorganic C (IC) that approach the analytical limits of standard laboratory measures. We applied 1st derivative Visible and Near-Infrared (VNIR) reflectance PLS regression modeling to soil samples obtained from six sites with similar soils across three counties in north central Montana, with five completely random 30% test sets selected for model validation. We obtained–relative to estimated SEL (Standard Error of Laboratory reference measurements) of 1.07 and 0.97 g kg−1 for OC and IC, respectively–SECV (calibration Standard Error of Cross-Validation) values of 1.04–1.20 and 1.54–1.63 g kg−1, and SEP (validation Standard Error of Prediction) values of 1.09–1.27 and 1.43–1.63 g kg−1. These results, together with validation RPD (Residual Prediction Deviation) values ≥2, could suggest a stable, effective PLS calibration that could be applied to similar soils in the same physiographic region. However, when we attempted to predict soil C for each of the six sites in turn using the remaining five sites for calibration, the models failed completely at two of the six sites and gave inconsistent results at a third site despite pre-screening for spectral similarity. “One-off” local calibrations for this study required ∼20–35% of the full samples, which could be prohibitively expensive for many applications. The results of this study demonstrate that “pseudo-independent” validation (random selection of non-independent test samples) can overestimate predictive accuracy relative to independent validation. The spatial structure of calibration and validation samples matters a great deal. Greater care needs to be taken to ensure that validation samples are independent to a degree that matches intended model use.