Using the correct intervals for prediction: A tutorial on tolerance intervals for ordinary least-squares regression

The use of regression models for prediction purposes is one of the most common applications of chemometrics. In most cases, only reporting the point predictions is unsatisfactory and intervals are needed to quantify the uncertainty involved. It is common practice to use the 95% prediction interval for this goal and to interpret such an interval as if 95% of the future responses will be contained within this interval. However, this is inappropriate, and it can result in a gross underestimation of the real uncertainty of the predicted response, especially in cases where the degrees of freedom is small. To correctly quantify the uncertainty of a prediction, so-called tolerance intervals should be used. Although the theoretical background on tolerance intervals is well documented in the statistical literature, practical guidelines to calculate and use tolerance intervals in real-world applications are lacking. Another less known concept is that of simultaneous versus non-simultaneous intervals. In this tutorial, we explain the origin, interpretation and practical calculation of confidence, prediction, and tolerance intervals, both in their simultaneous and non-simultaneous form. Depending on the case, equations for exact or approximate tolerance intervals are provided. The accuracy of the approximations is discussed. A MATLAB program is available to estimate the exact width of tolerance intervals in all cases involving ordinary least squares regression.