Estimating standard errors of accuracy assessment statistics under cluster sampling

Cluster sampling is a viable sampling design for collecting reference data for the purpose of conducting an accuracy assessment of land-cover classifications obtained from remotely sensed data. The formulas for estimating various accuracy parameters such as the overall proportion of pixels correctly classified, the kappa coefficient of agreement, and userʹs and producerʹs accuracy are the same under cluster sampling and simple random sampling, but the formulas for estimating standard errors differ between the two designs. If standard error formulas appropriate for cluster sampling are not employed in an accuracy assessment based on this design, the reported variability of map accuracy statistics is likely to be grossly misleading. The proper standard error formulas for common map accuracy statistics are derived for one-stage cluster sampling. The validity of these standard error formulas is verified by a small simulation study, and the standard errors computed according to the usual simple random sampling formulas are shown to underestimate the true cluster sampling standard errors by 20–70% if the intracluster correlation is moderate.