Validation methods for calibrating software effort models

COCONUT calibrates effort estimation models using an exhaustive search over the space of calibration parameters in a Cocomo I model. This technique is much simpler than other effort estimation method yet yields PRED levels comparable to those other methods. Also, it does so with less project data and fewer attributes (no scale factors). However, a comparison between COCONUT and other methods is complicated by differences in the experimental methods used for effort estimation. A review of those experimental methods concludes that software effort estimation models should be calibrated to local data using incremental holdout (not jack knife) studies, combined with randomization and hypothesis testing, repeated a statistically significant number of times.