MaxMin: An Unbiased Outlier Detection Method

Purpose To study the outlier detection methods and supply practical tools for data mining. Methods A novel parametric outlier detection method called maximum minimum (MaxMin) was presented to solve the crucial outlier detection issue, i.e., the estimation of the distribution parameters of the target normal distribution ´contaminated´ with the outliers. Through an iterative rejection sampling procedure, a normal parametric probability density function was obtained to maximize the minimum ratio of nonparametric probability density to parametric probability density which is calculated based on each single sample. Simulation datasets, consisting of 2000 samples following the standard normal distribution contaminated by different portions of outlier following a Weibull distribution, were analysed to evaluate the performance of the MaxMin method. Results The simulation revealed that the distribution parameters could be precisely and stably estimated regardless of the proportion of the outliers in the whole dataset. The MaxMin procedure was mathematically demonstrated to be able to obtain the unbiased estimation of the distribution parameters when the outliers were not in the kernel region of the target distribution (e.g. [μ - σ, μ + σ]). Conclusion The unbiased estimation of the target distribution can be obtained by using MaxMin method under a relatively weak assumption. The outlier detection method has high precision and robustness.