On determination of the number of signals in spatially correlated noise

A method for determining the number of signals in a correlated noise field using two well-separated linear arrays of receivers was given by Zhang and Wong (1993). In this paper, we improve on this method with the use of new penalty functions. Three criteria are given, and it is proved that for a large class of penalty functions, the probability of incorrect detections by each of the new criteria is exponentially decreasing when the moment-generating function of the squared Euclidean norm of the observation vector is finite at some point. It is also proved that with these new criteria, the estimates of the number of signals are strongly consistent. Randomized penalty functions for the three criteria, based on samples, are presented, and their uses are then shown to give consistent estimation of the number of signals. The finite sample behavior of the proposed approaches are studied by Monte Carte simulation