Regression-based parameter optimization for binary output systems

Binary Output Systems (BOSs) generate Bernoulli distributed outputs with the given parameter. Such systems are quite common in various fields, and the system performance is usually measured by success rate or correct rate. Traditional parameter optimization methods utilize system performance approximations calculated by averaging the binary outputs. The binary outputs are used only once in the approximation process, and little about the internal relationship between different binary outputs is considered. In this article, we propose a novel method named Iterative Binary Regression (IBR) for parameter optimization of BOSs. IBR tackles the binary outputs directly and utilizes every binary output repeatedly in the regression process. This feature makes IBR particularly effective when the amount of available binary outputs is small. Considering the distribution of the binary outputs, we propose regression methods based on Least Squared Estimation (LSE), Weighted Least Squared Estimation (WLSE) and Maximum Likelihood Estimation (MLE) for IBR. Numerical comparison with Simultaneous Perturbation Stochastic Approximation (SPSA) and Blind Random Search on hypothesized and real BOSs is provided to show the effectiveness of IBR.