Regularization parameter tuning optimization approach in logistic regression

Under regression analysis methods, logistic regression comes and it got popular since it has proved its effectiveness in modelling categorical outcomes as a function of either continuous -real value- or categorical - yes vs. no- variables. The coefficients of this prediction function are based on a data set that is used to shape this function. However, sometimes the dataset, which is used to generate the prediction function of the logistic regression, would have some odds and need to be smoothened to avoid under or over-fitting. Thus, a mathematical regularization part has been introduced to be added to the cost function of logistic regression and it mainly contains an important parameter which is called the regularization parameter that would have to be determined. Often, this regularization parameter is pre-set or pre-expected by the code developer. Few random values would be tested and judged by the accuracy rate of the prediction function applied on the testing set. Obviously, it´s neither effective nor practical approach to choose a parameter that would concretely play an important role in varying the accuracy of the prediction function. In this paper, we propose a mathematical approach to tune the regularization parameter in order to achieve the highest accuracy rate possible. Our idea is basically vectorizing the value of the regularization parameter that is fed to the cost function for the purpose of testing its accuracy. Hence, a vectorized value of the accuracy would be produced and each will correspond to a specific value of the regularization parameter. Lastly, the regularization parameter value which produced the highest accuracy rate would be the obvious suitable choice for the prediction function.