Computation networks to implement optimization

This paper presents threshold binary networks and block-based gradient estimation networks to optimize the square error objective function and recover the regularized least squares (LS) solution. The threshold binary networks consist of linear processing elements with threshold nonlinearities to produce binary outputs. The objective function is expressed at the bit-level and the optimization takes place on partitions of the networks. Partitions are active only if the optimization takes place at the locations in that partition. The optimization process switches between active and inactive partitions to continue the progress toward deeper and more stable minima (lower error energy/cost) of the objective function. Block-based gradient networks use local estimates of the gradient and in-place convolution operations to recover the LS estimate. Regularization controls the rate of convergence to the LS estimate.