Adaptive RLS algorithm for nonlinearity modeling in the nanometrology system

The periodic nonlinearity in the nanometrology systems based on the laser heterodyne interferometers mainly arises from imperfect laser source and misalignment of their optical setup. The accuracy of the nanometric displacement measurements can be effectively limited by the periodic nonlinearity. In this paper, we model the periodic nonlinearity in a modified laser heterodyne interferometer by adaptive recursive least square (RLS) algorithm. It is shown that this approach can obtain optimal modeling parameters of the nonlinearity. The results show that the RLS algorithm has faster conversions speed and lower steady state mean square error (MSE) in the nonlinearity modeling, comparing the neural network approach.