A gradient method for unconstrained optimization in noisy environment

Abstract

A gradient method for solving unconstrained minimization problems in noisy environment is proposed and analyzed. The method combines line-search technique with Stochastic Approximation (SA) method. A line-search along the negative gradient direction is applied while the iterates are far away from the solution and upon reaching some neighborhood of the solution the method switches to SA rule. The main issue is to determine the switching point and that is resolved both theoretically and practically. The main result is the almost sure convergence of the proposed method due to a finite number of line-search steps followed by infinitely many SA consecutive steps. The numerical results obtained on a set of standard test problems confirm theoretical expectations and demonstrate the efficiency of the method. © 2013 IMACS.