DETERMINATION OF ACCELERATED FACTORS IN GRADIENT DESCENT ITERATIONS BASED ON TAYLOR’S SERIES
Abstract
In this paper the efficiency of accelerated gradient descent methods regarding the way of determination of accelerated factor is considered. Due to the previous researches we assert that the use of Taylor’s series of posed gradient descent iteration in calculation of accelerated parameter gives better final results than some other choices. We give a comparative analysis of efficiency of several methods with different approaches in obtaining accelerated parameter. According to the achieved results of numerical experiments we make a conclusion about the one of the most optimal way in defining accelerated parameter in accelerated gradient descent schemes.
References
Andrei, N. 2006. An acceleration of gradient descent algoritham with backtracing for unconstrained optimization. Numer. Algor, 42.
Andrei, N. 2008. An unconstrained optimization test functions collection. Advanced Modeling and Optimization, 10(1).
Fletcher, R., & Reeves, C. 1964. Function minimization by conjugate gradients. Comput. J., 7, pp. 149-154.
Petrović, M. 2015. An Accelerated Double Step Size Method In Unconstrained Optimization. Applied Mathematics and Computation, .
Petrović, M., Rakočević, V., Kontrec, N., Panić, S., & Ilić, D. 2016. Hibridization of accelearted gradient descent method. Numer. Algor., . under review.
Petrović, M., & Stanimirović, P. 2014. Accelerated Double Direction Method For Solving Unconstrained Optimization Problems. Mathematical Problems in Engineering, 2014, pp. 309-319.
Polak, E., & Ribiére, G. 1969. Note sur la convergence de méthodes de directions conjuguées. Revue Francaise Informat. Reserche Opérationnelle, 16, pp. 35-43.
Polyak, B.T. 1969. The conjugate gradient method in extreme problems. USSR Comp. Math. Math. Phys., 9, pp. 94-112.
Stanimirović, P.S., & Miladinović, M.B. 2010. Accelerated gradient descent methods with line search. Numer. Algor, 54, pp. 503-520.
Stanimirović, P.S., Milovanović, G.V., Petrović, M., & Kontrec, N.Z. 2015. Transformation of Accelerated Double Step Size Method for Unconstrained Optimization. Mathematical Problems in Engineering, .
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.