COMPARATIVE PERFORMANCE ANALYSIS OF SOME ACCELERATED AND HYBRID ACCELERATED GRADIENT MODELS
Abstract
We analyze a performance profile of several accelerated and hybrid accelerated methods. All comparative methods are at least linearly convergent and have satisfied numerical characteristics regarding tested metrics: number of iterations, CPU time and number of function evaluations. Among the chosen set of methods we numerically show which one is the most efficient and the most effective. Therewith, we derived a conclusion about what type of method is more preferable to use considering analyzed metrics.
References
Andrei, N. 2006. An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numerical Algorithms, 42(1), pp. 63-73. doi:10.1007/s11075-006-9023-9
Andrei, N. 2008. An unconstrained optimization test functions collection. Advanced Modeling and Optimization, 10 (1), pp. 147-161.
Khan, S. 2013. A Picard-Mann hybrid iterative process. Fixed Point Theory and Applications, 2013(1), p. 69. doi:10.1186/1687-1812-2013-69
Panić, S., Petrović, M.J., & Carević, M.M. 2018. Initial Improvement Of The Hybrid Accelerated Gradient Descent Process. Bulletin of the Australian Mathematical Society, 98(02), pp. 331-338. doi:10.1017/s0004972718000552
Petrović, M. J. 2015. An Accelerated Double Step Size model in unconstrained optimization. Applied Mathematics and Computation, 250, pp. 309-319. doi:10.1016/j.amc.2014.10.104
Petrović, M., & Kontrec, N. 2017. Determination of accelerated factors in gradient decent iterations based on Taylor's series. University thought. Nat. Sci., 7(1), pp. 41-45. doi:10.5937/univtho7-14337
Petrović, M., Rakočević, V., Kontrec, N., Panić, S., & Ilić, D.. 2018. Hybridization of accelearted gradient descent method. Numerical Algorithms, 79(3), pp 769–786, doi:10.1007/s11075-017-0460-4
Petrović, M. J., & Stanimirović, P. S. 2014. Accelerated Double Direction Method for Solving Unconstrained Optimization Problems. Mathematical Problems in Engineering, pp. 1-8. doi:10.1155/2014/965104
Picard, E. 1890. Memoire sur la theorie des equations aux derivees partielles et la methode des approximations successives. J. Math. Pures Appl., 6, pp. 145-210.
Stanimirović, P. S., & Miladinović, M. B. 2010. Accelerated gradient descent methods with line search. Numerical Algorithms, 54(4), pp. 503-520. doi:10.1007/s11075-009-9350-8
Stanimirović, P. S., Milovanović, G. V., Petrović, M. J., & Kontrec, N. Z. 2015. A Transformation of Accelerated Double Step Size Method for Unconstrained Optimization. Mathematical Problems in Engineering, pp. 1-8. doi:10.1155/2015/283679
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.