COMPARATIVE PERFORMANCE ANALYSIS OF SOME ACCELERATED AND HYBRID ACCELERATED GRADIENT MODELS

  • Milena Petrović Faculty of mathematics and natural sciences, University of Priština, Kosovska Mitrovica
  • Milica Ivanović Faculty of mathematics and natural sciences, University of Priština, Kosovska Mitrovica
  • Marijana Đorđević Faculty of mathematics and natural sciences, University of Priština, Kosovska Mitrovica
DOI: https://doi.org/10.5937/univtho9-18174
Keywords: Gradient descent methods, Line search, Convergence rate,

Abstract


We analyze a performance profile of several accelerated and hybrid accelerated methods. All comparative methods are at least linearly convergent and have satisfied numerical characteristics regarding tested metrics: number of iterations, CPU time and number of function evaluations. Among the chosen set of methods we numerically show which one is the most efficient and the most effective. Therewith, we derived a conclusion about what type of method is more preferable to use considering analyzed metrics.

Author Biography

Milica Ivanović, Faculty of mathematics and natural sciences, University of Priština, Kosovska Mitrovica
Faculty of mathematics and natural sciences, University of Priština, Kosovska Mitrovica

References

Andrei, N. 2006. An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numerical Algorithms, 42(1), pp. 63-73. doi:10.1007/s11075-006-9023-9

Andrei, N. 2008. An unconstrained optimization test functions collection. Advanced Modeling and Optimization, 10 (1), pp. 147-161.

Khan, S. 2013. A Picard-Mann hybrid iterative process. Fixed Point Theory and Applications, 2013(1), p. 69. doi:10.1186/1687-1812-2013-69

Panić, S., Petrović, M.J., & Carević, M.M. 2018. Initial Improvement Of The Hybrid Accelerated Gradient Descent Process. Bulletin of the Australian Mathematical Society, 98(02), pp. 331-338. doi:10.1017/s0004972718000552

Petrović, M. J. 2015. An Accelerated Double Step Size model in unconstrained optimization. Applied Mathematics and Computation, 250, pp. 309-319. doi:10.1016/j.amc.2014.10.104

Petrović, M., & Kontrec, N. 2017. Determination of accelerated factors in gradient decent iterations based on Taylor's series. University thought. Nat. Sci., 7(1), pp. 41-45. doi:10.5937/univtho7-14337

Petrović, M., Rakočević, V., Kontrec, N., Panić, S., & Ilić, D.. 2018. Hybridization of accelearted gradient descent method. Numerical Algorithms, 79(3), pp 769–786, doi:10.1007/s11075-017-0460-4

Petrović, M. J., & Stanimirović, P. S. 2014. Accelerated Double Direction Method for Solving Unconstrained Optimization Problems. Mathematical Problems in Engineering, pp. 1-8. doi:10.1155/2014/965104

Picard, E. 1890. Memoire sur la theorie des equations aux derivees partielles et la methode des approximations successives. J. Math. Pures Appl., 6, pp. 145-210.

Stanimirović, P. S., & Miladinović, M. B. 2010. Accelerated gradient descent methods with line search. Numerical Algorithms, 54(4), pp. 503-520. doi:10.1007/s11075-009-9350-8

Stanimirović, P. S., Milovanović, G. V., Petrović, M. J., & Kontrec, N. Z. 2015. A Transformation of Accelerated Double Step Size Method for Unconstrained Optimization. Mathematical Problems in Engineering, pp. 1-8. doi:10.1155/2015/283679

Published
2019/01/26
Issue
Vol. 9 No. 1 (2019)
Section
Original Scientific Paper

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.