The convergence properties of some descent conjugate gradient algorithms for optimization models

Volume 22, Issue 3, pp 204--215 http://dx.doi.org/10.22436/jmcs.022.03.02

Publication Date: July 31, 2020 Submission Date: May 20, 2020 Revision Date: June 03, 2020 Accteptance Date: June 15, 2020

Download PDF

Download XML

• Export citations
  • CITATIONS & REFERENCES
  • EndNote (.ENW)
  • Mendeley, JabRef (.BIB)
  • Papers, Zotero, Reference Manager, RefWorks (.RIS)
  • ARTICLE CITATION
  • EndNote (.ENW)
  • Mendeley, JabRef (.BIB)
  • Papers, Zotero, Reference Manager, RefWorks (.RIS)
  • REFERENCES
  • EndNote (.ENW)
  • Mendeley, JabRef (.BIB)
  • Papers, Zotero, Reference Manager, RefWorks (.RIS)

• 1298 Downloads
• 2364 Views

Authors

I. M. Sulaiman - Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Besut Campus, 22200, Malaysia. M. Mamat - Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Besut Campus, 22200, Malaysia. A. E. Owoyemi - Department of General Studies, Federal College of Agricultural Produce Technology, Hotoro GRA extension, Kano, Nigeria. P. L. Ghazali - Faculty of Business and Management, Universiti Sultan Zainal Abidin, Terengganu, Malaysia. M. Rivaie - Department of Computer Science and Mathematics, Universiti Teknologi Mara, Terengganu, Malaysia. M. Malik - Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Besut Campus, 22200, Malaysia.

Abstract

The three-term conjugate gradient (CG) algorithms are among the efficient variants of CG method for convex and nonconvex functions. This is because most three-term algorithms are constructed using the classical CG method whose numerical performance has been tested and convergence proved. In this paper, we present a modification of RMIL$+$ CG method proposed by Dai [Z. Dai, Appl. Math. Comput., \(\bf 267\) (2016), 297--300] based on the convergence analysis of RMIL (2012) CG method. Interestingly, the modified method possesses sufficient descent condition and the global convergence prove was established using exact minimization condition. We further extended the results of the modified RMIL$+$ to construct a three-term CG algorithm and also show that the method satisfies the sufficient descent condition under the strong Wolfe line search. Preliminary numerical results are reported based on known benchmark problems which show that the proposed methods are efficient and promising compare to other CG methods.

Share and Cite

Keywords

• Conjugate gradient method
• three-term CG algorithm
• line searches
• optimization models

MSC

•  65K05
•  65K10

References

• [1] M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal., 5 (1985), 121--124
  • View Article
  • Google Scholar


• [2] A. Y. Al-Bayati, W. H. Sharif, A new three-term non-linear conjugate gradient method for unconstrained optimization, Can. J. Sci. Eng. Math., 1 (2010), 108--124
  • View Article
  • Google Scholar


• [3] N. Andrei, A simple three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 241 (2013), 19--29
  • View Article
  • Google Scholar


• [4] E. M. L. Beale, A derivative of conjugate gradients, in: Numerical Methods for Nonlinear Optimization, Academic Press, London, 1972 (1972), 39--43
  • Google Scholar


• [5] Z. Dai, Comments on a new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 267 (2016), 297--300
  • View Article
  • Google Scholar


• [6] Y. H. Dai, Y. Yuan, Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal., 16 (1996), 155--164
  • View Article
  • MathSciNet
  • Google Scholar


• [7] Y. H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (2000), 177--182
  • View Article
  • MathSciNet
  • Google Scholar


• [8] E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201--213
  • View Article
  • Google Scholar


• [9] R. Fletcher, Practical Methods of Optimization, John Wiley & Sons, New York (1989)
  • View Article
  • Google Scholar


• [10] R. Fletcher, C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), 149--154
  • View Article
  • Google Scholar


• [11] P. L. Ghazali, I. Mohd, M. Mamat, W. M. A. W. Ahmad, Mathematical Modelling in Family Takaful, J. Appl. Sci., 11 (2011), 3381--3388
  • View Article
  • Google Scholar


• [12] M. R. Hestenes, E. L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. National Bureau Standards, 49 (1952), 409--436
  • View Article
  • Google Scholar


• [13] K. Kamfa, M. Y. Waziri, I. M. Sulaiman, M. Mamat, S. S. Abas, An Efficient Hybrid BFGS-CG Search Direction for Solving Unconstrained Optimization Problems, J. Adv. Res. Dyn. Control Syst., 12 (2020), 1035--1041
  • View Article
  • Google Scholar


• [14] J. K. Liu, Y. M. Feng, L. M. Zou, Some three-term conjugate gradient methods with the inexact line search, Calcolo, 55 (2018), 1--16
  • View Article
  • Google Scholar


• [15] G. Liu, J. Han, H. Yin, Global convergence of the Fletcher-Reeves algorithm with an inexact line search, Appl. Math. J. Chinese Univer., 10 (1995), 75--82
  • View Article
  • Google Scholar


• [16] Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms, J. Optim. Theory Appl., 69 (1991), 129--137
  • View Article
  • Google Scholar


• [17] J. K. Liu, Y. X. Zhao, X. L. Wu, Some three-term conjugate gradient methods with the new direction structure, Appl. Numer. Math., 150 (2020), 433--443
  • View Article
  • Google Scholar


• [18] M. Mamat, I. M. Sulaiman, M. Malik, F. Sukono, Z. A. Zakaria, An Efficient Spectral Conjugate Gradient Parameter with Descent Condition for Unconstrained Optimization, J. Adv. Res. Dyn. Control Syst., 12 (2020), 2487--2493
  • View Article


• [19] M. F. McGuire, P. Wolfe, Evaluating a restart procedure for conjugate gradients, IBM Research Center, Yorktown Heights (1973)
  • View Article
  • Google Scholar


• [20] O. O. Y. Osman, The convergence properties of RMIL$+$ conjugate gradient method under the strong Wolfe line search, Appl. Math. Comput., 367 (2020), 18 pages
  • View Article
  • Google Scholar


• [21] E. Polak, G. Ribiere, Note sur la convergence de directions conjugées, Rev. Francaise Informat Recherche Opertionelle, 3e annee, 16 (1969), 35--43
  • Google Scholar


• [22] B. T. Polyak, The conjugate gradient method in extremal problems, USSR Comput. Math. Math. Phys., 9 (1969), 94--112
  • View Article
  • Google Scholar


• [23] M. J. D. Powell, Nonconvex Minimization Calculations and the Conjugate Gradient Method, in: Numerical analysis, Springer-Verlag, Berlin, 1984 (1984), 122--141
  • View Article
  • Google Scholar


• [24] M. Rivaie, M. Mamat, W. J. Leong, M. Ismail, A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 218 (2012), 11323--11332
  • View Article
  • Google Scholar


• [25] I. M. Sulaiman, M. Mamat, A. Abashar, M. Rivaie, S. M. Fatma, Z. Salleh, The global convergence properties of an improved conjugate gradient method, Appl. Math. Sci., 9 (2015), 1857--1868
  • View Article
  • Google Scholar


• [26] I. M. Sulaiman, M. Mamat, A. O. Umar, K. Kamfa, E. N. Madi, Some Three-Term Conjugate Gradient Algorithms with Descent Condition for Unconstrained Optimization Models, J. Adv. Res. Dyn. Control Syst., 12 (2020), 2494--2501
  • View Article
  • Google Scholar


• [27] I. M. Sulaiman, S. Supian, M. Mamat, New class of hybrid conjugate gradient coefficients with guaranteed descent and efficient line search, IOP Conference Series: Materials Science and Engineering, 2019 (2019), 7 pages
  • View Article
  • Google Scholar


• [28] I. M. Sulaiman, U. A. Yakubu, M. Mamat, Application of spectral conjugate gradient method for Solving unconstrained optimization problems, Int. J. Optim. Control Theor. Appl., 10 (2020), 198--205
  • View Article
  • Google Scholar


• [29] L. Zhang, W. Zhou, D. Li, A descent modified Polak-Ribiere-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629--640
  • View Article
  • Google Scholar


• [30] L. Zhang, W. Zhou, D. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Methods Soft., 22 (2007), 697--711
  • View Article
  • Google Scholar


• [31] G. Zoutendijk, Nonlinear programming, computational methods, in: Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970 (1970), 37--86
  • View Article
  • Google Scholar