# The convergence properties of some descent conjugate gradient algorithms for optimization models

Volume 22, Issue 3, pp 204--215
Publication Date: July 31, 2020 Submission Date: May 20, 2020 Revision Date: June 03, 2020 Accteptance Date: June 15, 2020
• 64 Views

### Authors

I. M. Sulaiman - Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Besut Campus, 22200, Malaysia. M. Mamat - Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Besut Campus, 22200, Malaysia. A. E. Owoyemi - Department of General Studies, Federal College of Agricultural Produce Technology, Hotoro GRA extension, Kano, Nigeria. P. L. Ghazali - Faculty of Business and Management, Universiti Sultan Zainal Abidin, Terengganu, Malaysia. M. Rivaie - Department of Computer Science and Mathematics, Universiti Teknologi Mara, Terengganu, Malaysia. M. Malik - Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu, Besut Campus, 22200, Malaysia.

### Abstract

The three-term conjugate gradient (CG) algorithms are among the efficient variants of CG method for convex and nonconvex functions. This is because most three-term algorithms are constructed using the classical CG method whose numerical performance has been tested and convergence proved. In this paper, we present a modification of RMIL$+$ CG method proposed by Dai [Z. Dai, Appl. Math. Comput., $\bf 267$ (2016), 297--300] based on the convergence analysis of RMIL (2012) CG method. Interestingly, the modified method possesses sufficient descent condition and the global convergence prove was established using exact minimization condition. We further extended the results of the modified RMIL$+$ to construct a three-term CG algorithm and also show that the method satisfies the sufficient descent condition under the strong Wolfe line search. Preliminary numerical results are reported based on known benchmark problems which show that the proposed methods are efficient and promising compare to other CG methods.

### Share and Cite

##### ISRP Style

I. M. Sulaiman, M. Mamat, A. E. Owoyemi, P. L. Ghazali, M. Rivaie, M. Malik, The convergence properties of some descent conjugate gradient algorithms for optimization models, Journal of Mathematics and Computer Science, 22 (2021), no. 3, 204--215

##### AMA Style

Sulaiman I. M., Mamat M., Owoyemi A. E., Ghazali P. L., Rivaie M., Malik M., The convergence properties of some descent conjugate gradient algorithms for optimization models. J Math Comput SCI-JM. (2021); 22(3):204--215

##### Chicago/Turabian Style

Sulaiman, I. M., Mamat, M., Owoyemi, A. E., Ghazali, P. L., Rivaie, M., Malik, M.. "The convergence properties of some descent conjugate gradient algorithms for optimization models." Journal of Mathematics and Computer Science, 22, no. 3 (2021): 204--215

### Keywords

• three-term CG algorithm
• line searches
• optimization models

•  65K05
•  65K10

### References

• [1] M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal., 5 (1985), 121--124

• [2] A. Y. Al-Bayati, W. H. Sharif, A new three-term non-linear conjugate gradient method for unconstrained optimization, Can. J. Sci. Eng. Math., 1 (2010), 108--124

• [3] N. Andrei, A simple three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 241 (2013), 19--29

• [4] E. M. L. Beale, A derivative of conjugate gradients, in: Numerical Methods for Nonlinear Optimization, Academic Press, London, 1972 (1972), 39--43

• [5] Z. Dai, Comments on a new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 267 (2016), 297--300

• [6] Y. H. Dai, Y. Yuan, Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal., 16 (1996), 155--164

• [7] Y. H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (2000), 177--182

• [8] E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201--213

• [9] R. Fletcher, Practical Methods of Optimization, John Wiley & Sons, New York (1989)

• [10] R. Fletcher, C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), 149--154

• [11] P. L. Ghazali, I. Mohd, M. Mamat, W. M. A. W. Ahmad, Mathematical Modelling in Family Takaful, J. Appl. Sci., 11 (2011), 3381--3388

• [12] M. R. Hestenes, E. L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. National Bureau Standards, 49 (1952), 409--436

• [13] K. Kamfa, M. Y. Waziri, I. M. Sulaiman, M. Mamat, S. S. Abas, An Efficient Hybrid BFGS-CG Search Direction for Solving Unconstrained Optimization Problems, J. Adv. Res. Dyn. Control Syst., 12 (2020), 1035--1041

• [14] J. K. Liu, Y. M. Feng, L. M. Zou, Some three-term conjugate gradient methods with the inexact line search, Calcolo, 55 (2018), 1--16

• [15] G. Liu, J. Han, H. Yin, Global convergence of the Fletcher-Reeves algorithm with an inexact line search, Appl. Math. J. Chinese Univer., 10 (1995), 75--82

• [16] Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms, J. Optim. Theory Appl., 69 (1991), 129--137

• [17] J. K. Liu, Y. X. Zhao, X. L. Wu, Some three-term conjugate gradient methods with the new direction structure, Appl. Numer. Math., 150 (2020), 433--443

• [18] M. Mamat, I. M. Sulaiman, M. Malik, F. Sukono, Z. A. Zakaria, An Efficient Spectral Conjugate Gradient Parameter with Descent Condition for Unconstrained Optimization, J. Adv. Res. Dyn. Control Syst., 12 (2020), 2487--2493

• [19] M. F. McGuire, P. Wolfe, Evaluating a restart procedure for conjugate gradients, IBM Research Center, Yorktown Heights (1973)

• [20] O. O. Y. Osman, The convergence properties of RMIL$+$ conjugate gradient method under the strong Wolfe line search, Appl. Math. Comput., 367 (2020), 18 pages

• [21] E. Polak, G. Ribiere, Note sur la convergence de directions conjugées, Rev. Francaise Informat Recherche Opertionelle, 3e annee, 16 (1969), 35--43

• [22] B. T. Polyak, The conjugate gradient method in extremal problems, USSR Comput. Math. Math. Phys., 9 (1969), 94--112

• [23] M. J. D. Powell, Nonconvex Minimization Calculations and the Conjugate Gradient Method, in: Numerical analysis, Springer-Verlag, Berlin, 1984 (1984), 122--141

• [24] M. Rivaie, M. Mamat, W. J. Leong, M. Ismail, A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 218 (2012), 11323--11332

• [25] I. M. Sulaiman, M. Mamat, A. Abashar, M. Rivaie, S. M. Fatma, Z. Salleh, The global convergence properties of an improved conjugate gradient method, Appl. Math. Sci., 9 (2015), 1857--1868

• [26] I. M. Sulaiman, M. Mamat, A. O. Umar, K. Kamfa, E. N. Madi, Some Three-Term Conjugate Gradient Algorithms with Descent Condition for Unconstrained Optimization Models, J. Adv. Res. Dyn. Control Syst., 12 (2020), 2494--2501

• [27] I. M. Sulaiman, S. Supian, M. Mamat, New class of hybrid conjugate gradient coefficients with guaranteed descent and efficient line search, IOP Conference Series: Materials Science and Engineering, 2019 (2019), 7 pages

• [28] I. M. Sulaiman, U. A. Yakubu, M. Mamat, Application of spectral conjugate gradient method for Solving unconstrained optimization problems, Int. J. Optim. Control Theor. Appl., 10 (2020), 198--205

• [29] L. Zhang, W. Zhou, D. Li, A descent modified Polak-Ribiere-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629--640

• [30] L. Zhang, W. Zhou, D. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Methods Soft., 22 (2007), 697--711

• [31] G. Zoutendijk, Nonlinear programming, computational methods, in: Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970 (1970), 37--86