Global convergence of modified conjugate gradient methods with application in conditional model regression function

Articles in Press

Document Type : Research Article

Authors

• C. Yacine
• Abd Elhamid

Laboratory Informatics and Mathematics (LIM), Mohamed Cherif Messaadia University, Souk Ahras, 41000, Algeria.

10.22067/ijnao.2024.85389.1344

Abstract

The conjugate gradient method is one of the most important ideas in sci-entific computing. It is applied to solving linear systems of equations and nonlinear optimization problems. In this paper, based on a variant of the Hestenes–Stiefel (HS) method and Polak–Ribière–Polyak (PRP) method, two modified conjugate gradient methods (named MHS∗ and MPRP∗) are presented and analyzed. The search direction of the presented methods fulfills the sufficient descent condition at each iteration. We establish the global convergence of the proposed algorithms under normal assumptions and strong Wolfe line search. Preliminary elementary numerical experi-ment results are presented, demonstrating the promise and the effective-ness of the proposed methods. Finally, the proposed methods are further extended to solve the problem of conditional model regression function.

Keywords

• Conjugate gradient method
• Inexact line search
• Descent condi-tion
• Global convergence
• Numerical comparisons
• Conditional model regres-sion
• Kernel estimator

Main Subjects

• Optimization

---

References

[1] Andrei, N. An unconstrained optimization test functions, Adv. Model. Optim. 10 (2008), 147–161.

[2] Awwal, A.M., Sulaiman, I.M., Malik, M., Mamat, M. and Kumam, P.K. Sitthithakerngkiet, A spectral RMIL conjugate gradient method for un-constrained optimization with applications in portfolio selection and mo-tion control, IEEE Access, 9 (2021), 75398–75414.
[3] Boente, G. and Fraiman, R. Nonparametric regression estimation, J. Multivariate Anal. 29 (1989), 180–198.
[4] Bongartz, I., Conn, A.R., Gould, N. and Toint, P.L. Constrained and unconstrained testing environment, ACM Trans. Math. Softw. 21 (1995), 123–160.
[5] Collomb, G., Hardle, W. and Hassani, S. A note on prediction via es-timation of the conditional mode function, J. Stat. Plan. Inference, 15 (1987), 227–236.
[6] Dai, Y.H. and Yuan, Y. A nonlinear conjugate gradient method with a strong global convergence property, SIAM J, Optim. 10 (1999), 177–182.
[7] Dai, Y.H. and Yuan, Y. Nonlinear conjugate gradient methods, Shanghai Scientific and Technical Publishers, 2000.
[8] Dolan, E.D. and Morè, J. J. Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), 201–213.
[9] Du, X.W., Zhang, P. and Ma, W. Some modified conjugate gradient methods for unconstrained optimization, J. Comput. Appl. Math. 305 (2016), 92–114.
[10] Fletcher, R. Practical methods of optimization, Wiley, New York, 1987.
[11] Fletcher, R. and Reeves, C. Function minimization by conjugate gradi-ents, Comput. J. 7 (1964), 149–154.
[12] Hager, W.W. and Zhang, H. A survey of nonlinear conjugate gradient methods, Pacific J. Optim. 2 (2006), 35–58.
[13] Hestenes, M.R. and Stiefel, E.L. Methods of conjugate gradients for solv-ing linear systems, J. Res. Nation. Bur. Stand. 49 (1952), 409–436.

[14] Huang, H. A new conjugate gradient method for nonlinear unconstrained optimization problems, J. Hunan Univ. 44 (2014), 141–145.
[15] Huang, H., Wei, Z. and Yao, S. The proof of the sufficient descent con-dition of the Wei-Yao-Liu conjugate gradient method under the strong Wolfe-Powell line search, Appl. Math. Comput. 189 (2007), 1241–1245.
[16] Liu, Y. and Storey, C. Efficient generalized conjugate gradient algo-rithms, J. Optim. Theory Appl. 69 (1991), 129–137.
[17] Ma, G., Lin, H., Jin, W. and Han, D. Two modified conjugate gradi-ent methods for unconstrained optimization with applications in image restoration problems, J.Appl. Math. Comput. (2022), 1–26.
[18] Mehamdia, A.E., Chaib, Y. and Bechouat, T. Two modified conju-gate gradient methods for unconstrained optimization and applications, RAIRO - Oper. Res. 57 (2023), 333–350.
[19] Polak, E. and Ribière, G. Note sur la convergence de directions con-juguée, Revue Francaise d’Informatique et de Recherche Opérationnelle, 16 (1969), 35–43.
[20] Polyak, B.T. The conjugate gradient method in extreme problems, Com-put. Math. Math. Phys. 9 (1969), 94–112.
[21] Powell, M.J.D. Restart procedures for the conjugate gradient method, Math. Program.12 (1977), 241–254.
[22] Powell, M.J.D. Nonconvex minimization calculations and the conjugate gradient method, Lect. Notes Math. 1066 (1984), 122–141.
[23] Samanta, M. and Thavaneswaran, A. Nonparametric estimation of the conditional mode, Commun. Stat. - Theory Methods, 16 (1990), 4515–4524.
[24] Wei, Z., Yao, S. and Liu, L. The convergence properties of some new con-jugate gradient methods, Appl. Math.Comput. 183 (2006), 1341–1350.
[25] Yao, S., Wei, Z. and Huang, H. A notes about WYL’s conjugate gradient method and its applications, Appl. Math. Comput. 191 (2007), 381–388.

[26] Zhang, L. An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation, Appl. Math. Comput. 6 (2009), 2269–2274.
[27] Zoutendijk, G. Nonlinear programming computational methods, Integer and Nonlinear Programming, (1970), 37–86.