An efficient hybrid conjugate gradient method for unconstrained optimization and image restoration problems

Articles in Press

Document Type : Research Article

Authors

• C. Souli ¹
• R. Ziadi ¹
• I. Lakhdari ²
• A. Leulmi ¹

¹ Laboratory of Fundamental and Numerical Mathematics (LMFN), University Ferhat Abbas Setif 1, Algeria.

² Laboratory of Mathematical Analysis, Probability and Optimizations, Biskra University, Algeria.

10.22067/ijnao.2024.88087.1449

Abstract

The conjugate gradient (CG) method is an optimization technique known for its rapid convergence; it has blossomed into significant developments and applications. Numerous variations of CG methods have emerged to en-hance computational efficiency and address real-world challenges. In this work, a novel conjugate gradient method is introduced to solve nonlinear unconstrained optimization problems. Based on the combination of PRP (Polak–Ribière–Polyak), HRM (Hamoda–Rivaie–Mamat) and NMFR (new modified Fletcher–Reeves) algorithms, our method produces a descent di-rection without depending on any line search. Moreover, it enjoys global convergence under mild assumptions and is applied successfully on various standard test problems as well as image processing. The numerical results indicate that the proposed method outperforms several existing methods in terms of efficiency.

Keywords

• Unconstrained optimization
• Hybrid conjugate gradient
• Global convergence
• Image restoration

Main Subjects

• Optimization

---

References

[1] Abdelrahman, A., Mohammed, M., Yousif, O.O. and Elbashir, M.K. Nonlinear conjugate gradient coefficients with exact and strong Wolfe line searches techniques, J. Math. 2022(1), (2022) 1383129.
[2] Al-Baali, M. Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal. 5 (1985) 121–124.

[3] Andrei, N. Another nonlinear conjugate gradient algorithm for uncon-strained optimization, Optim. Methods Softw. 24 (2008) 89–104.
[4] Andrei, N. An unconstrained optimization test functions, Adv. Modeling Optim. 10, (2008) 147–161.
[5] Dai, Y.H., Han, J.Y., Liu, G.H., Sun, D.F., Yin X. and Yuan, Y. Con-vergence properties of nonlinear conjugate gradient methods, SIAM J. Optim. 10 (1999) 348–358.
[6] Dai, Y.H. and Liao, L.Z. New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001) 87–101.
[7] Dai, Y.H. and Yuan, Y. A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim. 10 (1999) 177–182.
[8] Dai, Y.H. and Yuan, Y. An efficient hybrid conjugate gradient method for unconstrained optimization, Annal. Oper. Res. 103(1) (2001) 33–47.
[9] Djordjević, S.S. New hybrid conjugate gradient method as a convex com-bination of LS and CD methods, Filomat 31 (2017), 6, 1813-1825.
[10] Dolan, E.D.and Moré, J.J. Benchmarking optimization software with performance profiles, Math. Program. 91, (2002) 201–213.
[11] Fletcher, R. Practical methods of optimization, 2nd Ed., J. Wiley, Sons, New York, USA, 1987.
[12] Fletcher, R. and Reeves, C.M. Function minimization by conjugate gra-dients, Comput. J. 7, (1964) 149–154.
[13] Gilbert, J.C.and Nocedal, J. Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim. 2 (1) (1992) 21–42.
[14] Hamoda, M., Mamat, M., Rivaie, M. and Salleh, Z. A conjugate gra-dient method with Strong Wolfe–Powell line search for unconstrained optimization, Appl. Math. Sci. 10 (15), (2016) 721–734.

[15] Hestenes, M.R. and Stiefel, E.L. Methods of conjugate gradients for solv-ing linear systems, J. Res. Natl. Bur. Stand. 49, (1952) 409–436.
[16] Ibrahim, Y.I. and Khudhur, H.M. Modified three-term conjugate gradi-ent algorithm and its applications in image restoration, J. Electr. Eng. Comput., 28(2022) 1510–1517.
[17] Liu, J.K. and Li, S.J. New hybrid conjugate gradient method for uncon-strained optimization, Appl. Math. Comput. 245 (2014) 36–43.
[18] Liu, Y. and Storey, C. Efficient generalized conjugate gradient algo-rithms, Part 1, Theory, J. Optim. Theory. Appl. 69, (1991) 129–137.
[19] Polak, E. and Ribiere, G. Note sur la convergence de méthodes de di-rections conjuguées, Revue Française d’informatique et de Recherche Opérationnelle, Série Rouge 3, (1969) 35–43.
[20] Rivaie, M. Mamat, M., June, L.W. and Mohd, I. A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 218(22) (2012) 11323–11332.
[21] Souli, C., Ziadi, R., Bencherif-Madani, A.and Khudhur, H.M. A hybrid CG algorithm for nonlinear unconstrained optimization with application in image restoration, J. Math. Mod. (2024) 301–317.
[22] Sulaiman, I.M., Bakar, N.A., Mamat, M., Hassan, B.A., Malik, M. and Ahmed, A.M.A new hybrid conjugate gradient algorithm for optimization models and its application to regression analysis J. Electr. Eng. Comput. 23(2021) 1100–1109.
[23] Touati-Ahmed D. and Storey, C. Efficient hybrid conjugate gradient tech-nique, J. Optim. Theory. Appl. 64, (1990) 379–397.
[24] Wei, Z.X., Yao, S.W. and Liu, L.Y. The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 (2) (2006) 1341–1350.
[25] Zhang, L. An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation, J. Appl. Math. Comput. 215(6), (2009) 2269–2274.

[26] Zheng, X.Y., Dong, X.L. Shi, J.R.and Yang, W. Further comment on an-other hybrid conjugate gradient algorithm for unconstrained optimization by Andrei, Numer. Algorithm 84 (2019) 603–608.
[27] Ziadi, R. and Bencherif-Madani, A. A mixed algorithm for smooth global optimization, J. Math. Model., 11(2)(2023) 207–228.
[28] Ziadi, R. and Bencherif-Madani, A. A Perturbed quasi-Newton algorithm for bound-constrained global optimization, J. Comp. Math., (2023) 1–29.
[29] Ziadi, R., Ellaia, R. and Bencherif-Madani, A. Global optimization through a stochastic perturbation of the Polak-Ribiére conjugate gradient method, J. Comput. Appl. Math., 317,(2017) 672–684.