Iranian Journal of Numerical Analysis and Optimization

Iranian Journal of Numerical Analysis and Optimization

An efficient hybrid three-term conjugate gradient method with practical applications

Document Type : Research Article

Authors
Laboratory of Fundamental and Numerical Mathematics, Department of Mathematics, Faculty of Sciences, Setif1 University-Ferhat Abbas , 19000, Algeria.
Abstract
The conjugate gradient method is widely used for large-scale unconstrained optimization due to its efficiency in iteration number and computing time. In this paper, we present a new parameter $\beta_{k}$ by hybridizing the Liu-Storey parameter $\beta_{k}^{LS}$ and its modification $\beta_{k}^{DLS}$. We also use a hybrid three-term technique ensuring sufficient descent based on strong Wolfe conditions. We prove that the new direction possesses the descent property and that the corresponding algorithm is globally convergent. Numerical experiments on test problems and some applications, such as image restoration and sparse signal recovery, further show that our approach is efficient, consistently outperforming other approaches in terms of convergence speed and solution quality.
Keywords
Subjects

[1] Andrei, N., An unconstrained optimization test functions collection, Adv. Model. Optim., 10
(2008), 147–161.
[2] Andrei, N., Nonlinear conjugate gradient methods for unconstrained optimization, Springer
Optim. Appl., 158 (2020), 455–466.
[3] Ben Hanachi, S., Sellami, B. and Belloufi, M., New iterative conjugate gradient method for
nonlinear unconstrained optimization, RAIRO Oper. Res., 56 (2022), 2315–2327.
[4] Cai, J., Chan, R. and Morini, B., Minimization of an edge-preserving regularization func-
tional by conjugate gradient type methods, in Image Processing Based on Partial Differential
Equations, Math. Vis., Springer, Berlin–Heidelberg, 2007.
[5] Candès, E., Romberg, J. and Tao, T., Robust uncertainty principles: exact signal reconstruc-
tion from highly incomplete frequency information, IEEE Trans. Inf. Theory, 52(2) (2006),
489–509.
[6] Fletcher, R., Practical methods of optimization: unconstrained optimization, Vol. 1, Wiley,
New York, 1987.
[7] Chan, R., Ho, C. and Nikolova, M., Salt-and-pepper noise removal by median-type noise
detectors and detail-preserving regularization, IEEE Trans. Image Process., 14(10) (2005),
1479–1485.
[8] Dai, Z. and Wen, F., Another improved Wei–Yao–Liu nonlinear conjugate gradient method
with sufficient descent property, Appl. Math. Comput., 218(13) (2012), 7421–7430.
[9] Delladji, S., Belloufi, M. and Sellami, B., New hybrid conjugate gradient method as a convex
combination of FR and BA methods, J. Inf. Optim. Sci., 42(3) (2021), 591–602.
[10] Djordjević, S., New hybrid conjugate gradient method as a convex combination of HS and
FR methods, J. Appl. Math. Comput., 2(9) (2018), 366–378.
[11] Djordjević, S., New hybrid conjugate gradient method as a convex combination of LS and
CD methods, Filomat, 31(6) (2017), 1813–1825.
[12] Djordjević, S., New hybrid conjugate gradient method as a convex combination of LS and
FR methods, Acta Math. Sci. Ser. B, 39(1) (2019), 214–228.
[13] Dolan, E. and Moré, J., Benchmarking optimization software with performance profiles,
Math. Program., 91 (2002), 201–213.
[14] Dai, Y. and Yuan, Y., An efficient hybrid conjugate gradient method for unconstrained
optimization, Ann. Oper. Res., 103 (2001), 33–47.
[15] Dai, Y. and Yuan, Y., A nonlinear conjugate gradient method with a strong global conver-
gence property, SIAM J. Optim., 10(1) (1999), 177–182.
[16] Fletcher, R. and Reeves, C., Function minimization by conjugate gradients, Comput. J.,
7(2) (1964), 149–154.
[17] Gould, N., Orban, D. and Toint, P., CUTEr and SifDec: a constrained and unconstrained
testing environment, revisited, ACM Trans. Math. Softw., 29(4) (2003), 373–394.
[18] Hemici, Y. E., Khelladi, S. and Benterki, D., New hybrid conjugate gradient method for
nonlinear optimization with application to image restoration problems, Kybernetika, 60(4)
(2024), 535–552.
[19] Hestenes, M. and Stiefel, E., Methods of conjugate gradients for solving linear systems, J.
Res. Natl. Bur. Stand., 49(6) (1952), 409–436.
[20] Hager, W. and Zhang, H., A survey of nonlinear conjugate gradient methods, Pac. J. Optim.,
2 (2006), 35–58.
[21] Jiang, X., Liao, W., Yin, J. and Jian, J., A new family of hybrid three-term conjugate
gradient methods with applications in image restoration, Numer. Algorithms, 91(1) (2022),
161–191.
[22] Liu, Y. and Storey, C., Efficient generalized conjugate gradient algorithm. Part I: theory, J.
Optim. Theory Appl., 69(1) (1991), 129–137.
[23] Ma, G., Lin, H., Jin, W. and Han, D., Two modified conjugate gradient methods for un-
constrained optimization with applications in image restoration problems, J. Appl. Math.
Comput., 68 (2022), 4733–4758.
[24] Malik, M., Sulaiman, I., Abubakar, A., Ardaneswari, G. and Sukono, A., A new family of
hybrid three-term conjugate gradient method for unconstrained optimization with application
to image restoration and portfolio selection, AIMS Math., 8(1) (2023), 1–28.
[25] Moré, J., Garbow, B. and Hillstrom, K., Testing unconstrained optimization software, ACM
Trans. Math. Softw., 7(1) (1981), 17–41.
[26] Mtagulwa, P. and Kaelo, P., A convergent modified HS–DY hybrid conjugate gradient method
for unconstrained optimization problems, J. Inf. Optim. Sci., 40(1) (2019), 97–113.
[27] Ouaoua, M. L., Khelladi, S. and Benterki, D., New parameter of conjugate gradient method
for unconstrained nonlinear optimization, Stat. Optim. Inf. Comput., 13(6) (2025), 2382–
2390.
[28] Polyak, B., The conjugate gradient method in extreme problems, USSR Comput. Math.
Math. Phys., 9 (1969), 94–112.
[29] Polak, E. and Ribière, G., Note sur la convergence des méthodes de directions conjuguées,
Rev. Française Inform. Rech. Opér., 16 (1969), 35–43.
[30] Rivaie, M., Mustafa, M. and Abdelrhaman, A., A new class of nonlinear conjugate gradient
coefficients with exact and inexact line searches, Appl. Math. Comput., 268 (2015), 1152–
1163.
[31] Sellami, B. and Chaib, Y., A new family of globally convergent conjugate gradient methods,
Ann. Oper. Res., 241 (2016), 497–513.
[32] Sellami, B. and Chaib, Y., New conjugate gradient method for unconstrained optimization,
RAIRO Oper. Res., 50 (2016), 1013–1026.
[33] Stanimirović, P., Ivanov, B. and Mosić, D., A survey of gradient methods for solving non-
linear optimization, Electron. Res. Arch., 28(4) (2020), 1573–1624.
[34] Sulaiman, I. M., Kaelo, P., Khalid, R. and Nawawi, M. K. M., A descent generalized RMIL
spectral gradient algorithm for optimization problems, Int. J. Appl. Math. Comput. Sci., 34(2)
(2024), 225–233.
[35] Yang, X., Luo, Z. and Dai, X., A global convergence of LS–CD hybrid conjugate gradient
method, Adv. Numer. Anal., 2013 (2013), Article ID 5.
[36] Zhang, Y. and Zheng, B., Two new Dai–Liao-type conjugate gradient methods for uncon-
strained optimization problems, J. Optim. Theory Appl., 175 (2017), 502–509.
[37] Zoutendijk, G., Nonlinear programming, computational methods, in Integer and Nonlinear
Programming, North-Holland, Amsterdam, 1970, pp. 37–86.
Send comment about this article
Enter Name.
Enter a valid email address.
Enter a vaid affiliation.
Enter comments (At leaset 10 words)
CAPTCHA Image
Enter Security Code Correctly.