A modified Liu-Storey scheme for nonlinear systems with an application to image recovery

Document Type : Research Article

Authors

• A.I. Kiri
• M.Y. Waziri
• K. Ahmed

Department of Mathematical Sciences, Bayero University, Kano, Nigeria.

10.22067/ijnao.2022.75413.1107

Abstract

Like the Polak-Ribi`ere-Polyak (PRP) and Hestenes-Stiefel (HS) meth-ods, the classical Liu-Storey (LS) conjugate gradient scheme is widely be-lieved to perform well numerically. This is attributed to the in-built capa-bility of the method to conduct a restart when a bad direction is encoun-tered. However, the scheme’s inability to generate descent search direc-tions, which is vital for global convergence, represents its major shortfall. In this article, we present an LS-type scheme for solving system of mono-tone nonlinear equations with convex constraints. The scheme is based on the approach by Wang et al. (2020) and the projection scheme by Solodov and Svaiter (1998). The new scheme satisfies the important condition for global convergence and is suitable for non-smooth nonlinear problems. Fur-thermore, we demonstrate the method’s application in restoring blurry im-ages in compressed sensing. The scheme’s global convergence is established under mild assumptions and preliminary numerical results show that the proposed method is promising and performs better than two recent meth-ods in the literature.

Keywords

• Nonlinear Monotone Equations
• Line search
• Projection method
• Signal processing
• Convex constraint
• Image de-blurring

20.1001.1.24236977.2023.13.1.3.4

Main Subjects

• Optimization

---

References

[1] Ahookhosh, M. Amini, K. and Bahrami, S. Two derivative-free projec-tion approaches for systems of large-scale nonlinear monotone equations, Numer. Algor. 64(2013), 21-42.
[2] Barzilai, J. and Borwein, J.M. Two point step size gradient method, IMA J. Numer. Anal. 8(1988),141-148.
[3] Birgin, E.G. and Martinez, J.M. A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2001), 117-128.
[4] Dennis, J. and Mor ́e, J. Quasi-Newton methods, motivation and theory, SIAM Review, Soc. Ind. Appl. Math. 19(1)(1977), 46-89.
[5] Dirkse, S.P. and Ferris, M.C. A collection of nonlinear mixed complemen-tarity problems, Optim. Methods Softw. 5(1995)319-345.
[6] Dolan, E.D. and Moré, J.J. Benchmarking optimization software with performance profiles, Math. Program. 91(2002), 201-2013.
[7] Figueiredo, M. Nowak, R. and Wright, S.J. Gradient projection for sparse reconstruction, application to compressed sensing and other inverse prob-lems, IEEE J-STSP IEEE Press, Piscataway, NJ. (2007), 586-597.
[8] Fletcher, R. Practical method of Optimization, Volume 1: Unconstrained Optimization, 2nd ed., Wiley, New York, 1997
[9] Gao, P. and He, C. An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints, Calcolo 55(53)(2018),1-17.
[10] Halilu, A.S. Majumder, A. Waziri, M.Y. Awwal, A.M. and Ahmed, K. On solving double direction methods for convex constrained monotone nonlinear equations with image restoration, Comput. Appl. Math. 40, 239 (2021).
[11] Halilu, A.S. Majumder, A. Waziri, M.Y. and Ahmed, K. Signal re-covery with convex constrained nonlinear monotone equations through conjugate gradient hybrid approach, Math. comput. Simulation, http://doi.org/10.1016/j.matcom.2021.03.020. (2021).

[12] Halilu, A.S. Majumder, A. Waziri, M.Y. Ahmed, K. and Awwal, A.M. Motion control of the two joint planar robotic manipulators through accel-erated Dai-Liao method for solving system of nonlinear equatiions, Eng. Comput. https://doi.org/10.1108/EC-06-2021-0317
[13] Hager, W.W. and Zhang, H. A new conjugate gradient method with guranteed descent and an efficient line search, SIAM J. Optim. 16 (2005), 170-192.
[14] Hestenes, M.R. and Stiefel, E.L. Methods of conjugate gradients for solv-ing linear systems, J. Res. Nat. Bur. Standards, 49(1952), 409-436.
[15] Hively, G.A. On a class of nonlinear integral equations arising in trans-port theory, SIAM J. Numer. Anal. 9 (1978), 787-792.
[16] Ibrahim, A.H. Deepho, J. Abubakar, A.B. Aremu, K.O. A Modified Liu-Storey-conjugate descent hybrid projection method for convex constrained nonlinear equations and image restoration, Numer. Alg. Cont. Optim. doi:10.3934/naco.2021022.
[17] Kelly, C. Iterative methods for optimization, Frontiers Appl. Math. 1999, DOI:10.1137/1.9781611970920 Corpus ID: 123596970.
[18] Koorapetse, M. Kaelo, P. An efficient hybrid conjugate gradient-based projection method for convex constrained nonlinear monotone equations, J. inter. math. 22 (6)(2019), 1031-1050.
[19] La Cruz, W. Martínez, J.M. and Raydan, M. Spectral residual method without gradient information for solving large-scale nonlinear systems, Theory and Experiments, Citeseer, Technical Report RT-04-08(2004).
[20] La Cruz, W. A Spectral algorithm for large-scale systems of nonlin-ear monotone equations, Numer. Algor. DOI 10.1007/s1107s-017-0299-8. (2017).
[21] Li, X. Shi, J. Dong, X. and Yu, J. A new conjugate gradient method based on Quasi–Newton equation for unconstrained optimization, J. Comput. Appl. Math. https://doi.org/10.1016/j.cam.2018.10.035
[22] Liu, Y. and Storey, C. Efficient generalized conjugate gradient algo-rithms, Part 1: Theory, J. Optim. Theory Appl. 69(1991), 129-137.
[23] Liu, J.K. and Li, S.J. A projection method for convex constrained monotone nonlinear equations with applications, Comput. Math. Appl. 70(10)(2015), 2442-2453.
[24] Meintjes, K. and Morgan, A.P. A methodology for solving chemical equi-librium systems, Appl. Math. Comput. 22(1987), 333-361.

[25] Mompati, S. and Kaelo, P. Globally convergent three-term conjugate gra-dient projection methods for solving nonlinear monotone equations, Arab. J. Math. 7(2018), 289-301.
[26] Muhammad, L. and Waziri, M.Y. An Alternative three-term conjugate gradient algorithm for systems of nonlinear equations, Intern. J. Math. Model. Comput. 07(02)(2017),145-157.
[27] Nakamura, W. Narushima, Y. and Yabe, H. Nonlinear conjugate gradi-ent methods with sufficient descent properties for unconstrained optimiza-tion., J. Ind. Manag. Optim. 9(3)(2013),595-619.
[28] Pang, J.S. Inexact Newton methods for the nonlinear complementarity problem, Math. Program. 36 (1986), 54-71.
[29] Polak, E. and Ribi ́ere, G. Note Sur la convergence de directions con-jugèes, Rev. Francaise Informat. Recherche Operationelle, 3e Ann`ee, 16 (1969), 35-43.
[30] Polyak, B.T. The conjugate gradient method in extreme problems, USSR Comp. Math. Math. Phys. 9 (1969), 94-112.
[31] Sabi’u, J. Shah, A. Waziri, M.Y. and Ahmed, K. Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving mono-tone nonlinear equations with convex constraint, Int. J. Comput. Methods. hppt://doi.org/10.1142/S0219876220500437 (2020).
[32] Solodov, M.V. and Svaiter, B.F. A globally convergent inexact Newton method for systems of monotone equations, in: Reformulation: Nons-mooth, Piecewise Smooth, Semismooth and Smoothing Methods, Kluwer Academic Publishers, 1998, pp. 355-369.
[33] Sugiki, K. Narushima, Y. and Yabe, H. Globally convergent three-term conjugate gradient methods that use secant conditions and generate de-scent search directions for unconstrained optimization, J. Optim. Theory Appl. 153(2012), 733-757.
[34] Wang, C. Wang, Y. and Xu, C. A projection method for a system of nonlinear monotone equations with convex constraints, Math. Methods Oper. Res. 66(1)(2007), 33-46.
[35] Wang, X.Y. Li, X.J. and Kou, X.P. A self-adaptive three-term conju-gate gradient method for monotone nonlinear equations with convex con-straints, Calcolo DOI 10.1007/s10092-015-0140-5.
[36] Wang, Z. Li, P. Li, X. and Pham, H. A modified three-term type CD con-jugate gradient algorithm for unconstrained optimization problems, Hin-dawi Mathematical Problems in Engineering Volume 2020, Article ID 4381515, 14 pages https://doi.org/10.1155/2020/4381515.

[37] Waziri, M.Y. Ahmed, K. and Sabi’u, J. A family of Hager-Zhang conju-gate gradient methods for system of monotone nonlinear equations, Appl. Math. Comput. 361(2019), 645-660.
[38] Waziri, M.Y. Ahmed, K. and Sabi’u, J. A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations, Arab. J. Math. 9(2020), 443-457.
[39] Waziri, M.Y. Ahmed, K. and Sabi’u, Descent Perry conjugate gradient methods for systems of monotone nonlinear equations, Numer. Algor. 85(2020), 763-785.
[40] Waziri, M.Y. Ahmed, K. Sabi’u, J. and Halilu, A.S. Enhanced Dai-Liao conjugate gradient methods for systems of monotone nonlinear equations, SeMA J. 78(2020), 15-51.
[41] Waziri, M.Y. Usman, H. Halilu, A.S. and Ahmed, K. Modified matrix-free methods for solving systems of nonlinear equations, Optimization. 70(2021), 2321-2340
[42] Waziri, M.Y. and Ahmed, K. Two descent Dai-Yuan conjugate gradient methods for systems of monotone nonlinear equations, J. Sci. Comput. (2022) 90:36. https://doi.org/10.1007/s10915-021-01713-7.
[43] Waziri, M.Y. Ahmed, K. Halilu, A.S. Awwal, A.M. Modified Dai-Yuan iterative scheme for nonlinear systems and its Application, Numer. Alg. Control Optim. doi:10.3934/naco.2021044.
[44] Waziri, M.Y. Ahmed, K. and Halilu, A.S. Adaptive three-term family of conjugate residual methods for system of monotone nonlinear equations, Sao Paulo J. Math. Sci. https://doi.org/10.1007/s40863-022-00293-0
[45] Xiao, Y. Wang, Q. and Hu, Q. Non-smooth equations based method for l1-norm problems with applications to compressed sensing, Nonlinear Anal. Theory Methods Appl. 74(11)(2011), 3570-3577.
[46] Xiao, Y. and Zhu, H. A conjugate gradient method to solve convex con-strained monotone equations with applications in compressive sensing, J. Math. Anal. Appl. 405(1)(2013), 310-319.
[47] Yuan, G. Modified nonlinear conjugate gradient methods with suffi-cient descent property for large-scale optimization problems, Optim. Lett. 3(2009), 11-21.