Iranian Journal of Numerical Analysis and Optimization

Iranian Journal of Numerical Analysis and Optimization

Enhancing conjugate gradient performance via a diagonal quasi-Newton strategy: applications in adaptive filtering and power allocation

Document Type : Research Article

Author
Department of Mathematical Sciences, Yazd University, Yazd, Iran.
Abstract
Unconstrained optimization is fundamental to scientific computing. While nonlinear conjugate gradient (CG) methods are valued for their low memory requirements and simplicity, their convergence may be slow. Conversely, quasi-Newton methods like BFGS achieve faster superlinear convergence but become computationally prohibitive for large-scale problems. To address this trade-off, this paper introduces a novel hybrid algorithm that combines a diagonal quasi-Newton (DQN) approach with a conjugate gradient method. This framework leverages the complementary strengths of both techniques, maintaining modest memory usage while accelerating convergence. Numerical experiments on several benchmark problems, adaptive filter design, and power allocation, demonstrate the proposed method's effectiveness. Our MATLAB-based evaluations show that the hybrid approach consistently outperforms standard CG and BFGS methods.
Keywords
Subjects

[1] Aini, N., Hajar, N., Mamat, M., Zull, N. and Rivaie, M., Hybrid quasi-Newton and conjugate
gradient method for solving unconstrained optimization problems, J. Eng. Appl. Sci. 12(18)
(2017), 4627–4631.
[2] Andrei, N., An unconstrained optimization test functions collection, Adv. Model. Optim. 10
(2008), 147–161.
[3] Andrei, N., Accelerated conjugate gradient algorithm with finite difference Hessian/vector
product approximation for unconstrained optimization, J. Comput. Appl. Math. 230(2)
(2009), 570–582.
[4] Broyden, C. G., The convergence of a class of double-rank minimization algorithms. Part I.
The new algorithm, J. Inst. Math. Appl. 6 (1970), 76–90.
[5] Cao, H. and Li, W., A diagonal quasi-Newton method based on automatic differentiation for
nonlinear equations, Calcolo 62(2) (2025), 1–17.
[6] Davidon, W. C., Variable metric method for minimization, Technical Report ANLC5990
(Revised), Argonne National Laboratory, Argonne, 1959.
[7] Dolan, E. D. and Moré, J. J., Benchmarking optimization software with performance profiles,
Math. Program. 91 (2002), 201–213.
[8] Enshaei, S., Leong, W. H. and Farid, M., Diagonal quasi-Newton method via variational
principle under generalized Frobenius norm, Optim. Methods Softw. 31(6) (2016), 1258–
1271.
[9] Fletcher, R., A new approach to variable metric algorithms, Comput. J. 13(3) (1970), 317–
322.
[10] Fletcher, R. and Powell, M. J. D., A rapidly convergent descent method for minimization,
Comput. J. 6 (1963), 163–168.
[11] Goldfarb, A., A family of variable metric methods derived by variational means, Math.
Comput. 24(109) (1970), 23–26.
[12] Haykin, S., Adaptive filter theory, Prentice-Hall, Inc., Upper Saddle River, NJ, 1996.
[13] Huynh, D. Q. and Hwang, F. N., An accelerated structured quasi-Newton method with
a diagonal second-order Hessian approximation for nonlinear least squares problems, J.
Comput. Appl. Math. 442 (2024), 115718.
[14] Liu, M., Zhang, J., Xiong, K., Zhang, M., Fan, P. and Letaief, K. B., Effective user clustering
and power control for multiantenna uplink NOMA transmission, IEEE Trans. Wireless
Commun. 21(11) (2022), 8995–9009.
[15] Monson, H., Statistical digital signal processing and modeling, John Wiley & Sons, Hoboken,
NJ, 1996.
[16] Nocedal, J. and Wright, S. J., Numerical optimization, Springer Series in Operations Re-
search and Financial Engineering, Springer, New York, 2006.
[17] Osinuga, I. and Stanimirovic, P., Another hybrid BFGS-conjugate gradient algorithm for
unconstrained optimization, AIP Conf. Proc. 2852 (2023), 130006.
[18] Polyak, B. T., The conjugate gradient method in extremal problems, USSR Comput. Math.
Math. Phys. 9(4) (1969), 94–112.
[19] Rivaie, M., Mamat, M., Leong, W. and Ismail, M., A new conjugate gradient coefficient for
large scale nonlinear unconstrained optimization, Int. J. Math. Anal. 6(23) (2012), 1131–
1146.
[20] Shanno, D.F.d, Conditions of quasi-Newton methods for function minimization, Math. Com-
put. 24(111) (1970), 647–650.
[21] Sim, H. S., Leong, W. J., Chen, C. Y. and Ibrahim, S. N. I., Diagonal quasi-Newton up-
dating formula using log-determinant norm, in: Advances in Industrial and Applied Math-
ematics: Proceedings of the 23rd Malaysian National Symposium of Mathematical Sciences
(SKSM23), (2016), 1–7.
[22] Zhu, Z., Zhu, X. and Tan, Z., An accelerated conjugate gradient method with adaptive two-
parameter with applications in image restoration, Comput. Appl. Math. 43 (2024), Article
116.
[23] Zola, E. and Martin-Escalona, I., A robust user association, backhaul routing, and switching
off model for a 5G network with variable traffic demands, IEEE Access 8 (2020), 96714–
96726.
[24] Zoutendijk, G., Nonlinear programming, computational methods, in: J. Abadie (ed.), Integer
and Nonlinear Programming, North-Holland Publishing Company, Amsterdam and London,
(1970), 37–86.
Send comment about this article
Enter Name.
Enter a valid email address.
Enter a vaid affiliation.
Enter comments (At leaset 10 words)
CAPTCHA Image
Enter Security Code Correctly.