A conjugate gradient algorithm based on a combined descent scheme for two-layer neural network training and unconstrained optimization

Document Type : Research Article

Authors

Chalabia Tebbal ¹

Mounia Laouar ²

Raouf Ziadi ¹

Abd Elhamid Mehamdia ³

¹ Laboratory of Fundamental and Numerical Mathematics (LMFN), Department of Mathematics, University Setif - 1 - Ferhat Abbas, Setif, Algeria.

² Laboratory of Partial Differential Equations, Department of Mathematics, Faculty of Mathematics and Computer Science, University of Batna 2, Batna, Algeria.

³ University M’Hamed Bougara of Boumerdes, Boumerdes 35000, Algeria..

10.22067/ijnao.2026.99592.1903

Abstract

Conjugate gradient methods are considered among the most efficient methods for solving large-scale optimization problems. Among them, the CD (Conjugate-Descent) variant has attracted the attention of many researchers thanks to its straightforward iterative process and low memory requirements. In the present work, we propose an enhancement of the CD method using an efficient combined descent scheme to address large-scale unconstrained optimization problems, with a specific application to training a two-layer neural network model. The sufficient descent condition is analyzed under the strong Wolfe line search conditions, and global convergence is established under mild assumptions. The performance of the suggested algorithm is evaluated on various two-layer neural network models, with the number of weights ranging from 32 to 4080. Furthermore, to validate its effectiveness, the approach is tested on a broad set of unconstrained benchmark functions. Comparisons with recent conjugate gradient methods confirm the robustness and efficiency of the proposed method.

Keywords

Conjugate gradient method

Conjugate-Descent method

global convergence

neural networks

two-layer neural network training model

Subjects

• Optimization

---