Convergence results for gradient flow and gradient descent systems in the artificial neural network training

Abstract: The field of artificial neural network (ANN) training has garnered significant attention in recent years, with researchers exploring various mathematical techniques for optimizing the training process. In particular, this paper focuses on advancing the current understanding of gradient flow and gradient descent optimization methods. Our aim is to establish a solid mathematical convergence theory for continuous-time gradient flow equations and gradient descent processes based on mathematical anaylsis tools.