In this course, you’ll learn theoretical foundations of optimization methods used for training deep machine learning models. Why does gradient descent work? Specifically, what can we guarantee about ...

High-performance optimization algorithms are essential in deep learning. However, understanding the behavior of optimization (i.e., learning process) remains challenging due to the instability and ...

Dr. James McCaffrey of Microsoft Research explains stochastic gradient descent (SGD) neural network training, specifically implementing a bio-inspired optimization technique called differential ...