Neural Network Mathematics: Exploring the Math behind Gradient Descent
Math
| Expert
- 12 videos | 1h 53m 45s
- Includes Assessment
- Earns a Badge
Because neural networks comprise thousands of neurons and interconnections, one can assume training a neural network involves millions of computations. This is where a general-purpose optimization algorithm called gradient descent comes in. Use this course to gain an intuitive and visual understanding of how gradient descent and the gradient vector work. As you advance, examine three neural network activation functions, ReLU, sigmoid, and hyperbolic tangent functions, and two variants of the ReLU function, Leaky ReLU and ELU. In examining variants of the ReLU activation function, learn how to use them to deal with deep neural network training issues. Finally, implement a neural network from scratch using TensorFlow and basic Python. When you're done, you'll be able to illustrate the mathematical intuition behind neural networks and be prepared to tackle more complex machine learning problems.
WHAT YOU WILL LEARN
-
Discover the key concepts covered in this course
Outline how gradient descent works
Summarize how to compute the gradient vector of partial derivatives
Recall the characteristics of activation functions
Illustrate step, sigmoid, and tangent activation functions
Illustrate relu, leaky relu, and elu activation functions -
Describe how unstable gradients can be mitigated using variants of the relu activation function
Create a simple neural network with one neuron for regression
Illustrate the impact of learning rate and number of epochs of training
Illustrate the classification dataset
Write python code from scratch to represent and train a single neuron
Summarize the key concepts covered in this course
IN THIS COURSE
-
2m 22s
-
11m 21s
-
3. Computing Gradients12m 24s
-
4. Activation Functions12m 43s
-
5. Visualizing Common Activation Functions11m 33s
-
6. Visualizing the ReLU Function and Its Variants9m 3s
-
7. Mitigating Issues in Neural Network Training10m 8s
-
8. Simple Regression Using TensorFlow12m 7s
-
9. Learning Rate and Number of Epochs8m 10s
-
10. Exploring Datasets and Setting up Utilities10m 31s
-
11. Training a Simple Neural Network from Scratch11m 25s
-
12. Course Summary1m 59s
EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.
Digital badges are yours to keep, forever.
