Covers: theory of Activation Functions

0- What is Activation Function?
- Why we use Activation functions with Neural Networks?
- Sigmoid or Logistic Activation Function
- Tanh or hyperbolic tangent Activation Function
- ReLU (Rectified Linear Unit) Activation Function
- Leaky ReLU
- Derivatives of activation functions

The online article explains different activation functions and their derivatives

Sagar Sharma

0 comment

Contributors

- Objectives
- The user of this shortlist will be able to understand the concept of hidden layers and the most common activation functions
- Potential Use Cases
- You are looking to understand the mathematical foundations for Deep Learning
- Who is This For ?
- BEGINNERDeep Learning practitioners interested in Mathematical foundations

Click on each of the following **annotated items** to see details.

Resources5/6

ARTICLE 1. An Introduction to Neural Networks

- What are neurons?
- How to code a neuron?
- How to combine neurons in a neural network?
- How to train a neural network?

15 minutes

ARTICLE 2. Overview of Neurons and Activation Functions

- What is an Artificial Neural Network?
- What is a neuron?
- What is an Activation Function?
- What are the most common activation functions?

5 minutes

VIDEO 3. Activation Function

- What is an activation function?
- Which activation function could work better?
- What are the most common activation functions?
- Which activation function should I use for the output layer?
- When to use the most common activation functions?

11 minutes

ARTICLE 4. Activation Functions in Neural Networks

- What is Activation Function?
- Why we use Activation functions with Neural Networks?
- Sigmoid or Logistic Activation Function
- Tanh or hyperbolic tangent Activation Function
- ReLU (Rectified Linear Unit) Activation Function
- Leaky ReLU
- Derivatives of activation functions

10 minutes

VIDEO 5. Layers in a Neural Network explained

- What are the type of layers in ANN?
- What kind of calculations occur within a hidden layer?
- How to build a hidden layer in Keras?

6 minutes

ARTICLE 6. Simplified Guide to Hidden Layer Neural Networks by DL Practitioner

- What is a multilayer perceptron (MLP)?
- What's the role of Feedforward and back propagation algorithms?
- Chain rule
- Gradient Descend Algorithm

15 minutes

0 comment