Browsed by
Tag: Neural Network

What is Gradient Descent in machine learning?

What is Gradient Descent in machine learning?

Comments 3 comments

The process of repeatedly nudging an input of a function by some multiple of the negative gradient is called gradient descent. It’s a way to converge towards some local minimum of a cost function basically valley in a graph.  According to Wikipedia, Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum or minimum cost of a function using gradient descent, one takes steps proportional…

Read full Article Read More

What is Activation Functions in Neural Network (NN)?

What is Activation Functions in Neural Network (NN)?

Comments 0 Comment

In Artificial Neural Network (ANN), the activation function of a neuron defines the output of that neuron given a set of inputs. For Neural Network to achieve maximum predictive power, we must apply activation function in the hidden layers. An activation function allows the model to capture non-linearities. Image 1 below from study.com gives examples of linear function and reduces nonlinear function. If the relationships in the data are not straight line relationships we will…

Read full Article Read More