Gradient Descent is an iterative machine learning optimization algorithm used to reduce the cost function. In machine learning, we use cost function to know the accuracy of our model by using different evaluation metrics to compare our predictions with the known targets. If you need more details on the evaluation metrics, visit my this post OR if you want to try this in Collab or Binder, visit this link. Gradient Descent can be best explained with the classical analogy of mountain. A person is stuck in the mountains and is trying to get down (i.e. trying to find the global minimum). There is heavy fog such that visibility is extremely low. Therefore, the path down the mountain is not visible, so they must use local information to find the minimum. They can use the method of gradient descent, which involves looking at the steepness of the hill at their current position, then proceeding in the direction with the steepest descent (i.e. downhill). If they were tr