How a Neural Network Learns
Backpropagation calculates gradients. Gradient descent uses them to update the parameters.
Gradient descent
The loss surface shows how the loss changes with the parameters. The gradient points toward increasing loss, so gradient descent steps in the opposite direction. A large learning rate can overshoot the minimum.

Backpropagation
The forward pass produces a prediction and a loss. Backpropagation applies the chain rule from the output layer back through the hidden layers to find how each weight and bias affects that loss.

Source: Neural Networks Bike Demand, v3. Equation images and diagrams are reproduced from the PowerPoint.