Quick Recap An artificial neural network is a combination of artificial neurons which does some math and try to estimate a mathematical function. This estimation process is called training or fitting. Basic training mechanism The math involved in ANN is mostly MAC (Multiply-Accumulate) operations where the input is multiplied by weights and biases are added to the product. One of the activation functions is applied to the output and it is forwarded to the next layer and the same process continues until it reaches to the end layer. This process is called feed-forward. After the end layer calculation, the output computed by the network is compared with the actual output. The difference between actual output and estimated output is calculated using a function called loss function. Common loss functions these days are Mean Squared Error, Mean Absolute Error, Root MSE, Cross-Entropy etc. The error calculated using loss function is propagated backward throughout the neura...