Deep Learning: A Quick Overview

Photo by William Stitt on Unsplash

This photo is from 1980, and the reason why we are dwelling into history is that neural networks along with deep learning have been around for quite some time. They have started picking up now and impacting the world right now, but if you look back into the 80s you will realize that even though they computers weren’t invented in 60s and 70s they really caught on to a trend and got the wind in 80, So people started talking about them a lot. There was done a lot of research in that area and everybody thought that deep learning or neural networks this new thing that is going to impact the world and is going to change everything and will solve all the world problems. And then it kind of slowly died for the next decade. So what happened why didn’t the neural networks survived and not change the world? The reason was they were not just good enough, they are not that good at predicting things, and not that good at modeling or they were not a good invention or is there any other reason. So actually there is another reason and the reason is in front of us is the fact that technology was not up to the right standard to facilitate the neural network.

In order for neural networks and deep learning to work properly you need two things, one is you need Data and you need a lot of  Data and other is you need a processing power you need a strong computer to process that Data and facilitate the neural network’s. So let's have a look at how data or storage of data has evolved over the years, here we got three years in the above image, how does storage look back in 1956 there is the hard drive and that is 5MB hard drive been transporting to some other place. A company had to pay 2500$ of those days’ dollars to just to rent it not buy it and just for one month.
After that in 1980, the situation improved a little bit, so you have 10MB for 3500$ and was still very expensive and has only 10MB and can store only one pic of nowadays.
And today in 2017 we have got 256GB for 150$ which can fit on your finger. If we compare these years across the board and before even taking price and size to consider, just take compare the capacity from 1956 to 1980 the capacity increased about double and then from the 1980s to 2017 it increased about 25600x times. And the length of the period is not that much but a huge jump in technological progress.
Here we have a chart of hard drive cost per gigabyte (USD).

Over the years this going to go even further right now scientist are even looking into using DNA for storage. From 10 to 20 years from now, everybody is going to use DNA storage if we go in this direction. Here are some stats.

So all of this brings us to what is deep learning?

here is the image of Geoffery Hinton and is known as the God Father of deep learning and he did research on deep learning in the 80s and he has done lots of work on Deep learning and published lots of research papers he now works in Google.
The idea behind deep learning is to look at the human brain, how the human brain operates. because human brain seems to be a very powerful tool in this planet for learning and adapting skills and applying it if the computer can copy that then we can just leverage what natural selection had already decided for us for all that algorithm.
So in the brain, we have neurons and us hundred of billions of neuron in the human brain and each neuron is connected with thousands of neurons with its neighbor. In below pic, this is what it looks like.

This is an actual section of the human brain this is called the cerebellum.and is responsible for balance so you can see there are billion and billion of neurons so that is what we are going to recreate.
So how do we create this on the computer?

In the above image: We will create an artificial structure called an artificial neural network. Where we have nodes or neurons, so we are going to have some neuron for input values. Which will take some input values in the input layer and we will have an output layer which will show the predicted result. And in between, we have hidden layers so before giving output it will go through billion and billions of neurons before it gives us output, so this is the whole concept behind it so we are going to model the brain. so we need this hidden layer. the input layer neurons are connected to hidden layer neurons and hidden layers neuron is then connected to output layers neurons.
So why is it called deep learning?

Now, we take this to next level we separate the input and output layers and we have not just one hidden layer, we have lots of lots hidden layers, and then we connect everything just like the human brain. Interconnect everything and that’s how input values are processed through all hidden layer just like the human brain and then we have output value. Now we are talking about deep learning.