What is a Neural Network?
In recent years, Neural Networks have become tremendously popular among artificial intelligence researchers and practitioners.
We will attempt to explain what a Neural Network is in layman’s terms and argue why we believe it is important to have a basic understanding of the technology.
If you haven’t already, please start by reading our article “What is Machine Learning?”.
A Machine Learning algorithm
A Neural Network is a particular kind of Machine Learning algorithm. In the last 10 to 5 years, Neural Networks have outperformed other learning algorithms in a vast number of domains.
Having a single type of algorithm replace specialized systems across multiple fields is exciting for researchers, because it means that progress in Neural Networks yields progress in many different areas of research.
Two different perspectives
Neural Networks can be explained with two different perspectives.
1) By using the human brain as a metaphor, we can gain a general level of understanding of the core idea behind the networks.
2) By diving into the mathematics of neural networks, we can understand how they work on a deeper level.
We will attempt to describe the first perspective and gain insights into the core idea behind the networks.
Using the emails classification problem introduced in our previous blogpost “What is Machine Learning?”, we will explain how a Neural Network can learn to identify emails that requires censorship.
The forward and backward step
Neural Networks use a forward and a backward step. In the forward step, an email is processed by the system, and the system predicts whether it should be censored.
The system determines this by running the email through all its neurons. The neurons process the information and sends their individual predictions to the next layer in the neural network. Eventually, the final layer is reached, and all the neurons combine their information to a single prediction.
The correct answer is then provided to the network (that is, whether the email should be censored or not), and the backwards step can begin.
In the backward step, all the neurons are adjusted slightly to accommodate the new information. The processed is reversed, so the final layer in the system sends the information backwards, to the previous layer, until the first layer is reached. Every time a neuron receives this information, it is adjusted slightly, to accommodate for the information in the email.
When all the neurons have been reached, the backward step is done and the system is ready to process the next email.
In the beginning, the system produces random output. But after thousands of cycles, the neurons are eventually adjusted to classify the emails correctly.
How the updating of neurons work
Understanding what it means to take the derivative of a function is a prerequisite to understanding neural networks.
Taking the derivative is the process of finding the speed at which a function change. If you move from one point to another, the derivative of your position is the speed at which you move. Let’s say that you’re sitting in an accelerating car. If you plot the distance you have traveled, it will start out slowly, and gradually move faster and faster. If you instead measure the speed (velocity) at which you are traveling (the derivative of the acceleration) we can see that your speed is increasing with a constant rate. Furthermore, if we measure the acceleration at any given point in time (the derivative of the speed), we see that it remains constant through time.
When the system makes a prediction, it does so by combining the output from neurons in the last layer of the neural networks. By taking the derivative with regard to the output of each neuron, we can determine how that neuron affected to the final answer (just as acceleration affects speed, and speed affects distance). If the neuron makes a positive contribution to reaching the correct conclusion, we increase its effect slightly. If it made a negative contribution, we decrease it. The neurons in the final layer then repeats this process, by taking the derivative with regard to the neurons in the previous layer until eventually, the first layer of the system is reached.
When that happens, all the neurons have been adjusted to accommodate for the information in the email.
And that’s all there is to it. But repeating this process millions of times, the neurons will eventually be adjusted to perform complex predictions. Because the entire process is automated, computers can do this without any human intervention. Machine Learning researchers starts the process in the morning, and in the evening, the system has learned to classify emails.