Let's Build Skynet: Building your 1st Neural Network

In the previous post, we discussed about the inner workings of a Neural Network. Now that we have an understanding of how the data goes around, let's try building a simple Neural Network.

We'll be coding in Python.

Before we begin, let's go over the things you'd need to start coding the network, apart from the Judgement day dreams we've all been having...

• First, let's start with python, head here. Select your OS and follow the instructions.
• Next, let's get something to code in, I'd suggest something like VSCode or Atom. If you're just starting out, you can use Notepad++.

Now with all that done, let's dive straight into coding! Be sure to grab a drink...

Well then, we'll start with the imports...

	from numpy import exp, array, dot
	import random

view raw imports.py hosted with ❤ by GitHub

numpy is a library that, as the suggests is about numbers. From here, we're going to import functions that will let us compute the exponential function, dot products and to create arrays.
random is no random import, random is very essential, as it's a library that contains functions to generate random numbers. We'll get into why we need random numbers here, in a bit...

Before we begin design, let's have a look at the network that we're trying to build.

So we've 4 inputs, and 1 output. We'll represent the inputs and outputs as matrices. Which would give us something like...

Here's the code for the whole program, we'll break it down in a bit... I'd recommend giving it one read and trying to understand the overall flow.

	from numpy import exp, array, dot
	import random
	#Our neural network/ single layer perceptron has 3 inputs and 1 output
	class NeuralNetwork():
	def __init__(self):
	#A seed is provided so that we can generate the same random numbers each time the program runs
	random.seed(1)
	#We assign 3 random numbers, in a 3x1 matrix, whose values lie between -1 and 1
	self.weights = 2 * random.random((3, 1)) - 1
	#This is our activation function
	#The sigmoid function takes a number 'x', and normalises it between 0 and 1
	def sigmoid(self, x):
	return 1 / (1 + exp(-x))
	#This is the derivative of our activation function, also called the Gradient
	#This shows how confident we're of the current weight values
	def sigmoid_derivative(self, x):
	return x * (1 - x)
	#This is the training set, we try adjusting our weights through each iteration.
	#This is done by trial and error
	def train(self, training_set_inputs, training_set_outputs, number_of_training_iterations):
	for training_step in range(number_of_training_iterations):
	#We pass the input data(training data) through our network
	output = self.predict(training_set_inputs)
	#Error is calculated as: Desired Output - Actual Output
	error = training_set_outputs - output
	#To adjust the weights, we do the following:
	#Multiply the error(obtained above) by the input and again by the gradient(derivative) of the Sigmoid Curve
	adjustment = dot(training_set_inputs.T, error * self.sigmoid_derivative(output))
	#We Adjust the weights
	self.weights += adjustment
	#This is where the neural network starts 'predicting'
	def predict(self, inputs):
	#We pass the input data/training data) through our network
	return self.sigmoid(dot(inputs, self.weights))
	if __name__ == "__main__":
	#Initialize a single layer perceptron
	neural_network = NeuralNetwork()
	#We start with Randomized weights
	print("Random weights: ")
	print(neural_network.weights)
	#We take four 3x1 matrices as our imput(training) data
	#And 1 output value, which is like the answer to the problem we're trying to solve/model
	training_set_inputs = array([[1, 0, 1], [0, 1, 1], [1, 0, 0], [1, 1, 1]])
	training_set_outputs = array([[0, 1, 1, 0]]).T
	#Train the network with the training data
	#Here we're doing the training 10,000 times
	#With each run, we make adjustments to our weights
	#Try different values, and see how this helps our output
	neural_network.train(training_set_inputs, training_set_outputs, 10000)
	#We print our weights after the training has been finished
	#Compare the weights from numerous runs, to see variations
	print("New weights after training: ")
	print(neural_network.weights)
	#Here we can ask the network to finally predict values for a given input
	print("Predicting labels for: [0, 1, 0] -> ?: ")
	print(neural_network.predict(array([0, 1, 0])))

view raw SLP.py hosted with ❤ by GitHub

Now, that looks daunting at first, but it's pretty simple once you start breaking it down...
First, let's look at how our input and weights interact.

We just perform the dot product on the 2 matrices.

dot(inputs, self.weights)

view raw dot.py hosted with ❤ by GitHub

That wasn't so hard now was it ? Onward to the next one then...

For weight values, we initially set random values for them, instead of writing down random values each time, we make use of the random library.

	#A seed is provided so that we can generate the same random numbers each time the program runs
	random.seed(1)
	#We assign 3 random numbers, in a 3x1 matrix, whose values lie between -1 and 1
	self.weights = 2 * random.random((3, 1)) - 1

view raw random.py hosted with ❤ by GitHub

We use activation functions at this point. An activation function squeezes your output from a neuron to a range of values. We're using the Sigmoid function, which will give you outputs in the range of 0 to 1. This makes processing the outputs easier, and even computation easier when you go for more than 1 layer.

	#This is our activation function
	#The sigmoid function takes a number 'x', and normalises it between 0 and 1
	def sigmoid(self, x):
	return 1 / (1 + exp(-x))

view raw sigmoids.py hosted with ❤ by GitHub

Now that we're done understanding initialization of our data, let's try to understand how a model is 'trained'. Always remember that, you can't teach your dog tricks in one go...

You'll have to give it some time, sit down for multiple sessions, unless your dog gets it in one go 'cause it's a terminator bot from the future maybe 🤔

Even machines need time and practice to get stuff right. They've to be trained 'n' number of times.
So we want our algorithm to run a few times, too much is bad too. This you've to figure out by some experimentation, here we'll do it 10,000 times.

	for training_step in range(number_of_training_iterations):
	#We pass the input data(training data) through our network
	output = self.predict(training_set_inputs)
	#Predict is basically dot(inputs, self.weights)

view raw training.py hosted with ❤ by GitHub

Now, going with our dog analogy, each time you show your dog how to go fetch, you'd probably have to go fetch all by yourself. And then look at how the dog does it. And then based on the mistakes it makes, you try correcting it. It's the same with machines too (atleast when you're doing supervised learning)

We calculate error, this is obtained from the formula:

Error = Desired Value - Actual Value

	#Error is calculated as: Desired Output - Actual Output
	error = training_set_outputs - output

view raw error.py hosted with ❤ by GitHub

Well, we're calculating the error, so it's like you know your dog isn't running after the ball so you've to show him that was the mistake he's making. Similarly, we know there's a variation in the Desired and Actual Values, so we've to correct those mistakes too. We do this by 'adjusting' the weights. We follow another simple formula...

Adjustment = Error * Inputs * Gradient

Gradient, is a number that tells us how confident we're with the current weight values, it is obtained by computing the derivative of the activation function you're using, here it's the Sigmoid function.

	#This is our activation function
	#The sigmoid function takes a number 'x', and normalises it between 0 and 1
	def sigmoid(self, x):
	return 1 / (1 + exp(-x))
	#This is the derivative of our activation function, also called the Gradient
	#This shows how confident we're of the current weight values
	def sigmoid_derivative(self, x):
	return x * (1 - x)

view raw sigmoid.py hosted with ❤ by GitHub

Now, have a look at the code from above

	from numpy import exp, array, dot
	import random
	#Our neural network/ single layer perceptron has 3 inputs and 1 output
	class NeuralNetwork():
	def __init__(self):
	#A seed is provided so that we can generate the same random numbers each time the program runs
	random.seed(1)
	#We assign 3 random numbers, in a 3x1 matrix, whose values lie between -1 and 1
	self.weights = 2 * random.random((3, 1)) - 1
	#This is our activation function
	#The sigmoid function takes a number 'x', and normalises it between 0 and 1
	def sigmoid(self, x):
	return 1 / (1 + exp(-x))
	#This is the derivative of our activation function, also called the Gradient
	#This shows how confident we're of the current weight values
	def sigmoid_derivative(self, x):
	return x * (1 - x)
	#This is the training set, we try adjusting our weights through each iteration.
	#This is done by trial and error
	def train(self, training_set_inputs, training_set_outputs, number_of_training_iterations):
	for training_step in range(number_of_training_iterations):
	#We pass the input data(training data) through our network
	output = self.predict(training_set_inputs)
	#Error is calculated as: Desired Output - Actual Output
	error = training_set_outputs - output
	#To adjust the weights, we do the following:
	#Multiply the error(obtained above) by the input and again by the gradient(derivative) of the Sigmoid Curve
	adjustment = dot(training_set_inputs.T, error * self.sigmoid_derivative(output))
	#We Adjust the weights
	self.weights += adjustment
	#This is where the neural network starts 'predicting'
	def predict(self, inputs):
	#We pass the input data/training data) through our network
	return self.sigmoid(dot(inputs, self.weights))
	if __name__ == "__main__":
	#Initialize a single layer perceptron
	neural_network = NeuralNetwork()
	#We start with Randomized weights
	print("Random weights: ")
	print(neural_network.weights)
	#We take four 3x1 matrices as our imput(training) data
	#And 1 output value, which is like the answer to the problem we're trying to solve/model
	training_set_inputs = array([[1, 0, 1], [0, 1, 1], [1, 0, 0], [1, 1, 1]])
	training_set_outputs = array([[0, 1, 1, 0]]).T
	#Train the network with the training data
	#Here we're doing the training 10,000 times
	#With each run, we make adjustments to our weights
	#Try different values, and see how this helps our output
	neural_network.train(training_set_inputs, training_set_outputs, 10000)
	#We print our weights after the training has been finished
	#Compare the weights from numerous runs, to see variations
	print("New weights after training: ")
	print(neural_network.weights)
	#Here we can ask the network to finally predict values for a given input
	print("Predicting labels for: [0, 1, 0] -> ?: ")
	print(neural_network.predict(array([0, 1, 0])))

view raw SLP.py hosted with ❤ by GitHub

Seems much simpler doesn't it ? It's the same code, go check...
I've packaged it into modules, as it's easier to move the code to production or to be used as an import, how you write the code is up to you...

Now, here's some stuff you can try out:

• Try different training lengths
• Try fixed weights instead of random
• And try adding more data.