Neural Networks Taken Apart: The working

Now that we've drawn visual comparisons to the biological and artificial neural networks, it's time we understand how they work. Let's take a single neuron and see what happens.

Now, while this model was useful to draw comparisons, for us to understand how it works, we need to add some math into the mix, after all, you need math if you want to make sense of what's going on.

Okay, here's the math-ified version...

So again, on the left hand side we've input and on the right hand side we've output. Here's notation:
X₁ to X_n  represent the inputs. (Let them be numbers between 0 and 1)
W₁ to W_n represent the weights. (Let them be numbers between 0 and 1)

Σ refers to the summation of all the inputs.

So Σ gives us the sum of the products of all inputs and their corresponding weights.

So now, lets try to make sense of all this, here's an example. If you were getting a tattoo done, the pain starts out a bit slow, and then keeps rising. But how do you measure the pain ? We compare things to state magnitude for things like pain, we'd say 'it hurt more than an injection', what's that like ? 'hurts more than a thorn'. Wait what ? 

So if I strictly asked you to rate it on a scale of 1 to 10, it'd be similar to something above. Now, your neurons would be receiving these pain signals of magnitude 2 initially say, they'd pass this value across the network, until it reaches your brain, and since your brain is fine with 2 you don't do much. But once it gets over a threshold (say 5), your brain would force you to want to move your hand away. This is your response to pain over level 5. 

Now, your entire arm would be sending pain signals from each point that hurts, say from 'n' places, and each signal has a value of 'X'. So your neurons would sum up all the signals/X values to get your total pain. Hence the summation 'Σ'. Now this Σ is sent as a signal to your brain and then it sends back info on what to do, which we call output.

See it's not all that hard now is it ? Now you understand why all the Xs exist and what the Σ does. Then what about the weights ? 

Well the weights are where the magic happens...

Each time an input is multiplied with it's corresponding weight, it's value is increased or decreased depending on the value of the weight. Here's an example to explain why this is important...

Now consider you're learning to ride a bicycle.

Now, at first we start by taking off the training wheels, this is the 1st time you'll be learning to balance, and depending on the surface you're rollin' on, sorry riding on, you've gotta make tiny adjustments to make sure you don't fall. You'll probably develop methods like using your legs to act as counter weights or hands and so on. At 1st you start with random movements, until you figure the right one.These adjustments are what sharpen your instincts for the road later. It's the same with weights, at 1st the weights are randomly assigned (sometimes even set to just 1). Later, depending on the output, the weights are optimized using a number of techniques like gradient descent that I covered here. 

With all these modifications made, your algorithm is ready to help humanity fight cancer or rid earth of it's cancer (see what I did there ?)

This is just the beginning, there's still more to cover, more to make this better, and more to even replace this. So stay tuned.