Thoughts on Neuronal Networks

This has been on my ToDo Heap for some time now: I started to implement some libraries for neuronal networks in nearly all languages I know. I'm no expert on this topic and I haven't actually been able to implement all the fascinating stuff so far, but there is still enough time left until the entropy death of the universe.

First of all, I should explain, what a (artificial) neuronal network is. Well, obviously it's about wiring neurons together. I like to imagine them as persistent functions: They provide a value, which can be computed with some inputs. As with functions, the input might itself be the value of a neuron, else one couldn't write for example abs(sin(x)) to compute the absolute value of the sine of a value x. On the other hand, often the input stays the same, so why not just keep the old value without recalculating, if the input didn't change?

And of course when the value of one neuron changes, there is a wave of updates: A changes, B has A as input and thus changes, C has B as input, etc. Quite like a normal program actually, you do some calculation, until your line of dependencies ends.

That's a boring program! It's just a straight line, we don't have any loops yet. But we could do that anyway, and there breaks the function metaphor: We could just wire them back to themselves.

Take for example the neuron A with the simple rule add your inputs. Now we connect

1 => A
A => A

and maybe add some things that do something useful. Now A might start off as 0 and update. It becomes 1 and update. 2 and update... That's just an infinite loop counting up, but one can easily imagine, that it is much better with neurons like 1 if the first input is greater than the second or 0 else.

For now that's where I am. There is of course more to it, as it is a possibility for creating artificial intelligence and dealing with computer learning, but you should probably read more about this, if you are interested. I might write more about this, once I have more time.