COURSES:
- HB12HRB Regulation of Human Body Function
- HB22NSM Neuroscience of Sensation and Movement
TOPIC: Neural Networks
LECTURER: Dr. Andrew Bendrups, School of Human Biosciences
NEURAL NETWORKS
© A P Bendrups, 2000
WHAT ARE NEURAL NETWORKS?
They are layers of interconnected neurones, in which each neurone in a layer is connected to some or all of the neurones in the next layer.
Mathematicians and computer programmers have been interested in them for a long time, for theoretical reasons. Biologists have only recently started paying serious attention.
DO THEY ACTUALLY EXIST IN THE NERVOUS SYSTEM?
Yes, it certainly looks that way. The cortex of the brain is clearly a layer of cells; different areas of cortex connect with each other and there is a lot of convergence and divergence in these connections. Other structures in the brain, like the thalamus, may not look like a layer, but probably still have connections like a "neural network" (as defined above). It's just that the layers in the thalamus are scrunched up into balls (nuclei), like a sheet of paper can be scrunched up into a ball.
For example, neural nets may link sensory inputs from a surface like the skin or the retina of the eye, to a sensory cortex, and through intermediate layers of neurones, to a final layer of motor neurones controlling muscles.
WHAT DO THEY DO?
Mathematicians and computer programmers have explored this area extensively, both in theory and practice. They have designed artificial neural nets: with pen and paper using the language of maths, or as computer programs called "neural net simulations".
They have discovered that neural nets can do amazing things: diagnose illness, predict stock exchange movements, recognise faces or fingerprints and so on...
Neural nets are very likely to lead to the manufacture of truly intelligent robots, capable of problem solving and stimulating conversation. At the moment, robots with artificial brains based on neural nets are probably only about as smart as a rat, at the most.
HOW DO THEY WORK?
Neural nets simply link any input pattern with any output pattern. In simulated nets, the input pattern is represented by a series of numbers. So is the output pattern. These numbers have to be given some meaning by the programmer.
For example, the input numbers might represent the sign and symptoms of an illness and the output numbers might represent the names of various diseases. The numbers have to be decoded by the programmer for the neural net to be of any use to anyone. A net that does diagnosis could be called an "expert system".
In a biological net, the meaning is provided by anatomy. Real sense organs are connected at the input end and muscles and glands at the output end. Instead of numbers typed into a computer keyboard, we have action potentials in nerve fibres: the number of action potentials firing per second in a real neurone is roughly equivalent to the number you might feed into a computer-simulated net.
Actually, numbers in a computer are really no less meaningful than action potentials in a nerve fibre. Either way they only represent or "encode" things happening in the world outside; in neither case are they the "real thing".
On the output side, it's only when the action potentials cause movement that they produce an effect in the outside world. Numbers in a computer can similarly be used to produce movement of an artificial body.
Most commonly, artificial nets are used to produce print-outs of information, so it's more like a person writing down a decision.
Let's look at an example: the recognition of a human face. In this case a complex pattern of activity in a few million sensory cells in the eye provides the input. The output may be a person's name. Different output neurones could represent different names.
How can the net recognise a particular face and link it to a particular name? Only through training. The net, whether real or artificial, learns through experience.
Actually, biological nets probably have some links built in, as part of normal development. They have been "trained" through evolution over millions of years. The neural circuits which control reflexes and fixed action patterns could be regarded as very simple "preprogrammed" neural nets.
HOW DOES TRAINING WORK?
Initially the synapses in a net may be about the same, or they could be randomly different in strength: some weaker, some stronger. The strength of a synapse (ie. how effective it is in producing a response in its target cell) is called the weight of the synapse, in artificial intelligence jargon.
In training, the weights are modified until the input produces the desired output responses. In this way the pattern representing Mary's face may be gradually taught to activate the output representing Mary's name.
A single neural net can store information about many faces and identify them accurately; but it may make mistakes, particularly during training and even after it has been well trained (as humans do). Also, if it contains too few neurones, it will have a limited capacity and may become "saturated" if you try to teach it too much: then it will make lots of errors.
It's interesting that artificial nets make errors similar to children learning language. When a particular artificial net was trained to learn the rules for past tense, it first learned the irregular examples correctly (eg. go and went), but as training progressed it started to generalise (eg. go and goed): it had learned a common rule, add "ed" to the present stem. Later, it learned the exceptions properly. Children go through similar phases.
SO HOW DO YOU TRAIN A NET?
With artificial nets there is a slow way and a fast way.
The slow way is perhaps easier to understand: you temporarily change the weight of one synapse at a time and see whether you get a better result. If you do get a better result, you keep the change. If you don't get a better result, you leave the weight as it was. You keep doing this at random for a very long time...
The faster way is called "back propagation". It is based on a rather complex (to a non-mathematician) calculation in which you start at the output layer and work out how much the error changes when you change the weight of each synapse. You repeat this process working backwards through the layers to the input layer and use this information to adjust the weights.
Backprop is fast, but no-one has found a biological equivalent, so it's probably only relevant to computers and artificial intelligence. It's a bit scary because it means that when smart robots arrive on the scene, they may be a lot faster at learning and thinking than us. Hopefully they will treat us kindly.
HOW DO YOU TRAIN BIOLOGICAL NETS?
This probably works by association, as in associative learning at the level of the single neurone. For example, where interaction occurs between "teacher" and "learner" synapses.
Also by repetition, as in non-associative learning.
Signals from parts of the brain concerned with the emotions of pleasure may act through "teacher" synapses and control the changing of the weights of synapses, so that behaviours leading to pleasure become reinforced (strengthened).
Pain presumably works the other way, by decreasing the strength of synapses involved in dangerous behaviour which happens to cause pain.
Just as associative learning in a few neurones can forge a link between a bell ringing and salivation (as Pavlov showed in his experiments on dogs), a similar process operating through large nets may link the image of a face with the sound of that person's name. In a similar way, it may help you to choose the right tennis stroke to hit a winner.
Understanding neural nets will probably be the secret to understanding the complex functions of the brain, but research in this field still has a long way to go. The role of neural nets in human brain function is still controversial and speculative.
