Neural Networks - Notes

Dr I F Wilde

Neural Networks - Notes (pdf file 753Kb) or (ps file 1436Kb)

These notes are based on lectures given some years ago in the Mathematics Department at King's College London (as part of the MSc programme in Information Processing and Neural Networks). An attempt has been made to present a reasonably logical (mathematical) account of some of the basic ideas of the "artificial intelligence" aspects of the subject.

- further available material.

Brains have been around for quite a time now, so it would be nice to know how they work and very nice to be able to build machines which mimic their function (or even some aspects of their function).

Electrical engineers are concerned with the nuts and bolts of real, working (and income-generating) artificial intelligence systems, for example, those involved with fingerprint or retina recognition. Fuzzy Logic has also been incorporated into such systems (Bart Kosko).

Theoretical considerations can be split into a small mathematical part and a somewhat larger part involving a certain amount of mathematical symbolism but yet unhampered by the logic or rigour of mathematics. If the words "neuron" and "fires or does not fire" are replaced by the words "spin" and "points up or down", then neural networks are transformed into "spin glasses" - so-called disordered systems.

An interesting point of view of the application of neural networks to statistical problems has been offered by D. Ripley in the collection "Networks and Chaos - Statistical and Probabilistic Aspects", edited by O,E. Barndorff-Nielsen, J.L. Jensen and W.S. Kendall, Chapman and Hall, London, 1993.

An overview of some rigorous mathematical results (as well as some remarks on lesser-mathematical methods) has been given by D. Petritis (in Ann. Inst. Henri Poincare, 64, 255-288, 1996.) This article contains many references including some to further rigorous work - a selection being that of: M. Aizenman, J. Lebowitz and D. Ruelle, A. Bovier, J. Bricmont, A. van Enter, J. Frohlich, C. Newman, M. Talagrand, B. Zegarlinski.