|
Neural Network Methodology
Self-organisation
using the Generative Topographic Map with selective smoothing
Alfredo
Vellido,
Wael El-Deredy, and P.J.G.
Lisboa
The
Generative Topographic Mapping (GTM) is a non-linear latent variable model
introduced by Markus Svense, Chris Bishop and Chris Williams as a probabilistic
reformulation of self-organizing maps (SOM). The complexity of this model is
mostly determined by the number and form of basis functions generating the
non-linear mapping from latent space to data space, but it can be further
controlled by adding a regularization term to increase the stiffness of the
mapping and avoid data over-fitting. In our research, we show that the map
smoothing can be improved by introducing multiple regularization terms, one
associated with each of the basis functions. A similar technique to that of
Automatic Relevance Determination (ARD), developed by David MacKay and Radford
Neal, our Selective Map Smoothing (SMS) locally controls the stiffness of the
mapping depending on length scales of the underlying manifold, while optimizing
the effective number of active basis functions.
Continuous logic algebra as a mechanism for rule induction from
trained neural networks
T. Etchells, P.J.G. Lisboa and D.C. Pountney
Automatic decision support uses numerical models, which invariably
result in complex decision boundaries in order to maximise classification accuracy. These
are often non-linear and result from a statistical or neural network analysis of the data.
Continuous logic algebra is being explored as a mechanism to derive explicit,
multivariate, Boolean rules to describe the behaviour of analogue decision systems.
Extracted rules for the diagnosis of Progressive Leucenphalopathy.
Corresponding extracted rules for control subjects.
Publications:
Vellido, A., El-Deredy, W., and Lisboa,
P.J.G., Selective Smoothing of the Generative Topographic Mapping. IEEE Transactions on Neural Networks,
14, 4, July 2003.
Lisboa,
P.J.G., Etchells, T.A and Pountney, D.C. ‘Minimal MLPs do not model the
XOR logic’ Neurocomputing, Rapid Communication, 48, 1-4,1033-1037, 2002.(pdf)
Lisboa. P.J.G., Vellido, A. and Wong, H. 'Bias reduction in
skewed binary classification with Bayesian neural networks' Neural Networks, 13,
407-410. 2000.(pdf)
Perantonis S.J. and Lisboa P.J.G. 'Higher-Order Networks for
Invariant Recognition of Hand-Written Numerals' I.E.E.E. Transactions on Neural Networks,
2, 3, 541-551, 1992.
Lisboa P.J.G. and Perantonis S.J. 'Complete Solution of the
Local Minima in the XOR Problem' Network: Computation in Neural Systems, 2, 1, 119-124,
1991.
Lisboa P.J.G. and Perantonis S.J. 'Convergence of Recursive
Associative Memories Obtained Using the Multi-layered Perceptron' Journal of Physics A,
17, 4039-4053, 1990.
Books and book chapters
Lisboa, P.J.G. 'Industrial Use of
Safety-Related Artificial Neural Networks', HSE/Liverpool John
Moores University
HSE Books, 2001. (Contract research report 327/2001) ISBN 0717619710
Taylor, M.J. and Lisboa, P.J.G. (eds.) 'Techniques and
Application of Neural Network' Ellis Horwood, London, 308pp, 1993.
Lisboa P.J.G. (ed.) 'Neural Networks: Current Applications'
Chapman and Hall, London, 304pp, 1992.
Conferences
Agogino, A., Ghosh, J., Perantonis, S.J., Virvilis, V. ,
Petridis, S. and Lisboa. P.J.G. 'The role of multiple, linear-projection based
visualization techniques in RBF classification of high-dimensional data' to appear in
International Joint Conference on Neural Newtoks, Como, 24-27 July, 2000.(ps)
Lisboa, P.J.G., El-Deredy, W., Vellido, A., Etchells, T. and
Pountney, D.C. 'Automatic Variable Selection and Rule Extraction Using Neural Networks',
Proc. of the 15th IMACS World Congress on Scientific Computation, Modelling and Applied
Mathematics, Berlin, 461-466, 1997.
|
