What are prototypes in RBF networks?

Question

I am currently reading Boosting the Performance of RBF Networks with Dynamic Decay Adjustment by Michael R. Berthold and Jay Diamond (online) to understand how Dynamic Decay Adjustment (DDA; a constructive trainining algorithm for RBF networks). Doing so, I stumbled over the word prototype a couple of times:

Unfortunately PRCE networks do not adjust the standard deviation of their prototypes individually, using only one global value for this parameter.

[...]

This paper introduces the Dynamic Decay Adjustment (DDA) algorithm which utilizes the constructive nature of the PRCE algorithm together with independent adaptation of each prototype's decay factor.

[...]

PNNs are not suitable for large databases because they commit one new prototype for each training pattern they encounter, eeffectively becoming a referential memory scheme.

I've tried to find it in the only resource they referenced in this context (D.L. Reilly, L.N. Cooper, C. Elbaum: "A Neural Model for Category Learning"), but sadly I don't have access to that one.

I found an explanation on https://chrisjmccormick.wordpress.com/2013/08/15/radial-basis-function-network-rbfn-tutorial/:

An RBFN performs classification by measuring the input’s similarity to examples from the training set. Each RBFN neuron stores a “prototype”, which is just one of the examples from the training set. When we want to classify a new input, each neuron computes the Euclidean distance between the input and its prototype. Roughly speaking, if the input more closely resembles the class A prototypes than the class B prototypes, it is classified as class A.

So a prototype is just the parameters (center and radius, assuming Gaussians are used) of an RBFs neuron?

Rephrasing the first quoted sentence, does it mean that the RBF networks usually only learn the center and the radius is fixed?

My question is if I understood it correct. Please add a reference (which is not a random blog article) which makes it more clear.

Answer 1

Kernel based methods work by constructing a feature vector based on some distance metric of an input vector to one or more example vectors, which are determined during training.These are the prototypes in your case (also known as regressors in some contexts). They are essentially vectors in the space of the independent variables.

Radial Basis functions essentially "convert" the (Euclidean) distance between input vector and prototype to a similarity value, usually [0,1]. Some of them, such as the Gaussian, have parameters such as the bandwidth.

In RBF Networks, there are a number of different options of determining the prototypes, for instance by Clustering at the space of independent variables, or by Orthogonal Least Squares (references at the bottom).

Like you correctly assumed, the first sentence is explaining that in many cases of training algorithms (the ones mentioned above included), the bandwidth of the RBF is constant and determined a-priori What is subject to determining is the actual location of the prototypes. In this context, the prototype would refer to the value composition in the input space, but not to the radius (which is fixed anyway).

I have not read the paper in question, but it seems that the authors are proposing some method to individually adjust the bandwidths of the RBFs. In this case, the radius may have been included as well into the notion of the prototype, but it is specific to the research in question.

References

J. Moody and C. J. Darken, “Fast Learning in Networks of Locally-Tuned Processing Units,” Neural Comput., vol. 1, no. 2, pp. 281–294, Jun. 1989.

S. Chen, C. F. N. Cowan, and P. M. Grant, “Orthogonal least squares learning algorithm for radial basis function networks,” IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302–309, Mar. 1991.