6
$\begingroup$

I try to understand the difference between radial basis neuron network and support vector regression with Gaussian Kernel. I watched Andrew Ng presentation about kernels in SVN, I have read also an introduction to RBFNN in Bishop's book and Wikipedia's article about SVR.

As I understand it correctly in both cases I try to find a function from features space to R by computation of gaussians of a distances between data from training set to some centers distributed in feature space (activation of rbfs). Next I try to find the best set of linear combination of coefficients, to approximate my data as a linear combination of activated RBFs.

However in some papers these methods are mentioned as two different, competitive methods.

So my question is what exactly is the difference?

$\endgroup$
1
$\begingroup$

Your description of the model functioning is correct. Structurally, both models are representatives of the so called kernel methods. As such they are very similar, the same in many cases.

What is completely different between the two methods is the way the kernel centers and linear coefficients are derived. My experience with SVM is limited, so I cannot go into the details of the training method, but in summary is consists of solving the constrained optimization problem of finding the best approximation subject to maximum coefficients. This is achieved by linear programming or through a method like Sequential Minimal Optimization (SMO).

Training RBFNs, on the other hand, is a different story. Identifying the kernels is done in one of two ways usually; either by using a clustering algorithm or by using Orthogonal Least Squares (OLS). In either case, the linear coefficients are identified as a second step through least squares (LS).

The differences in training mean that, even through in structure the resulting models may be the same, in functioning they will most probably be completely different, due to the different fitting procedures.

Some references:

J. C. Platt, “Fast Training of Support Vector Machines Using Sequential Minimal Optimization,” Adv. kernel methods, pp. 185–208, 1998.

J. Moody and C. J. Darken, “Fast Learning in Networks of Locally-Tuned Processing Units,” Neural Comput., vol. 1, no. 2, pp. 281–294, Jun. 1989.

S. Chen, C. F. N. Cowan, and P. M. Grant, “Orthogonal least squares learning algorithm for radial basis function networks,” IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302–309, Mar. 1991.

$\endgroup$
0
$\begingroup$

SVR work only in margin of training data but RBFNN must be optimized weight

RBF NN may be fall in local minima opposed to SVR

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.