3
$\begingroup$

I'm trying to use a custom kernel that accepts 3 arguments, with the SVM in sk-learn:

def k_gaussian(_x1, _x2, _sigma):
    normsq = np.square(np.linalg.norm(_x1-_x2))
    return np.exp(- normsq/(2 * np.square(sigma)))

According to the documentation, a custom kernel must only have two arguments, which the svm.SVC class will handle automatically with the given input data. We are told to pass the custom kernel in a form like:

clf = svm.SVC(kernel=my_kernel)

However, I'm working on an assignment which requires us to run experiments on SVM performance with varying values of _sigma.

How can I achieve that in this case? Can I pass in something like?:

clf = svm.SVC(kernel=k_gaussian(_sigma=2)) 

Would things like decorators help me here?

| improve this question | | | | |
$\endgroup$
7
$\begingroup$

That can be done with a closure like:

Code:

def build_k_gaussian(sigma):

    def k_gaussian(_x1, _x2):
        diff = _x1[:, np.newaxis] - _x2
        normsq = np.square(np.linalg.norm(diff, axis = 2))
        return np.exp(- normsq / (2 * np.square(sigma)))

    return k_gaussian

clf = svm.SVC(kernel=build_k_gaussian(sigma=2))

How does this work?

The function k_gaussian is defined when build_k_gaussian() is called. k_gaussian will be able to access the value of sigma from when the function was created. This is known as a closure.

So in the end, build_k_gaussian returns a function that takes two parameters, which is what the kernel parameter required.

According to the Using Python functions as kernels on scikit-learn:

Your kernel must take as arguments two matrices of shape (n_samples_1, n_features), (n_samples_2, n_features) and return a kernel matrix of shape (n_samples_1, n_samples_2).

So you need to apply the kernel function on all pairs of samples, therefore diff broadcasts the _x1 matrix and subtracts all samples in _x2 from all samples in _x1. You have to calculate the norm along axis=2.

| improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.