Questions tagged [kernel]

Kernel functions are a class of functions which transform the original data into a new space in which the classes of the data are easier to separate by a kernel algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
9 views

Kernel Trick and Inner Product Preservation

I understand that the point of using the kernel trick is to project the problem onto a higher dimensional space, where the problem is linearly separable. In this explanation, https://www.quora.com/...
0
votes
0answers
10 views

how use RBF for primal model of svm?

I know if we want to solve primal model of non-linear SVM, we have to generate new features. for example for kernel (1+xz)^2 for primal problem for any pair of features x1 and x2 we have to generate: ...
1
vote
0answers
14 views

Designing Custom Kernel from my Mathematical model

I derived a mathematical model for a porous system and the final function looks like this , after going through the Mercers Theorem and it condition for a kernel i would love to write a SVM kernel ...
2
votes
1answer
35 views

Keras: Misunderstanding the Conv2D's param “filters”?

From https://keras.io/layers/convolutional/ (Conv2D): ...
2
votes
1answer
27 views

Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?

Kernel ridge regression associate a regularization parameter $C$ with weight term ($\beta$): $\text{Minimize}: {KRR}=C\frac{1}{2} \left \|\beta\right\|^{2} + \frac{1}{2}\sum_{i=1}^{\mathcal{N}}\left\|...
1
vote
1answer
196 views

Kernel PCA and K largest eigenvectors

How can one prove that the optimal kPCA solution $a^*=\{a_1...a_K\}$ are the $k$-largest Eigenvectors of the (centered) kernel matrix $K$? I referred to a lot of resources and couldn't find a proper ...
0
votes
0answers
7 views

Kernel approximation of kernels dependent on sum and differences of inputs

As we know, kernels $\kappa(x_i, x_j)$ with values depending on $x_i - x_j$ only can be approximated quite easily to be equal to $z_\omega(x_i)z_\omega(x_j)$ However what do we in case of kernel ...
0
votes
0answers
6 views

Why kernel perceptron relies more on the training data than kernel SVM?

In UCSD machine learning course, it is said that: "for Kernel Perceptron, the solution is likely to depend on more of the training points than the ...
1
vote
0answers
26 views

Implementing SVM with Gaussian Kernel

This is referencing Prof. Andrew Ng's course on machine learning. In the part that details implementing an SVM with the Gaussian kernel, we are supposed to use all the training examples as our ...
3
votes
1answer
128 views

Does increasing kernel size in a CNN result in higher accuracy on the training set?

In a convolutional neural network, does increasing the size of kernel always result in better training set accuracy? For example, if I use 5x5 kernels in a CNN instead of 3x3 ones, will it always ...
0
votes
0answers
17 views

OSError in Kaggle using [Fastai]

I am trying to use fastai (v 1.0.52) in Kaggle and have been getting the following error very time call the tabular_learner or conv_learner This error does not occurs when using ...
5
votes
1answer
71 views

Is there any conceptual relationship between 'kernel' in SVM and 'kernel' in convolution neural net?

In SVM, we have kernel function that maps an input raw data space into a higher dimensional feature space In CNN, we also have a 'kernel' mask that travels the input raw data space (image as a matrix)...
0
votes
0answers
30 views

How to fit a Gaussian Process with a product kernel K(x,a)=K_1(x,x')K_2(a,a') with scikit-learn?

I have a training dataset in the form of $(x, a, y)$ where $x$ and $a$ are two arrays of features and $y$ is the target outcome. I am interested in fitting a Gaussian Process with a product kernel $...
0
votes
0answers
18 views

Finding the kernel in which the data will have the highest ratio of leading eigenvalue to trace

I have a method which works well with the data where a ratio of a leading eigenvalue or of first few leading eigenvalues to trace is high. Is it possible to find a kernel transformation which is ...
1
vote
1answer
58 views

Computation of kernel matrix using radial basis kernel in svm

I want to compute a kernel matrix using RBF on my own. The training data is multidimensional. My query is whether we will apply $$e^{-\gamma(x-y)^2}$$ for each dimension and then sum the values across ...
0
votes
1answer
96 views

Does Convolution kernel size affect number of channels?

I am going through Dilated Residual Network blog post. In this, Under 2.Multi-scale Context aggregation heading, author mentioned this. The last one is the 1×1 ...
2
votes
1answer
29 views

Positive semidefinite kernel matrix from Gower distance

I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
1
vote
1answer
34 views

Basis expansion for regression using neural network?

I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data ...
1
vote
1answer
159 views

Channels in convolutional layer

I usually see convolutions performed over all the channels of the input. For example a $3x3$ kernel is really a $3x3xN$ kernel for a an input with $N$ channels, thus resulting in a single output ...
2
votes
2answers
68 views

CNN strategy in recognizing spinned images

I wrote my CNN code from scratch with some convolution kernels. But my CNN can't recognize flipped/spinned images correctly when there are only a few convolution kernels (3*3). My convolution kernels ...
2
votes
1answer
62 views

RBF kernel can classify two classes as in figure?

As you can see, I have some points (belonging to red and blue class), and I would to use an RBF kernel but I think that an RBF kernel can make points linearly separable only if they are located in ...
3
votes
3answers
74 views

How to understand features impact in a non linear case?

I give a simple example: I have a set of houses with different features (# rooms, perimeter, # neighbours, etc...), almost 15, and a price value for each house. The features are also quite correlated (...
0
votes
1answer
36 views

Mercer's Theorem importance

I understood that Mercer's Theorem extends the definition of kernels also for infinite input space. In Machine Learning realm our training set is always finite and hence the input space is always ...
1
vote
1answer
345 views

Question about “1x3 and 3x1 conv is equivalent to 3x3 conv”

I see a lot of sites talk that we can substitute 1x3 conv + 3x1 conv for 3x3 conv. In order to demonstrate easily, we use a 3x3 image as an example. From the point of view of parameters, I know that ...
0
votes
2answers
48 views

What advantage does Guassian kernel have than any other kernels, such as linear kernel, polynomial kernel and so on?

Guassian kernel is so important in SVM as we know. The parameter gamma is designed for this kind of kernel. My question is what makes Guassian kernel so unique? ...
0
votes
1answer
39 views

How to choose PCA or KernelPCA a priori?

I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA. If I understood correctly, I use PCA whenever I want to reduce the number of features ...
1
vote
0answers
16 views

kernel fisher discriminant

I have been using LDA to try to build a disease prognosis using medical data for a group of patients and a group of controls, and after limited success I decided to try to use kernel Fisher ...
1
vote
2answers
94 views

Intuition behind the fact that SVM uses only measure of similarity between examples for classification

I have read about SVM and although I did not understand the math behind it completly, I know that it produces decision plane with maximum margin between examples of different classes and role of ...
1
vote
0answers
21 views

Is the hypothesis space spanned by kernel evaluations on datapoints equivalent to the hypothesis space of linear functionals in the feature space?

when studying kernel methods a few years ago I got a bit confused with the concepts of feature space, hypothesis space and reproducing kernel Hilbert space. Recently, I thought a little about ...
3
votes
1answer
1k views

How do I interpret the length-scale parameter of the RBF kernel?

According to the Scikit-Learn documentation for the RBF kernel: The length scale of the kernel. If a float, an isotropic kernel is used. If an array, an anisotropic kernel is used where each dimension ...
1
vote
1answer
123 views

Unable to understand Kernel Ridge regression

I am trying to read kernel ridge regression from this link But , I am unable to get the intution behind the derivation. Can anyone please help me ?
1
vote
0answers
34 views

Nyström approximation of the non-linear mapping $\phi$ for a RBF kernel - what is the impact of weak duality?

For SVM, it is better to solve the problem in the primal for very large data-sets. However, the non-linear mapping $\phi$ for a RBF kernel is not explicit. Approximation methods for $\phi$ like the ...
1
vote
2answers
168 views

How to tune bandwidth in machine learning kernel model?

Gaussian kernel $k(x,y) = \exp(-\lVert x-y \rVert^2/\sigma^2)$ has a hyperparameter $\sigma$. I know grid search cross validation, but this would require a lot of computation since computational ...
0
votes
1answer
41 views

Bandwidth selection Kernel Density Estimation

I want to do KDE on data that are not necessarily normal using Gaussian kernels. In KDE in wikipedia an expression for the bandwidth is given when the underlying distribution of the data is gaussian. ...
1
vote
1answer
45 views

What Kernel is suitable for the following data for SVM classification?

I have the following 2 class data, as shown below. . Its a hand crafted example using two ellipse equations. I want to know what might be a recommended kernel to be used with this problem if I want ...
1
vote
2answers
58 views

How do we decide which kernel needs to be used in SVM?

How do we decide which kernel needs to be used for a particular dataset? Is there any criteria needs to be followed? And also what is the criteria to select C and gamma values? Kindly excuse me if ...
1
vote
1answer
552 views

SVM with polynomial kernel: strange results with high degree?

Currently I'm working in WEKA, using the SMO classifier (an implementation of SVM). For an assignment I am requested to use a polynomial kernel, and report the results for degrees varying from 1 up to ...
0
votes
1answer
403 views

Finding a kernel with feature transformation

Suppose we have feature transformation $\Phi(x) = [1, x_1, x_2, x_1x_2]$. Now we want to find the kernel corresponding to $\Phi$. What I have done is using kernel decomposition, we have: $$ K(x, y) = ...
1
vote
0answers
22 views

How is Kernel Matrix on a distribution defined?

Consider the following words taken from the lecture notes: The Hilbert-Schmidt Independence Criterion (HSIC) measures the dependence of the two random variables $X$ and $Y$. An empirical estimate of ...
0
votes
1answer
30 views

Kernel with complex vectors

I have a question regarding my machine learning lecture where we had to decide whether $$K(x,y)=x_1y_1-x_2y_2$$ is a valid kernel (e.g. for a SVM). My intuition would say that it is a valid kernel ...
2
votes
1answer
543 views

Under what conditions should an autoencoder be chosen over kernel PCA?

I've recently been looking at autoencoders and kernel PCA for unsupervised feature extraction. Lets consider just for a moment linear PCA. Its my understanding that if a autoencoder (with a single ...
12
votes
1answer
9k views

back propagation in CNN

I have the following CNN: I start with an input image of size 5x5 Then I apply convolution using 2x2 kernel and stride = 1, that produces feature map of size 4x4. Then I apply 2x2 max-pooling with ...
2
votes
1answer
82 views

What algorithm can predict structured outputs of arbitrary size?

I have a collection of graph objects of variable size (input) which are each paired to another graph of variable size (output). The task is, given an input graph, produce the most likely output graph. ...
5
votes
2answers
643 views

Kerns LSTM kernel

I am trying to understand how the weight matrix in an LSTM cell is used. An LSTM unit has several weight matrix: Wf, Wi, Wc, Wo like below: ( from http://colah....