Questions tagged [kernel]

Kernel functions are a class of functions which transform the original data into a new space in which the classes of the data are easier to separate by a kernel algorithm.

Filter by
Sorted by
Tagged with
18
votes
1answer
16k views

back propagation in CNN

I have the following CNN: I start with an input image of size 5x5 Then I apply convolution using 2x2 kernel and stride = 1, that produces feature map of size 4x4. Then I apply 2x2 max-pooling with ...
8
votes
0answers
78 views

What are practical differences between kernel k-means and spectral clustering?

I've been lately wondering about kernel k-means and spectral clustering algorithms and their differences. I know that spectral clustering is a more broad term and different settings can affect the ...
6
votes
1answer
84 views

Is there any conceptual relationship between 'kernel' in SVM and 'kernel' in convolution neural net?

In SVM, we have kernel function that maps an input raw data space into a higher dimensional feature space In CNN, we also have a 'kernel' mask that travels the input raw data space (image as a matrix)...
6
votes
3answers
125 views

Why spectral clustering results in disjointed cluster?

I'm working on a project where I have to dynamically cluster the position of objects with respect to one coordinate. So I'm essentially dealing with subsequent frames and each frame represents a one-...
5
votes
1answer
3k views

How do I interpret the length-scale parameter of the RBF kernel?

According to the Scikit-Learn documentation for the RBF kernel: The length scale of the kernel. If a float, an isotropic kernel is used. If an array, an anisotropic kernel is used where each dimension ...
5
votes
3answers
1k views

Kerns LSTM kernel

I am trying to understand how the weight matrix in an LSTM cell is used. An LSTM unit has several weight matrix: Wf, Wi, Wc, Wo like below: ( from http://colah....
3
votes
1answer
72 views

Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?

Kernel ridge regression associate a regularization parameter $C$ with weight term ($\beta$): $\text{Minimize}: {KRR}=C\frac{1}{2} \left \|\beta\right\|^{2} + \frac{1}{2}\sum_{i=1}^{\mathcal{N}}\left\|...
3
votes
1answer
1k views

Does increasing kernel size in a CNN result in higher accuracy on the training set?

In a convolutional neural network, does increasing the size of kernel always result in better training set accuracy? For example, if I use 5x5 kernels in a CNN instead of 3x3 ones, will it always ...
3
votes
3answers
90 views

How to understand features impact in a non linear case?

I give a simple example: I have a set of houses with different features (# rooms, perimeter, # neighbours, etc...), almost 15, and a price value for each house. The features are also quite correlated (...
3
votes
1answer
35 views

What is a good method for detecting local minims and maxims?

I'm using kernel density estimation in order to compute probability density function for item (triangles in the figure) occurrence. Using this output, i want to find all the local minims and maxims. ...
2
votes
2answers
69 views

CNN strategy in recognizing spinned images

I wrote my CNN code from scratch with some convolution kernels. But my CNN can't recognize flipped/spinned images correctly when there are only a few convolution kernels (3*3). My convolution kernels ...
2
votes
1answer
556 views

Keras: Misunderstanding the Conv2D's param “filters”?

From https://keras.io/layers/convolutional/ (Conv2D): ...
2
votes
2answers
70 views

How do we decide which kernel needs to be used in SVM?

How do we decide which kernel needs to be used for a particular dataset? Is there any criteria needs to be followed? And also what is the criteria to select C and gamma values? Kindly excuse me if ...
2
votes
1answer
826 views

Under what conditions should an autoencoder be chosen over kernel PCA?

I've recently been looking at autoencoders and kernel PCA for unsupervised feature extraction. Lets consider just for a moment linear PCA. Its my understanding that if a autoencoder (with a single ...
2
votes
1answer
860 views

Question about “1x3 and 3x1 conv is equivalent to 3x3 conv”

I see a lot of sites talk that we can substitute 1x3 conv + 3x1 conv for 3x3 conv. In order to demonstrate easily, we use a 3x3 image as an example. From the point of view of parameters, I know that ...
2
votes
1answer
94 views

What algorithm can predict structured outputs of arbitrary size?

I have a collection of graph objects of variable size (input) which are each paired to another graph of variable size (output). The task is, given an input graph, produce the most likely output graph. ...
2
votes
0answers
20 views

KLMS in machine learning

As Least Mean Square is a very popular choice to be used in combination with neural networks topologies, what would be the most common machine learning algorithms (and easily) to combine with Kernel ...
2
votes
0answers
31 views

How to implement SVM from scratch?

I am trying to build a SVM from scrath and I would like to maximize this Lagrarian expression: I know what variables means but I would like to know how this maximization is implemeted. Should I start ...
2
votes
0answers
29 views

Kernel engineering, valid kernels, multipliying by constant =0?

I am reading Bishop, Pattern Recognition and Machine Learning. In the chapter about kernels rules are given for constructing kernels from existing valid kernels. The first one being let k(x,y) be a ...
2
votes
0answers
39 views

Implementing a Kernel Adaptive Filtering model explained in a paper

In this paper, Stock price prediction using kernel adaptive filtering within a stock market interdependence approach, the authors propose a method for predicting stock prices by combining the ...
2
votes
0answers
19 views

Designing Custom Kernel from my Mathematical model

I derived a mathematical model for a porous system and the final function looks like this , after going through the Mercers Theorem and its condition for a kernel I would love to write an SVM kernel ...
2
votes
1answer
46 views

Positive semidefinite kernel matrix from Gower distance

I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
2
votes
1answer
147 views

RBF kernel can classify two classes as in figure?

As you can see, I have some points (belonging to red and blue class), and I would to use an RBF kernel but I think that an RBF kernel can make points linearly separable only if they are located in ...
2
votes
2answers
62 views

Many separation line using RBF kernel in SVM

Below is my code, it take a range of a number, creates a new column label that contains either -1 or 1. In case the number is ...
1
vote
2answers
22 views

PCA vs.KernelPCA: which one to use for high dimensional data?

I have a dataset which contains a lot of features (>>3). For computational reasons, I would like to apply a dimensionality reduction. At this point I could use different techniques: standard PCA ...
1
vote
1answer
121 views

Mercer's Theorem importance

I understood that Mercer's Theorem extends the definition of kernels also for infinite input space. In Machine Learning realm our training set is always finite and hence the input space is always ...
1
vote
2answers
311 views

How to tune bandwidth in machine learning kernel model?

Gaussian kernel $k(x,y) = \exp(-\lVert x-y \rVert^2/\sigma^2)$ has a hyperparameter $\sigma$. I know grid search cross validation, but this would require a lot of computation since computational ...
1
vote
1answer
676 views

Computation of kernel matrix using radial basis kernel in svm

I want to compute a kernel matrix using RBF on my own. The training data is multidimensional. My query is whether we will apply $$e^{-\gamma(x-y)^2}$$ for each dimension and then sum the values across ...
1
vote
1answer
268 views

Channels in convolutional layer

I usually see convolutions performed over all the channels of the input. For example a $3x3$ kernel is really a $3x3xN$ kernel for a an input with $N$ channels, thus resulting in a single output ...
1
vote
1answer
145 views

Unable to understand Kernel Ridge regression

I am trying to read kernel ridge regression from this link But , I am unable to get the intution behind the derivation. Can anyone please help me ?
1
vote
1answer
50 views

What Kernel is suitable for the following data for SVM classification?

I have the following 2 class data, as shown below. . Its a hand crafted example using two ellipse equations. I want to know what might be a recommended kernel to be used with this problem if I want ...
1
vote
1answer
788 views

SVM with polynomial kernel: strange results with high degree?

Currently I'm working in WEKA, using the SMO classifier (an implementation of SVM). For an assignment I am requested to use a polynomial kernel, and report the results for degrees varying from 1 up to ...
1
vote
1answer
32 views

How to use a RBF kernel to create a “Kernel Space” using the similarity of each pair of point?

I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper. Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of ...
1
vote
0answers
15 views

What is the kernel matrix used for in the kernel trick?

I have $n$ linearly inseperable datapoints, $x_1 \dots , x_n$. I use the kernel trick to map and compute the dot product in higher dimensions (without actually mapping / transforming the data). ...
1
vote
1answer
226 views

Kernel PCA and K largest eigenvectors

How can one prove that the optimal kPCA solution $a^*=\{a_1...a_K\}$ are the $k$-largest Eigenvectors of the (centered) kernel matrix $K$? I referred to a lot of resources and couldn't find a proper ...
1
vote
0answers
39 views

Implementing SVM with Gaussian Kernel

This is referencing Prof. Andrew Ng's course on machine learning. In the part that details implementing an SVM with the Gaussian kernel, we are supposed to use all the training examples as our ...
1
vote
1answer
43 views

Basis expansion for regression using neural network?

I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data ...
1
vote
0answers
20 views

kernel fisher discriminant

I have been using LDA to try to build a disease prognosis using medical data for a group of patients and a group of controls, and after limited success I decided to try to use kernel Fisher ...
1
vote
2answers
243 views

Intuition behind the fact that SVM uses only measure of similarity between examples for classification

I have read about SVM and although I did not understand the math behind it completly, I know that it produces decision plane with maximum margin between examples of different classes and role of ...
1
vote
0answers
23 views

Is the hypothesis space spanned by kernel evaluations on datapoints equivalent to the hypothesis space of linear functionals in the feature space?

when studying kernel methods a few years ago I got a bit confused with the concepts of feature space, hypothesis space and reproducing kernel Hilbert space. Recently, I thought a little about ...
1
vote
0answers
39 views

Nyström approximation of the non-linear mapping $\phi$ for a RBF kernel - what is the impact of weak duality?

For SVM, it is better to solve the problem in the primal for very large data-sets. However, the non-linear mapping $\phi$ for a RBF kernel is not explicit. Approximation methods for $\phi$ like the ...
1
vote
0answers
22 views

How is Kernel Matrix on a distribution defined?

Consider the following words taken from the lecture notes: The Hilbert-Schmidt Independence Criterion (HSIC) measures the dependence of the two random variables $X$ and $Y$. An empirical estimate of ...
0
votes
1answer
652 views

Does Convolution kernel size affect number of channels?

I am going through Dilated Residual Network blog post. In this, Under 2.Multi-scale Context aggregation heading, author mentioned this. The last one is the 1×1 ...
0
votes
1answer
48 views

Understanding SVM Kernels

Following Andrew Ng's machine learning course, he explains SVM kernels by manually selecting 3 landmarks and defining 3 gaussian function based on them. Then he says that we are actually defining 3 ...
0
votes
1answer
30 views

Kernel Trick and Inner Product Preservation

I understand that the point of using the kernel trick is to project the problem onto a higher dimensional space, where the problem is linearly separable. In this explanation, https://www.quora.com/...
0
votes
1answer
58 views

How to choose PCA or KernelPCA a priori?

I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA. If I understood correctly, I use PCA whenever I want to reduce the number of features ...
0
votes
1answer
52 views

Bandwidth selection Kernel Density Estimation

I want to do KDE on data that are not necessarily normal using Gaussian kernels. In KDE in wikipedia an expression for the bandwidth is given when the underlying distribution of the data is gaussian. ...
0
votes
0answers
19 views

Kernel approximation of a function known only point-wise?

Assume that I have a set of $N$ points $x_i, i=1,...,N,$ in some space $\mathbb{R}^D$, and corresponding point-wise (scalar) function evaluations $f(x_i)$. It is my goal to approximate the unknown ...
0
votes
0answers
16 views

Bottleneck Distance

Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar? Also, does it make sense to compute the bottleneck distance using $H_0$ ...
0
votes
1answer
14 views

Why do we use 2D kernel for RGB data?

I have recently started kearning CNN and I coukdnt understand that why are we using a 2D kernel like of shape (3x3) for a RGB data in place of a 3D kernel like of shape (3x3x3)? Are we sharing the ...