Questions tagged [kernel]

Kernel functions are a class of functions which transform the original data into a new space in which the classes of the data are easier to separate by a kernel algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
48 views

How to choose a kernel function and a feature mapping function?

Although, after extensive of reading, I know the concepts of support vector machines pretty well by now, I have trouble translating the concept of the kernel function $K$ and the feature mapping ...
1
vote
2answers
256 views

Intuition behind the fact that SVM uses only measure of similarity between examples for classification

I have read about SVM and although I did not understand the math behind it completly, I know that it produces decision plane with maximum margin between examples of different classes and role of ...
-1
votes
0answers
11 views

extrapolation in SVM model cauchy [closed]

I am using a SupportVectorMachine model type Cauchy. The model was created with minimum value to predict is zero, but running ...
0
votes
0answers
15 views

SVD Kernel and Linear Algebra Kernel, is there a conceptual difference?

Is the term kernel used in Sklearn to execute the SVD machine learning algorithm conceptually related to the notion of a kernel in linear algebra ( null space )? Or do they happen to use this same ...
0
votes
0answers
10 views

Has anyone succeeded in finding a good Scala/Spark kernel for Jupyter?

The ones I've tried so far Almond: Works very well for just Scala, but you have to import dependencies, and it gets tedious after a while. And unfortunately can't run when using Spark with YARN ...
2
votes
2answers
67 views

Many separation line using RBF kernel in SVM

Below is my code, it take a range of a number, creates a new column label that contains either -1 or 1. In case the number is ...
12
votes
1answer
130 views

What are practical differences between kernel k-means and spectral clustering?

I've been lately wondering about kernel k-means and spectral clustering algorithms and their differences. I know that spectral clustering is a more broad term and different settings can affect the ...
2
votes
1answer
232 views

Kernel PCA and K largest eigenvectors

How can one prove that the optimal kPCA solution $a^*=\{a_1...a_K\}$ are the $k$-largest Eigenvectors of the (centered) kernel matrix $K$? I referred to a lot of resources and couldn't find a proper ...
0
votes
1answer
31 views

Support Vector Machine (SVM) for classification problem based on Earth Mover's Distance (EMD)

I would like to run SVM for my classification problem using the Earth Mover's Distance (EMD) as a distance measurement. As I understood the documentation for Python scikit-learn (https://scikit-learn....
1
vote
1answer
45 views

Basis expansion for regression using neural network?

I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data ...
0
votes
0answers
22 views

Kernel approximation of a function known only point-wise?

Assume that I have a set of $N$ points $x_i, i=1,...,N,$ in some space $\mathbb{R}^D$, and corresponding point-wise (scalar) function evaluations $f(x_i)$. It is my goal to approximate the unknown ...
19
votes
1answer
17k views

back propagation in CNN

I have the following CNN: I start with an input image of size 5x5 Then I apply convolution using 2x2 kernel and stride = 1, that produces feature map of size 4x4. Then I apply 2x2 max-pooling with ...
2
votes
0answers
22 views

KLMS in machine learning

As Least Mean Square is a very popular choice to be used in combination with neural networks topologies, what would be the most common machine learning algorithms (and easily) to combine with Kernel ...
0
votes
0answers
17 views

Bottleneck Distance

Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar? Also, does it make sense to compute the bottleneck distance using $H_0$ ...
0
votes
1answer
16 views

Why do we use 2D kernel for RGB data?

I have recently started kearning CNN and I coukdnt understand that why are we using a 2D kernel like of shape (3x3) for a RGB data in place of a 3D kernel like of shape (3x3x3)? Are we sharing the ...
1
vote
2answers
26 views

PCA vs.KernelPCA: which one to use for high dimensional data?

I have a dataset which contains a lot of features (>>3). For computational reasons, I would like to apply a dimensionality reduction. At this point I could use different techniques: standard PCA ...
0
votes
0answers
33 views

Gaussian Process Classifier and specifying kernel

I am using scikitlearn's gaussian process classifier and either I don't think I understand how the kernel is used (more likely), or there is an error in the module (less likely). In short, the ...
0
votes
2answers
100 views

What advantage does Guassian kernel have than any other kernels, such as linear kernel, polynomial kernel and so on?

Guassian kernel is so important in SVM as we know. The parameter gamma is designed for this kind of kernel. My question is what makes Guassian kernel so unique? ...
0
votes
0answers
17 views

How do I create these Kernel functions in Python for Gaussian Process Regression?

I have a dataset of 1031 observed samples of 7 features that form the X and one target variable that forms the Y. I am using Gaussian Process Regressor to train my models. I want to use anisotropic ...
0
votes
0answers
16 views

How to create anisotropic exponential and gaussian correlation function in Python for kernel?

I have a dataset of 1000 observed samples of 6 features that form the X and one target variable that forms the Y. I am using kriging or Gaussian Process Regressor to train my models. I would like to ...
0
votes
0answers
9 views

Preparing L2 norm kernel to be use in SVR

I have a pandas DataFrame and I want to use SVR with one of DataFrame's columns ("Age"). In this SVR model, I want to use the L2 norm. To do this, I created a custom kernel function called ...
0
votes
0answers
10 views

Using kernel estimation to find similarity/difference between two feature sets for binary classification

I am trying to train a binary classifier using word vectors. I have the tfidf vectors for each sentence in my training set. Before applying binary classification algorithms, I just want to check ...
1
vote
1answer
33 views

How to use a RBF kernel to create a “Kernel Space” using the similarity of each pair of point?

I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper. Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of ...
2
votes
0answers
31 views

How to implement SVM from scratch?

I am trying to build a SVM from scrath and I would like to maximize this Lagrarian expression: I know what variables means but I would like to know how this maximization is implemeted. Should I start ...
0
votes
0answers
36 views

Kernel Density Estimation for bimodal distribution with Python

I have a bimodal distribution for the range [-0.1, 0.1] which can be viewed here: I want to train/fit a Kernel Density Estimation (KDE) on the bimodal distribution as shown in the picture and then, ...
0
votes
1answer
51 views

Understanding SVM Kernels

Following Andrew Ng's machine learning course, he explains SVM kernels by manually selecting 3 landmarks and defining 3 gaussian function based on them. Then he says that we are actually defining 3 ...
0
votes
0answers
20 views

GP derivative in GpyTorch

I am working on a project using GP-regression models to model transition and measurements models in a Kalman Filter. This means I need to be able to sample from the derivative of the original GP model....
0
votes
0answers
8 views

How do you prevent multiple kernels in a CNN from recognizing the same feature?

I've been reading Rosebrock's "Deep Learning for Computer Vision with Python", and he mentions that in a CNN, one of the layers is a set of $K$ kernels that each activate when they see a specific ...
1
vote
0answers
16 views

What is the kernel matrix used for in the kernel trick?

I have $n$ linearly inseperable datapoints, $x_1 \dots , x_n$. I use the kernel trick to map and compute the dot product in higher dimensions (without actually mapping / transforming the data). ...
0
votes
0answers
63 views

Generalized quadratic loss learning

I'm studying a binary classification task with an objective function, derived from SVM, defined so: $\vec{\xi}' S \vec{\xi}$ with: $y_i (f(\vec{x}_i)) >= 1 - \xi_i, i=1..l$ and: $\xi_i >=0,...
0
votes
0answers
24 views

Using rbf_kernel with two 2D numpy arrays causes Python 3 kernel to die

I am trying to calculate the maximum mean discrepancy between two datasets, X, Y, where the entries of X, Y are of type ...
0
votes
0answers
14 views

Deploy kernel from Github to Azure ML

Is there a way to have my kernel in Github and have an automatic deployment system to deploy the model to Azure ML with some tests after the deployment or in the time of deployment? I have read this ...
0
votes
0answers
24 views

Code freezes and never returns when linear_kernel (sklearn.metrics.pairwise) is used on 20M Movielens dataset

I'm fairly new to ML/AI, i'm trying learn the content based recommendation - here is my source code - https://github.com/jaganlal/content-based-recommender I'm using MovieLens 20M dataset - tags.csv ...
2
votes
1answer
951 views

Question about “1x3 and 3x1 conv is equivalent to 3x3 conv”

I see a lot of sites talk that we can substitute 1x3 conv + 3x1 conv for 3x3 conv. In order to demonstrate easily, we use a 3x3 image as an example. From the point of view of parameters, I know that ...
3
votes
1answer
35 views

What is a good method for detecting local minims and maxims?

I'm using kernel density estimation in order to compute probability density function for item (triangles in the figure) occurrence. Using this output, i want to find all the local minims and maxims. ...
2
votes
0answers
30 views

Kernel engineering, valid kernels, multipliying by constant =0?

I am reading Bishop, Pattern Recognition and Machine Learning. In the chapter about kernels rules are given for constructing kernels from existing valid kernels. The first one being let k(x,y) be a ...
6
votes
3answers
129 views

Why spectral clustering results in disjointed cluster?

I'm working on a project where I have to dynamically cluster the position of objects with respect to one coordinate. So I'm essentially dealing with subsequent frames and each frame represents a one-...
5
votes
3answers
1k views

Kerns LSTM kernel

I am trying to understand how the weight matrix in an LSTM cell is used. An LSTM unit has several weight matrix: Wf, Wi, Wc, Wo like below: ( from http://colah....
2
votes
0answers
19 views

Designing Custom Kernel from my Mathematical model

I derived a mathematical model for a porous system and the final function looks like this , after going through the Mercers Theorem and its condition for a kernel I would love to write an SVM kernel ...
2
votes
0answers
42 views

Implementing a Kernel Adaptive Filtering model explained in a paper

In this paper, Stock price prediction using kernel adaptive filtering within a stock market interdependence approach, the authors propose a method for predicting stock prices by combining the ...
0
votes
0answers
29 views

Gradient equations of gaussian kernel discriminant trained with gradiant descent

I am having a hard time trying to find the gradient equations for the weight $\alpha^t$ and $w_0$ for a gaussian kernel discriminant trained with gradient descent with the following error function $$E(...
0
votes
0answers
20 views

In SVM, is the support set still small if kernel trick is used?

In SVM, we classify y based on whether f(x) > 0 or f(x) < 0. I understand that in SVM with f(x) being linear in x, the support set is typically small (i.e., the number of support vectors is much ...
0
votes
1answer
33 views

Kernel Trick and Inner Product Preservation

I understand that the point of using the kernel trick is to project the problem onto a higher dimensional space, where the problem is linearly separable. In this explanation, https://www.quora.com/...
1
vote
1answer
147 views

Unable to understand Kernel Ridge regression

I am trying to read kernel ridge regression from this link But , I am unable to get the intution behind the derivation. Can anyone please help me ?
3
votes
1answer
80 views

Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?

Kernel ridge regression associate a regularization parameter $C$ with weight term ($\beta$): $\text{Minimize}: {KRR}=C\frac{1}{2} \left \|\beta\right\|^{2} + \frac{1}{2}\sum_{i=1}^{\mathcal{N}}\left\|...
2
votes
1answer
607 views
1
vote
0answers
40 views

Implementing SVM with Gaussian Kernel

This is referencing Prof. Andrew Ng's course on machine learning. In the part that details implementing an SVM with the Gaussian kernel, we are supposed to use all the training examples as our ...
1
vote
1answer
126 views

Mercer's Theorem importance

I understood that Mercer's Theorem extends the definition of kernels also for infinite input space. In Machine Learning realm our training set is always finite and hence the input space is always ...
3
votes
1answer
1k views

Does increasing kernel size in a CNN result in higher accuracy on the training set?

In a convolutional neural network, does increasing the size of kernel always result in better training set accuracy? For example, if I use 5x5 kernels in a CNN instead of 3x3 ones, will it always ...
6
votes
1answer
92 views

Is there any conceptual relationship between 'kernel' in SVM and 'kernel' in convolution neural net?

In SVM, we have kernel function that maps an input raw data space into a higher dimensional feature space In CNN, we also have a 'kernel' mask that travels the input raw data space (image as a matrix)...