Questions tagged [kernel]
Kernel functions are a class of functions which transform the original data into a new space in which the classes of the data are easier to separate by a kernel algorithm.
93
questions
0
votes
0
answers
16
views
LeaveOneOut CV for Bandwidth selection of Kernel Density Estimation
I've taken this code in order to try optimization of bandwidth_selection with GridSearchCV (while implementing LeaveOneOut logics within this CV: "LeaveOneOut() is equivalent to KFold(n_splits=n)&...
- 133
2
votes
1
answer
29
views
how is the feature mapping for the kernel trick "found"?
I did not find detailed explanations for this on wikipedia or other sites.
When I have a dataset that is not linearly separable and apply the "kernel trick" - how do I know if a mapping to ...
- 23
1
vote
1
answer
35
views
Unexpected behaviour of Scikit-Learn SVR
I'm using Scikit-learn to fit a support vector regression on a really simple dataset of car stopping distances vs car speed.
My code for applying SVR to this dataset is:
...
- 113
0
votes
0
answers
6
views
Literature suggestions: Guarantees for GPs with subset of data
I am currently trying to solve the following problem:
Given a fixed set of query points (or if possible even better: a region where I want to obtain accurate predictions), approximate the GP such that ...
- 1
0
votes
0
answers
10
views
Remove feature from kernel matrix
I am pretty new to machine learning so please bear with me :)
I am trying to do a binary classification task using an SVM with precomuted kernel (in python using sklearn).
I created my train kernel ...
- 1
0
votes
0
answers
13
views
What is normalized winning frequency in kernel self organizing map(SOM)?
In the k-means based kernel SOM, proposed by MacDonald and Fyfe (2000), the update of the mean is based on a soft learning algorithm
...
0
votes
1
answer
161
views
Using Maximum Mean Discrepancy (MMD) to Compare Kernel Density Estimates (KDEs)
I'm interested in comparing two Kernel Density Estimates (KDEs) and I've come across the Maximum Mean Discrepancy (MMD) metric ...
- 103
1
vote
2
answers
188
views
Kernel Kmeans formula
I'm trying to implement the Kernel Kmeans algorithm but I struggle with the following formula :
Let's say I have a case in one dimension with three points : 1, 5, 9. Let's say I want two clusters. ...
- 23
0
votes
1
answer
75
views
Is Maximum Mean Discrepancy (MMD) suitable for comparing distributions with different sample sizes?
I'm working on a project where I need to compare the similarity of two probability distributions using MMD. However, the two datasets have different sample sizes. I've read that MMD can be affected by ...
- 103
1
vote
1
answer
96
views
Kernel Kmeans implementation
I'm currently trying to implement the Kernel Kmeans from scratch. At the time I'm writing this post, my implementation is perfectly working on nested circles dataset or even on Iris dataset (see ...
- 23
0
votes
0
answers
14
views
Multiple kernel SVM is equal to one ANN - Is Kernel SVM better that one ANN?
I'm comparing multiple Kernel SVM with one neural network, e.g one ANN with one hidden layer.
I have succesfully trained a neural network by using multiple Kernel ...
- 115
0
votes
0
answers
13
views
SVM kernel for detecting if a substring appears in some given string
I'm trying to do the exercise in 16.1 in the book Understanding Machine Learning, Ben-David, et al. formulated as follows:
Consider the task of learning to find a sequence of characters
("...
- 155
0
votes
1
answer
170
views
Why linear kernel regression is equivalent to plain linear regression?
I am trying to understand either intuitively/geometricaly and/or mathematicaly why the followings are equivalent:
Classic Ordinay Least Squares linear regression
Linear-kernelized Ordinary Least ...
- 101
0
votes
2
answers
147
views
Why don't we increase the parameter from 64 to 128 in this CNN model?
I'm looking at an example lab from a coursera course titled Intro to Tensorflow. In this CNN model, they're gradually increasing the no. of filters from 16 to 32 and then 64. Why don't we increase it ...
0
votes
1
answer
261
views
Epanechnikov kernel smoothing and Priestley-Chao (PC) kernel estimate
I wrote the Python code below to try to automate the application of kernel smoothing using the Epanechnikov kernel with a bandwidth of h = 0.4 calculating the Priestley-Chao kernel estimate of the ...
- 1
1
vote
0
answers
36
views
A mathematician from the outside looking in
I am wondering if anybody could give a survey of applications of approximation theory to data science. One application I am familiar with are, for example, wavelet neural networks. Does anybody know ...
- 111
0
votes
1
answer
75
views
Kernel ridge regression (KRR), accuracy scale?
What does a good range for the accuracy score look like for the KRR model?
For example, RMSE produces a value between 0 and 1, where values closer to 0 represent better fitting models. What's the ...
- 103
2
votes
1
answer
97
views
memory bound for kernel tricks in machine learning
Based on Andrew Ng's lecture on Kernel,
you use training samples (referred as landmarks l) and use them during prediction to construct the higher dimensional representation of the given sample.
This ...
- 121
2
votes
0
answers
22
views
why layer of dimension 1 is outputting image of size n
I am studying a model where landmarks from an image are calculated. The work comes from Convolutional Experts Constrained Local Model for 3D Facial Landmark Detection.
I need to confirm why the ...
- 21
2
votes
1
answer
369
views
What is custom SVM kernel?
What is custom kernel in the Support Vector Machine. How is it different from Polynomial kernel. How to implement a custom kernel. Can you provide a code to implement a custom kernel.
- 21
1
vote
1
answer
286
views
What are the benefits of using spectral k-means over simple k-means?
I have understood why k-means can get stuck in local minima.
Now, I am curious to know how the spectral k-means helps to avoid this local minima problem.
According to this paper A tutorial on ...
- 133
1
vote
0
answers
17
views
Bayesian Linear Regression using the Kernel Trick vs Constructing features using Kernels as Prototypes
How different is it to do Bayesian linear regression using the GP approach (kernel trick) versus constructing features using kernels to prototypes? As far as I know, this very basic question is ...
- 23
2
votes
2
answers
201
views
Kernel trick derivation: why this simplification is incorrect?
I am trying to derive kernel trick from linear regression, and I have a mistake in the very end, which leads to an expression too simple.
Basic linear regression
For a basic linear regression (with no ...
- 327
1
vote
1
answer
98
views
which type of machine learning algorithms perform better at extrapolation (in general)
Assuming that:
the problem lies in the field of natural science, i.e. relationships between variables are physics-based and does not change depending on context
its a regression based model
Would it ...
- 43
1
vote
1
answer
544
views
Can a CNN have a different number of convolutional layers and kernel and what does it mean?
So if I have $3$ RGB channels, $6$ convolutional layers and $4$ kernels, does this mean that each kernel does a convolution on each channel and so the input for the next convolution will be $3 \times ...
- 13
0
votes
1
answer
639
views
When should I use 'rbf' and 'polynomial' kernel trick in machine learning algo?
I have a problem about hate-speech classification using support-vector machine algorithm.
The task is to identify the sentence that contains 'positive' or 'negative' sentiment.
Which is the best ...
- 3
5
votes
1
answer
516
views
Theoretical differences between KPCA and t-SNE?
I (think I) understand the underlying principles of most dimensionality reduction methods (MDS, IsoMap, t-SNE, Spectral Embedding, Diffusion maps, etc...).
Some of the algorithms I use the most are ...
- 153
0
votes
0
answers
201
views
Radial Basis Function (Gaussian) kernel question
I have followed the mathematics of the RBF kernel and understand how to show that the input space is mapped into an infinite-dimensional feature space. However, I'm struggling to make sense of the ...
- 13
2
votes
0
answers
544
views
Does linear kernel make SVM a linear model?
I have deleloped several SVR models for my case study using the linear kernel, and those models were optimized using the RMSE as criterion. Now Im searching for additional evaluation metrics and it ...
- 43
1
vote
1
answer
1k
views
Can someone clarify what the linear assumption of PCA is?
0
For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some ...
- 11
3
votes
1
answer
624
views
Why are neural networks equivalent to kernel methods?
I read a recent paper by Pedro Domingos, claiming that Every Model Learned by Gradient Descent Is Approximately a Kernel Machine.
I wanted to understand the key idea a little better. Why are neural ...
- 31
1
vote
0
answers
111
views
How to evaluate KDE against histogram?
I am currently testing some approaches for density estimation, and I think the basic approach of histograms may not be the best option to me and KDE is certainly a good alternative to go. While ago I ...
- 300
2
votes
0
answers
223
views
How do you choose a kernel for a discontinuous function in Gaussian Process Regression? [closed]
I'm doing Gaussian Process Regression and created a series of functions by gluing other functions together on random places. Here's an example:
Perhaps this one is to complicated, but all the ...
- 131
1
vote
1
answer
562
views
Kernel Density in Scikit Learn
I'm trying to understand how does the KernelDensity class in scikit-learn work. Consider the following two cases which build a kernel from two different arrays (a). I'm wondering why the result of ...
- 229
0
votes
1
answer
120
views
Should kernel size always be a prime number?
Should kernel size always be a prime number? E.g. (3,3) (5,5) (7,7).
While tinkering with sklearn.preprocessing.KernelCenterer(), I noticed that I could only get it ...
- 539
2
votes
1
answer
121
views
what are the main differences between parametric and non-parametric machine learning algorithms?
I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
- 21
1
vote
0
answers
16
views
In classical kernel regression, is there a task which responds almost exclusively to a single kernel choice?
I'm curious if there is any well-known kernel regression/classification task which can only be "solved" using a specific choice of kernel?
2
votes
0
answers
154
views
Non semi positive definite kernel matrix
What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite?
This is my flow of thought: In kernel methods $w.x$ is ...
- 21
4
votes
1
answer
1k
views
Has anyone succeeded in finding a good Scala/Spark kernel for Jupyter?
The ones I've tried so far
Almond: Works very well for just Scala, but you have to import dependencies, and it gets tedious after a while. And unfortunately can't run when using Spark with YARN ...
- 141
2
votes
1
answer
141
views
KLMS in machine learning
As Least Mean Square is a very popular choice to be used in combination with neural networks topologies, what would be the most common machine learning algorithms (and easily) to combine with Kernel ...
2
votes
1
answer
1k
views
Why do we use 2D kernel for RGB data?
I have recently started kearning CNN and I coukdnt understand that why are we using a 2D kernel like of shape (3x3) for a RGB data in place of a 3D kernel like of shape (3x3x3)?
Are we sharing the ...
- 267
1
vote
2
answers
174
views
PCA vs.KernelPCA: which one to use for high dimensional data?
I have a dataset which contains a lot of features (>>3). For computational reasons, I would like to apply a dimensionality reduction. At this point I could use different techniques:
standard PCA
...
- 147
1
vote
1
answer
109
views
How to use a RBF kernel to create a "Kernel Space" using the similarity of each pair of point?
I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper.
Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of ...
- 33
2
votes
1
answer
160
views
How to implement SVM from scratch?
I am trying to build a SVM from scrath and I would like to maximize this Lagrarian expression:
I know what variables means but I would like to know how this maximization is implemeted. Should I start ...
0
votes
1
answer
234
views
Understanding SVM Kernels
Following Andrew Ng's machine learning course, he explains SVM kernels by manually selecting 3 landmarks and defining 3 gaussian function based on them. Then he says that we are actually defining 3 ...
- 155
1
vote
1
answer
278
views
How to choose a kernel function and a feature mapping function?
Although, after extensive of reading, I know the concepts of support vector machines pretty well by now, I have trouble translating the concept of the kernel function $K$ and the feature mapping ...
- 11
1
vote
0
answers
231
views
GP derivative in GpyTorch
I am working on a project using GP-regression models to model transition and measurements models in a Kalman Filter. This means I need to be able to sample from the derivative of the original GP model....
- 21
1
vote
0
answers
31
views
What is the kernel matrix used for in the kernel trick?
I have $n$ linearly inseperable datapoints, $x_1 \dots , x_n$. I use the kernel trick to map and compute the dot product in higher dimensions (without actually mapping / transforming the data).
...
- 21
0
votes
1
answer
301
views
Support Vector Machine (SVM) for classification problem based on Earth Mover's Distance (EMD)
I would like to run SVM for my classification problem using the Earth Mover's Distance (EMD) as a distance measurement. As I understood the documentation for Python scikit-learn (https://scikit-learn....
2
votes
2
answers
146
views
Many separation line using RBF kernel in SVM
Below is my code, it take a range of a number, creates a new column label that contains either -1 or 1.
In case the number is ...
- 605