Questions tagged [kernel]

Kernel functions are a class of functions which transform the original data into a new space in which the classes of the data are easier to separate by a kernel algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Kernel trick derivation: why this simplification is incorrect?

I am trying to derive kernel trick from linear regression, and I have a mistake in the very end, which leads to an expression too simple. Basic linear regression For a basic linear regression (with no ...
0
votes
1answer
39 views

which type of machine learning algorithms perform better at extrapolation (in general)

Assuming that: the problem lies in the field of natural science, i.e. relationships between variables are physics-based and does not change depending on context its a regression based model Would it ...
2
votes
1answer
36 views

Can a CNN have a different number of convolutional layers and kernel and what does it mean?

So if I have $3$ RGB channels, $6$ convolutional layers and $4$ kernels, does this mean that each kernel does a convolution on each channel and so the input for the next convolution will be $3 \times ...
0
votes
1answer
34 views

When should I use 'rbf' and 'polynomial' kernel trick in machine learning algo?

I have a problem about hate-speech classification using support-vector machine algorithm. The task is to identify the sentence that contains 'positive' or 'negative' sentiment. Which is the best ...
2
votes
0answers
39 views

Theoretical differences between KPCA and t-SNE?

I (think I) understand the underlying principles of most dimensionality reduction methods (MDS, IsoMap, t-SNE, Spectral Embedding, Diffusion maps, etc...). Some of the algorithms I use the most are ...
0
votes
0answers
13 views

What is meant by data dependent kernel?

I was reading this research paper Isolation kernel and it's effect on SVM wherein they mention in the paper that data dependent kernels depend directly on the data.Is there a simple explanantion that ...
0
votes
0answers
17 views

Radial Basis Function (Gaussian) kernel question

I have followed the mathematics of the RBF kernel and understand how to show that the input space is mapped into an infinite-dimensional feature space. However, I'm struggling to make sense of the ...
1
vote
0answers
34 views

Does linear kernel make SVM a linear model?

I have deleloped several SVR models for my case study using the linear kernel, and those models were optimized using the RMSE as criterion. Now Im searching for additional evaluation metrics and it ...
0
votes
0answers
12 views

Is it efficient to use kernel trick in primal form of SVM?

I know we can use Kernel trick in the primal form of SVM. So the hypothesis will be - and optimization objective - We can optimize the above equation using gradient descent, but in this equation ...
1
vote
1answer
137 views

Can someone clarify what the linear assumption of PCA is?

0 For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some ...
0
votes
0answers
18 views

How do I extract the kernel matrix for a classifier created using `sklearn.svm.SVC`?

I am currently using the kernels that come with sk-learn support vector machine library. How do I extract the kernel matrix for a classifier created using ...
0
votes
0answers
15 views

Difficulty understanding the dimension differences in kernel PCA

In Kernel PCA, the kernel trick works because we can show that there is an equivalency between eigenvectors of the kernel matrix and eigenvectors of the covariance matrix. I know the math to go from ...
0
votes
0answers
8 views

State-of-the-art methods for out-of-sample-extension

I'm using a kernel based dimensionality reduction algorithms, and interested in extending out-of-sample data points for further analysis. I've been using the Nystrom method for this task, and some ...
2
votes
0answers
45 views

Why are neural networks equivalent to kernel methods?

I read a recent paper by Pedro Domingos, claiming that Every Model Learned by Gradient Descent Is Approximately a Kernel Machine. I wanted to understand the key idea a little better. Why are neural ...
1
vote
0answers
26 views

How to evaluate KDE against histogram?

I am currently testing some approaches for density estimation, and I think the basic approach of histograms may not be the best option to me and KDE is certainly a good alternative to go. While ago I ...
1
vote
0answers
60 views

How do you choose a kernel for a discontinuous function in Gaussian Process Regression? [closed]

I'm doing Gaussian Process Regression and created a series of functions by gluing other functions together on random places. Here's an example: Perhaps this one is to complicated, but all the ...
1
vote
1answer
77 views

Kernel Density in Scikit Learn

I'm trying to understand how does the KernelDensity class in scikit-learn work. Consider the following two cases which build a kernel from two different arrays (a). I'm wondering why the result of ...
0
votes
1answer
26 views

Should kernel size always be a prime number?

Should kernel size always be a prime number? E.g. (3,3) (5,5) (7,7). While tinkering with sklearn.preprocessing.KernelCenterer(), I noticed that I could only get it ...
0
votes
0answers
982 views

Jupyter notebook kernel dies when reading CSV file on M1 Mac

Odd Jupyter notebook problem on M1 mac here. Using these steps I was able to get Jupyter Notebooks to run in the first place. It worked fine executing the following simple code: ...
1
vote
1answer
35 views

what are the main differences between parametric and non-parametric machine learning algorithms?

I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
1
vote
0answers
14 views

In classical kernel regression, is there a task which responds almost exclusively to a single kernel choice?

I'm curious if there is any well-known kernel regression/classification task which can only be "solved" using a specific choice of kernel?
0
votes
0answers
11 views

A question about dual representations, kernels and notations used in Bishop's book

I'm having hard times about kernel functions and dual representations on 'Pattern recognition from and machine learning' by Bishop. Here it is the page I'm trying to understand: Of course setting the ...
2
votes
0answers
21 views

Non semi positive definite kernel matrix

What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite? This is my flow of thought: In kernel methods $w.x$ is ...
4
votes
1answer
386 views

Has anyone succeeded in finding a good Scala/Spark kernel for Jupyter?

The ones I've tried so far Almond: Works very well for just Scala, but you have to import dependencies, and it gets tedious after a while. And unfortunately can't run when using Spark with YARN ...
2
votes
1answer
37 views

KLMS in machine learning

As Least Mean Square is a very popular choice to be used in combination with neural networks topologies, what would be the most common machine learning algorithms (and easily) to combine with Kernel ...
1
vote
1answer
194 views

Why do we use 2D kernel for RGB data?

I have recently started kearning CNN and I coukdnt understand that why are we using a 2D kernel like of shape (3x3) for a RGB data in place of a 3D kernel like of shape (3x3x3)? Are we sharing the ...
1
vote
2answers
63 views

PCA vs.KernelPCA: which one to use for high dimensional data?

I have a dataset which contains a lot of features (>>3). For computational reasons, I would like to apply a dimensionality reduction. At this point I could use different techniques: standard PCA ...
1
vote
1answer
61 views

How to use a RBF kernel to create a "Kernel Space" using the similarity of each pair of point?

I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper. Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of ...
2
votes
0answers
39 views

How to implement SVM from scratch?

I am trying to build a SVM from scrath and I would like to maximize this Lagrarian expression: I know what variables means but I would like to know how this maximization is implemeted. Should I start ...
0
votes
1answer
86 views

Understanding SVM Kernels

Following Andrew Ng's machine learning course, he explains SVM kernels by manually selecting 3 landmarks and defining 3 gaussian function based on them. Then he says that we are actually defining 3 ...
1
vote
1answer
74 views

How to choose a kernel function and a feature mapping function?

Although, after extensive of reading, I know the concepts of support vector machines pretty well by now, I have trouble translating the concept of the kernel function $K$ and the feature mapping ...
1
vote
0answers
81 views

GP derivative in GpyTorch

I am working on a project using GP-regression models to model transition and measurements models in a Kalman Filter. This means I need to be able to sample from the derivative of the original GP model....
1
vote
0answers
19 views

What is the kernel matrix used for in the kernel trick?

I have $n$ linearly inseperable datapoints, $x_1 \dots , x_n$. I use the kernel trick to map and compute the dot product in higher dimensions (without actually mapping / transforming the data). ...
1
vote
0answers
70 views

Generalized quadratic loss learning

I'm studying a binary classification task with an objective function, derived from SVM, defined so: $\vec{\xi}' S \vec{\xi}$ with: $y_i (f(\vec{x}_i)) >= 1 - \xi_i, i=1..l$ and: $\xi_i >=0, i=1....
0
votes
1answer
101 views

Support Vector Machine (SVM) for classification problem based on Earth Mover's Distance (EMD)

I would like to run SVM for my classification problem using the Earth Mover's Distance (EMD) as a distance measurement. As I understood the documentation for Python scikit-learn (https://scikit-learn....
2
votes
2answers
82 views

Many separation line using RBF kernel in SVM

Below is my code, it take a range of a number, creates a new column label that contains either -1 or 1. In case the number is ...
3
votes
1answer
43 views

What is a good method for detecting local minims and maxims?

I'm using kernel density estimation in order to compute probability density function for item occurrence. Using this output, i want to find all the local minims and maxims. I'm interested in different ...
2
votes
0answers
40 views

Kernel engineering, valid kernels, multipliying by constant =0?

I am reading Bishop, Pattern Recognition and Machine Learning. In the chapter about kernels rules are given for constructing kernels from existing valid kernels. The first one being let k(x,y) be a ...
12
votes
1answer
220 views

What are practical differences between kernel k-means and spectral clustering?

I've been lately wondering about kernel k-means and spectral clustering algorithms and their differences. I know that spectral clustering is a more broad term and different settings can affect the ...
6
votes
3answers
244 views

Why spectral clustering results in disjointed cluster?

I'm working on a project where I have to dynamically cluster the position of objects with respect to one coordinate. So I'm essentially dealing with subsequent frames and each frame represents a one-...
2
votes
0answers
84 views

Implementing a Kernel Adaptive Filtering model explained in a paper

In this paper, Stock price prediction using kernel adaptive filtering within a stock market interdependence approach, the authors propose a method for predicting stock prices by combining the ...
0
votes
1answer
57 views

Kernel Trick and Inner Product Preservation

I understand that the point of using the kernel trick is to project the problem onto a higher dimensional space, where the problem is linearly separable. In this explanation, https://www.quora.com/...
2
votes
0answers
22 views

Designing Custom Kernel from my Mathematical model

I derived a mathematical model for a porous system and the final function looks like this , after going through the Mercers Theorem and its condition for a kernel I would love to write an SVM kernel ...
2
votes
1answer
1k views

Keras: Misunderstanding the Conv2D's param "filters"?

From https://keras.io/layers/convolutional/ (Conv2D): ...
3
votes
1answer
146 views

Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?

Kernel ridge regression associate a regularization parameter $C$ with weight term ($\beta$): $\text{Minimize}: {KRR}=C\frac{1}{2} \left \|\beta\right\|^{2} + \frac{1}{2}\sum_{i=1}^{\mathcal{N}}\left\|...
2
votes
2answers
278 views

Kernel PCA and K largest eigenvectors

How can one prove that the optimal kPCA solution $a^*=\{a_1...a_K\}$ are the $k$-largest Eigenvectors of the (centered) kernel matrix $K$? I referred to a lot of resources and couldn't find a proper ...
1
vote
0answers
53 views

Implementing SVM with Gaussian Kernel

This is referencing Prof. Andrew Ng's course on machine learning. In the part that details implementing an SVM with the Gaussian kernel, we are supposed to use all the training examples as our ...
4
votes
1answer
5k views

Does increasing kernel size in a CNN result in higher accuracy on the training set?

In a convolutional neural network, does increasing the size of kernel always result in better training set accuracy? For example, if I use 5x5 kernels in a CNN instead of 3x3 ones, will it always ...
6
votes
1answer
148 views

Is there any conceptual relationship between 'kernel' in SVM and 'kernel' in convolution neural net?

In SVM, we have kernel function that maps an input raw data space into a higher dimensional feature space In CNN, we also have a 'kernel' mask that travels the input raw data space (image as a matrix)...
0
votes
0answers
20 views

Finding the kernel in which the data will have the highest ratio of leading eigenvalue to trace

I have a method which works well with the data where a ratio of a leading eigenvalue or of first few leading eigenvalues to trace is high. Is it possible to find a kernel transformation which is ...