Questions tagged [svm]
Support Vector Machines (SVM) are a popular supervised machine learning algorithm that can be used for classification or regression.
580
questions
2
votes
1
answer
324
views
Plotting SVM hyperplane margin
I'm trying to understand how to plot SVM hyperplane and its margins by this example:
https://scikit-learn.org/stable/auto_examples/svm/plot_svm_margin.html
And I got stuck at the plotting the ...
- 21
1
vote
1
answer
75
views
How can I weight each point in one-class SVM?
I want to give weights to some data points
Specifically, these are points related to anomalies
(I'm implementing one-class SVM for anomaly detection)
Exactly, I want to consider some data points that ...
0
votes
2
answers
67
views
SVMs: where comes the 1 and -1 in hyperplane equations and where is b?
Concering Support Vector Machines (SVM):
it is always mentioned that $\textbf{w}^T\textbf{x}_i - b >= 1$, for $\textbf{x}_i$ of class 1 (i.e. $y=1$)
and $\textbf{w}^T\textbf{x}_i - b <= -1$, ...
- 303
1
vote
0
answers
209
views
Does linear kernel make SVM a linear model?
I have deleloped several SVR models for my case study using the linear kernel, and those models were optimized using the RMSE as criterion. Now Im searching for additional evaluation metrics and it ...
- 33
1
vote
0
answers
40
views
One-class SVM formula
Recently I have been studying one-class SVM and am a little bit confused about the offset $\rho$. The common optimization problem is to find a function $f(x)= w^\top x-\rho$ by solving $$\begin{array}...
- 11
1
vote
1
answer
87
views
Gaussian Kernel faster than Linear
I have a Dataset with 580 samples and 7 features. I compared the time between three kernels: Linear, Quadratic and Gaussian and using RandomizedSearchCV as the following:
...
- 123
3
votes
1
answer
183
views
Implementing a weighted support vector machine in python
I have the following problem.
The minimization problem of the SVM that I want to solve is:
$$
\min_{w, b} \frac{1}{2}w^{T}w + \sum^{m}_{i=1}C_{i}xi_{i}
$$
Subject to:
$$
y_{i}(w^{T}x_{i} - b) \geq 1 - ...
- 33
0
votes
0
answers
24
views
what is the effect of initial weights on model training for different algorithms
If I do a model training on a dataset with three different algorithms Logistic Regression (L1), SVM(S1) and a Neural Net(N1). If I train the models again with the same data set and same parameters ...
- 525
1
vote
1
answer
178
views
regression with noisy target vairable
How can I approach a regression problem where the input data is not noisy but the target variable is noisy? Are there any regression algorithms that are robust to a noisy target variable?
Also, is it ...
- 35
2
votes
1
answer
392
views
Should bag of words in training set include test set data when doing text classification?
I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
- 21
0
votes
1
answer
71
views
How the Support Vector Machine will perform if the bias b = 0 in the equation of hyperplane?
We have a soft margin linear SVM and the equation is as follows :
How the SVM will perform if b = 0, means the hyperplane is passing through the origin ?
- 13
0
votes
1
answer
59
views
Non IID variables and SVM Classifier
I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
- 101
2
votes
1
answer
81
views
Linear Learning Machines
I was reading about Linear Learning Machines (LLMs) and learned that it is closely related with SVMs. Would like to know an example of any concrete problems that can be classified by LLM as I couldn't ...
- 123
0
votes
1
answer
35
views
What if Training and testing dataset comes from the same source?
I am working on a classification problem in which I have to distinguish between healthy and damaged plates. when I use the combination of k-means clustering and SVM algorithm together with 10-fold ...
0
votes
1
answer
39
views
Get the prediction probability using prediction function
I'm new to SVM models. I took custom SVM classifier from the github. In there standard predict function was overwritten by custom predict function.
...
- 1
1
vote
0
answers
144
views
Interpreting one-class SVM
I am new to SVM (one-class) and was practically investigating it. Got some weird result that I can not explain. Let me demonstrate by some small reproducible code and visualization:
...
- 261
2
votes
2
answers
428
views
Is it acceptable to use label encoding for nominal categorical data when one hot encoding would create too many features?
I'm working on a short data science project to compare the accuracy of different classification methods. The groups decided to use and compare Random Forest, Naive Bayes and SVM.
The dataset we are ...
- 21
1
vote
1
answer
29
views
Support Vector Classifier and cost C
Applying SVC (Support Vector Classifier) to the 1-d data shown here:
What will be the support vectors for the parameter cost C=0 and C=Infinity?
As far as I read about SVM and hyper parameter C I ...
- 111
1
vote
1
answer
60
views
Why is the optimal C chosen by GridSearchCV so small?
I'm trying to use GridSearchCV to select the optimal C value in this simple SVM problem with non-separable samples. The issue I'm having is that when I run the code the optimal C is chosen to be ...
- 13
1
vote
1
answer
49
views
Need help understanding Hard SVM quadratic program equation
This is from the textbook "Understanding Machine Learning" by Shalev-Schwarz p. 169.
Can anyone help me understand why the solutions to this optimization problem need to be divided by the ...
- 113
0
votes
1
answer
31
views
Steps to fit a Machine learning model for prediction of up and down market movement
I have around 5 years of data of an index containing many features on a daily basis. I want to classify whether the index will move up or down the next trading day (up or down movement is determined ...
1
vote
0
answers
29
views
Finding the dual to an optimization problem on an unsupervised dataset [closed]
We consider the unsupervised dataset $x_1,..x_N \in R^d$ and the optimization problem: $$min_w \,\frac{1}{2}{\left\lVert w \right\rVert}^2,$$ subject to constraints:$$\forall_{i=1}^N: \phi(x_i)^Tw\...
- 11
0
votes
0
answers
165
views
Results of quadratic SVM in Matlab are different from the results obtained in Python
I am trying to replicate a quadratic SVM classifier from Matlab to Python, however I am having different results regarding the accuracy. In Matlab the accuracy is 0.8955 meanwhile in Python the ...
0
votes
1
answer
82
views
How important is outcome variable scaling in SVM regression?
Should I scale outcome variable for SVM regression? What is the magnitude of impact of outcome variable scaling in SVM regression?
- 157
-2
votes
3
answers
58
views
How to decrease $R^2$ value and change it to positive value [closed]
I'm working on a data, and use regression , as you see bellow:
from sklearn.svm import SVR
regressor = SVR(kernel = 'linear')
regressor.fit(trainX,trainY)
above ...
- 13
0
votes
1
answer
368
views
1
vote
0
answers
21
views
Machine learning model (neural network or SVM) for unequal feature matrices size
I have feature matrices obtained from visual bags of words model for various dictionary sizes. Example, Nx5, Nx10, …., Nx15000. Where N is the number of samples and 5, 10, …15000 are the visual ...
- 11
3
votes
2
answers
656
views
Preprocessing: StandardScaler() Do we really need mean to be zero?
For instance, many elements used in the objective function of a
learning algorithm (such as the RBF kernel of Support Vector Machines
or the l1 and l2 regularizers of linear models) assume that all
...
- 31
0
votes
1
answer
16
views
1
vote
1
answer
948
views
Bert and SVM classification
I'm trying to understand the concepts in the title and how they fit into the task of binary classification. According to my understanding so far, you can encode text using various feature-extraction ...
- 111
3
votes
2
answers
624
views
Understanding Learning Curves
I would like to clarify my understanding of learning curves with two example plots below. I am experimenting with small data sets here between 500 and 1500 samples to clarify my understanding.
My ...
- 133
2
votes
2
answers
206
views
One-Class classification
I am currently using an one class classification svm and I am trying to boost the classification results by employing more than one svm-occ with varying gamma parameters and combine these decisions ...
user110508
1
vote
1
answer
585
views
Why does using a standard scalar on my tf idf matrix make it perform better?
I have a TF-IDF matrix transformed on a list of tweets from a data set I am using. I have a pipeline where I initiate a StandardScalar and then next have my SVM with a linear kernel and auto gamma as ...
- 11
2
votes
0
answers
61
views
Intuition behind One Class SVM (Scholkopf)
I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space.
Why separation from origin with a hyperplane ...
- 139
1
vote
0
answers
36
views
SVM is margin determined by nearest datapoint or nearest datapoints?
I am studying support vector machines and different resources seem to define the margin differently. Some define the margin as 2 times the distance to the nearest datapoint.
Others define the margin ...
- 141
3
votes
2
answers
3k
views
0
votes
0
answers
1k
views
UserWarning: No contour levels were found within the data range
I am running the exact example give in this SVM example of Scikit learn without any modification.
I get the following warning.
...
2
votes
1
answer
456
views
How to find the initial hyperplane in a Support Vector Machine (SVM)?
As far as I understand Support Vector machines, we are trying to find the optimal hyperplane, out of all hyperplanes that are equidistant from the support vectors. There are an infinite number of ...
- 141
1
vote
1
answer
193
views
Support Vector Machines (SVM) vs Minimum distance between two convex hulls [closed]
Imagine that we have two sets of seperable points, X and Y, in the plane (R^2) that we want to classify. To find the optimal line (hyperplane) that separates these two sets of points we can:
Run a ...
- 141
0
votes
0
answers
46
views
SVM overfitting with consistent validation results
I have some imbalanced (1400 samples of which 250 are +ve) data for a binary classification problem and I am running an SVM grid search optimising for precision. I am trying 3,4,5,6,7,and 8 stratified ...
1
vote
0
answers
72
views
SVM with gradient descent
The constrained optimization problem in SVM is given by
min 1/2 ||w||^2
s.t y(i)(w^T x(i) + b >= 1 for all i
Now converting this to an unconstrained optimization problem gives the lagriangian L as ...
- 11
0
votes
1
answer
33
views
Why multi-class SVM can't detect some classes?
I use PCA and multi-class SVM for classifying 4-class problem in the Python environment. But in results, I see some differences in detection rate (Unweighted Accuracy in this problem). For example:
...
2
votes
0
answers
100
views
Non semi positive definite kernel matrix
What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite?
This is my flow of thought: In kernel methods $w.x$ is ...
- 21
1
vote
1
answer
111
views
SVM - Making sense of distance derivation
I am studying the math behind SVM.
The following question is about a small but important detail during the SVM derivation.
The question
Why the distance between the hyperplane $w*x+b=0$ and data ...
- 21
1
vote
0
answers
508
views
SVM on BERT-Embeddings with very small dataset does not converge
I am trying to reproduce the results from this paper where they use a linear SVM on top of BERT-Embeddings for text-classification. They use the average of the token-embeddings which results in a 768 ...
- 91
1
vote
2
answers
390
views
How much data do you need to build a classifier?
I would like to ask you what a good size of dataset would be for building a classifier.
I know that there are datasets of 1000 obs and datasets of 1m obs. But I also read papers where classifiers were ...
- 295
1
vote
0
answers
276
views
Why Imblearn pipeline gives very different results when used scaler and under sampling method swapped
I am using the Kaggle's credit card fraud detection dataset (https://www.kaggle.com/mlg-ulb/creditcardfraud)
In order to create a balanced datasets I was testing RandomUnderSampler() and NearMiss(). I ...
- 11
1
vote
2
answers
36k
views
ValueError: y should be a 1d array, got an array of shape (285, 30) instead [closed]
I am using this data set below and I am trying to find the support vector machine of the data set. Also
I have my code and error below as well.
http://scikit-learn.org/stable/modules/generated/sklearn....
- 13
0
votes
1
answer
40
views
GridSearchCV Acting Weird
I am using GridSearchCV to find the best combination of parameters for SVM. However, the parameters chosen by GridSeasrchCV do not seem to be the best ones. I tried some parameters randomly and they ...
- 63
1
vote
1
answer
145
views
Is Recursive Feature Elimination finding best features subset?
On a set of 9 features I have applied Recursive Feature Elimination (RFE) algorithm using SVM estimator, following approach from (1). When requesting a subset of size 1 to be found, then RFE returned ...
- 687