# Questions tagged [svm]

Support Vector Machines (SVM) are a popular supervised machine learning algorithm that can be used for classification or regression.

160 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
251 views

### How sklearn SVM find the initial hyperplane before Optimisation?

The optimization goal of the SVM is to maximize the distance between the positive and negative hyperplanes. But before optimizing, how does sklearn first find the positive and negative support vectors ...
• 261
506 views

### Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? EDIT: I have built a website that tries to answer this question with means of embedding / visually clustering data according ...
• 151
271 views

### Support Vector Regression trained with data sets

I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
• 41
2k views

### Which algorithm is used in sklearn SGDClassifier when modified huber loss is used?

The documentation says: The loss function to be used. Defaults to ‘hinge’, which gives a linear SVM. The ‘log’ loss gives logistic regression, a probabilistic classifier. ‘modified_huber’ is ...
• 141
2k views

### why the accuracy of LDA model is always changing and also is high

Let’s explain the whole goal firstly, then go through the question. I am using topic modeling like LAtent Dirichlet Allocation and NMF to extract the topic from a collection of documents. My dataset ...
• 331
185 views

### How to mitigate the hierarchical error propagation in tree-structured classification

Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$ We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
• 41
247 views

### feature weights in structured support vector machine

I like to find the weight vector for input-space features in a structured SVM. The idea is to identify the most important set of input-space features (based on the magnitude of their corresponding ...
• 31
29 views

### Is There any Open Source Implementation of Large-Scale SVM?

Question: Is anyone aware of a publicly accessible python package for large scale SVMs? Thanks! Why the Question Ought to be Answerable: As has been noted (e.g., here), the SVM problem can be ...
81 views

### Feature selection and model performance

Featuretools provides an automated way to generate features from your data, by providing relationships within your data and applying their so-called deep feature synthesis. It generates features like ...
• 121
94 views

### memory bound for kernel tricks in machine learning

Based on Andrew Ng's lecture on Kernel, you use training samples (referred as landmarks l) and use them during prediction to construct the higher dimensional representation of the given sample. This ...
• 121
513 views

### random_state on train_test_split() appears to have large effect in performance metrics?

To summarize the problem: I have a data set with ~1450 samples, 19 features and a binary outcome where classes are fairly balanced (0.51 to 0.49). I split the data into a train set and a test set ...
• 31
289 views

### How do I use wavelet transform for feature extraction correctly?

I'm trying to classify words based on EMG signals using a support vector machine as my model. My dataset includes 15 classes (words) with 230 repetitions and 1000 features each. I already merged all ...
• 21
723 views

### Input shape of dataset in hybrid CNN-SVM classifier

I am working on hybrid CNN-SVM for classification task, where I aim to use CNN for feature extraction and SVM for classification. So after the training of my CNN model as below: ...
• 135
414 views

### Why is the leave-one-out error for support vector machines equal to the number of support vectors divided by the number of training examples?

Elementary question about support vector machines. Given a support vector machine classifier and a linearly separable dataset. Why is the leave-one-out cross validation error said to be bounded by the ...
• 21
510 views

### Does linear kernel make SVM a linear model?

I have deleloped several SVR models for my case study using the linear kernel, and those models were optimized using the RMSE as criterion. Now Im searching for additional evaluation metrics and it ...
• 43
71 views

### Intuition behind One Class SVM (Scholkopf)

I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space. Why separation from origin with a hyperplane ...
• 149
151 views

### Non semi positive definite kernel matrix

What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite? This is my flow of thought: In kernel methods $w.x$ is ...
156 views

### How to implement SVM from scratch?

I am trying to build a SVM from scrath and I would like to maximize this Lagrarian expression: I know what variables means but I would like to know how this maximization is implemeted. Should I start ...
17 views

### Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
364 views

### Why are the regions/decision boundaries overlapping with multi-class classification using SVM in sci-kit?

I am using the SVM in scikit-learn library for doing multiclass classification. I am wondering why these regions (decision boundaries) are overlapping (as seen in the picture below)? Could someone ...
• 31
473 views

### Naive Bayes and Support Vector Machine (NBSVM) Classification

I am relatively new to datascience and have a question about NBSVM. I have a two class problem and text data (headlines from the newspaper). I want to use NBSVM to predict whether a headline has the ...
70 views

### Version of Perceptron

If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
• 51
17 views

### Confusion regarding prediction results of SVM and ANN on feature vectors

I am making a custom image classifier using Transfer Learning on Inception V3. I have 3 classes of images with ~6K images each. The input dimension of the network is 500X500 and the output of the ...
419 views

### Sequence classification using oneClass SVM

In the code below, I'm using a sequence to sequence approach as a prediction model for anomaly detection. The data set I'm working with is ADFA-LD. The training phase is done using only normal ...
• 634
149 views

### Structured Support Vector Machine (Joint Feature Map)

I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine) The theory's clear, but I need a tangible example to make everything more concrete. ...
40 views

### I have data of some movies and their subtitles.I want to classify them based on their ratings

I will convert the subtitles into vectors and use them as features to classify the movies into different categories based on their ratings.The problem that I am facing is my feature vector is much ...
41 views

### Solution of quadratic optimization in support vector machines

In support vector machines, the minimization problem with inequality constraints can be converted to a minimization problem of Lagrange multipliers with equality constraints by KKT condition and ...
• 237
143 views

### Non-linear Support Vector Regression issue - Sklearn Python 3.6

I am fairly new to Sklearn and machine learning and have encountered an issue when using SVR with an RBF kernel. Below is my predicted data compared directly with my real data: I do not know what I ...
• 75
1k views

### How can I find anomalies in each row of data?

I have some reported data I want to spot anomalies on. The columns are a facility name then monthly reports of that given facility. ...
32 views

### When should the bias b be updated with weights w and when should it be updated seperately?

It seems in some Machine Learning models, the bias term $b$ is updated just like other weights $w_i, i=1...n$. For example, in Logistic Regression, using SGD, $b \ \text{or} \ w_0$ is updated with: ...
178 views

### How to create a global model with personalized features for multi-label classification problem

I'm trying to predict additional recipients of a message given the content of the message (like subject and body) and the current recipients of the message. for ex: I have 4 users in the system U1, ...
• 133
102 views

### What are some machine learning problems that can be attacked with continuous multiobjective optimization?

I am working on continuous vector optimization, and hence continuous multiobjective optimization is a particular case. I am interested in finding applications in machine learning for this problems. Is ...
• 123
91 views

### How to select samples for a trainings set

My dataset contains half a million unlabeled entries with over 100 binary features. A third of these features are present in less than 1000 samples. I want to classify a few samples by hand (into ...
• 21
1k views

### fitting classifier object of type 'int' has no len()

We have LDA topic modeling whose purpose is to generate a number of topics given a set of documents. So each document can belong to various topics. Also, we can evaluate the model we have created. one ...
• 331
10k views

### Found array with dim 3. Estimator expected <= 2

I am using LDA over a simple collection of documents. My goal is to extract topics, then use the extracted topics as features to evaluate my model. I decided to use multinomial SVM as the evaluator. ...
• 331
190 views

### Non-linear transformations input dataset for support vector machines

I have two classes (A,B) that I would like to classify using a SVM. Say that I have a class C and a function f. Can I do this: ...
• 121
146 views

### Many separation line using RBF kernel in SVM

Below is my code, it take a range of a number, creates a new column label that contains either -1 or 1. In case the number is ...
• 605
1 vote
32 views

### How does ROC work with SVM?

Could someone please explain how ROC works with SVM? Specifically i'm using RocCurveDisplay.from_predictions(y_test, y_pred, ax=ax[1]) which works fine. Since the ...
1 vote
29 views

### SVM, Is the slack value always equal to the alpha value for points within the margin?

I'm preparing for an exam and I got stuck on this question. I understand that the alpha values 'affects' how much influence corresponding data point has on the position of the decision boundary, and ...
• 11
1 vote
140 views

### What is ROC curve based on for SVM?

I was studying about the ROC curves for Logistic regression. There is a threshold in this method that determines the classification. By changing this threshold we get different confusion matrices and ...
• 111
1 vote
12 views

### Boosting the effect of some of the features in SVM

I'm doing text classification with SVM. I'm using Tfidf vectorization. In addition to the text vectors, I have a context data denoting the possible outcomes of the prediction. For example, I have a ...
• 121
1 vote
34 views

### Can we train of a binary classifier with "A" to classify "a"?

I have a maybe naive question about the appropriateness of using binary classifications. This is a hypothetical example, so forgive me if it is too coarse. Let's say I want to train a support vector ...
• 11
1 vote
12 views

### Precision and AUROC for which class values

I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms. Can anyone ...
• 361
1 vote
22 views

### Strong bias from Linear SVR meta model

I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% ...
• 121
1 vote
187 views

### Why would a Linear SVR model greatly outperform a Linear Regression model on model stacking

I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% ...
• 121
1 vote
449 views

### Should I train multiple models or single model with respective data for multiple parameters?

In one of my projects I have 7 output classes. However, while this makes up a baseline set of results, I also want to test the impact of some experimental parameters. To avoid any information ...
• 113
1 vote
23 views

### How to estimate a best fitting line that separates variables on a scatterplot by some third binary outcome variable with 95% accuracy?

0 I've been thinking about this and haven't found a non-brute force method of doing it. I have a series of scatterplots depicting the relationship between two predictor variables and a third binary ...
• 27
1 vote
22 views

### SVM Marginal line query

In SVM, why are marginal lines drawn such that they touch some data points (support vectors)? Isn't the classification more accurate when no point is touched?
• 317
1 vote