Questions tagged [svm]
Support Vector Machines (SVM) are a popular supervised machine learning algorithm that can be used for classification or regression.
185
questions with no upvoted or accepted answers
6
votes
2
answers
63
views
How sklearn SVM find the initial hyperplane before Optimisation?
The optimization goal of the SVM is to maximize the distance between the positive and negative hyperplanes. But before optimizing, how does sklearn first find the positive and negative support vectors ...
- 261
4
votes
2
answers
1k
views
How can I use two different datasets as a training model for svm
I know that you're supposed to scale your test data using the parameters (mean and stdev) from your training data. This is relatively simple; but what if the number of samples is limited in one ...
- 51
4
votes
2
answers
259
views
Support Vector Regression trained with data sets
I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
- 41
4
votes
1
answer
475
views
Illustrating the dimensionality reduction done by a classification or regression model
Tl;DR: You can predict something, but how do you explain the prediction?
Your usual classification/regression setup
Lets say the data is a classic regression/classification problem: several numerical ...
- 141
3
votes
1
answer
174
views
Support Vector Machines with soft margin: solving the dual form
I am currently struggling with finding an analytical solution for the $\alpha_k$.
I have derived the following constrained optimization problem:
$$
L = \sum_{i=1}^{N} \alpha_i - \frac{1}{2} \sum_{i=1}^...
- 31
3
votes
0
answers
2k
views
why the accuracy of LDA model is always changing and also is high
Let’s explain the whole goal firstly, then go through the question.
I am using topic modeling like LAtent Dirichlet Allocation and NMF to extract the topic from a collection of documents.
My dataset ...
- 321
3
votes
1
answer
2k
views
Which algorithm is used in sklearn SGDClassifier when modified huber loss is used?
The documentation says:
The loss function to be used. Defaults to ‘hinge’, which gives a
linear SVM. The ‘log’ loss gives logistic regression, a probabilistic
classifier. ‘modified_huber’ is ...
- 131
3
votes
0
answers
160
views
How to mitigate the hierarchical error propagation in tree-structured classification
Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$
We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
- 41
3
votes
0
answers
232
views
feature weights in structured support vector machine
I like to find the weight vector for input-space features in a structured SVM. The idea is to identify the most important set of input-space features (based on the magnitude of their corresponding ...
- 31
2
votes
0
answers
22
views
Force positive coefficients for Logistic Regression and LinearSVC
Do you know what is the best way to force positive coefficients with Logistic Regression and Linear SVC using scikit learn?
for instance
...
- 145
2
votes
0
answers
75
views
Feature selection and model performance
Featuretools provides an automated way to generate features from your data, by providing relationships within your data and applying their so-called deep feature synthesis. It generates features like ...
- 121
2
votes
1
answer
43
views
memory bound for kernel tricks in machine learning
Based on Andrew Ng's lecture on Kernel,
you use training samples (referred as landmarks l) and use them during prediction to construct the higher dimensional representation of the given sample.
This ...
- 121
2
votes
0
answers
198
views
random_state on train_test_split() appears to have large effect in performance metrics?
To summarize the problem: I have a data set with ~1450 samples, 19 features and a binary outcome where classes are fairly balanced (0.51 to 0.49).
I split the data into a train set and a test set ...
- 31
2
votes
1
answer
92
views
How do I use wavelet transform for feature extraction correctly?
I'm trying to classify words based on EMG signals using a support vector machine as my model. My dataset includes 15 classes (words) with 230 repetitions and 1000 features each. I already merged all ...
- 21
2
votes
1
answer
417
views
Input shape of dataset in hybrid CNN-SVM classifier
I am working on hybrid CNN-SVM for classification task, where I aim to use CNN for feature extraction and SVM for classification.
So after the training of my CNN model as below:
...
- 125
2
votes
0
answers
363
views
Why is the leave-one-out error for support vector machines equal to the number of support vectors divided by the number of training examples?
Elementary question about support vector machines.
Given a support vector machine classifier and a linearly separable dataset. Why is the leave-one-out cross validation error said to be bounded by the ...
- 21
2
votes
1
answer
392
views
Should bag of words in training set include test set data when doing text classification?
I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
- 21
2
votes
0
answers
61
views
Intuition behind One Class SVM (Scholkopf)
I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space.
Why separation from origin with a hyperplane ...
- 139
2
votes
0
answers
100
views
Non semi positive definite kernel matrix
What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite?
This is my flow of thought: In kernel methods $w.x$ is ...
- 21
2
votes
1
answer
67
views
How to implement SVM from scratch?
I am trying to build a SVM from scrath and I would like to maximize this Lagrarian expression:
I know what variables means but I would like to know how this maximization is implemeted. Should I start ...
2
votes
0
answers
15
views
Connection between prob output LogisticReg/SVM and ROC
I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
- 31
2
votes
1
answer
291
views
How can I imporve accuracy for text classification and mapping using SVM?
I am working on a problem where I need to predict the text corresponding to another text in my training data file. For example: if I have value like the software in one of my columns and another ...
user79322
2
votes
0
answers
261
views
Why are the regions/decision boundaries overlapping with multi-class classification using SVM in sci-kit?
I am using the SVM in scikit-learn library for doing multiclass classification. I am wondering why these regions (decision boundaries) are overlapping (as seen in the picture below)?
Could someone ...
- 31
2
votes
1
answer
438
views
Naive Bayes and Support Vector Machine (NBSVM) Classification
I am relatively new to datascience and have a question about NBSVM. I have a two class problem and text data (headlines from the newspaper). I want to use NBSVM to predict whether a headline has the ...
2
votes
1
answer
58
views
Version of Perceptron
If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
- 51
2
votes
0
answers
15
views
Confusion regarding prediction results of SVM and ANN on feature vectors
I am making a custom image classifier using Transfer Learning on Inception V3. I have 3 classes of images with ~6K images each. The input dimension of the network is 500X500 and the output of the ...
2
votes
0
answers
385
views
Sequence classification using oneClass SVM
In the code below, I'm using a sequence to sequence approach as a prediction model for anomaly detection. The data set I'm working with is ADFA-LD. The training phase is done using only normal ...
- 604
2
votes
1
answer
193
views
Dataset where svm performance is significantly different from random forest
Is there a specific dataset where svm performs significantly better or worse than random forest?
I know that the performance could depend on the dataset but is there a specific dataset?
2
votes
0
answers
137
views
Structured Support Vector Machine (Joint Feature Map)
I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine)
The theory's clear, but I need a tangible example to make everything more concrete.
...
2
votes
0
answers
35
views
I have data of some movies and their subtitles.I want to classify them based on their ratings
I will convert the subtitles into vectors and use them as features to classify the movies into different categories based on their ratings.The problem that I am facing is my feature vector is much ...
- 107
2
votes
0
answers
39
views
Solution of quadratic optimization in support vector machines
In support vector machines, the minimization problem with inequality constraints can be converted to a minimization problem of Lagrange multipliers with equality constraints by KKT condition and ...
- 227
2
votes
0
answers
136
views
Non-linear Support Vector Regression issue - Sklearn Python 3.6
I am fairly new to Sklearn and machine learning and have encountered an issue when using SVR with an RBF kernel. Below is my predicted data compared directly with my real data:
I do not know what I ...
- 75
2
votes
2
answers
1k
views
How can I find anomalies in each row of data?
I have some reported data I want to spot anomalies on. The columns are a facility name then monthly reports of that given facility.
...
- 143
2
votes
0
answers
24
views
When should the bias b be updated with weights w and when should it be updated seperately?
It seems in some Machine Learning models, the bias term $b$ is updated just like other weights $w_i, i=1...n$. For example, in Logistic Regression, using SGD, $b \ \text{or} \ w_0$ is updated with:
$$...
- 352
2
votes
0
answers
172
views
How to create a global model with personalized features for multi-label classification problem
I'm trying to predict additional recipients of a message given the content of the message (like subject and body) and the current recipients of the message.
for ex: I have 4 users in the system U1, ...
- 133
2
votes
0
answers
101
views
What are some machine learning problems that can be attacked with continuous multiobjective optimization?
I am working on continuous vector optimization, and hence continuous multiobjective optimization is a particular case. I am interested in finding applications in machine learning for this problems. Is ...
- 123
2
votes
0
answers
80
views
How to select samples for a trainings set
My dataset contains half a million unlabeled entries with over 100 binary features. A third of these features are present in less than 1000 samples.
I want to classify a few samples by hand (into ...
- 21
2
votes
0
answers
1k
views
fitting classifier object of type 'int' has no len()
We have LDA topic modeling whose purpose is to generate a number of topics given a set of documents. So each document can belong to various topics.
Also, we can evaluate the model we have created. one ...
- 321
2
votes
0
answers
9k
views
Found array with dim 3. Estimator expected <= 2
I am using LDA over a simple collection of documents. My goal is to extract topics, then use the extracted topics as features to evaluate my model.
I decided to use multinomial SVM as the evaluator.
...
- 321
2
votes
0
answers
188
views
Non-linear transformations input dataset for support vector machines
I have two classes (A,B) that I would like to classify using a SVM. Say that I have a class C and a function f. Can I do this:
...
- 121
2
votes
2
answers
132
views
Many separation line using RBF kernel in SVM
Below is my code, it take a range of a number, creates a new column label that contains either -1 or 1.
In case the number is ...
- 585
1
vote
0
answers
23
views
SVM, Is the slack value always equal to the alpha value for points within the margin?
I'm preparing for an exam and I got stuck on this question.
I understand that the alpha values 'affects' how much influence corresponding data point has on the position of the decision boundary, and ...
- 11
1
vote
0
answers
27
views
What is ROC curve based on for SVM?
I was studying about the ROC curves for Logistic regression. There is a threshold in this method that determines the classification. By changing this threshold we get different confusion matrices and ...
- 111
1
vote
0
answers
10
views
Boosting the effect of some of the features in SVM
I'm doing text classification with SVM. I'm using Tfidf vectorization. In addition to the text vectors, I have a context data denoting the possible outcomes of the prediction. For example, I have a ...
- 121
1
vote
1
answer
31
views
Can we train of a binary classifier with "A" to classify "a"?
I have a maybe naive question about the appropriateness of using binary classifications. This is a hypothetical example, so forgive me if it is too coarse.
Let's say I want to train a support vector ...
- 11
1
vote
0
answers
7
views
Precision and AUROC for which class values
I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms.
Can anyone ...
- 359
1
vote
0
answers
15
views
Strong bias from Linear SVR meta model
I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% ...
- 121
1
vote
0
answers
98
views
Why would a Linear SVR model greatly outperform a Linear Regression model on model stacking
I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% ...
- 121
1
vote
0
answers
304
views
Should I train multiple models or single model with respective data for multiple parameters?
In one of my projects I have 7 output classes. However, while this makes up a baseline set of results, I also want to test the impact of some experimental parameters.
To avoid any information ...
- 111
1
vote
0
answers
22
views
How to estimate a best fitting line that separates variables on a scatterplot by some third binary outcome variable with 95% accuracy?
0
I've been thinking about this and haven't found a non-brute force method of doing it. I have a series of scatterplots depicting the relationship between two predictor variables and a third binary ...
- 27