2
$\begingroup$

I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve a similarity problem. Using GridSearchCV, I am finding the best hyper parameters using the scoring as "R2". The best hyperparameters are C=1, cache_size=200, coef0=0.0, degree=3, epsilon=0.001, gamma=0.005, kernel='rbf', shrinking=True, tol=0.001.

I fit the training data and training label as model.fit(X_train, Y_train)

Now I use test data in the same model as: prediction = model.predict(X_test)

The reason I am using SVM-Regression is to find similarity between two inputs. However, for some of the test data, the prediction contains value in negative (less than 0) and for all the same vs same comparison it returns the value as 1.09469178. I am expecting the value to be between 0 and 1. Is this normal or am I doing something wrong?

$\endgroup$
4
$\begingroup$

I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve similarity problem.

Wait a second, if you're using support-vector regression, then you're not using logistic regression. These two are very different algorithms. They aren't even applicable to the same type of problem. Support-vector regression is used when you are predicting a continuous target, whereas logistic regression (despite the name) is a classification algorithm.

However, for some of the test data, the prediction contains value in negative (less than 0) and for all the same vs same comparison it returns the value as 1.09469178.

There's nothing unusual about this if you're using support-vector regression. A support-vector machine can (in theory) output any real number.

Logistic regression, on the other hand, is a sigmoid function. It will take as input any real number and output a result between 0 and 1. Maybe you meant to use scikit's LogisticRegression model rather than SVR?

The reason I am using SVM-Regression is to find similarity between two inputs.

Could you expound on your use-case a bit more? Support-vector regression isn't really meant to be used to compute similarity.

I'm speculating that you have a training set of (X, y) pairs (where y is a label between 0 and 1). You are training a model to output $\hat{y}$, a prediction of y. To find the similarity of two inputs, $X_i$ and $X_j$, you pass them both through the model and measure the difference in the $\hat{y}_i$ and $\hat{y}_j$. Is that right?

If so, I think this is a really roundabout way of computing similarity, and is unlikely to yield better results that a more straightforward method. I'm not sure that you need a machine learning algorithm at all.

Have you looked into other similarity metrics that might be suitable for your problem? For similarity based on Euclidean distance, you can compute $\frac{1}{1 + d(X_i, X_j)}$ (where $d$ is the euclidean distance function). Cosine similarity might be a good choice if you care more about the similarity in direction of two input vectors.

$\endgroup$
  • 1
    $\begingroup$ Well done answer: both corrects the misunderstanding (LogR vs SVR) and answers the (somewhat fuzzy) intent behind the question. $\endgroup$ – javadba Jun 18 at 17:40
  • $\begingroup$ @Zachdj Thank you for your answer. I am extracting a similarity vector for a pair of inputs and the label is just 0 for different input and 1 for same input. I am not providing any label in-between. You could think of some kind of sentence similarity. The problem is not the similarity but the final similarity value going beyond the range(0, 1). $\endgroup$ – ErAcube Jun 19 at 13:43
0
$\begingroup$

This is normal: unless your training data covers the population very well, the test set is bound to contain instances which slightly deviate from the cases seen in the training data. With any regression method, this might cause predicted values to go slightly out of range. If the application requires normalized values, these deviations should be programmatically corrected post-process (i.e. anything negative changed to 0 and anything higher than 1 changed to 1).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.