1
$\begingroup$

I am having a dataset which all the features are from 0 to 1(real numbers) and the output is 0 or 1(integer numbers). Example:

var1   var2   var3  output
0.01    0.1    0.7       1
0.01    0.1    0.7       1
 0.1    0.2    0.3       0
 0.2    0.4    0.4       0
 0.4    0.1    0.9       1    

Which classification algorithm is recommended when the variables are 'normalized' from 0 to 1? Does SVM or logistic regression 'react' good in these types of data?

I have noticed that most of the people that are using SVM,NN or logistic regression when making feature scaling they use stardadization( (value-mean)/std ) ). Is there a reason not to rescale the values between from 0 to 1?

$\endgroup$
1
$\begingroup$

Re-scaling or any other form of standardization/normalization is very useful when dealing with models that are trained with gradient descent: (SVM,NN, LogReg).

This question explains the effect of normalization on the gradient rather well: https://stats.stackexchange.com/questions/111467/is-it-necessary-to-scale-the-target-value-in-addition-to-scaling-features-for-re

Decision trees for example are invariant to linear transformations, therefore feature scaling, in theory, does not effect the model in any way.

In the case of your set-up, you have a binary classification problem.

I would recommend trying out first a linear classifier on your data. Logistic regressions is a good first choice. If you decide that LogReg is not working as well as you would like it to then you can move on to more complicated models. I would recommend using decision trees coupled with a gradient boosting model.

In problems that are similar to what you described, specifically when there is no inherent structure in the data (as there is in image classification, speech recognition, etc.), gradient boosting models tend to outperform neural networks.

Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.