2
$\begingroup$

I have a prepossessed data set ready and the corresponding labels (8 classes). I've already done KFold cross validation with K=10 with some classifiers such as DT,KNN,NB and SVM and now I want to do a linear regression model, but not sure how it goes with the KFold , is it even possible or for the regression I should just divide the set on my own to a training and testing sets ?

The 8 classes are ages (for NLP problem) , so I want to check both options of classification and regression.

Improve this question
$\endgroup$
2
  • $\begingroup$ if your labels are categorical values (8 classes as you mentioned) you should try logistic regression, not a linear regression. Because its a classification problem, not a regression. $\endgroup$ – tenshi Jul 4 '18 at 10:03
  • $\begingroup$ @tenshi , I agree, that's why I'm also trying classification , but my task's requirements are linear regression and not logistic (following this article : (repository.cmu.edu/cgi/…)) $\endgroup$ – M.F Jul 4 '18 at 10:07
4
$\begingroup$

Cross-validation

An extremely important concept to understand is that:

Cross-validation works independently of the model you use.

Cross-validation is just the process of splitting your data into multiple pairs of training and test sets. Once this is done, you can train that data using whatever model you like. In sklearn, which I assume you are using given your tags, you can use cross-validation with any classifier/regressor you'd like.

Cross-validation is way to evaluate how well the model will perform on data that it hasn't seen before. When you train your model on all your data, you do not have any data that the model hasn't seen before to test on. This is why you cross-validate so that you can get multiple estimates of what the performance might be. This is more robust than just splitting your data once between a training and test set because in this case, it might happen that your test set is easier very difficult or very hard. By cross-validating, you smooth this out.

Linear vs Logistic regression

Apparently, your output is ages, and therefore, you could use linear or logistic regression. If you use linear regression, you will try to guess the age directly. If you use logistic regression, you could round the output to the closest integer to obtain the proper target classes.

I would advise you to use logistic regression because it will allow your model to learn better. If you use linear regression, your classes are all as different as each other (from a mathematical point of view). However, my guess is that the 13 class is closer to 14 than it is to 20+.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, that's actually useful. My main misunderstanding is the concept of averaging the CV result - for example with K=10 and KNN, I take 10 accuracy scores and average them at the end. How this averaging is done with linear regression ? how the model keeps in mind the 10 runs ? $\endgroup$ – M.F Jul 4 '18 at 11:58
  • $\begingroup$ The scores will also be averaged. Cross-validation works the same regardless of the model. Whether you use KNN, linear regression, or some crazy model you just invented, cross-validation will work the same way. $\endgroup$ – Valentin Calomme Jul 4 '18 at 12:00
  • $\begingroup$ Won't I get 10 different LR models (and 10 plots) ? With 10 different mse ? how that's averaging ? ( e.g how do I present the plot if I get 10 different ones) $\endgroup$ – M.F Jul 4 '18 at 12:12
  • $\begingroup$ I updated my answer to add more details. You will indeed get different LR models. That's the point of cross-validation, training multiple models on different training sets so that we can get a good estimate of what the performance might be. Cross-validation is just there to evaluate how well the model will perform. You are only interested in the average performance. $\endgroup$ – Valentin Calomme Jul 4 '18 at 12:18
0
$\begingroup$

Till now you have done classification using DT, KNN, NB and SVM. Your task is of classification as you have 8 output classes. Linear regression is not for classification problems, if you want you can go for Logistic Regression (that too, in the same way with K fold CV as you did for other methods).

Improve this answer
$\endgroup$
6
  • $\begingroup$ As I commented before, for this task I need linear regression , following this article : (repository.cmu.edu/cgi/…) I have 8 datasets ,each containing texts written by teenagers at the ages of 13 to 20, so the regression output can be 16.4 for example . Again, this is for a small research so I do need the regression anyway. $\endgroup$ – M.F Jul 4 '18 at 10:10
  • $\begingroup$ Can you post a part of your output ? $\endgroup$ – Ankit Seth Jul 4 '18 at 10:16
  • $\begingroup$ which output ? of the classification ? it's just accuracy for now... (which is low due too weak features, and that's not important for the moment...) $\endgroup$ – M.F Jul 4 '18 at 10:20
  • $\begingroup$ No, I am asking about the output data, like you said age. $\endgroup$ – Ankit Seth Jul 4 '18 at 10:28
  • $\begingroup$ Not sure I get what you mean - for each text the output should be it's author's age- from the labels (13,14,15,16,17,18,18,20+) $\endgroup$ – M.F Jul 4 '18 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.