I have a data set predicting a continuous variable, $Y$. I have $15$ to $20$ potential feature variables most of which are categorical, some of which are ordinal or categorical. These have been converted to numerical values. I have two questions.

  1. Is linear regression suitable in this case?
  2. If the variables do not show linear relationships with $Y$, is linear regression still suitable? Otherwise, which algorithms, hopefully existing in scikit-learn, might work?

1 Answer 1


You can employ the linear regression algorithm even for categorical data. The point is that whether your data is learnable or not. For instance, take a look at your data, and see whether an expert can really find the output by taking a look at the input vector. If it's possible, your task can be learnt using linear regression method.

About linearity, the point is that linear regression can also learn nonlinear mappings. You just have to provide enough higher order polynomials of the current feature space you have which is not an easy task. For instance, you can expand your current feature space by adding the square of each feature to the current feature space. You will observe that it may have better performance than the usual case if your mapping is not linear, but you may still have error. Consequently, you have to supply more polynomial features, but you do not know which to use.

An alternative to linear regression which does not need to add extra features is multi layer neural networks (MLP). You can simply use them which can learn nonlinear mappings. You can take a look at the official page of SKlearn for applying them. Furthermore, you can take a look at here for applying them.

  • $\begingroup$ Very nice answer thanks. Re learnable: do you mean just look at the categories, and the output? There are 20 or so so I do not know how anyone could figure out if the 20 would predict the output. Or am I missing something? $\endgroup$
    – schoon
    May 5, 2019 at 14:39
  • 1
    $\begingroup$ @schoon To show you what I meant I use an example. Suppose your data contains images and you have to say whether they are cats or dogs. Learnable in this features space means that you yourself, as an expert, can recognize the entities though the images may be a little noisy. But if you have wrong labels for your data or if you have a very noisy dataset, the model cannot perform well. $\endgroup$ May 5, 2019 at 14:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.