1
$\begingroup$

I have a dataset which has categorical variable class. I am trying to solve a regression problem

I am not understanding whether I should build a model on entire dataset and consider variable class as one of the input variable or for each class should I build a different model altogether.

What are general rules which can help me deciding between two approaches.

This is a sample of how my data looks like

+------------+----+-----+-----------------+
|   Class    | X1 | X2  |     Speed       |
+------------+----+-----+-----------------+
| Class1     | 12 | 123 |              10 |
| Class2     | 14 | 120 |              32 |
| Class3     | 15 |  34 |              12 |
|   .        |  . |   . |               . |
|   .        |  . |   . |               . |
|   .        |    |     |                 |
|  Class 300 | 23 |  13 |              45 |
+------------+----+-----+-----------------+

Class is the input categorical variable and I have around 300 classes. The output variable is Speed. I am trying to predict the speed with variables Class, X1, X2.

Should I build a model for each Class separately. So when I know input type is Class1 I will select model built for Class1. When input type is Class2 I will use model built for Class2 and so on. Also the values in Class variable can repeat meaning Class1 can come 4 times, Class2 can come 8 times etc

Other way I was thinking is to include Class itself as a variable and just build one model.

I don't know which would be the correct way for it

$\endgroup$
2
$\begingroup$

Build one model including the class variable as a categorical feature.

Since it is a high cardinality feature, there are different techniques that you can use:

  1. One hot encoding (would create around 300 variables based on different classes)
  2. Label encoding (would be a single variable but will assign weights to different classes - not ideal)
  3. Hashing trick - use a hashing function to reduce the number of features
  4. Embeddings
  5. Cluster the different classes together
| improve this answer | |
$\endgroup$
1
$\begingroup$

It seems to me that building 300 models is too much brute force.

I suggest performing an unsupervised clustering to check if your (x1,x2,speed) are indeed grouped in those 300 classes. Doing so, you may get and idea if the feature Class is important for the regression problem.

In that sense, imagine you discover that this classes can be grouped in 5 new super-classes. Then, building 5 models seems to be a much more practical and correct approach

You can also check if including the Class feature as a categorical feature, works for a single regression model

| improve this answer | |
$\endgroup$
1
$\begingroup$

Sometimes, it's reasonable to build different models for different classes. But as a first approach try to use one model.

If your categorical data are ordinals you can encode them as natural numbers. Otherwise, try one-hot encode, but use threshold, because you have too many classes and you'll have too many dimension.

For example, you can one-hot encode all the classes that have more than 10 samples in them and dismiss all the rest classes. In any case, there's no sense to encode class that have only 1-2 samples.

It's reasonable to use different models for each class if you use different approach. For example, for one class tree-based models may work the best, for the other SVM. You can also build different models, if one model predicts well for high values of some independent variable, and the other for low.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.