I need some feedback on a problem I have been working on. I am working with a fairly balanced dataset with all categorical features, and a categorical outcome (classification problem). The data has no continuous numerical features. To predict my outcome on testset I am using xgboost algorithm. Since I have all categorical predictors I am using one-hot encoding to handle my categorical features. Now I am a bit worried that I might be missing something in the process, so I wanted to check if I have all categorical features with a binary outcome is this a valid approach? I don't see any other way to deal with this problem.

FYI the categorical variables are not things like ZIP codes, IDs...they are actually relevant to the outcome...e.g. smoker (yes/no) | high bp (yes/no)

What do you think?

  • $\begingroup$ What would you do with a continuous response? $\endgroup$
    – Dave
    Jun 13 at 23:56
  • $\begingroup$ yes it wont help, and it doesn't make sense either. I was just thinking something aloud.. :) $\endgroup$ Jun 14 at 0:14
  • $\begingroup$ But what would you do if you had these categorical features and wanted to predict a continuous quantity? $\endgroup$
    – Dave
    Jun 14 at 0:42
  • $\begingroup$ I mean you can still use the same approach, only the nature of the problem changes a bit. From classification to regression. With one caveat that you might have to scale or normalize a few things. The base case logistic regression with no continuous variable comes to mind $\endgroup$ Jun 14 at 0:47

1 Answer 1


I don't see any problem doing classification with purely categorical features, as far as the features are relevant.

And as always, some precautions dealing with categorical features:

  1. The choice of model. Some models can handle categorical features off-the-shelf (e.g. tree-based algorithm), some are specifically designed for (e.g. CatBoost). These models may ease your feature engineering work, and probably better accuracy.

  2. Cardinality. Sometimes a categorical feature can take a lot of values (plus unknown/unseen ones), which can be a problem. You should think ahead about what to do in these cases.

  • $\begingroup$ yes what you said is true. I have been running xgboost, and catboost on my data to see how they perform. Thankfully the cardinality is fairly low in my data - so I got lucky there. Thanks $\endgroup$ Jun 14 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.