2
$\begingroup$

I am working on a dataset from Kaggle (housing price prediction). I have done some pre-processing on the data (missing values, category aggregation, selecting ordinal vs one-hot). I am trying to implement a pipeline to streamline the code. The pipeline consists of a ColumnTransformer with two components: one component contains a standard scaler applied to numerical and ordinal features; the second component has a one-hot encoder for the remaining set of features. I am passing this transformer to a GridSearchCV object to tune hyperparameters. In this case, it is a LASSO model. So, I am trying to tune the coefficient of the penalty term. The problem is some of the one-hot encoded features are highly skewed with the count in mostly one category. When GridSearchCV tries to run cross-validation, it raises an error saying that unknown categories are found while validating the model. I think this happens because while fitting the one-hot encoder the train set doesn't contain data points with specific labels that show up in the validation set. One obvious way to handle this would be to fit a one-hot encoder, keep it aside and then build a pipeline and carry on with the grid search (/validation) steps. This seems a bit disconnected to me considering the notion of pipeline was defined for exactly this purpose. Maybe I am missing something here. Is there a better (/efficient) way to achieve the above rather than separating the one-hot encoder from the pipeline?

enter image description here

For reference, the data for the above histogram,

  • Category Counts
  • CompShg 1434
  • Tar&Grv 11
  • Tar&Grv 11
  • WdShngl 6
  • WdShake 5
  • Roll 1
  • ClyTile 1
  • Metal 1
  • Membran 1
$\endgroup$

1 Answer 1

1
$\begingroup$

You can use handle_unknown='ignore' in the OneHotEncoder; levels present in the test set but not the train set will be encoded as all-zeros rather than raising an error.

But... that example you provide, I rather doubt it's worth it to keep all the levels. The coefficient learned for such a small level's dummy variable will be overly specialized (overfit). Consider alternatives.

$\endgroup$
3
  • $\begingroup$ The rationale behind 'ignore' for test data makes sense because that's something for which literally, we have no information. Sure, you may be right that coefficient for a rare type (eg. Metal) may result in overfitting, but that would be taken care of by the regularization term. I mean that's the whole point of using a LASSO model here. I guess my question is geared more towards the design of API. You can surely come up with examples similar to above where the counts are relatively less skewed and depending on the size of validation set you end up in a similar situation. $\endgroup$ Commented Mar 19, 2020 at 17:16
  • $\begingroup$ A continuation of my previous comment. But, even in such a case, I guess the probability of this happened would be very low considering the validation set (and the train set) will be randomly sampled. I wonder if the developers already thought of this edge case while designed the one-hot encoder. $\endgroup$ Commented Mar 19, 2020 at 17:27
  • 1
    $\begingroup$ I explored my problem a bit more and realized that although LASSO is ideally a feature selection technique, due to the skewed counts it becomes much harder to even train a good LASSO model. So, to even reach a viable model for feature selection, it seems like a good idea to reduce the number of categories. Aggregating the rare categories into a new group (called 'other') seems like a good idea. $\endgroup$ Commented Mar 21, 2020 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.