in sklearn example there is a code
numeric_features = ["age", "fare"] numeric_transformer = Pipeline( steps=[("imputer", SimpleImputer(strategy="median")), ("scaler", StandardScaler())] ) categorical_features = ["embarked", "sex", "pclass"] categorical_transformer = Pipeline( steps=[ ("encoder", OneHotEncoder(handle_unknown="ignore")), ("selector", SelectPercentile(chi2, percentile=50)), ] ) preprocessor = ColumnTransformer( transformers=[ ("num", numeric_transformer, numeric_features), ("cat", categorical_transformer, categorical_features), ] )
Later on the preprocessor is used in Pipeline before putting data into LogisticRegression model (as one can see in the link mentioned at the beginning).
In categorical_transformer they use OneHotEncoder and then SelectPercentile. Imagine a situation: p_class is encoded, for example, into p_class_1, p_class_2 and p_class_3, p_class_4 and let's say that SelectPercentile removes p_class_2, p_class_3 and p_class_4. We are left only with p_class_1.
1.Isn't it in a way loss of information that potential model does not see whether an observation is of class 2 or 3 or 4?
2.Could one intepret it as "is p_class_1 or is not p_class_1"?
3.Is such feature selection after OneHotEncoding a good practise?
At this moment, from unexperienced person's point of view, I think that such action can improve learning time, reduce overfitting, but on the other hand the intepretation of the model is a little bit harder.