in sklearn example there is a code

numeric_features = ["age", "fare"]
numeric_transformer = Pipeline(
    steps=[("imputer", SimpleImputer(strategy="median")), ("scaler", StandardScaler())]

categorical_features = ["embarked", "sex", "pclass"]
categorical_transformer = Pipeline(
        ("encoder", OneHotEncoder(handle_unknown="ignore")),
        ("selector", SelectPercentile(chi2, percentile=50)),
preprocessor = ColumnTransformer(
        ("num", numeric_transformer, numeric_features),
        ("cat", categorical_transformer, categorical_features),

Later on the preprocessor is used in Pipeline before putting data into LogisticRegression model (as one can see in the link mentioned at the beginning).
In categorical_transformer they use OneHotEncoder and then SelectPercentile. Imagine a situation: p_class is encoded, for example, into p_class_1, p_class_2 and p_class_3, p_class_4 and let's say that SelectPercentile removes p_class_2, p_class_3 and p_class_4. We are left only with p_class_1.
1.Isn't it in a way loss of information that potential model does not see whether an observation is of class 2 or 3 or 4?
2.Could one intepret it as "is p_class_1 or is not p_class_1"?
3.Is such feature selection after OneHotEncoding a good practise?
At this moment, from unexperienced person's point of view, I think that such action can improve learning time, reduce overfitting, but on the other hand the intepretation of the model is a little bit harder.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.