in sklearn example there is a code

numeric_features = ["age", "fare"]
numeric_transformer = Pipeline(
    steps=[("imputer", SimpleImputer(strategy="median")), ("scaler", StandardScaler())]

categorical_features = ["embarked", "sex", "pclass"]
categorical_transformer = Pipeline(
        ("encoder", OneHotEncoder(handle_unknown="ignore")),
        ("selector", SelectPercentile(chi2, percentile=50)),
preprocessor = ColumnTransformer(
        ("num", numeric_transformer, numeric_features),
        ("cat", categorical_transformer, categorical_features),

Later on the preprocessor is used in Pipeline before putting data into LogisticRegression model (as one can see in the link mentioned at the beginning).
In categorical_transformer they use OneHotEncoder and then SelectPercentile. Imagine a situation: p_class is encoded, for example, into p_class_1, p_class_2 and p_class_3, p_class_4 and let's say that SelectPercentile removes p_class_2, p_class_3 and p_class_4. We are left only with p_class_1.
1.Isn't it in a way loss of information that potential model does not see whether an observation is of class 2 or 3 or 4?
2.Could one intepret it as "is p_class_1 or is not p_class_1"?
3.Is such feature selection after OneHotEncoding a good practise?
At this moment, from unexperienced person's point of view, I think that such action can improve learning time, reduce overfitting, but on the other hand the intepretation of the model is a little bit harder.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.