1
$\begingroup$

in sklearn example there is a code

numeric_features = ["age", "fare"]
numeric_transformer = Pipeline(
    steps=[("imputer", SimpleImputer(strategy="median")), ("scaler", StandardScaler())]
)

categorical_features = ["embarked", "sex", "pclass"]
categorical_transformer = Pipeline(
    steps=[
        ("encoder", OneHotEncoder(handle_unknown="ignore")),
        ("selector", SelectPercentile(chi2, percentile=50)),
    ]
)
preprocessor = ColumnTransformer(
    transformers=[
        ("num", numeric_transformer, numeric_features),
        ("cat", categorical_transformer, categorical_features),
    ]
)

Later on the preprocessor is used in Pipeline before putting data into LogisticRegression model (as one can see in the link mentioned at the beginning).
In categorical_transformer they use OneHotEncoder and then SelectPercentile. Imagine a situation: p_class is encoded, for example, into p_class_1, p_class_2 and p_class_3, p_class_4 and let's say that SelectPercentile removes p_class_2, p_class_3 and p_class_4. We are left only with p_class_1.
1.Isn't it in a way loss of information that potential model does not see whether an observation is of class 2 or 3 or 4?
2.Could one intepret it as "is p_class_1 or is not p_class_1"?
3.Is such feature selection after OneHotEncoding a good practise?
At this moment, from unexperienced person's point of view, I think that such action can improve learning time, reduce overfitting, but on the other hand the intepretation of the model is a little bit harder.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.