0
$\begingroup$

This is my code below, I need to use SMOTENC to balance the dataset, which means I have to use the pipeline from the imblearn library. However, it does not recognize the CountVectorizer features

from imblearn.pipeline import Pipeline 
# from sklearn.pipeline import Pipeline

 vectorizer_params = dict(ngram_range=(1, 2), min_df=200, max_df=0.8)

 categorical_features = ['F1','F2','F3','F4']
 categorical_transformer = OneHotEncoder(handle_unknown="ignore")

 textual_feature = ['F5']
 text_transformer = Pipeline(
    steps=[
        ("squeez", FunctionTransformer(lambda x: x.squeeze())),
        ("vect", CountVectorizer(**vectorizer_params)),
        ("tfidf", TfidfTransformer()),
        ("toarray", FunctionTransformer(lambda x: x.toarray())),
    ]
    )

preprocessor = ColumnTransformer(
    transformers=[
        ("cat", categorical_transformer, categorical_features),
        ("txt", text_transformer, textual_feature),
    ]
)

sgd_log_pipeline = Pipeline(
    [
        ("preprocessor", preprocessor),
        ('smote', SMOTENC(random_state=11,categorical_features=[4,5,6,7])),
        ("clf", SGDClassifier()),
    ]
)
$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.