I am working on a multiclass text classification problem. I want to use the top k features based on mutual information (
mutual_info_classif) for training my model.
I started this project on ML models:
tfidf for feature extraction and then used
mutual_info_classif for feature selection.
svc = Pipeline([('tfidf', TfidfVectorizer()), ('ig', SelectKBest(mutual_info_classif,k=1000)), ('clf',LinearSVC(multi_class='ovr')), ]) svc.fit(X_train, y_train) y_pred = svc.predict(X_test)
This was pretty straight-forward.
Next, I started to work on RNN (more specifically a simple LSTM model)and that is where I am having a problem.
I have used pre-trained word embeddings from GloVe (300d) to get features from my data. The embedding matrix I feed into the embedding layer of my RNN has the shape (4293,300), 4293 is the number of unique words found in my data and 300 is the dimension.
My questions are: Is there any way that I can use the top 1000 words (features) out of these 4293 based on
mutual_info_classif? Is it even possible to do so? If yes, then should it be done before making the embedding_matrix or after?