I would like to perform a multiclass-multioutput classification task, on vectorized textual data. I started by using a random forest classifier in a multioutput startegy:
forest = RandomForestClassifier(random_state=1) multi_target_forest = MultiOutputClassifier(forest, n_jobs=-1) multi_target_forest.fit(X_train, y_train) y_pred_test = multi_target_forest.predict(X_test)
When looking on the feature importance for the individual estimators (multi_target_forest.estimators_ ) I've noticed that some features in my dataset are very relevant and useful for some tasks, but are disrupting for another class. Example:
Task 1: classify documents for Date (q1, q2, q3, q4) Task 2: classify document for Version (preliminary, final, amendment)
For task 1, features related to dates, such as 'April', are very useful. However, for the second task, the feature 'April' gets a high importance but is a consequence of overfitting to a small dataset. Knowing this I would like to actively remove such features.
Is there a way to control which features are used for every task?
I could just explicitly train separate classifiers for every task, but is that equivalent to multioutput-multiclass? or is there some joined probability calculation going on, that I'll be missing?