I have a dataset with a large number of text features, where the target variable has three classes.
I have encoded the features using tf-idf. This has resulted in a dataset with an extremely large number of features.
I have tried performing classification using lasso logistic regression (in sklearn, LogisticRegression(penalty='l1')), which automatically sets some features' coefficients to 0 and thus performs some kind of feature selection for me.
The sklearn implementation of LogisticRegression() performs multiclass classification using 'one-versus-rest'. That is to say, it trains three separate classifiers which predict one particular class against the other two, and then assigns each data point to the class whose predictor gave the highest probability to.
The coefficients of some of the features from my dataset were set to $0$ in all three classifiers. Others, however, are set to non-zero numbers in all three.
Is it correct to think of those with coefficients set to $0$ by all three classifiers as uninformative, in the sense that they add no new information given the presence of all the others?
Does it make sense to compare the magnitude of the non-zero coefficients, across the three classifiers, in order to find out which features are most informative?