1
$\begingroup$

In my use case of text classification (identify the author from a subset of 10 authors), I find that post all processing with trigrams, there are a 100 thousand and odd features with nearly 50k observations/samples to be trained.

It leaves me baffled to decide on the number of neurons and layers to configure for my MLPClassifier of scikit-learn. If I try a number like 250 neurons with just a single layer, my laptop hangs in about 5 minutes :(

Your advice based on your experience is highly appreciated.

$\endgroup$

1 Answer 1

1
$\begingroup$

As you know, you can basically reduce the number of features by choosing best discriminant ones. If you used scikit-learn vectorizers like TF-IDF, they have the parameter max_features which chooses $n$ best features for you. But my point is something else:

All BoW models which I guess you used are unsupervised. I strongly recommend that you use mutual information between features and targets (you may also search "Supervised TF-IDF" to get more insight) and choose your best features more effectively. At the end you need to choose $n$ neurons as input layer but with this way you may be able to get the most out of less number of features.

If you are familiar with autoencoders, they are also a pretty strong way to reduce the number of features effectively.

Hope it helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.