Best Python NLP library for supervised topic classification

Question

I have a labeled dataset that I have ingested into a dataframe. It consists of news articles,

>>> df.columns

Index(['title', 'headline', 'byline', 'dateline', 'text', 'copyright',
       'country', 'industry', 'topic', 'file'],
      dtype='object')

where the text column contains the body (text) of the article and the topic column contains a list of associated topics.

I want to train a model from this dataset to predict the article topics. I was considering using transformers (https://huggingface.co/transformers/index.html) to do this, along with tensorflow, but I from what I know of transformers, it's not really good for this.

What would be the best NLP library to perform this task with high accuracy?

Answer 1

My team created a PyPi package called Happy Transformer. Happy Transformer is built on top of Hugging Face's Transformers library to provide a simple interface to implement Transformer models. I suggest you take a look at the text classification section.

https://happytransformer.com/

https://github.com/EricFillion/happy-transformer

Answer 2

The recommendation is going to be still Huggingface transformers. You extract the features from BERT, then, some dense layers and then, feed them into sigmoid layer with unit equal to number of classes. Pose it as a multi-label classification.

Answer 3

I suggest hugging face is a good try although I have little experience using it. If it ultimately does not work well, you can also try NLTK, gensim and SpaCy. They are all widely used in NLP.

Here is a demo notebook that I found: https://www.kaggle.com/thebrownviking20/topic-modelling-with-spacy-and-scikit-learn

Hope it helps and best luck to your exploration!