Text classification analysis based on similarity

Question

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the models and the steps involved.

I have multiple datasets, each of them on a particular topic. These datasets include news and fake news, manually flagged at the moment. I have collected texts from different sources about similar topics (using keywords) and now I would like to try to build a model that can allow me to classify a news as real or fake automatically.

I have thought that it could have been useful to study the frequency of words and punctuation, and also similarity, trying to group similar texts based on same conditions (for example, similarity between texts as in plagiarism). I have used similarity (Jaccard or cosine) for comparing texts rather than words, but I do not know if this is the right approach and how I should create a model based on that.

Probably the best method would be to build a logistic regression model using a binary variable, but I also read a lot of naive bayesian modes and nn models.

Do you have any example on how I should apply text similarity for classification or further relevant information about that?

Answer 1

I shall give you some hints here.

It depends on your data, my advice is that you try several algorithms.

If you have binary classification you can use both classification (try Logistic regression, SVM, RandomForest) or regression (try XGBoost or any gradient boosting algorithm, there are several in scikit learn), and then seek for a boudary in order to decide wheter a value is 0 or 1. Depending on your data, it might work very well (I've got 0.8 fscore on text classification using RandomForest and performing an important pre-processing before). Do not forget to do an appropriate preprocessing. Concerning pre-processing, in scikit-learn you have CountVectorizer and TfIdfVectorizer, very useful.

Do not use Naive Bayes, not on text and NLP.

Well, the best method for that is using Sequence models (especially LSTM), parsing text as an succession of words or characters. You can use in this case Sequences available in Keras.