1
$\begingroup$

I am analysing a bunch of tweets and I want to understand which political party the authors support.

I am using Mathematica but I am thinking to re-write my code in Python. If you have any suggestion, do not esitate.

Let say, for example that I have five parties labelled by a number: Party1, Party2,...,Party5 and I provide a list of tweets which are for sure associated to the parties. This list is roughly trainingList = {tweet1-> "Party1",tweet2->"Party2",...,tweet5->"Party5",tweetK->"Neutral"}. Moreover, I added a class "Neutral" because some tweets are not classifiable. Actually, I provide more than one tweet par party. In fact, the trainingList is much longer.

Notice that I remove stop-words from the tweets and I apply a stemming-words algorithm to simplify the expressions. Then, every tweets is a list of potentially-significant words.

I create the classifier function, providing a prior (the explicit for of the prior is not important)

c = Classify[trainingList, ClassPriors-> prior]

Then, I want to do some checks. I take a bunch of tweets belonging to other party-supporters and try 

c[{word1,word2,word3,word4,word5,word7},"Probabilities"]

which gives me a list of probabilities $p_i$. For example, 

<| 'Party1' -> p1, 'Party2' -> p2, 'Party3' -> p3, 'Party4' -> p4, 'Party5' -> p5, 'Neutral' -> p6 |>

Question1 Is there a way to understand how the algorithm associate these probabilites? More specifically, how each word is associated to a party

Question2 Is there a way to see which are the most frequent n-grams of words associated to a given party (class)?

Improve this question
$\endgroup$
3
  • $\begingroup$ The classification method specifies how features (tokens or words) are associated with class (party) probability. Alas, I'm not familiar with NLP methods in Mathematica. Do you happens to know what method Classify uses? Naive Bayes would be a logical candidate. $\endgroup$
    – S van Balen
    Commented Feb 27, 2018 at 9:37
  • $\begingroup$ @SvanBalen I can actually choose this method. You can have a look at the list of methods here reference.wolfram.com/language/ref/Classify.html $\endgroup$
    – apt45
    Commented Feb 27, 2018 at 9:38
  • $\begingroup$ So the direct answer to your 1st question is: your tokens are treated as features which the algorithm of choice learns to associate to the target class on basis of a corpus of provided examples. The strengths of these associations are represented as probabilities, sooner or later (N.B sooner, DecisionTree later fi). The exact way they do this vary with the method. As to your 2nd question: Your classifiers also tokenize the text to features, in this step you should be able to specify ngrams, feature importance will show you the correlation. $\endgroup$
    – S van Balen
    Commented Feb 27, 2018 at 11:07

3 Answers 3

Reset to default
1
$\begingroup$

I think you are asking, for an arbitrary trained classifier, how do you find its mapping between tweet-words and party-probabilities? There are a couple of model-agnostic ways to do this, amounting to checking which features make the biggest difference. The following two Python libraries implement one or more of these algorithms, starting from a paper written the LIME authors:

  1. LIME shows which words matter for a given classification. You could send it single-word texts to get their average importance, but of course in better text models, context will matter. enter image description here

  2. ELI5 includes an alternate LIME implementation, and also several special functions for some common scikit-learn classifiers.

enter image description here

@CarlRynegardh also suggests a general method, given a (word x latent-feature) matrix that can be extracted from many classifiers, but given you seemed unfamiliar with that approach, I think you will get more value from exploring these libraries. Also, I think they will better account for context effects, if you want to consider more than single words, and at least ELI5 takes advantage of some specific model features.

For getting started doing this in Python, you're likely to get a lot of opinions. I think most would recommend the anaconda python distribution and the SciPy ecosystem including NumPy, matplotlib, and pandas. For general machine learning my go-to package is scikit-learn. For deep networks, the two main competitors seem to be Keras/TensorFlow and FastAI/PyTorch. The second is more Pythonic. (I believe it also currently has the edge in classifier performance, but not as much support for deploying in production. Both are still evolving quickly.)

Coming from Mathematica, I expect you'll want to work in the jupyter notebook environment. Both LIME and ELI5 have several example notebooks, and GitHub has a built-in notebook viewer, so you can read them in your browser when following the links above.

The fastest way to start may be to use a cloud service that provides instant access to a jupyter notebook pre-configured with all the libraries. Plenty of options from clouds big and small. I'm using Paperspace, but I hear Google Collab is free and Just Works (TM).

Improve this answer
$\endgroup$
0
$\begingroup$

Take a few words you know are linked with e.g republicans and with democrats. Extract their word embedding. That is the vector representation of the word which will be high dimensional. This will assume you have trained word embedding, either pretrained, or even better trained them yourself on your data. The word embeddings could have been trained/extracted through e.g LSTM training or matrix factorisations.

On the word embeddings, do dimension reduction(PCA as an example) to 2d. Plot! Those who are close together will be "similar" for the objective they are trained for.

Pretrained word embeddings exist to download. However, they might not be the optimal embeddings for your objective, but better than nothing still.

Improve this answer
$\endgroup$
3
  • $\begingroup$ What do you mean by training of word embedding? $\endgroup$
    – apt45
    Commented Feb 27, 2018 at 9:54
  • $\begingroup$ In order to classify, train each word needs to be transformed into a vector with numerical values. This vector representation can be trained in order to make more sense than if you for example would use a "one-hot" encoding for a word. en.wikipedia.org/wiki/Word2vec $\endgroup$
    – Carl Rynegardh
    Commented Feb 27, 2018 at 10:01
  • $\begingroup$ Ps. This is the solution. $\endgroup$
    – Carl Rynegardh
    Commented Mar 3, 2018 at 15:59
0
$\begingroup$

I believe that "Sequence Classification using LSTM RNN" is the exact answer to your problem.

Check out this tutorial, it's one of the best:

https://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.