0
$\begingroup$

I'm trying to train a classifier to classify text from a chat between 2 users so later on I can predict who of the two users is more likely to say X sentence/word. To get there I mined the text from the chat log and ended up with two arrays of words, UserA_words and UserB_words.

Which classifier should I use fot this purpose and what structure the training data should have? I've researched for the bag of words structure but dont know exactly how to train a classifier with data in that format.

To clarify this last point, for now I have the data in a dict like {"hello":34, "how":12} and so on, being the terms word:frequency of each user. As far as I know, there is no way to use this two dicts as a classifier fit input. So, how do I transform this 2 dicts into an array that I can use to train a classifier (let's say I want to use a gaussian Naive Bayes just for the sake of the example)

Improve this question
$\endgroup$
12
  • $\begingroup$ Train 2 word2vec style language models -- one for each users. And... that is it, more or less. You can ask the language models the probabilities of each saying the word/sequence of words. (The only trick is to keep the language models, rather than just the word embedding) $\endgroup$
    – Frames Catherine White
    Oct 24, 2016 at 13:10
  • $\begingroup$ @Oxinabox I don't fully understand what do you mean with "keep the language models", I'm kinda new to all this, could you explain? $\endgroup$
    – xgrimau
    Oct 24, 2016 at 13:29
  • $\begingroup$ Are you familiar with a language model? (Or with word2vec?) If not, perhaps go and do a bit more reading, or maybe ask another question "What is a language model, and how does word2vec differ from a traditional language model". $\endgroup$
    – Frames Catherine White
    Oct 24, 2016 at 14:23
  • $\begingroup$ @Oxinabox I'm not, will do my research on the term as you said. Thank you for your insights! $\endgroup$
    – xgrimau
    Oct 24, 2016 at 14:29
  • 1
    $\begingroup$ @Oxinabox Actually I have a fair amount of it, 50k-60k words per user. I guess it should be enough. $\endgroup$
    – xgrimau
    Oct 24, 2016 at 14:39

1 Answer 1

Reset to default
0
$\begingroup$

You're asking what ML representation you should use for user-classification of chat text. User-classification is not the usual text-processing task.

It's not strictly necessary to semantically understand what the user is saying, only how they're saying it; so we look for telltale features indicative of a specific user. And we don't necessarily need to use, or solely rely on, the usual text-processing representations like bag-of-words, word-counts, TFIDF and word-vector.

Here are some features which are predictive of the user:

  • character length, word length, sentence length of each comment
  • typing speed (esp. if you have timestamps in seconds)
  • ratio of punctuation (e.g. 17 punctuation symbols in 80 chars = 17/80)
  • ratio of capitalization
  • ratio of numerals
  • ratio of whitespace
  • character n-grams (and notice these can pick up e.g. l0ser, f##k, :-) )
  • use of Unicode (emojis, symbols e.g. stars)
  • ratio of specific punctuation (e.g. how many '.', '!', '?', '*', '#' )
  • word-counts, esp. anything statistically anomalous, foreign, slang, insults
  • anything else you can think of that seems predictive for these two users, e.g. number of misspelled words per sentence (may be actual typos, or come from predictive swiping on a cellphone)
Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for your answer, but by now I'm more focused on trying if its possible to extract some sort of conclusions ouf of the word frequency feature. (Want to point out that all your insights have been pretty interesting un order to understand what kind of data would be really useful in my case) $\endgroup$
    – xgrimau
    Oct 26, 2016 at 21:12
  • $\begingroup$ I'm offering you my experience that the other stuff can be more predictive, and is certainly very easy to compute and try out. If you insist on using bag-of-words, you're throwing away lots of features. $\endgroup$
    – smci
    Oct 26, 2016 at 21:50
  • $\begingroup$ Decided to follow your advice and extract the "word length, sentence length and ratio of whitespaces (Just to keep it simple by now). So given this new scenario, you would pass something like this, to the algorithm?: array = [[8,15,9],[3,24,6],...] for both of the users? Or It has to be grouped in a single array including the features from the two of them? Thank you in advance $\endgroup$
    – xgrimau
    Oct 26, 2016 at 22:15
  • $\begingroup$ When you say "pass to the algorithm", it doesn't need to be a vector; they are entirely separate features, so separate columns. Personally I'd suggest using a tree (random-forest or xgboost), but you could try logistic regression, or anything else. If you insist on using Naive Bayes, and insist on passing it a vector, you're really just arbitrarily picking constraints and reducing optimality. As to "does it have to be grouped in a single array including the features", yes, creating one big Y matrix will make your life saner. $\endgroup$
    – smci
    Oct 27, 2016 at 1:18
  • $\begingroup$ Will do some research on the random-forest aproach. With the data I mentioned above, Thank you for your patience. $\endgroup$
    – xgrimau
    Oct 27, 2016 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.