Classification engine with multiple Textual independent fields

Question

I have made a classification engine with only one independent field (Comments) and classified them in multiple dependent variables. Now I want to have multiple independent variables in training data (comments,state,age,gender etc).

Currently i am using python 3.6.

---

Example of what I have done:

Comments                                Classified
(The car wash service is good)          Positive

Example of what I want to do:

 Comments                          Gender    City   Age       Classified
 (The car wash service is good)    Male      LA     40        Positive

As you can see in the second example we have 4 independent variables affecting the outcome(Classified sentiment). I want to implement a classification engine based on this use case. Please suggest how should i move ahead...???

Edit:- I have used SGDClassifier from sklearn.linear_model.

I have first passed list of comments as vectors which i have vectorized using TfidfVectorizer. Below is the code of vectorization:-

from sklearn.feature_extraction.text import TfidfVectorizer

vectorizer = TfidfVectorizer(min_df=5,
                                 max_df = 0.8,
                                 sublinear_tf=True,
                                 use_idf=True)

train_vectors = vectorizer.transform(Train_data_Comments_list)

then I pass these vectors to train along with their classified labels. Below is the code(I have used partial_fit as i want to train again and again):-

classifier = SGDClassifier.partial_fit(train_vectors, Train_Labels)

Then I use the the classifier to classify rest of the vectors.

test_vectors = vectorizer.transform(Test_data_comments_list)
prediction_liblinear = classifier.predict(test_vectors)

If I combine all the fields value in my second Example and then form a single vector from that and then pass those vectors for classification, would that be the ideal way of doing it?

Answer 1

If I combine all the fields value in my second Example and then form a single vector from that and then pass those vectors for classification, would that be the ideal way of doing it?

In short, yes. You can convert each input variable into a vectorised feature separately, then concatenate those vectors into one long vector of features per example. Most statistical machine learning models don't care (or "understand") that the feature data has been formed in different ways, it is just treated numerically.

You do need to pay attention to each input data type in order to decide how best to convert to a usable feature. There is no single "best" way to do this, and this is partly an art - you add the "science" part of Data Science by testing your ideas and taking measurements, e.g. accuracy.

These are some quick thoughts on how you might prepare the data into features from each column:

• Comments. You have already converted this, you are using a variant of "bag of words" to convert text into a fixed length numerical vector. This is a common approach and hard to beat. If you have a lot of data you can look into more sophisticated models that take account of word order, such as Recurrent Neural Networks, but that's a whole new subject area for now.

• Gender. You can convert this into a simple vector, using one hot encoding - this will have two or more columns.

• City. You could one hot encode this also. It might also be worth grouping cities and having a smaller feature vector. E.g. group by state/country before one hot encoding.

• Age. Either group and one hot encode, or scale to a smaller number, by dividing by e.g. 50 (this helps algorithms like SGD by keeping distance metrics similar between different feature types). I suspect grouping in typical demographic splits (e.g. 0-11, 12-17, 18-24, 25-34 etc) and one hot encoding would work well for sentiment analysis, because that would capture some generalised differences between uses of text expression.