Classification engine with multiple Textual independent fields

I have made a classification engine with only one independent field (Comments) and classified them in multiple dependent variables. Now I want to have multiple independent variables in training data (comments,state,age,gender etc).

Currently i am using python 3.6.

Example of what I have done:

Comments                                Classified
(The car wash service is good)          Positive


Example of what I want to do:

 Comments                          Gender    City   Age       Classified
(The car wash service is good)    Male      LA     40        Positive


As you can see in the second example we have 4 independent variables affecting the outcome(Classified sentiment). I want to implement a classification engine based on this use case. Please suggest how should i move ahead...???

Edit:- I have used SGDClassifier from sklearn.linear_model.

I have first passed list of comments as vectors which i have vectorized using TfidfVectorizer. Below is the code of vectorization:-

from sklearn.feature_extraction.text import TfidfVectorizer

vectorizer = TfidfVectorizer(min_df=5,
max_df = 0.8,
sublinear_tf=True,
use_idf=True)



then I pass these vectors to train along with their classified labels. Below is the code(I have used partial_fit as i want to train again and again):-

classifier = SGDClassifier.partial_fit(train_vectors, Train_Labels)


Then I use the the classifier to classify rest of the vectors.

test_vectors = vectorizer.transform(Test_data_comments_list)
prediction_liblinear = classifier.predict(test_vectors)


If I combine all the fields value in my second Example and then form a single vector from that and then pass those vectors for classification, would that be the ideal way of doing it?

• @Sean PLease suggest. I have added examples to clarify my question. – gulshan raj Sep 26 '17 at 9:43
• I think it is important to know what ML library (or algorithm if you have built it from scratch in Python) you have used for the first classifier. Probably that library can be extended to support the additional features, but there might be limitations. For instance, if you are using something pre-trained or made only to work with natural language, it might be harder to extend than if you had built and trained your own classifier on a "bag of words" model and a standard classifier library. – Neil Slater Sep 26 '17 at 19:32
• @NeilSlater I have edited my question along with the details of the way i am doing this classification. Please suggest – gulshan raj Sep 28 '17 at 6:00
• The question is clear now to me, I think it can be answered easily, and I have voted to reopen it. By the way I would consider what you have done as built and trained your own classifier on a "bag of words" model and a standard classifier library, as TfidfVectorizer creates a slightly more sophisticated version of "bag of words" where the words are scored according to frequency. – Neil Slater Sep 28 '17 at 7:06

If I combine all the fields value in my second Example and then form a single vector from that and then pass those vectors for classification, would that be the ideal way of doing it?

In short, yes. You can convert each input variable into a vectorised feature separately, then concatenate those vectors into one long vector of features per example. Most statistical machine learning models don't care (or "understand") that the feature data has been formed in different ways, it is just treated numerically.

You do need to pay attention to each input data type in order to decide how best to convert to a usable feature. There is no single "best" way to do this, and this is partly an art - you add the "science" part of Data Science by testing your ideas and taking measurements, e.g. accuracy.

These are some quick thoughts on how you might prepare the data into features from each column:

• Comments. You have already converted this, you are using a variant of "bag of words" to convert text into a fixed length numerical vector. This is a common approach and hard to beat. If you have a lot of data you can look into more sophisticated models that take account of word order, such as Recurrent Neural Networks, but that's a whole new subject area for now.

• Gender. You can convert this into a simple vector, using one hot encoding - this will have two or more columns.

• City. You could one hot encode this also. It might also be worth grouping cities and having a smaller feature vector. E.g. group by state/country before one hot encoding.

• Age. Either group and one hot encode, or scale to a smaller number, by dividing by e.g. 50 (this helps algorithms like SGD by keeping distance metrics similar between different feature types). I suspect grouping in typical demographic splits (e.g. 0-11, 12-17, 18-24, 25-34 etc) and one hot encoding would work well for sentiment analysis, because that would capture some generalised differences between uses of text expression.