1
$\begingroup$

I am doing feature engineering right now for my classification task. In my dataframe I have a column with text messages. I decided to create a binary feature which depends on whether or not in this text were words "call", "phone", "mobile", "@gmail", "mail" "facebook". But now I wonder should I create separate binary features for each word (or group of words) or one for all of them. How to check which solution is better. Is there any metric and how usually people do in practice. Thanks)

$\endgroup$

2 Answers 2

0
$\begingroup$

You should be creating binary features for each group of words. So if you have n groups you should create n-1 features. If you just create one feature it will have 1s for all rows where any word is found and 0 otherwise which will not make sense.

$\endgroup$
0
$\begingroup$

The most common way to describe what you are doing is one-hot encoding ngrams. The goal is to create the features for a classification algorithm to learn. Thus, create as many separate features that could possibly be useful. This is often limited by how often an ngram appears in the data.

The usefulness of features can be assessed through cross-validation. First, pick an evaluation metric. Common evaluation metrics for text classification are precision, recall, or F-score. Second, divide the data using cross-validation. See if certain feature combinations result in higher evaluation metric performance on the validation dataset. Again, this process is often limited by the amount of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.