I have a dataset with three columns "message", "city" and "has_info". Here is a sample of it:

message                                    city         has_info
ill be there soon, call me 313-972-0310  New-York          1
use this email john***@gmail.com         Boston            1
ok. you can check it                     Boston            0
.................................        .......           ..
i love it                                Miami             0

has_info column is binary column which defines whether or not some contact information was mentioned in column "message" (1 if there was, 0 if wasn't). I have train and test pandas dataframes like that. And I want to make classifier to predict target value "has_info" in test dataset.

I turned feature "city" into categorical one and created couple of new features as well, like number of words in message for example. I also used bag of words method by finding up to 1000 most frequent tokens in train dataset and sorting them by number of occurences (highest first). So it will create 1000 additional features.

All of it however gave me only AUC value 0.85.

I wanted to know, if there any other (better) method for this particular case? Maybe I should just manually create list of red flag words (phone, mail, number, call, text, etc.) and based on them create dummy variables whether they occurred in message or not? Is there any other nlp solution that can probably give me at least 0.9 AUC?

My train dataset has 900000 rows in it, so its very large.

Thanks in advance


1 Answer 1


What type of information would you see as contact information? If it's just phone number and email address I would probably use a simple rule based classifier using (regex) pattern matching, which will probably already get you quite far. In addition you could use specific keywords that are often related to contacting someone such as call, send, mail, etc., for this you can use your existing training dataset to see what these types of words are for your use case.

  • $\begingroup$ it also can be a link to social network account. so you think no nlp package are useful here? $\endgroup$
    – Ir8_mind
    Jan 24, 2022 at 19:10
  • $\begingroup$ Depending on how difficult the problem is branching out to NLP packages might be helpful, but I would first start with the most simply solution and build out from there. $\endgroup$
    – Oxbowerce
    Jan 25, 2022 at 8:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.