0
$\begingroup$

I have a bunch of categorical, part of speech data that I want to collapse into fewer categories. np.where() won't do because I want to have 6 categories at the end: noun, verb, adjective, adverb, preposition, and other.

I've found out that I can use pandas.replace() in combination with a dictionary to do this.

So, I've made the following dictionary:

mappings = {"NN" : "noun", "NNS" : "noun", "NNP" : "noun",
            "VB" : "verb", "VBD" : "verb", "VBG" : "verb", "VBN" : "verb", "VBP" : "verb", "VBZ" : "verb",
            "JJ" : "adj", "JJR" : "adj", "JJS" : "adj",
            "RB" : "adv", "RBR" : "adv", "RBS" : "adv",
            "IN" : "prep"}

The problem is, there are a LOT more parts of speech present in the data. Is there a way for me to shove all of those other parts of speech into an "other" category, or will I have to manually type in all of the other possible parts of speech?

$\endgroup$

1 Answer 1

0
$\begingroup$

You could use numpy select function

You would need to adapt but it would be something like this:

nouns = ["NN","NNS","NNP"]
verbs = ["VB","VBD","VBG","VBN","VBP","VBZ"]
adjs = ["JJ","JJR","JJS"]
advs = ["RB","RBR","RBS"]
preps = ["IN"]

condlist = [
           df.my_colum.isin(nouns),
           df.my_colum.isin(verbs),
           df.my_colum.isin(adjs),
           df.my_colum.isin(advs),
           df.my_colum.isin(preps),
           ]


choicelist = ["noun","verb","adj","adv","prep"]


df["gruop"] = np.select(condlist= condlist, choicelist= choicelist, default = "other")
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.