I would like to write a NN that can classify different kinds of words(e.g. nouns,verbs,names) and am struggling to find information on how to do feature extraction on single words.For example, i would like the NN to learn that "street" is a noun, while "How would i go about doing that? I'm really nwe to this and searching for it always seems to yield only examples of text feature extraction, which is not what I'm looking for.

Thank you in advance kind strangers!

  • 1
    $\begingroup$ Why would you want a ML model to do this? There are already libraries in various different programming languages that can give you the actual correct answer immediately rather than trying to train a NN to statistically guess based on letter combinations. I don't think it would be particularly insightful (in terms of practicality or as an exercise) to train a model to predict whether a given combination of letters is a noun/particle/verb/adjective/etc. $\endgroup$ – Jason K Lai Oct 2 '19 at 18:29
  • $\begingroup$ See TextStructure. For example on "street". $\endgroup$ – Edmund Oct 2 '19 at 22:30

I suppose you could try to learn a mapping from characters (or character-level n-grams) to part-of-speech. This would be analogous to document classification. Instead of a document, you have a single word. And instead of a sequence of words, you have a sequence of letters.

With this formulation, most of the tricks which extract features from text could also be applied to extract features from characters (although your mileage may vary). At the simple end of the scale, you could try "old-school" techniques like bag-of-words and TF-IDF (except you would have a bag-of-characters, and it would be Character-Frequency Inverse-Word-Frequency). On the complex end of the scale, you could try to learn embeddings of characters or character n-grams.

However, before you get started, there are a few things I think you should keep in mind:

  1. Do you think there is enough information in sub-word features to classify parts of speech? I think it's pretty unlikely (at least for English). Your model might learn some of the easy cases (e.g. an "-ly" suffix often indicates that the word is an adverb), but I don't think it would perform well in general.

  2. This is actually a multilabel classification problem, because there are many words which can serve as more than one part-of-speech. Also, names (proper nouns) are a subclass of nouns.

  3. What happens if you feed your model a non-existent word? Should your model try to classify it as a part-of-speech, or do you want it to recognize non-existent words?

  4. If you're new to machine learning, consider putting this aside for a little while. Jumping headfirst into a difficult classification problem with deep learning is a hard, confusing way to learn.

Finally, are you doing this for experimental/fun reasons, or is this intended to be part of a production application? If it's the former, then go for it! Worst case: it doesn't work but you learn something.

But if this project is for anything more serious, then you should not train a ML model to solve this problem. Your model will never outperform a dictionary :)

  • 1
    $\begingroup$ ah thank you for the explanation, that was really helpful. Yeah, this was just an idea i had, that i wanted to try. I thought doing that might be a good basis for classifying different kinds of character-sequences down the line. I am aware of the low information of sub-word features(the most difficult name i could think of of the top of my head is "Caren"), this was mainly just meant as an exercise for classifying character-sequences in general , because it was what i could most easily find online. $\endgroup$ – Apocalypse Meow Oct 3 '19 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.