1
$\begingroup$

Suppose that I have a file which has thousands of skills starting from A-Z. Now, I would like to create a model that can group similar skills together (example neural network and SVM can group together). I know that I can use NLP for this problem, but I'm not sure about the algorithm that I can use to get the best result.

I'm new to NLP so any help is greatly appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

If all you have is the name/word, then I see only two ways:

  1. You can look at word similarity, basically by measuring how similar the spelling is. There are a number of possible metrics (e.g. Levenshtein distance). This will probably not get you very far.
  2. You can look at semantic similarity. You can use pre-trained word embeddings to map the words to a new vector space where you can calculate the distance between the word embeddings, e.g. with word2vec or other implementations. Alternatively, you can use a dictionary/corpus (e.g. https://www.nltk.org/index.html) that provides you a word taxonomy that allows you to compute semantic similarity between words via relationships like synonyms etc. See here for different approaches.
Improve this answer
$\endgroup$
3
  • $\begingroup$ Can you show me some example? $\endgroup$
    – Srikant Singh
    Commented May 9, 2019 at 21:05
  • $\begingroup$ what do you mean by pre-trained word embeddings? $\endgroup$
    – Srikant Singh
    Commented May 11, 2019 at 7:33
  • $\begingroup$ if you are unfamiliar with these concepts it's probably best to review them first. if you have a specific question about them, you can open a new question. otherwise your question is too broad to be answered in this type of forum :-) $\endgroup$
    – oW_
    Commented May 12, 2019 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.