I am relativity new to machine/deep learning and NLP. As a part of my Phd thesis I have scraped vast number of job vacancies (most of them are in Polish, and about 10% are in English ones) and then extracted required skills/competencies. As an output I got a vector of strings contains single skills/competences. I performed initial preparation (removed extra spaces, special symbols, stop words etc.) and built then a frequency distribution table to figure out the most demanded skills. The most commonly required skills were as follow:
prawo jazdy kat b ## driver license umiejętność pracy w zespole ## team work wykształcenie wyższe ## higher education czynne prawo jazdy kat b ## driver license znajomość pakietu ms office ## MS Office prawa jazdy kat b ## driver license prawo jazdy kategorii b ## driver license fluent english doświadczenie na podobnym stanowisku ## work experience dobra znajomość języka angielskiego ## English excellent english skills doświadczenie ## work experience
As you may see some of these competencies are quite similar. The problem is that job offers were provided by a different employers. European classification of skills/competences qualifications and occupations define about 3500 different competencies for instance.
My aim is to create trivial classifier using logistic regression, SVM or random forest that would be able to classify skills in real time using pre trained model. The problem is that I can not label vector of unique skills because of vast number of skills within its synonyms. I spent a while trying to figure out a solution.
So, the idea was to perform cluster analysis and to put similar skills into separate groups (clusters) and then label this clusters. In this case I would have much less group to label. Moreover, I would have a lot of synonyms within each group, which would increase precision of classification (I guess). Using labelled data set I would be able to trained model, and then to classify future job vacancies (skills). However, as I said I am new to machine learning, so I am not sure whether my solution would be good enough.
So, I am looking for advice/tutorials/links that would help me to solve my problem rather than for complete solutions. I can use both R and Python. Would be appreciated for any help.