I have a question with automatic natural language processing, I would like to automatically visualize skills (communication, marketing, statistics, etc.) taken from a document.

I already have a defined list of 10 skills and I have texts extracted from the documents. These documents are written with 1 to 3 skills in mind. According to my defined list, I would like to be able to extract the relevant skills from the document.

I would like to know what are the ideal methods to achieve my goal? (Like bag of words, word embeddings, etc)


To be more precise these papers are written on personal topics based on Kolb's model which is experiential learning in 4 phases which are Concrete experience, Reflective Observation, Abstract Conceptualisation and Active Experimentation. An example of a topic might be "How can I improve my company's marketing campaign?" And the topic is written based on the 4 phases of Kolb's model by explaining one's experience, making an analysis of one's experience, coming up with concept ideas with various theories and formulating hypotheses to test the ideas. It is easy to guess that this topic is aimed at marketing skill. Each document targets one or up to three skills.

And what I want to achieve, a way to say which skills are targeted by this paper based on my defined list of skills.

Here are some examples of the skills that are defined in the list: self-initiative, team leadership, marketing, project management, courage to make choices, etc.

I hope I have been explicit enough.

  • $\begingroup$ Welcome to DataScienceSE. Can you please give an example of the kind of document and which content you want to extract? I don't know if this is relevant to you but I remember this possibly related question. $\endgroup$
    – Erwan
    Jul 5 at 22:05
  • $\begingroup$ @Erwan, thanks for the link but it is not relevant to me. I gave an example as you can see to be as specific as possible. $\endgroup$
    – talohsa
    Jul 7 at 9:23
  • $\begingroup$ As far as I understand it looks like text classification, i.e. trying to predict which of the predefined 10 skills correspond to a text. Classification is supervised so you would need a training set, i.e. sample of documents labelled with their corresponding skills. Note that since each document can have multiple classes this would be multi-label classification. Do you or can you have a training set? $\endgroup$
    – Erwan
    Jul 8 at 10:02
  • $\begingroup$ @Erwan, This is a good idea, yes. I didn't think of that because I was so fixated on NLP methods. But isn't it too much for a classification algorithm to classify 10 classes? Because it is not impossible that new skills are defined later. Or is it a good idea to train a skill-based classification model, i.e. to do a binary classification? For example, I want to train an algorithm for the skill "marketing" so I add a target variable for the binary classification to put true for documents targeting the skill "marketing" and false for documents targeting other skills. $\endgroup$
    – talohsa
    Jul 10 at 9:41
  • $\begingroup$ 10 classes is not that many but it depends on the data, i.e. whether it contains enough indications to distinguish between them. I think you're right that binary classification for every label (i.e. multi-label classification) makes sense for this case, because it's possible that a document corresponds to several skills and it also makes it easier to add a class later. But for any kind of classification you will need a labelled training set of course. $\endgroup$
    – Erwan
    Jul 10 at 13:23

Browse other questions tagged or ask your own question.