1
$\begingroup$

I am working on a small project and I would like to use the word2vec technique as a text representation method. I need to classify patents but I have only a few of them labelled and to increase the performance of my ML model, I would like to increase the corpus/vocabulary of my model by using a large amount of patents. The question is, once I have train my word embedding feature, how to use this larger corpus with my training data - my labelled data?

My data set is composed by 2000 patents which are labelled.

The patents used to train my word embedding corpus are 3 millions (some of my 2000 labelled patents are already included in this larger corpus) which I trained using Gensim.

Do you have any suggestions on how to do it?

Thank you very much in advance.

Rob

Improve this question
$\endgroup$
2
  • $\begingroup$ Hi Rob, welcome to the community :) Word2Vec is trained without any labelling necessary, so you can train it on the biggest dataset you can get. For your classification task, you can use the word embeddings from Word2Vec to train a model on top which will learn to classify using your labelled dataset. $\endgroup$
    – Adam Oudad
    Commented Jul 15, 2020 at 19:27
  • $\begingroup$ Thank you very much Adam. I have already trained the Word2Vec but I am not able yet to use it as corpus for my labelled dataset using Python. $\endgroup$
    – Rob C
    Commented Jul 16, 2020 at 14:46

1 Answer 1

Reset to default
0
$\begingroup$

Use large amount of un-label data to finetune the BERT based model. You can train BERT in unsupervised manner. Then, use that bert to get embeddings of your input text of label data and train a classifier.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi Uday, thank you for your reply. Do you have a link that you could suggest me to use Bert running on Python? Cheers $\endgroup$
    – Rob C
    Commented Jul 16, 2020 at 14:48
  • $\begingroup$ Check transformers library from huggingface @RobC $\endgroup$
    – Uday
    Commented Jul 16, 2020 at 15:28
  • $\begingroup$ Thank you I will have a look at it. $\endgroup$
    – Rob C
    Commented Jul 17, 2020 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.