2
$\begingroup$

In computer vision, if we don't have a large training set, a common method is to start with a pre-trained model for some related task (e.g., ImageNet) and fine-tune that model to solve our problem.

Can something similar be done with natural language processing problems? I have a boolean classification problem on sentences and don't have a large enough training set to train a RNN from scratch. In particular, is there a good way to fine-tune a LSTM or 1D CNN or otherwise do transfer learning? And, if we want to do classification on sentences, is there a reasonable pre-trained model to start from?

$\endgroup$

2 Answers 2

2
$\begingroup$

This paper might be useful....

https://arxiv.org/abs/1801.06146

$\endgroup$
1
1
$\begingroup$

Consider few-shot learning approaches, recently found guys that practice such approaches for their information extraction models, claiming you will need somewhere between 8 training examples. But it depends on the cases you are looking to train and the expected satisfied precision. https://huggingface.co/knowledgator

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.