In computer vision, if we don't have a large training set, a common method is to start with a pre-trained model for some related task (e.g., ImageNet) and fine-tune that model to solve our problem.
Can something similar be done with natural language processing problems? I have a boolean classification problem on sentences and don't have a large enough training set to train a RNN from scratch. In particular, is there a good way to fine-tune a LSTM or 1D CNN or otherwise do transfer learning? And, if we want to do classification on sentences, is there a reasonable pre-trained model to start from?
