I have trained two text classification models using GPU on Azure. The models are the following
- Bert (ktrain)
- Lstm Word2Vec (tensorflow)
Exaples of the code can be found here: nlp
I saved the models into files (.h5) for later use. The files are big e.g. 27,613kb for the lstm and 1.2 gb for bert.
I loaded the models and in a computer where only CPU is available. They both work fine but the
model.predict(text) function is super slow predicting the class of the text e.g. on average 1 tweet sized message per second.
Adding GPU on the computer is not an option. I wonder if there is another way to make it run faster? e.g. train the models in a different way (without compromising accuracy) or save the model in a different file format?