Initially posted in Stack Overflow.
I would like to implement a model which is actually 2 neural networks stacked together. However the size of these 2 architecture is too big to fit in GPU at the same time.
My idea was the following :
- Load the first model and run it for 1 batch
- Unload the first model from GPU, load the second model
- Run the second model from the output of the first model
- Unload the second model from GPU
- Repeat for every batch
I actually don't need to train the first model, since it's pre-trained. But I need to train second model.
Is it possible to do something like this ? Is my approach correct ? What are the pitfalls I should be aware of ? What about performance ?
I already tried the idea of computing the output of the first model for the whole dataset at first, and then use it as input for the second dataset. However the output of the first dataset is really big, and I don't have the available space for storing the whole pre-processed dataset. That is why I wanted to do it each batch.
After the very nice answer from @Gal Avineri, just one more precision : I would like to implements my architecture using only one GPU.