Cost Efficient Machine Learning Development Using Cloud GPUs

Question

Let's say I have a Juypter Notebook I am working on where I am analyzing, visualizing, testing, etc. various Machine Learning Models with different hyperparameters on some arbitrary data set or I am developing Machine Learning libraries and toolkits, all in Python. Let's also say that some of the computations I want to run using that data set need a powerful GPU in order to complete in a reasonable amount of time (I guess powerful is relative, I am talking in the range of a computer with 1 RTX 3090 or perhaps a 3080 ti).

That being said, while there are a few functions which are very computationally intense (say training a model or multiple models with variations of hyperparameters), much of the code I am writing is not. In addition, I am also working out how exactly I want to design my analysis, working out bugs, etc.

Due to the above, the amount of time I actually need to use a (relatively) expensive, powerful GPU is not high, however, it seems to me that all of the options I have found to do this task involve paying for a cloud VM or Juypter Server where I am paying for all of the time the VM/server is "up", which includes a lot of time where I am just editing my code, debugging my code, or running code which I could easily run on my current laptop.

I imagine the best solution to my problem is to do one of two things:

1. Find a service where I pay for only for Computation Time. So that when I am editing my code, thinking, etc. I do not pay for all the time that I am not exerting significant computational resources (or at least I am not paying much, I do not mind paying a small idle time fee on the order of less than 10 cents an hour or so).

2. Find a service that allows me to run code and develop fully on my home machine, but when I need to get the result of a computationally intense function, I use something comparable to a Google Cloud Function, where I essentially would be able to run just one function in the cloud. This could be used to say run a CUML training algorithm on a given dataset in a cloud GPU and then I can analyze the results on my home machine without paying for a cloud provider. This is not something I have been able to find using GPUs.

I have to imagine I am not the first low-budget Machine Learning Engineer (in training) to need to do this, so I am hoping there are resources available to accomplish what I am looking for in the general sense, whether it is one of the two options I listed above or an alternative which is similarly easy/simple to use and affordable.

Also, I don't anticipate these factors making a difference but to give some personalized details, my home machine uses Mac OS, I would strongly prefer to use NVIDIA gpus, and I am a college student (in case that matters for the purposes of a discount somewhere).

Notes:

Just to pre-empt a few responses people gave me in the past:

1. I can't just develop my code on my home machine and then run it on a remote machine because I need to be able to use the GPU in the debug loop.
2. I can't solve my problem just by a "downscale" of my computing needs by using smaller data sets in the phase where I am still debugging/editing, because even if I use smaller data sets for debugging I still need a GPU to run my code on and my home machine does not have a GPU.

Answer 1

I am in a similar boat (although not developing algorithms/libraries, just running DL models on some datasets). My laptop does not have a GPU and DL requires a GPU. So what I do is use free GPU's like Google Colab, Kaggle notebooks, Paperspace to run my models.

Most of my time is spent in EDA and feature engineering which does not require a GPU. So while I am performing these tasks I turn off the GPU on the platform I am using. When I actually want to train my model or tune HP, I simply turn on the GPU. This facility is provided by the platform you are using. So basically I am running DL models for free.

Although the GPU's provided by these platforms are not that fast but they are enough for most of the people provided you are not working with big data.

If you really need something faster, you could buy a subscription of Google Colab (10$ per month). I don't know what kind of GPU you would get for that but it would be much faster than the free version.

Another viable option is to rent a VM in Azure. The part where you do not require a GPU you could do that in your local machine and for the part where you require one, only that part you could execute in the VM. For example the EDA, feature engineering, feature selection etc all those steps could be done on your local machine and the actual training and HP tuning could be done on the VM. Hope it helps!

Cheers!