Using GPUs of other machines in the network with Keras

Question

My situation is as follows:

• I have a rather cheap laptop with Ubuntu 18.04 running on it that unfortunately is not powerful enough (old, cheap GPU) to train deep learning models with. I am located in city A.
• My father (located in city B) has two computers (with powerful GPUs) with Windows 10 Professional running on both of them that I could theoretically use at nighttime. I can connect to the network via VPN and have SSH root access to both machines.

I am now looking for a Keras setup that allows me to train neural networks from my local machine making use of the remote GPUs. Ideally, I would have a Jupyter notebook with all the logic running on my conputer and could somehow connect to a thin client (probably exposing some interface to CNTK) running on the remote machines. I'd prefer to keep the Windows machines as dumb as possible and would ideally only make use of their GPUs (with some local caching of course). Also, I would prefer to stick with Windows on the remote machines not having to set up some dual-boot with a Linux distro. Enabling the Linux subsystem would be fine though.

Any thoughts on how to tackle my problem?

Answer 1

I'm not sure how exactly you're imagining it but the way you're describing it is not possible. You can't perform some computation locally and some remotely and even if you could set it up it would be inefficient due to latency.

A lot of people (including myself) have similar situations and the way we deal with it is executing everything remotely. I.e. you have your data, your code and your notebooks all in the remote machine, from where you run them. However, you access these from your own PC. E.g. you might have a notebook running on your own browser, that actually executes its commands on the remote machine (think of it like google colab or kaggle notebooks).

---

How to set this up

First of all, creating a dual-boot linux isn't necessary. You should do it only if you feel that you're a lot more familiar Linux than Windows. What you need to do is:

1. Install all necessary NVIDIA tools so that you can run keras on GPU. These include installing (a) the proper graphics card drivers, (b) CUDA and (c) cuDNN
2. Setting up your python ecosystem, which needs to have all necessary libraries (jupyter, tenrosflow, keras, etc.). I'd suggest a virtual environment. At this point you should make sure keras runs properly on the GPU on your remote machine.
3. Setup an ssh connection that forwards everything sent to port 8888 of your remote machine to a port of your choosing (e.g. 9999) in your own machine: ssh -N -f -L localhost:8888:localhost:9999 username@remote_hostname.
4. Connect via ssh to your remote machine and run jupyter notebook (I prefer running it on the background or in a screen so that I don't have to keep the terminal always open). Take note of the token that it outputs to the screen as it won't log you in automatically.
5. Open your browser and type localhost:9999. You need to add the token here for authentication. Now you're using a notebook that's actually running on your remote machine!

Additionally, you can set up other tools to make your life easier. For instance, I have a FTP client for transferring data easier and have set up PyCharm to use a remote interpreter (i.e. execute code I write on the remote machine).

Answer 2

What you could do is, since you can ssh into the machine, setup an environment with only the necessary dependencies and run the training on it.

You could also look into Amazon Web Services with their GPU machines that can be quite affordable.