I am looking for a way to utilize a computer’s gpu without using cuda (or any installable software). The reason for this is I have an application where the neural network runs on a user’s computer and I can’t assume they will have the knowledge/ spend the time and effort to install CUDA. I see that it is possible to utilize the gpu using WebGL but from my reading, it sounds like due to other limitations with this approach it is not actually faster than using cpu. Does anyone know how to do this maybe using openCL or some other way to do it?
I'm pretty sure that you will need CUDA to use the GPU, given you have included the tag tensorflow. All of the
ops in tensorflow are written in C++, which the uses the CUDA API to speak to the GPU. Perhaps there are libraries out there for performing matrix multiplication on the GPU without CUDA, but I haven't heard of a deep learning framework that doesn't use CUDA, when executing on the GPU.
What you can perhaps do it find a solution that works using Docker. The latest version doesn't require any additional Nvidia software to be installed. It would mean your customer doesn't need to mess around with CUDA themselves, they just need docker installed, which is generally less than 5 minutes work. Here is the Tensorflow documentation for a starter.
In addition, using Docker is perhaps the most professional and isolated way to deliver code to somebody else's machine. There are many benefits and it works on Windows, Mac, Linux. Any changes you make can simply be pushed to their machine, without them having to change anything. It can also scale arbitrarily to multiple machines, running locally or in the cloud, the list goes on and on...
There are some options to run stuff on the GPU without having the user install CUDA:
- If the user has the nvidia driver installed, you could bundle the CUDA libraries with your application (along with any other indirect dependency, maybe cublas and stuff like that). This is what some deep learning libraries like PyTorch do.
- You could use tensorflow.js, which runs on the GPU via WebGL. According to their web site, running via WebGL can be 100x running on CPU.
- Use the driver nvidia driver API directly without CUDA.
The decision depends on many factors (expertise of the team, deadlines, etc). Without any knowledge on those factors, I would recommend bundling the CUDA libraries, assuming that the GPUs are nvidia and that you meet the CUDA license.
I would not go for docker. By default docker containers cannot access the underlying GPU. You need to install and configure the docker nvidia runtime and specify it when running the image. If your users cannot install CUDA, its probable they won't be able to deal with this either.