I am developing an application that would take feed from multiple IP cameras and run 3 different neural networks on it. I want to determine the number of such IP cameras that can I process parallelly on my GPU.
This is sort of like asking "how long is a rope?". To be able to answer how many neural networks you can run simultaniously on a single GPU one would require a lot of information. For example:
- How much memory does your GPU have?
- Are we talking training or inference?
- If we are talking training, is there a performance requirements and demand for a specific number of samples per batch?
- How big is each image that should be processed?
- How big is the neural network?
- Datatype: float16 or float32 or something else?
- Deeplearning-framework used
If you were to provide this information, one might be able to guess how many you can run in parallell, but it would still be hard.
Best way is if you simply try running your network and monitor the memory usage on your GPU, using for example
nvidia-smi, and find the peak memory usage and then simply check of many of that fits on the total GPU-memory. However note that you need to check the specifications for the framework e.t.c. to verify how they allocate GPU-memory (for example many frameworks allocate all the memory that's avaliable)