0
$\begingroup$

I am developing an application that would take feed from multiple IP cameras and run 3 different neural networks on it. I want to determine the number of such IP cameras that can I process parallelly on my GPU.

Improve this question
$\endgroup$
1
  • $\begingroup$ It depends on how much your networks consume in terms of memory. For example, with my GPU of 12GB RAM, I can only train two networks at a time because at the peak consumption point, they takes over 11GB RAM of GPU. To check your GPU status, use nvidia-smi. $\endgroup$
    – Giang Nguyen
    Commented Apr 22, 2020 at 6:00

1 Answer 1

Reset to default
1
$\begingroup$

This is sort of like asking "how long is a rope?". To be able to answer how many neural networks you can run simultaniously on a single GPU one would require a lot of information. For example:

  • How much memory does your GPU have?
  • Are we talking training or inference?
  • If we are talking training, is there a performance requirements and demand for a specific number of samples per batch?
  • How big is each image that should be processed?
  • How big is the neural network?
  • Datatype: float16 or float32 or something else?
  • Deeplearning-framework used
  • E.t.c.

If you were to provide this information, one might be able to guess how many you can run in parallell, but it would still be hard.

Best way is if you simply try running your network and monitor the memory usage on your GPU, using for example nvidia-smi, and find the peak memory usage and then simply check of many of that fits on the total GPU-memory. However note that you need to check the specifications for the framework e.t.c. to verify how they allocate GPU-memory (for example many frameworks allocate all the memory that's avaliable)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.