Seeking guidance on understanding graphics card parameters for deep learning training

Question

I am currently in the process of purchasing a new Nvidia graphics card for training deep learning models, and I have a few questions regarding the parameters involved and their relationship to the training process.

The graphics card specifications include parameters such as memory bandwidth, memory size, and theoretical performance in FLOPS. However, I'm unsure about how these parameters relate to the training process of deep learning models. Let me provide an example to illustrate my question better.

I am planning to use the Mask2Former model, which consists of 216 million parameters and has a theoretical performance of 868 billion FLOPs. Is there a formula or a thinking process I can use to get a rough idea about the training speed I can expect on a specific graphics card?

I would greatly appreciate it if someone could shed some light on this topic or provide some guidelines to help me make an informed decision. Thank you all in advance for your valuable insights and guidance. I look forward to learning from the community's expertise.

Answer 1

As in know from deeplearning when you are considering the specifications of a graphics card for training deep learning models, there are a few key parameters that play a role in determining the training speed and performance. While there is no direct formula to calculate the exact training speed but understanding these parameters will help you make an informed decision.

Some parameters are :

1. Memory Bandwidth
2. Memory Size
3. Theoretical Performance (FLOPS)

To get a better idea about the training speed you can expect on a specific graphics card i think you can consider the following guidelines:

1. Look for benchmarks and performance comparisons
2. Consider the memory requirements of your model
3. Take into account the complexity of the model
4. Evaluate the overall system configuration

Note: that deep learning training speed is influenced by several factors, including the specific optimization techniques used, data preprocessing, model architecture, and batch size. Therefore, it's essential to consider the graphics card specifications in conjunction with these factors to have a more accurate estimation of the training speed you can expect.