I am currently in the process of purchasing a new Nvidia graphics card for training deep learning models, and I have a few questions regarding the parameters involved and their relationship to the training process.
The graphics card specifications include parameters such as memory bandwidth, memory size, and theoretical performance in FLOPS. However, I'm unsure about how these parameters relate to the training process of deep learning models. Let me provide an example to illustrate my question better.
I am planning to use the Mask2Former model, which consists of 216 million parameters and has a theoretical performance of 868 billion FLOPs. Is there a formula or a thinking process I can use to get a rough idea about the training speed I can expect on a specific graphics card?
I would greatly appreciate it if someone could shed some light on this topic or provide some guidelines to help me make an informed decision. Thank you all in advance for your valuable insights and guidance. I look forward to learning from the community's expertise.