I have a question (beginner :D) that is related to throughput and inference rate. Can the throughput and inference rate change as the model is trained or are the values for these parameters fixed? Many thanks to everyone who can contribute to the answer.
During the training of a model, either the parameters are calculated algorithmically or updated using Backpropagation (depending upon the algorithm and/or model architecture).
However, the architecture/algorithm and number of parameters do not change. As you would appreciate, that the inference time of a model depends on the architecture and number of parameters for a given hardware, training the model more number of iterations or on a larger training data would have no impact on the model's inference time throughput.