As mentioned in this question, it could be useful to harness a GPU vs. CPU(s) for validation. For example, in cross-validation, where the number of validation examples can exceed the number of training examples in each epoch, this results in a need for speed in forward inferencing because the process of training to convergence must be replicated, especially when using Monte Carlo cross-validation.
The multi_gpu_model' function oftensorflow.keras.utils` has given me trouble (at least since TF2.2 came out), so I am not considering that at this time, although I would like to in the future for both training and validation (currently using only one GPU). Obviously it would be faster to use more than one GPU in parallel, but that is not the essence of my question; I think I will get a significant speed increase even using just one for now.
I could be completely mistaken that I will get a speed boost, because a significant part of the calculation time for each validation example is probably consumed by a custom loss function I am currently investigating. However, that exact same custom loss function (not available in tensorflow at present) is calculated at every training step within the epoch, and that takes less than half a second on the GPU, yet validation for 25 examples takes well over a minute with 10 CPU workers!
