0
$\begingroup$

I have two GPUs, NVIDIA GTX 1070 Ti. For using Keras with TensorFlow back-end, should I connect them with SLI or not? If not, then they will be treated separately, and one model will be trained on one card. These are the two options from what I understood so far. Thank you.

$\endgroup$
0
$\begingroup$

You need not connect GPUs via SLI. Keras and TensorFlow will take care of distributing batches across GPUs

https://keras.io/utils/#multi_gpu_model

Instead of SLI, if you use NV-link, Keras can use use GPU for merge as well. https://www.nvidia.com/en-us/data-center/nvlink/

From the documentation :

Specifically, this function implements single-machine multi-GPU data parallelism. It works in the following way:

Divide the model's input(s) into multiple sub-batches. Apply a model copy on each sub-batch. Every model copy is executed on a dedicated GPU. Concatenate the results (on CPU) into one big batch. E.g. if your batch_size is 64 and you use gpus=2, then we will divide the input into 2 sub-batches of 32 samples, process each sub-batch on one GPU, then return the full batch of 64 processed samples.

This induces quasi-linear speedup on up to 8 GPUs.

This function is only available with the TensorFlow backend for the time being.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy