# Multiple GPU in MXNet C++

I am trying to make MXNet (C++ API) learn, with a common sample in C++, on multiple GPU. According to this MXNet forum post, we need to aggregate manually the gradients that we fetch at the backpropagation time. Now, if I separate the gradients of each GPU, both networks are training. If I concatenate the weights, it doesn't work. (Like this) :

gradValuesCombined.insert(gradValuesCombined.end(), gradValues1.begin(), gradValues1.end());


However, if the gradients have the same batch_size, then summing the gradients works.

gradArray1[i] + gradArray2[i]


But, summing all the gradients like if it was a one-batch vector, doesn't work:

combinedOneDim[0] = 1;
int sizeOneBatch = combinedOneDim.Size();