Do grouped convolutions actually improve learning?

My Understanding of Grouped Convolutions

Let say we have some data with the dimensions [100,100,32] (lets ignore batch size and assume channels last) and we want to pass it to a convolutional layer with 64 filters. Without grouping, we could pass the input directly into the second convolutional layer and get one output with a shape [100,100,64].

Alternatively we can split the input into n groups. For example lets say n=2. This changes the input from 1 [100,100,32] tensor to 2 [100,100,16] tensors. Then we pass each input to a different convolutional layer with 64/n = 32 filters to get two outputs with a shape of [100,100,32]. These two inputs are than concatenated channel wise to get a single output with a shape of [100,100,64].

The Problem

If this is correct, I understand how this can be useful for distributed training on multiple GPUs/CPUs (which I believe is what AlexNet did). However, I have seen claims that grouped convolutions improve performance and I feel that these two approaches (grouped and ungrouped) are mathematically identically and should have no difference in performances.

My Reasoning

Each filter is made up of c kernels; where c is the number of channels in the input. This is because each kernel is applied to a single channel. There is no interaction between kernels in the same filter. Therefore, regardless of how the channels are placed (ie. in one stack or in a groups) the resulting filter will be the same.

Am I mistaken?

Places that claim that grouped convolutions improve performnces