There are 2 different levels of complexity in a network :
- Number of parameters
- Number of operations (FLOPs)
It is especially important to make a distinction when using CNN since a convolution kernel is applied on many different pixels, so a same weights will be used in different computations. The ratio $operations/parameters$ is approximately $1$ in a fully connected network, but in a CNN it is way more important.
About the number of parameters
This article explains very well the number of parameters of each CNN architecture, you should give it a look.
If you take a look at the tables of parameters of ResNet and VGG, you will notice that most of VGG parameters are on the last fully connected layers (about 120 millions of the 140 millions parameters of the architecture). This is due to the huge size of the output layer of the convolutional part.
The output size is 512 7*7 features maps, so it is the equivalent of a $512*7*7 = 25088$ size layer in a fully connected network. This is why the connection to the next layer (which has 4096 neurons) is very expensive and requires $25088*4096 = 100M$ parameters.
This is quite a particularity of VGG, this architecture has 70% of its parameters used for one layer.
Now if you compare with ResNet, ResNet use an avg pool on each feature maps at the end, so the number of outputs of the convolutional part is 512 values, which leads for fully connected part of the network to have $1000*512 = 512 000$ parameters (if we forget about the bias of each neuron).
This is why (at least in my brain) VGG has so many parameters while ResNet keep it fairly low.
About the number of operations (FLOPs)
As I mentionned previously, a layer with many parameters may not be the hardest to compute for the network, it depends if the parameters is only used once (fully connected layers) or more (convolutional layers), so the 100M parameters layer of VGG may not be the cause of its important number of operations.
I'm way less confident in my explanation here, but I think ResNet efficiency is due to the fact that Resnet quickly diminishes the size of feature maps (first layer divide it by 2 and second layer as well), so since we work on less pixels, we have less computations to do.
VGG applies 3x3 convolution kernels on each of the 224*224 pixel of the image on all 3 of its first layers, which is what is not efficient and requires a lot of computational power.
That is how I would explain ResNet having less computational complexity than VGG
Sorry for writing something so long, hope it helps.