Here some describe some points below :
Smaller datasets: Batch Renormalization relies on computing the mean
and standard deviation estimates across mini-batches. For smaller
datasets, this can lead to unreliable estimates of the mean and
standard deviation. In such cases, Batch Normalization may be a better
choice.
When training time is a priority: Batch Renormalization introduces additional computations, which can increase training time. If training
time is a priority, Batch Normalization may be a better choice as it
is computationally less expensive.
Non-stationary datasets: Batch Renormalization assumes that the statistical distribution of input features across mini-batches is
stationary. However, in some cases, such as in online learning or
datasets with significant concept drift, the statistical distribution
may change over time. In such cases, Batch Normalization may be more
appropriate.
Models with gated activations: Batch Renormalization may not be
well-suited for models with gated activations such as LSTM or GRU.
This is because the gating mechanism can interact with the Batch
Renormalization layer and affect the estimates of mean and variance.
while Batch Renormalization can improve the stability of the mean and standard deviation estimates across mini-batches, it is not always the best choice. The choice between Batch Normalization and Batch Renormalization depends on the characteristics of the dataset and the model architecture.