Background: While fitting neural networks with relu activation, I found that sometimes the prediction becomes near constant. I believe that this is due to the relu neurons dieing during training as stated here. (What is the "dying ReLU" problem in neural networks?)
Question: What im hoping to do is to implement a check in the code itself to check if the neurons are dead. After that, the code could refit the network if needed.
As such, what is a good citeria to check for dead neurons? Currently im thinking of checking for low variance in the prediction as a citeria.
If it helps, im using keras.