Consider the following network:
There are two weights, say $w_1$ and $w_2$, and two biases, $b_1$ and $b_2$. The hidden layer has a ReLU activation function $g^{(1)}$ and the output layer has a linear activation function $g^{(2)}$.
Say we have a dataset of $M$ datapoints, each of the form $(x^{[m]}, y^{[m]})$ (1D inputs/outputs)
The network's predictions are \begin{align*} \hat{y}^{[m]} = g^{(2)}\left(w_2 \left[g^{(1)} \left(w_1 x^{[m]} + b_1 \right)\right] + b_2\right) = g^{(2)}\left(w_2 {a_1^{(1)}}^{[m]}+ b_2\right) \,. \end{align*} Assuming a linear activation function in the output layer, this simplifies to \begin{align*} \hat{y}^{[m]} =w_2 \left[g^{(1)} \left(w_1 x^{[m]} + b_1 \right)\right] + b_2 = w_2 {a_1^{(1)}}^{[m]} + b_2 \,. \end{align*} The MSE loss is $$L_\text{MSE}=\frac{1}{M}\sum_{m=1}^M L_m = \frac{1}{M} \sum_{m=1}^M \left(\hat{y}^{[m]} - y^{[m]} \right)^2 \,.$$
The derivatives of the MSE loss with respect to the network parameters are: $$ \frac{\partial L_\text{MSE}}{\partial b_2} = \frac{1}{M} \sum_{m=1}^M \frac{\partial}{\partial b_2} \left[\left(\hat{y}^{[m]} - y^{[m]} \right)^2 \right] = \frac{1}{M} \sum_{m=1}^M 2\left(\hat{y}^{[m]} - y^{[m]} \right) \cdot 1 \, $$ $$\frac{\partial L_\text{MSE}}{\partial w_2} = \frac{1}{M} \sum_{m=1}^M \frac{\partial}{\partial w_2} \left[\left(\hat{y}^{[m]} - y^{[m]} \right)^2 \right] = \frac{1}{M} \sum_{m=1}^M 2\left(\hat{y}^{[m]} - y^{[m]} \right) {a_1^{(1)}}^{[m]} \,$$ $$\frac{\partial L_\text{MSE}}{\partial b_1} = \frac{1}{M} \sum_{m=1}^M \frac{\partial}{\partial b_1} \left[\left(\hat{y}^{[m]} - y^{[m]} \right)^2 \right] = \frac{1}{M} \sum_{m=1}^M 2\left(\hat{y}^{[m]} - y^{[m]} \right) w_2 \hspace{0.2em}g^{(1)^{'}}\left(w_1 x^{[m]} + b_1 \right) \,$$ $$ \frac{\partial L_\text{MSE}}{\partial w_1} = \frac{1}{M} \sum_{m=1}^M \frac{\partial}{\partial w_1} \left[\left(\hat{y}^{[m]} - y^{[m]} \right)^2 \right]= \frac{1}{M} \sum_{m=1}^M 2\left(\hat{y}^{[m]} - y^{[m]} \right) w_2 \hspace{0.2em}g^{(1)^{'}}\left(w_1 x^{[m]} + b_1 \right) x^{[m]} \,.$$
I'm trying to find a combination of parameters (weights and biases) and a dataset (of any size $M > 1$) such that $\displaystyle \frac{\partial L_\text{MSE}}{\partial b_2}$ and $\displaystyle\frac{\partial L_\text{MSE}}{\partial w_2}$ are 0 but $\displaystyle \frac{\partial L_\text{MSE}}{\partial b_1}$ and $\displaystyle \frac{\partial L_\text{MSE}}{\partial w_1}$ are not equal to 0.
However, I've been struggling to solve this problem and find a combination that works. Any help would be much appreciated.