I'm trying to create a model that, given a feature $x_i$, predicts $y_i$ such that $y_i=ax^2_i+bx_i+c$ by using backpropagation.
To do this, I'm using the ReLU activation function for each layer.
The output fits well when $x_i > 0$, but when it isn't it just outputs a flat line, as you can see in the picture:
Thinking about the reason, it seems pretty logical that when the output < 0 its derivative is also 0, hence the output is just the bias, but I cannot understand how it's possible to predict also negative values.
I'm not posting any code because I think the problem is more mathematical than due to programming, however if it's necessary I can post it.