I want to make a TensorFlow model that, given features $x$ and labels $y$ such that $y_i = ax_i^2+bx_i+c$, predicts reasonably well the equation.

x = np.arange(-1000, 1000, 0.74)
y = 1.3*x**2 + 5.3*x + 4

Now, here is the model:

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, activation="relu"),

model.compile(loss="mae", optimizer=tf.keras.optimizers.Adam(learning_rate = 0.01))
history = model.fit(tf.expand_dims(x, axis=-1), y, epochs = 100)

However, the model does not predict a parabola, but a straight line, as you see in the picture:

enter image description here

I've tried to add more layers, to increase or decrease the learning rate, but nothing sorts any kind of effect.

How can I fix it? Thanks.


1 Answer 1


You did not specify any activation function in your dense layers. When you stack multiple linear layers without any activation function, the end result is equivalent to a single linear layer (see this other answer for the mathematical proof). Therefore, the functions your network can learn are basically linear, with a final rectification from the ReLU.

You may add ReLU activations (or other activation, like tanh or sigmoid) to each of the intermediate layers to enable your network to model non-linear functions.

I would like to point out that when someone says "quadratic regression", it usually means that you are doing linear regression with extra variables computed by squaring the other variables. This is not what you are doing. You are modelling the quadratic function as if it was a black box, without using your knowledge of the underlying process.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.