Celsius to Fahrenheit conversion simple Neural Network

Question

This common example is a simple way to dabble into the most basic neural network (i.e. two neurons, one input and one output).

I have been playing around with this using TensorFlow and keras, and I am wondering, it should be very simple for the NN to learn the weight (1.8) and bias (32) with little to no effort.

However, I find that (albeit, using only 13 examples) after running for 500 epochs I can get a weight of 1.8 but the bias is always off (29.2, for example). Obviously there is no activation function, given this linear problem.

My question is, for such a simple case, will a NN be able to solve this problem exactly?

Answer 1

Hard to say without seeing code, but I can imagine several reasons:

• You are using an activation function. You want 'linear' or no activation
• Learning rate is much too low
• Wrong network (should be one dense layer with one weight)
• Poor batch size

This works pretty instantly:

from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Nadam

model = Sequential([Dense(1, activation=None, input_shape=[1])])
model.compile(loss='mean_squared_error', optimizer=Nadam(lr=0.1), metrics=['mean_absolute_error'])

F = list(range(-200,200))
C = [(f - 32) * 5 / 9 for f in F]

model.fit(F, C, batch_size=16, epochs=50)

print(model.predict(np.array([300])))
print(model.layers[0].get_weights())

[[148.88893]]
[array([[0.5555556]], dtype=float32), array([-17.777737], dtype=float32)]

300 F -> 148.8889 C, good, and it has learned the conversion is C = 5/9 * F - 17.7778, or C = 5/9 * (F - 32)