5
$\begingroup$

I have been trying to implement simple linear regression using neural networks in Keras in hope of understanding how to work in the Keras library. Unfortunately, I am ending up with a very bad model.

Here is the implementation:

from pylab import *
from keras.models import Sequential
from keras.layers import Dense

#Generate dummy data
data = data = linspace(1,2,100).reshape(-1,1)
y = data*5

#Define the model
def baseline_model():
   model = Sequential()
   model.add(Dense(1, activation = 'linear', input_dim = 1))
   model.compile(optimizer = 'rmsprop', loss = 'mean_squared_error', metrics = ['accuracy'])
   return model


#Use the model
regr = baseline_model()
regr.fit(data,y,epochs =200,batch_size = 32)
plot(data, regr.predict(data), 'b', data,y, 'k.')

The generated plot is as follows:

Plot

Can somebody point out the flaw in the above definition of the model (which could ensure a better fit)?

$\endgroup$
7
$\begingroup$

Your code works perfectly. The only problem is that the learning of the parameters is not finished. If you try with 10000 epochs this will works but this is way too much for this problem. As a matter of fact you can see that the loss is diminishing very slowly.

  • Solution : Increase the learning rate.

I set the batch size to one because this penalize the convergence speed here. Increasing the batch size is useful when you need to avoid overfitting. Here you want to overfit your data. Moreover I choose to use a simpler update rule with the SGD optimizer. With those changes, you will see that only 4 epochs are necessary to fit perfectly your data.

from pylab import *
from keras.models import Sequential
from keras.layers import Dense
from keras import optimizers

#Generate dummy data
data = data = linspace(1,2,100).reshape(-1,1)
y = data*5

#Define the model
def baseline_model():
   model = Sequential()
   model.add(Dense(1, activation = 'linear', input_dim = 1))
   sgd = optimizers.SGD(lr=0.2)
   model.compile(optimizer = sgd, loss = 'mean_squared_error', metrics = ['accuracy'])
   return model


#Use the model
regr = baseline_model()
regr.fit(data,y,epochs = 4,batch_size = 1)
plot(data, regr.predict(data), 'b', data,y, 'k.')
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.