Simple linear regression in PyTorch

Question

I am performing simple linear regression using PyTorch but my model is not able to properly fit over the training data. please look at the code to find the mistake.

Dataset is here

import torch
from torch import nn
from torch.optim import SGD, Adam
from torch.autograd import Variable

class Linear_Reg(nn.Module):
    def __init__(self):
        super(Linear_Reg, self).__init__()
        self.linear = nn.Linear(1,1)

    def forward(self, x):
        y_pred = self.linear(x)
        return y_pred

net = Linear_Reg()

Xt = Variable(torch.Tensor(X[:,0]))
yt = Variable(torch.Tensor(y))
Xt = Xt.view(-1,1)

criterion = nn.MSELoss()
optimizer = Adam(net.parameters(), lr=0.001)

EPOCHS = 500
for epoch in range(EPOCHS):
    pred_y = net(Xt)
    loss = criterion(pred_y, yt)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print('Eopch: {}, \t\t loss: {}'.format(epoch, loss.data.item()))

The loss decreases from ~68.88 to ~30.26

and the resulting fitting is this:

xxt = torch.arange(5,23)
with torch.no_grad():
    a = net(xxt.reshape(-1,1).float())
plt.scatter(X[:,1], y, s=30, c='r', marker='x', linewidths=1)
plt.plot(xxt.data.numpy(),a.data.numpy(), label='Linear regression (Gradient descent)')

What is the problem here?

Answer 1

I know this doesn't answer you 100%, but maybe it helps. Feel free to disregard.

I just implemented this in keras and didn't have any issues

model = Sequential([
    Dense(1,activation='linear',input_shape=(1,))
])
sgd = optimizers.Adam(lr=0.001)
model.compile(optimizer='adam',
              loss='mean_squared_error',
              metrics=['mae'])
model.fit(test_df.x, test_df.y, validation_split = .2,epochs=500, batch_size=1)

plt.scatter(test_df.x,test_df.y,s=30,c='r',marker='x')
x = np.linspace(4,25,100)
y = model.get_weights()[0][0][0]*x+model.get_weights()[1][0]
plt.plot(x, y)
plt.ylim([-5,50])
plt.xlim([4.5,23])

Is it possible you read your data in incorrectly maybe? Or maybe you are referencing it incorrectly when you set your Xt and yt?