2
$\begingroup$

I am performing simple linear regression using PyTorch but my model is not able to properly fit over the training data. please look at the code to find the mistake.

Dataset is here

import torch
from torch import nn
from torch.optim import SGD, Adam
from torch.autograd import Variable

class Linear_Reg(nn.Module):
    def __init__(self):
        super(Linear_Reg, self).__init__()
        self.linear = nn.Linear(1,1)
        
    def forward(self, x):
        y_pred = self.linear(x)
        return y_pred
    
net = Linear_Reg()

Xt = Variable(torch.Tensor(X[:,0]))
yt = Variable(torch.Tensor(y))
Xt = Xt.view(-1,1)

criterion = nn.MSELoss()
optimizer = Adam(net.parameters(), lr=0.001)

EPOCHS = 500
for epoch in range(EPOCHS):
    pred_y = net(Xt)
    loss = criterion(pred_y, yt)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print('Eopch: {}, \t\t loss: {}'.format(epoch, loss.data.item()))

The loss decreases from ~68.88 to ~30.26

and the resulting fitting is this:

xxt = torch.arange(5,23)
with torch.no_grad():
    a = net(xxt.reshape(-1,1).float())
plt.scatter(X[:,1], y, s=30, c='r', marker='x', linewidths=1)
plt.plot(xxt.data.numpy(),a.data.numpy(), label='Linear regression (Gradient descent)')

enter image description here

What is the problem here?

$\endgroup$
  • $\begingroup$ Check if the yt values are being altered by the gradient. Could be the case. $\endgroup$ – Carl Rynegardh May 3 '19 at 15:56
  • $\begingroup$ I checked them, they are not being altered. $\endgroup$ – thanatoz May 3 '19 at 16:24
  • $\begingroup$ Can you post the part of the code where you plot as well $\endgroup$ – Andy M May 3 '19 at 16:30
  • $\begingroup$ @AndyM, I have updated the code. $\endgroup$ – thanatoz May 3 '19 at 16:37
1
$\begingroup$

I know this doesn't answer you 100%, but maybe it helps. Feel free to disregard.

I just implemented this in keras and didn't have any issues

model = Sequential([
    Dense(1,activation='linear',input_shape=(1,))
])
sgd = optimizers.Adam(lr=0.001)
model.compile(optimizer='adam',
              loss='mean_squared_error',
              metrics=['mae'])
model.fit(test_df.x, test_df.y, validation_split = .2,epochs=500, batch_size=1)

plt.scatter(test_df.x,test_df.y,s=30,c='r',marker='x')
x = np.linspace(4,25,100)
y = model.get_weights()[0][0][0]*x+model.get_weights()[1][0]
plt.plot(x, y)
plt.ylim([-5,50])
plt.xlim([4.5,23])

enter image description here

Is it possible you read your data in incorrectly maybe? Or maybe you are referencing it incorrectly when you set your Xt and yt?

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks Andy, but my issue is only limited to PyTorch. Other frameworks and even implementing linear regression from scratch causes no issue. It would only help if you solve it using PyTorch. $\endgroup$ – thanatoz May 4 '19 at 4:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.