# Simple linear regression in PyTorch

I am performing simple linear regression using PyTorch but my model is not able to properly fit over the training data. please look at the code to find the mistake.

Dataset is here

import torch
from torch import nn

class Linear_Reg(nn.Module):
def __init__(self):
super(Linear_Reg, self).__init__()
self.linear = nn.Linear(1,1)

def forward(self, x):
y_pred = self.linear(x)
return y_pred

net = Linear_Reg()

Xt = Variable(torch.Tensor(X[:,0]))
yt = Variable(torch.Tensor(y))
Xt = Xt.view(-1,1)

criterion = nn.MSELoss()

EPOCHS = 500
for epoch in range(EPOCHS):
pred_y = net(Xt)
loss = criterion(pred_y, yt)
loss.backward()
optimizer.step()
print('Eopch: {}, \t\t loss: {}'.format(epoch, loss.data.item()))



The loss decreases from ~68.88 to ~30.26

and the resulting fitting is this:

xxt = torch.arange(5,23)
a = net(xxt.reshape(-1,1).float())
plt.scatter(X[:,1], y, s=30, c='r', marker='x', linewidths=1)


What is the problem here?

• Check if the yt values are being altered by the gradient. Could be the case. Commented May 3, 2019 at 15:56
• I checked them, they are not being altered. Commented May 3, 2019 at 16:24
• Can you post the part of the code where you plot as well Commented May 3, 2019 at 16:30
• @AndyM, I have updated the code. Commented May 3, 2019 at 16:37

I know this doesn't answer you 100%, but maybe it helps. Feel free to disregard.

I just implemented this in keras and didn't have any issues

model = Sequential([
Dense(1,activation='linear',input_shape=(1,))
])
loss='mean_squared_error',
metrics=['mae'])
model.fit(test_df.x, test_df.y, validation_split = .2,epochs=500, batch_size=1)

plt.scatter(test_df.x,test_df.y,s=30,c='r',marker='x')
x = np.linspace(4,25,100)
y = model.get_weights()[0][0][0]*x+model.get_weights()[1][0]
plt.plot(x, y)
plt.ylim([-5,50])
plt.xlim([4.5,23])



Is it possible you read your data in incorrectly maybe? Or maybe you are referencing it incorrectly when you set your Xt and yt?

• Thanks Andy, but my issue is only limited to PyTorch. Other frameworks and even implementing linear regression from scratch causes no issue. It would only help if you solve it using PyTorch. Commented May 4, 2019 at 4:50

The problem is with the optimizer. You used Adam optimizer which is mainly for large neural networks. What you need is simple stochastic gradient descent SGD.

import torch
from torch import nn
import numpy as np

class Linear_Reg(nn.Module):
def __init__(self):
super(Linear_Reg, self).__init__()
self.linear = nn.Linear(1,1)

def forward(self, x):
y_pred = self.linear(x)
return y_pred

net = Linear_Reg()

Xt = Variable(torch.Tensor(X))
yt = Variable(torch.Tensor(y))
Xt = Xt.view(-1,1)

criterion = nn.MSELoss()
optimizer = SGD(net.parameters(), lr=0.001)

EPOCHS = 100
for epoch in range(EPOCHS):