3
$\begingroup$

My problem is that in PyTorch I cannot reproduce the MSE loss that I have achieved in Keras.

I have trained the following model in Keras:

from keras.models import Sequential
from keras.layers import Dense

model = Sequential()
model.add(Dense(10))
model.add(Dense(1))
model.compile(optimizer = "adam", loss = "mean_squared_error")
model.fit(X_train, y_train,
      batch_size = 32,
      epochs = 200
     )

The shape of the training data is:

print(X_train.shape)
>>>(3550, 10)
print(y_train.shape)
>>>(3550,)

After training the MSE is ~0.15:

mse_train = model.evaluate(X_train, y_train)
>>>3550/3550 [==============================] - 0s 18us/step
print("Train MSE: ", mse_train)
>>>Train MSE:  0.1499910642017781

Then I initialize the same model in PyTorch:

import torch
import torch.nn as nn

class NN(nn.Module):
 def __init__(self):
  super(NN, self).__init__()
  self.dense1 = nn.Linear(10, 10)
  self.dense2 = nn.Linear(10, 1)

 def forward(self, x):
  out = self.dense1(x)
  out = self.dense2(out)
  return out

net = NN()
criterion = nn.MSELoss()

And assign the weights I have achieved in Keras:

from keras.models import load_model
keras_model = load_model(MODEL_PATH)

dense_weights = keras_model.layers[0].get_weights()
weights = torch.tensor(dense_weights[0].swapaxes(0,1))
bias = torch.tensor(dense_weights[1])
net.dense1.weight.data = weights
net.dense1.bias.data = bias

dense_weights = keras_model.layers[1].get_weights()
weights = torch.tensor(dense_weights[0].swapaxes(0,1))
bias = torch.tensor(dense_weights[1])
net.dense2.weight.data = weights
net.dense2.bias.data = bias

Now I try to calculate the MSE loss:

X_train_torch = torch.tensor(X_train, dtype=torch.float)
y_train_torch = torch.tensor(y_train, dtype=torch.float)

outputs = net(X_train_torch)
loss = criterion(outputs, y_train_torch)
print("Train loss: ", loss)
>>>Train loss:  0.338391376896338

The MSE is now ~0.34 and is twice as big as calculated in Keras.

What could be the reason? Is there a bug in my calculation?

$\endgroup$
0
$\begingroup$

The problem was that outputs and y_train_torch had different shapes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.