I am playing around with a deterministic function in order to understand machine learning as in this tutorial blog.
The program I am using a deterministic function $y = f(x)$ where $f(x) = x^2$. I get a beautiful plot with the ($x$, predicted $f(x)$) and ($x$, $f(x)$). $x$ and $f(x)$ are scaled using MinMaxScaler. It follows that the predicted $f(x)$ from the model is scaled.
So I would like to rescale the predicted $f(x)$ to its normal. Here is the cut-down code.
Summary: plots beautifully when the value is scaled, BUT when the data is re-scaled, I do not understand why the re-scaled values of $\hat{y} = predicted \ f(x)$ are the same.
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense
from numpy import asarray
from matplotlib import pyplot
# define data
x = asarray([i for i in range(1000)]); # This is x UNSCALED
y = asarray([a**2 for a in x]); # This is f(x) = x**2 UNSCALED
# reshape into rows and cols
x = x.reshape((len(x), 1)); # This is x UNSCALED
y = y.reshape((len(y), 1)); # This is f(x) unscaled
# scale data
x_s = MinMaxScaler()
x = x_s.fit_transform(x) # This is x SCALED
y_s = MinMaxScaler()
y = y_s.fit_transform(y) # This is f(x) SCALED - come back to it.
# fit a model
model = Sequential()
model.add(Dense(300, input_dim=1, activation='relu'));
model.add(Dense(1))
#compile a model
model.compile(loss='mse', optimizer='adam')
#fit a model
model.fit(x, y, epochs=277, batch_size=200, verbose=0)
mse = model.evaluate(x, y, verbose=0)
print("the value of the mse")
print(mse)
# predict
yhat = model.predict(x); #This is 'scaled' predicted value
# plot real vs predicted - the plots look beautiful
pyplot.plot(x,y,label='y')
pyplot.plot(x,yhat,label='yhat')
pyplot.legend()
print("the graph is printed on another window")
pyplot.show()
print("Printing the output of the scaled values of yhat, f(x) and x")
print("printing the first 10")
for i in range(10):
print(yhat[i],y[i],x[i])
print("printing the 10th to 20th")
for i in range(10):
print(yhat[i+10],y[i+10],x[i+10])
print("Printing the output of the unscaled values")
y_predicted = y_s.inverse_transform(yhat); # NOTE HERE using original UNSCALED y = f(x)'s y_s
y_expected = y_s.inverse_transform(y)
x_original = x_s.inverse_transform(x)
#print(y_predicted[0:10,].tolist(), x_original[0:10,].tolist())
print("Printing the first 10, predicted, expected, and x")
for i in range(10):
print(y_predicted[i], y_expected[i], x_original[i])
print("let's try some other arbitrary section, say 10:20")
#print(y_predicted[9:21,].tolist(),y_predicted[9:21,].tolist(), x_original[9:21,].tolist())
print("printing 10th to 20th, predicted, expected, and x")
for i in range(10):
print(y_predicted[i+10],y_expected[i+10], x_original[i+10])
But I don't know why the rescaled yhat to the original scale produces this result
Printing the output of the unscaled/re-scaled values Printing the first 10, predicted, expected, and x [1171.0186] [0.] [0.] [1171.0186] [1.] [1.] [1171.0186] [4.] [2.] [1171.0186] [9.] [3.] [1171.0186] [16.] [4.] [1171.0186] [25.] [5.] [1171.0186] [36.] [6.] [1171.0186] [49.] [7.] [1171.0186] [64.] [8.] [1171.0186] [81.] [9.] let's try some other arbitrary section, say 10:20 printing 10th to 20th, predicted, expected, and x [1171.0186] [100.] [10.] [1171.0186] [121.] [11.] [1171.0186] [144.] [12.] [1171.0186] [169.] [13.] [1171.0186] [196.] [14.] [1171.0186] [225.] [15.] [1171.0186] [256.] [16.] [1171.0186] [289.] [17.] [1171.0186] [324.] [18.] [1171.0186] [361.] [19.]
I would have expected the left-most column, which is the rescaled yhat = predicted f(x) to approximate the 2nd column, BUT ALL THE VALUES IN THE LEFT-MOST COLUMN ARE THE SAME.
I want to clarify that when I print the scaled values, the values of yhat(scaled) = predicted are all the same. So something is happening at the scaled value of yhat such that they're all the same. printing the first few scaled yhat, f(x) and x. Where they are displayed in groups of 3. Cannot use and in comments. [0.00117336] [0.] [0.], [0.00117336] [1.002003e-06] [0.001001], [0.00117336] [4.00801202e-06] [0.002002], [0.00117336] [9.01802704e-06] [0.003003],
I played around again with the experiment to determine the accuracy of deep learning being able to predict a deterministic function without using the formula y = f(x) = x**2.
I have an improved result, but need to work out why the estimated values, say the first five estimate predicted values seem out of place.
Yet the plots of (x,y) and (x,yhat) seemed to look quite close.
Here is the code to replicate the problem. I left out the import statements.
x = x.reshape((len(x),1))
y = y.reshape((len(y),1))
x_s = MinMaxScaler()
y_s = MinMaxScaler()
x_scaled = x_s.fit_transform(x)
y_scaled = y_s.fit_transform(y)
model = Sequential()
model.add(Dense(100,input_dim=1,activation='relu'))
model.add(Dense(1))
model.compile(loss='mse',optimizer='adam')
model.fit(x_scaled,y_scaled, epochs=100, batch_size=10,verbose=0)
mse = model.evaluate(x_scaled, y_scaled,verbose=0)
mse
1.0475558547113905e-05
yhat = model.predict(x_scaled)
yhat_original = y_s.inverse_transform(yhat)
#First five of yhat_original (yhat rescaled)
yhat_original[:5].T
array([[11.835742, 11.835742, 11.835742, 11.835742, 11.835742]]
#compared to first original 5 elements of y = 0,1,4,9,16
#Last five of yhat_original (yhat rescaled)
yhat_original[-5:].T
array([[8985.839, 9154.454, 9323.067, 9491.684, 9660.3 ]
#compared to last original 5 elements of y = 9025, 9216, 9409, 9604, 9801
#Now determine the RMS of the predicted and original values
cum_sum = 0
for i in range(len(yhat_original)):
cum_sum+= (y[i]-yhat_original[i])**2/len(yhat_original)
mse = sqrt(cum_sum)
mse
array([31.72189417])
pyplot.plot(x,y,label='y')
pyplot.plot(x,yhat_original,label='estimated')
pyplot.legend()
Result: results were more accurate at the higher end but remained inaccurate at the lower end.
Any advice please. Thank you.
