1
$\begingroup$

I'm trying to build a toy supervised learning model in order to understand it better but I'm making an error somewhere. I know this model doesn't make practical sense, but it should be possible to do.

The dataset is every increment of 0.01 from [0, 1], for both $x$ and $y$. So if $x_i$ is 0.01, $y_i$ is also 0.01. If $x_i$ is 0.71, $y_i$ is also 0.71. etc. So the task of the model would be to learn the pattern, which is; input a number and output the same number.

The feature and label is going into a dense layer with 10 neurons with a ReLU activation function, which is then going into a dense layer with 1 neuron with a ReLU activation function, because I want to predict 1 positive number.

It's not working. I have tried different things, changing the layers, neurons, activation functions, optimizer, etc., but I can't get it to work. So obviously something is wrong with my model. How would you build a model to do this?

       X     Y
0   0.01  0.01
1   0.02  0.02
2   0.03  0.03
3   0.04  0.04
4   0.05  0.05
..   ...   ...
95  0.96  0.96
96  0.97  0.97
97  0.98  0.98
98  0.99  0.99
99  1.00  1.00
import numpy as np
import pandas as pd
from math import floor
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Input, Dense


data = {
    "X": [(i+1)*0.01 for i in range(100)],
    "Y": [(i+1)*0.01 for i in range(100)]
}
df = pd.DataFrame(data)

X = df[['X']] # double brackets so dataframe can be directly input
Y = df[['Y']]

# seperate into train / test split
train_split = 0.7

train_size = floor(len(X) * train_split)
test_size = len(X) - train_size

X_train = X[0:train_size]
X_test = X[train_size:]

Y_train = Y[0:train_size]
Y_test = Y[train_size:]

# model
model = Sequential()

model.add(Input(shape=1))

model.add(Dense(units=10,
                activation='relu'))

model.add(Dense(units=1,
                activation='relu'))

model.compile(loss='mse',
              optimizer='adam',
              metrics=['mse'])

model.summary()

# train
history = model.fit(x=X_train,
                    y=Y_train,
                    epochs=100,
                    validation_split=0.5,
                    verbose=0)

# metrics
plt.plot(history.history['loss'], label='Training loss')
plt.plot(history.history['val_loss'], label='Validation loss')
plt.legend()

# make prediction
input_sample = X_test[1:1 + 1]
print(input_sample)
prediction = model.predict(input_sample, verbose=0)
print(prediction)
$\endgroup$

1 Answer 1

1
$\begingroup$

welcome to the forum. A quick glance and it looks like there are 2 or 3 issues with what you're trying to do.

  • Your model is too complex. For the identity function, you can simply have a single layer with a single neuron and get better performance.
  • You should shuffle your inputs, i.e. pass shuffle=True to model fit
  • You probably don't want a validation split of 0.5. Without shuffling, this means that your network would see 0.01 to 0.49 every time, and not see the rest of the inputs ever. With shuffling, it's better, but withholding half of your training data for validation seems high. hth.
$\endgroup$
1
  • 1
    $\begingroup$ It worked! I changed it to 1 layer with 1 neuron, added shuffle=True, changed the validation to validation_split=0.1, and trained it for 200 epochs and it's now predicting correctly. Thank you so much for answering. $\endgroup$
    – mathbike
    Aug 6, 2023 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.