How would you build a Supervised Learning model that predicts the next number in the sequence?

Question

I'm trying to build a toy supervised learning model in order to understand it better but I'm making an error somewhere. I know this model doesn't make practical sense, but it should be possible to do.

The dataset is every increment of 0.01 from [0, 1], for both $x$ and $y$. So if $x_i$ is 0.01, $y_i$ is also 0.01. If $x_i$ is 0.71, $y_i$ is also 0.71. etc. So the task of the model would be to learn the pattern, which is; input a number and output the same number.

The feature and label is going into a dense layer with 10 neurons with a ReLU activation function, which is then going into a dense layer with 1 neuron with a ReLU activation function, because I want to predict 1 positive number.

It's not working. I have tried different things, changing the layers, neurons, activation functions, optimizer, etc., but I can't get it to work. So obviously something is wrong with my model. How would you build a model to do this?

       X     Y
0   0.01  0.01
1   0.02  0.02
2   0.03  0.03
3   0.04  0.04
4   0.05  0.05
..   ...   ...
95  0.96  0.96
96  0.97  0.97
97  0.98  0.98
98  0.99  0.99
99  1.00  1.00

import numpy as np
import pandas as pd
from math import floor
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Input, Dense


data = {
    "X": [(i+1)*0.01 for i in range(100)],
    "Y": [(i+1)*0.01 for i in range(100)]
}
df = pd.DataFrame(data)

X = df[['X']] # double brackets so dataframe can be directly input
Y = df[['Y']]

# seperate into train / test split
train_split = 0.7

train_size = floor(len(X) * train_split)
test_size = len(X) - train_size

X_train = X[0:train_size]
X_test = X[train_size:]

Y_train = Y[0:train_size]
Y_test = Y[train_size:]

# model
model = Sequential()

model.add(Input(shape=1))

model.add(Dense(units=10,
                activation='relu'))

model.add(Dense(units=1,
                activation='relu'))

model.compile(loss='mse',
              optimizer='adam',
              metrics=['mse'])

model.summary()

# train
history = model.fit(x=X_train,
                    y=Y_train,
                    epochs=100,
                    validation_split=0.5,
                    verbose=0)

# metrics
plt.plot(history.history['loss'], label='Training loss')
plt.plot(history.history['val_loss'], label='Validation loss')
plt.legend()

# make prediction
input_sample = X_test[1:1 + 1]
print(input_sample)
prediction = model.predict(input_sample, verbose=0)
print(prediction)

Answer 1

welcome to the forum. A quick glance and it looks like there are 2 or 3 issues with what you're trying to do.

• Your model is too complex. For the identity function, you can simply have a single layer with a single neuron and get better performance.
• You should shuffle your inputs, i.e. pass shuffle=True to model fit
• You probably don't want a validation split of 0.5. Without shuffling, this means that your network would see 0.01 to 0.49 every time, and not see the rest of the inputs ever. With shuffling, it's better, but withholding half of your training data for validation seems high. hth.