How would you build a Supervised Learning model that predicts the next number in the sequence?

I'm trying to build a toy supervised learning model in order to understand it better but I'm making an error somewhere. I know this model doesn't make practical sense, but it should be possible to do.

The dataset is every increment of 0.01 from [0, 1], for both $$x$$ and $$y$$. So if $$x_i$$ is 0.01, $$y_i$$ is also 0.01. If $$x_i$$ is 0.71, $$y_i$$ is also 0.71. etc. So the task of the model would be to learn the pattern, which is; input a number and output the same number.

The feature and label is going into a dense layer with 10 neurons with a ReLU activation function, which is then going into a dense layer with 1 neuron with a ReLU activation function, because I want to predict 1 positive number.

It's not working. I have tried different things, changing the layers, neurons, activation functions, optimizer, etc., but I can't get it to work. So obviously something is wrong with my model. How would you build a model to do this?

       X     Y
0   0.01  0.01
1   0.02  0.02
2   0.03  0.03
3   0.04  0.04
4   0.05  0.05
..   ...   ...
95  0.96  0.96
96  0.97  0.97
97  0.98  0.98
98  0.99  0.99
99  1.00  1.00

import numpy as np
import pandas as pd
from math import floor
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Input, Dense

data = {
"X": [(i+1)*0.01 for i in range(100)],
"Y": [(i+1)*0.01 for i in range(100)]
}
df = pd.DataFrame(data)

X = df[['X']] # double brackets so dataframe can be directly input
Y = df[['Y']]

# seperate into train / test split
train_split = 0.7

train_size = floor(len(X) * train_split)
test_size = len(X) - train_size

X_train = X[0:train_size]
X_test = X[train_size:]

Y_train = Y[0:train_size]
Y_test = Y[train_size:]

# model
model = Sequential()

activation='relu'))

activation='relu'))

model.compile(loss='mse',
metrics=['mse'])

model.summary()

# train
history = model.fit(x=X_train,
y=Y_train,
epochs=100,
validation_split=0.5,
verbose=0)

# metrics
plt.plot(history.history['loss'], label='Training loss')
plt.plot(history.history['val_loss'], label='Validation loss')
plt.legend()

# make prediction
input_sample = X_test[1:1 + 1]
print(input_sample)
prediction = model.predict(input_sample, verbose=0)
print(prediction)


• It worked! I changed it to 1 layer with 1 neuron, added shuffle=True, changed the validation to validation_split=0.1, and trained it for 200 epochs and it's now predicting correctly. Thank you so much for answering. Commented Aug 6, 2023 at 16:48