1
$\begingroup$

I want to create a model with LSTM to predict a user the next purchase value. For this I have used I used a user's purchase history. I have created the model and it works well, but honestly, I don't know I do the Train/Test split on the proper way or not.

To do this, I have used (univariate) user's purchase history.(X-purchase history values, y-target purchase value) As a first step, I have created a sliding-window process that creates new data. (As you can see in the pic) In the original dataset, I had 1000 users with 2820 timestamps and 1 feature (purchase values), with the Sliding-window process I got 1000*2320 users with 500 timestamps and 1 feature.

X.shape -> OriginalDataShape (1000, 2820, 1)

X.shape -> ModifiedDataShape (2320000‬, 500, 1) enter image description here

# model
model = Sequential()
model.add(LSTM(50, activation='relu', input_shape=(500,1))) model.add(Dense(1)) model.compile(loss='mae', optimizer='adam')


# train
repeats = range(3)
scores = list()
for i in repeats:
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)
    model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=5, batch_size=32)
    pred = model.predict(X_test)
    score.append(metrics.mean_squared_error(pred,y_test))
print('Final score (Mse):')
print(score)

enter image description here

My questions are: It is the proper way or not? If it is not, do you have a suggestion or GitHub link for the solution?

$\endgroup$
4
$\begingroup$

The problem here is that you're shuffling the time-series before splitting it.
This way, every time-step in the test set might have a time-step close to it in the train set.

To avoid this, you can set shuffle=False in train_test_split (so that the train set is before the test set), or use Group K-Fold with the date as the group (so whole days are either in the train or test set).

You can read more in this question in Cross Validated

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.