1
$\begingroup$

I am trying to make a model to classify whether these patients can be diagnosed with dementia by their 35 days of biometric data.

A brief summary of a dataset is below.

as an input X_train data, it has 51 features and 148 user IDs. And collect 35 days of biometric data from patients. enter image description here

as an input y_train data, it has a list of 148 patient's diagnoses. It can be 'CN' 'MCI' and 'Dem', so it has 3 categories.

['CN',
 'MCI',
 'MCI',
 'CN',
 'MCI',
 'CN',
 'MCI',
...

So I made a model something like this.

model = Sequential()
model.add(LSTM(35,input_shape=(35, 51)))  #35 days, 51 features
model.add(Dropout(0.5))
model.add(Dense(100, activation='relu'))
model.add(Dense(3, activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

Which accuracy is really not that great, as you can see below it does not even pass validation accuracy 50%.

...
Epoch 200/200
2/2 [==============================] - 0s 131ms/step - loss: 0.0609 - sparse_categorical_accuracy: 0.9835 - val_loss: 3.2440 - val_sparse_categorical_accuracy: 0.4667

Is there any way to improve this model? Or should I have to rebuild the model?

$\endgroup$
0
$\begingroup$

It looks like a classic OVER-FITTING problem, the performances on the train set are really good and the performances on the validation set is lacking. You can plot a training graph that displays the train and validation loss and accuracy of each epoch. Example for such one: link

history = model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, verbose=0)
print(history.history.keys())
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

Another easy way to reduce the over-fitting is to reduce your model computational power/number or trained parameters, by reducing the number of neurons at each layer of your model. For example:

model = Sequential()
model.add(LSTM(35,input_shape=(35, 51)))  #35 days, 51 features
model.add(Dropout(0.5))
model.add(Dense(32, activation='relu'))
model.add(Dense(3, activation='softmax'))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.