1
$\begingroup$

Initial Information

I built a Neural Network Model (Logistic Regression) to classify Lung Cancer based on the patient's (user) symptoms

My dataset is kind of small (only about 276 data)

Here is the illustration for my dataset:

data.head()
GENDER AGE SMOKING YELLOW_FINGERS ANXIETY PEER_PRESSURE CHRONIC DISEASE FATIGUE ALLERGY WHEEZING ALCOHOL CONSUMING COUGHING SHORTNESS OF BREATH SWALLOWING DIFFICULTY CHEST PAIN LUNG_CANCER
M 69 1 2 2 1 1 2 1 2 2 2 2 2 2 YES
M 74 2 1 1 1 2 2 2 1 1 1 2 2 2 YES
F 59 1 1 1 2 1 2 1 2 1 2 2 1 2 NO
M 63 2 2 2 1 1 1 1 1 2 1 1 2 2 NO

Here's how I preprocessing the dataset:

  1. I drop the duplicate data
  2. I encode the [GENDER] and [LUNG CANCER] value
  3. I change the 2,1 value to 1,0
  4. I scale the age feature using StandardScaler()
  5. I resample the training set using RandomOverSampler().fit_resample

Here's my Neural Network Model:

model = Sequential(
    [
        Dense(3, activation = 'sigmoid', input_shape=[15]),
        Dense(1, activation = 'sigmoid'),
    ])

model.compile(
loss=tf.keras.losses.BinaryCrossentropy(),
optimizer=tf.keras.optimizers.Adam(learning_rate=0.01),
metrics=['accuracy'])

history = model.fit(X_train, y_train, 
                        epochs=50, batch_size=16,
                        validation_split=0.2, 
                        shuffle=True)

Here's the Training Result:

Model Accuracy Model Loss

Here's the test result:

test_loss, test_acc = model.evaluate(X_test, y_test)
print("Test loss:", test_loss)
print("Test accuracy:", test_acc)

3/3 [==============================] - 0s 4ms/step - loss: 0.1746 - accuracy: 0.9420

Test loss: 0.17462675273418427 Test accuracy: 0.9420289993286133

Here's the confusion matrix and classification report: Confusion Matrix

                precision    recall  f1-score   support

           0       0.98      0.95      0.97        60
           1       0.73      0.89      0.80         9

    accuracy                           0.94        69
   macro avg       0.86      0.92      0.88        69
weighted avg       0.95      0.94      0.94        69

My QUESTION:

  • Is my model result (accuracy) good enough considering I want to build a Cancer Detection app using this model?
  • If my model result (accuracy) is not good enough, how could I improve this model? What parameter should I tweak or maybe should I reduce or add the Neural layer?

Note: Please enlighten me, I'm kind of new to Machine Learning :)

$\endgroup$
2
  • $\begingroup$ Why don't you leave the model train for a longer time? From the loss curves it seems there's still room for improvement $\endgroup$
    – noe
    May 18, 2023 at 6:17
  • $\begingroup$ Thank you for the advice. I will try it :) $\endgroup$
    – Jonathan
    May 18, 2023 at 6:49

1 Answer 1

1
$\begingroup$
  1. It depends on the application. This is more of a business than technical question. Also, are you sure accuracy is a good metric for your application?

  2. Get more data, and try more different models; consider ensembling.

Do not spend more time to train or add more layers; it is already overfitting. Won't do you any good.

$\endgroup$
5
  • $\begingroup$ Hmm now that you said, I think Recall is a more suitable metric for my application. Thank you for the advice :) $\endgroup$
    – Jonathan
    May 18, 2023 at 6:53
  • $\begingroup$ Just Recall? It can get 100% Recall by guessing everyone has cancer. $\endgroup$
    – lpounng
    May 18, 2023 at 6:54
  • $\begingroup$ Hmm, precision too maybe? But I think recall metrics is a little bit more important than precision (because to avoid too many False Negative) for Cancer Detection (considering it's a rare disease) $\endgroup$
    – Jonathan
    May 18, 2023 at 7:01
  • $\begingroup$ May I ask something again? Should I split my training data (to have validation data) before resampling my training data? or It's okay to resample my training data first an then split it to have validation data? $\endgroup$
    – Jonathan
    May 18, 2023 at 7:08
  • $\begingroup$ I'd say before resampling, to make sure both test and validation set follow exactly same distribution as where they come from. But not 100% certain. $\endgroup$
    – lpounng
    May 18, 2023 at 7:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.