0
$\begingroup$

I have created a classification model for predicting data, and the problem is that the two classes are highly imbalanced

I have a problem. I have created a classification model for predicting data, and the problem is that the two classes are highly imbalanced. So, I dealt with it using the SMOTE+ENN technique. I applied SMOTE+ENN before splitting the data into training and test sets. The reason is that SMOTE generates synthetic data to balance the classes. I thought that performing SMOTE+ENN before splitting the data would create a representative state for the data

Currently, I am conducting research for a journal article, and I am unable to modify the model. The only thing I can do is to provide supporting research or reasoning as to why SMOTE+ENN is performed before splitting the training and test data. Can you please help me with some supporting arguments or rationales for this approach

Example:Can I provide the following rationale: "Performing SMOTE+ENN before splitting the data can still be effective because it aims to create a more balanced situation in the dataset by generating synthetic data through SMOTE that resembles the original data but with different statistical values. This means that there will be new data points introduced. At the same time, ENN helps reduce the redundancy of samples close to the minority class. I have also set the parameter to increase the data by only 10% and decrease it by 10%, which is a minimal change. Therefore, the model's performance remains relatively unchanged, and the interpretation of model evaluation only slightly varies."

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

You should not apply SMOTE-ENN before splitting. It has two big issues:

  1. Adding synthetic data in the test set will change the distribution of the data and the metrics you measure will not be representative of the true distribution.

  2. It will introduce a data leak. The fact that SMOTE-ENN will create data based on the entire dataset means that the training data of the model includes information about the test data.

So I would not try to rationalize it and try to fix the issue instead.

Improve this answer
$\endgroup$
7
  • $\begingroup$ I set the parameters for SMOTE+ENN as sampling_strategy=0.1/1. This means that the positive class was increased from 3,907 to 17,076, while the negative class was reduced from 248,023 to 228,854. I chose these parameters to avoid generating too many synthetic data points. The reason for using SMOTE+ENN with a sampling_strategy of 0.1/1 is to address the class imbalance issue and prepare the data for model training. It is as if we are seeking additional data points, but in reality, we are using SMOTE to generate synthetic data and reducing complexity using ENN. $\endgroup$
    – Hello is me
    Commented Jun 2, 2023 at 12:50
  • $\begingroup$ I hope this explanation provides a reasonable rationale for my approach. $\endgroup$
    – Hello is me
    Commented Jun 2, 2023 at 12:50
  • $\begingroup$ I understand and that makes sense. However, that should be confined to the training set. $\endgroup$
    – Simon Larsson
    Commented Jun 2, 2023 at 12:55
  • $\begingroup$ I have tried applying the SMOTE+ENN technique only to the training set and found that the precision and recall values were not as effective as expected. However, when I applied the aforementioned steps to the entire process, the precision and recall values improved. $\endgroup$
    – Hello is me
    Commented Jun 2, 2023 at 13:16
  • $\begingroup$ Unfortunately, I am unable to make any further modifications as the model has already been deployed in real-world applications. What I can do is provide a rationale for my actions and explain the steps I took to address the class imbalance issue using SMOTE+ENN. Do you think my reasoning is sufficient, or do you have any other suggestions or relevant research papers that could assist me? I'm at a loss and would appreciate your help. $\endgroup$
    – Hello is me
    Commented Jun 2, 2023 at 13:16
1
$\begingroup$

Unfortunately, we cannot really find a suitable reasoning, because the process is faulty. This is a common misconception in imbalanced data, however. Resampling methods should only be applied to the training partition, the test set must remain untouched and unseen until final validation. Take a look at this paper, it explains the issue thoroughly and inclusively evaluates the effects of doing the split before and after (including using SMOTE-ENN).enter image description here

Improve this answer
$\endgroup$
1
  • $\begingroup$ I understand everything you said, but the model cannot be modified anymore. So, I am looking for the best possible reasoning to present to my instructor. Do you think the aforementioned reasons can be considered good enough, or could you suggest better reasons to help me successfully complete this project? I kindly request your assistance please $\endgroup$
    – Hello is me
    Commented Jun 2, 2023 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.