0
$\begingroup$

I'm working on an audio classification experiment. In my original database, I have 1,412 records. To improve the performance of my models, I resorted to data augmentation, applying simple techniques such as noise addition and pitch reduction.

After this step, my dataset was 7060, including the original 1412 records. My big question is: how to proceed with the classification experiments now, considering that the results are better, however, I don't know if because of having original data and copies there is an "inflation" in the results.

Does anyone know what the best strategy might be for this situation? Is there any way to prevent copies and originals from being in different sets (training and testing)?

Sorry for the naive question.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Data augmentation must be done after splitting your data into training and testing sets to avoid data leakage.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I understand. But in my case, I have music files and I have copies generated and features extracted. In that case, what would you do to prevent the leak? $\endgroup$
    – Paulo Sergio Moreira
    Commented Jan 25, 2023 at 13:49
  • $\begingroup$ Don't you have any way of tracking the original files (e.g. based on file name)? Also, are those exact copies? $\endgroup$
    – noe
    Commented Jan 25, 2023 at 14:16
  • $\begingroup$ I have the file ID. Regarding copies, they are noise files, tone reduction, among other features. Unfortunately, I don't have so much programming ability to search and divide the data to avoid leakage. Do you have any solution so I can base myself? $\endgroup$
    – Paulo Sergio Moreira
    Commented Jan 25, 2023 at 15:37
  • 1
    $\begingroup$ You could post your specific programming problem in StackOverflow, which is more focused in programming matters. $\endgroup$
    – noe
    Commented Jan 25, 2023 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.