I'm trying to use CNNs on time series data (EEG), measured on different people. Each person has 10-20 recorded signals of different lengths and every subject has one global class assigned.
- Subject 01 has label 1 and 20 recorded EEG signals
- Subject 02 has label 0 and 7 recorded EEG signals
I've found an EEG specific CNN application where the signal is cut with fixed windows of 2-3 seconds and the label assigned is that of an event present in the signal, unlike what my problem is, having a global subject label. Thus I end up having up to 100-200 samples from the same subject with the same label.
My question is, does the fact that I train a model on many samples coming from the same subject, with the same label, and all samples are somehow related to each other, constitute a problem? I think I'm missing some theory here.
Another thing is, how should I split the data? By randomly splitting a training and test set, I end up having in the test set, data coming from a subject I've already trained the model on... or is that not an issue? Should I manually choose a couple of subjects to test on just to be safe?