2
$\begingroup$

I'm doing a classification of time sequenced sensor data (in Python), where I'm segmenting the sensordata into samples, with a certain window-length (e.g. 3 seconds). However, the samples are also overlapping each other. For example, the first sample is 0s $\rightarrow$ 3s, second is 2.7s $\rightarrow$ 5.7s...

I'm wondering now, how I can do a proper train test split for this samples. Right now, I see two ways to do this:

  1. Split the samples without shuffling them first and drop the sample "at the border", to avoid overlap between train and test data. However, this is not optimal because I would like to have samples from all over the dataset for both the training and the test set. Otherwise I would just test the classifier on a sequence of the data, that might be very different from the sequence used for training.

  2. Shuffle the samples first and then split them into training in test. This would result in an overlap between training and test data and would therefore produce overly optimistic results.

Does anyone have another idea how this could be done?

$\endgroup$
1
$\begingroup$

You can shuffle the samples first with a serial number attached to all the samples. Now, randomly select p% of the samples as test data and leave the (100-p)% samples as the training data. During this random selection, make sure that if a sample of serial #n has been selected as test, then #n-1 & #n+1 are not picked.

In that way,

  1. You are not testing the classifier on a sequence of the data.
  2. Avoid overlap between train & test samples.
$\endgroup$
0
$\begingroup$

Split the sample in odd and even in the order of the start timestamp.

There is no overlap between the two groups, as the only overlap is between the odd and even samples.

You may take the train data and the test data from one of the group without any disjunct data.

$\endgroup$
  • $\begingroup$ Yes that makes sense. But wouldn't I only be able to use one of the groups (odd or even) for the train test split then? If I would take e.g. the odd samples for training and the even samples for test, then I would have an overlap. But if I would only use the odd samples, and split this group into 80% training and 20% test, then I wouldn't use the other group at all (the even ones in this case) - which would also be a drawback $\endgroup$ – Frank Jun 26 '18 at 13:53
  • $\begingroup$ Correct, of course. This simple solution reduces the usable data to nearly a half. I updated the answert in this direction, but agree that this is possible suboptimal. $\endgroup$ – Marmite Bomber Jun 26 '18 at 14:09
  • $\begingroup$ @Frank you mean like: odds for train and even for test (or viceversa)? $\endgroup$ – Francesco Pegoraro Sep 24 '18 at 22:27
0
$\begingroup$

I would recommend you look into oversampling as a possible means of balancing your dataset. If there is unavoidable overlap (which it sounds like there is) then you can create a single dataset from all of the data, representing each measurement interval equally, bias is reduced from overlapping moments.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.