2
$\begingroup$

I'm doing a classification of time sequenced sensor data (in Python), where I'm segmenting the sensordata into samples, with a certain window-length (e.g. 3 seconds). However, the samples are also overlapping each other. For example, the first sample is 0s $\rightarrow$ 3s, second is 2.7s $\rightarrow$ 5.7s...

I'm wondering now, how I can do a proper train test split for this samples. Right now, I see two ways to do this:

  1. Split the samples without shuffling them first and drop the sample "at the border", to avoid overlap between train and test data. However, this is not optimal because I would like to have samples from all over the dataset for both the training and the test set. Otherwise I would just test the classifier on a sequence of the data, that might be very different from the sequence used for training.

  2. Shuffle the samples first and then split them into training in test. This would result in an overlap between training and test data and would therefore produce overly optimistic results.

Does anyone have another idea how this could be done?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
1
$\begingroup$

You can shuffle the samples first with a serial number attached to all the samples. Now, randomly select p% of the samples as test data and leave the (100-p)% samples as the training data. During this random selection, make sure that if a sample of serial #n has been selected as test, then #n-1 & #n+1 are not picked.

In that way,

  1. You are not testing the classifier on a sequence of the data.
  2. Avoid overlap between train & test samples.
Improve this answer
$\endgroup$
0
$\begingroup$

Split the sample in odd and even in the order of the start timestamp.

There is no overlap between the two groups, as the only overlap is between the odd and even samples.

You may take the train data and the test data from one of the group without any disjunct data.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Yes that makes sense. But wouldn't I only be able to use one of the groups (odd or even) for the train test split then? If I would take e.g. the odd samples for training and the even samples for test, then I would have an overlap. But if I would only use the odd samples, and split this group into 80% training and 20% test, then I wouldn't use the other group at all (the even ones in this case) - which would also be a drawback $\endgroup$
    – Frank
    Commented Jun 26, 2018 at 13:53
  • $\begingroup$ Correct, of course. This simple solution reduces the usable data to nearly a half. I updated the answert in this direction, but agree that this is possible suboptimal. $\endgroup$
    – Marmite Bomber
    Commented Jun 26, 2018 at 14:09
  • $\begingroup$ @Frank you mean like: odds for train and even for test (or viceversa)? $\endgroup$
    – Francesco Pegoraro
    Commented Sep 24, 2018 at 22:27
0
$\begingroup$

I would recommend you look into oversampling as a possible means of balancing your dataset. If there is unavoidable overlap (which it sounds like there is) then you can create a single dataset from all of the data, representing each measurement interval equally, bias is reduced from overlapping moments.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.