0
$\begingroup$

I want to divide my time series dataset into training and test sets. The data is seasonal and very noisy. When I randomly split, the test and train samples do not resemble in their distributions. Sometimes, train data get most of the noisy/peak points and sometimes these points go to test data. Can I split the data into train/test by calculating how much information the two sets hold. The information can be quantified by e.g. Shannon entropy or standard deviation etc. When I tried to search literature around this, I could not find any reference?

$\endgroup$
1
  • $\begingroup$ If you are dealing with time series dataset, would it be nature that you split the data using a certain point on the time line (before which will be the training data, and after which will be the validation/test data)? If there are seasonal trends, you may want to make sure that your training data covers a full cycle. $\endgroup$
    – Albert
    Jul 15, 2021 at 11:19

1 Answer 1

0
$\begingroup$

In theory, the very first thing to do should be fixing noises and seasonality. There are various ways of approaching the noises. If possible, try to understand the reason for noisy samples and extract the noise part from them manually. Or, use the smoothing algorithms to minimize the noises. In addition, you can create a dummy predictor variable that will be accountable for the noise part in the data. In time-series data, you can manually put 1 for the dates that you consider as noise and 0 for normal data points. You can read more about dummy variables here. However, it might be unhealthy to use dummy variables for improper cases.

To overcome seasonality, there are some powerful approaches, for example, X-13ARIMA-SEATS is a good method of extracting the seasonality from the time series.

If you want to do a split on the strength of some metric, you might encounter different possible scenarios. For example, if you would use some metric to split the data like variance or information then your test data would be very dissimilar compared to your train data. Thus, the test set would perform poorly since its characteristic is not like the train data. In other words, you might not objectively assess the performance of the model.

Lastly, as Albert mentioned in the comments, if you do a time-series prediction, then it would be better to split the data using the time. In other words, n consecutive samples would be your train data and m consecutive samples would be your test data.

$\endgroup$
1
  • $\begingroup$ Regarding your 3rd para: The way I am suggesting is not by maximizing the information in training set but by reducing the absolute difference between information that is contained in the training and test sets. For example if I split the whole data such that the difference in entropy of training and test is minimum. Does that make sense? Is there any literature around that? I understand the point you are making regarding seasonality/preprocessing etc. $\endgroup$ Jul 15, 2021 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.