What is and why use blocked cross-validation?

Question

I was reading about cross validation equivalents for time series data and found a variation called blocked cross validation. On the page I was reading it says the following:

"However, this may introduce leakage from future data to the model. The model will observe future patterns to forecast and try to memorize them. That’s why blocked cross-validation was introduced.(...) The second (margin) is between the folds used at each iteration in order to prevent the model from memorizing patterns from an iteration to the next."

From my understanding, iterations are independent, thus, the model is learned from scratch in each iteration. How can the model "memorize" patterns from one iteration to the next?

The first graph represents the problematic approach and the second represents the blocked cross validation solution.

Answer 1

Blocked time series cross-validation is very much like traditional cross-validation. As you know CV, takes a portion of the dataset and sets it aside only for testing purposes. The data can be taken from any part of the original data, beginning, middle, end, etc. It does not matter where because you assume the variance is the same throughout.

But since the time series data IS changing in some way, you cannot control one needs to chop it up finely. This keeps a slice of time (and its variables) held to one smaller section, hopefully where the variance or whatever does not change appreciably.

With the time-series-split that is shown above, one is using parts of the data for both purposes, training, and testing. Consider the first cv-iteration (#0), half is used for testing and half for training, ok fine. But when you move forward to CV-#2, one is now using The testing block (let's say it's red) over AGAIN in the next blue training set. Using different and growing pieces over and over again. Eventually using almost all of the data for training AND testing.