Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am recently working on an imbalanced binary classification problem where the data is time ordered. I would like to validate my model using training/validation splits that have the same imbalance ratio (similar to the stratified kfold of Scikit-learn)
I tried the timeseriesSplit of Scikit-learn. However, their implementation does not guarantee that both classes exist with a similar imbalance ratio over the training and data splits. Is there a way to do that?
their implementation does not guarantee that both classes exist with a similar imbalance ratio over the training and data splits. Is there a way to do that?
First I would check how the class ratio varies over time. If it does not vary significantly, you don't necessarily need to worry about stratification.
If it does, the way to properly split the data depends on the specifics of what you are trying to do.
n_splits resulted in a situation where each of them is all 0s or all 1s. Before you can decide on a split strategy, I would try to understand why the target variable is so seasonal. Once you do, you might be able to find the right features to use and the right split to ensure your cross-validation makes sense.
$\endgroup$
Mar 9 at 23:21