I am attempting to mirror a machine learning program by Ahmed Besbes, but scaled up for multi-label classification. It seems that any attempt to stratify the data returns the following error:
The least populated class in y has only 1 member, which is too few. The minimum number of labels for any class cannot be less than 2.
In my data set, I have 1 column which contains clean, tokenized text. The other 8 columns are for the classifications based on the content of that text. Just to note, column 1 - 4 have significantly more samples than 5 - 8 (more obscure classifications derived from the text).
Here is a generic sample from my code:
x = data['cleaned_text'] y = data[['car','truck','ford','chevy','black','white','parked', 'driving']] x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.1, random_state=42) print(x_train.shape, x_test.shape, y_train.shape, y_test.shape)
(6293,) (700,) (6293, 8) (700, 8)
train_test_split returns the error previously mentioned. Even when I limit y to just one column, I still get the error.
How can I stratify the data so that I give the program a fair look in the training set?