Creating quality data with sklearn.datasets.make_classification

Question

I'm doing some experiments on some svm kernel methods. My methodology for comparing those is having some multi-class and binary classification problems, and also, in each group, having some examples of p > n, n > p and p == n. However, finding some examples (5 or so for each of those subgroups) is really hard, so I want to generate them with sklearn. Said so, I don't know how to do it in a consistent and realistic way. I can generate the datasets, but I don't know which parameters set to which values for my purpose.

So basically my question is if there is a metodological way to perform this generation of datasets, and if so, which is.

Answer 1

---

A comprehensive guide on generating artificial datasets for testing purposes!!

---

1.) Blobs Classification Problem

The make_blobs() function can be used to generate blobs of points with a Gaussian distribution. You can control how many blobs to generate and the number of samples to generate, as well as a host of other properties.

The problem is suitable for linear classification problems given the linearly separable nature of the blobs. The example below generates a 2D dataset of samples with three blobs as a multi-class classification prediction problem. Each observation has two inputs and 0, 1, or 2 class values.

from sklearn.datasets import make_blobs
from matplotlib import pyplot
from pandas import DataFrame

 # generate 2d classification dataset
 # n_samples is the no of points, n_features is the no of features for each 
   sample and centers is the number of centers to generate, or the fixed center 
   locations
 X, y = make_blobs(n_samples=100, centers=3, n_features=2)
 
 # scatter plot, dots colored by class value
 df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
 colors = {0:'red', 1:'blue', 2:'green'}
 fig, ax = pyplot.subplots()
 grouped = df.groupby('label')
 for key, group in grouped:
    group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, 
               color=colors[key])
 pyplot.show()

Running the example generates the inputs and outputs for the problem and then creates a handy 2D plot showing points for the different classes using different colors.

2.) Moons Classification Problem

The make_moons() function is for binary classification and will generate a swirl pattern, or two moons.You can control how noisy the moon shapes are and the number of samples to generate. This test problem is suitable for algorithms that are capable of learning nonlinear class boundaries.

The example below generates a moon dataset with moderate noise.

from sklearn.datasets import make_moons
from matplotlib import pyplot
from pandas import DataFrame

# generate 2d classification dataset
X, y = make_moons(n_samples=100, noise=0.1)

# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
   group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()

3.) Circles Classification Problem The make_circles() function generates a binary classification problem with datasets that fall into concentric circles. Again, as with the moons test problem, you can control the amount of noise in the shapes.

This test problem is suitable for algorithms that can learn complex non-linear manifolds.

The example below generates a circles dataset with some noise.

from sklearn.datasets import make_circles
from matplotlib import pyplot
from pandas import DataFrame

# generate 2d classification dataset
X, y = make_circles(n_samples=100, noise=0.05)

# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
   group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()

4.) Imbalanced datasets

The make_classification function can be used to generate a random n-class classification problem. This initially creates clusters of points normally distributed (std=1) about vertices of an n_informative-dimensional hypercube with sides of length 2*class_sep and assigns an equal number of clusters to each class. It introduces interdependence between these features and adds various types of further noise to the data.

The complete example of defining the dataset and performing random oversampling (just one of the many methods) to balance the class distribution is listed below.

from collections import Counter
from sklearn.datasets import make_classification
from imblearn.over_sampling import RandomOverSampler

# define dataset
# here n_samples is the no of samples you want, weights is the magnitude of 
# imbalance you want in your data, n_classes is the no of output classes 
# you want and flip_y is the fraction of samples whose class is assigned 
# randomly. Larger values introduce noise in the labels and make the 
# classification task harder  
X, y = make_classification(n_samples=10000, weights=[0.99], n_classes = 2, 
                           flip_y=0)

# summarize class distribution
print(Counter(y))

# define oversampling strategy
oversample = RandomOverSampler(sampling_strategy='minority')

# fit and apply the transform
X_over, y_over = oversample.fit_resample(X, y)

# summarize class distribution
print(Counter(y_over))

Running the example first creates the dataset, then summarizes the class distribution. We can see that there are nearly 10K examples in the majority class and 100 examples in the minority class.

Then the random oversample transform is defined to balance the minority class, then fit and applied to the dataset. The class distribution for the transformed dataset is reported showing that now the minority class has the same number of examples as the majority class.

Before oversampling Counter({0:9900, 1:100})

After oversampling Counter({0:9900, 1:9900})