0
$\begingroup$

I'm doing some experiments on some svm kernel methods. My methodology for comparing those is having some multi-class and binary classification problems, and also, in each group, having some examples of p > n, n > p and p == n. However, finding some examples (5 or so for each of those subgroups) is really hard, so I want to generate them with sklearn. Said so, I don't know how to do it in a consistent and realistic way. I can generate the datasets, but I don't know which parameters set to which values for my purpose.

So basically my question is if there is a metodological way to perform this generation of datasets, and if so, which is.

$\endgroup$
2
  • $\begingroup$ This is not that clear to me whet you need, but If I'm not wrong you are looking for a way to generate reliable syntactic data. If so you can use pypi.org/project/sdgym and I know this is not Scikit-learn as your requirement but might be a good start $\endgroup$ Sep 13 at 22:58
  • $\begingroup$ @JulioJesus Gonna check it, thanks. I need some way to generate synthetic data with some restriction about p and n, due to the fact that I don't have any datasets with those restrictions. I could just try to generate them with sklearn methods, but I don't think that is a "reliable" way for my benchmarking purposes. I want to know if there is a known method for this kind of problem. $\endgroup$
    – Norhther
    Sep 13 at 23:19
0
$\begingroup$

A comprehensive guide on generating artificial datasets for testing purposes!!


1.) Blobs Classification Problem

The make_blobs() function can be used to generate blobs of points with a Gaussian distribution. You can control how many blobs to generate and the number of samples to generate, as well as a host of other properties.

The problem is suitable for linear classification problems given the linearly separable nature of the blobs. The example below generates a 2D dataset of samples with three blobs as a multi-class classification prediction problem. Each observation has two inputs and 0, 1, or 2 class values.

from sklearn.datasets import make_blobs
from matplotlib import pyplot
from pandas import DataFrame

 # generate 2d classification dataset
 # n_samples is the no of points, n_features is the no of features for each 
   sample and centers is the number of centers to generate, or the fixed center 
   locations
 X, y = make_blobs(n_samples=100, centers=3, n_features=2)
 
 # scatter plot, dots colored by class value
 df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
 colors = {0:'red', 1:'blue', 2:'green'}
 fig, ax = pyplot.subplots()
 grouped = df.groupby('label')
 for key, group in grouped:
    group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, 
               color=colors[key])
 pyplot.show()

Running the example generates the inputs and outputs for the problem and then creates a handy 2D plot showing points for the different classes using different colors.

enter image description here

2.) Moons Classification Problem

The make_moons() function is for binary classification and will generate a swirl pattern, or two moons.You can control how noisy the moon shapes are and the number of samples to generate. This test problem is suitable for algorithms that are capable of learning nonlinear class boundaries.

The example below generates a moon dataset with moderate noise.

from sklearn.datasets import make_moons
from matplotlib import pyplot
from pandas import DataFrame

# generate 2d classification dataset
X, y = make_moons(n_samples=100, noise=0.1)

# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
   group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()

enter image description here

3.) Circles Classification Problem The make_circles() function generates a binary classification problem with datasets that fall into concentric circles. Again, as with the moons test problem, you can control the amount of noise in the shapes.

This test problem is suitable for algorithms that can learn complex non-linear manifolds.

The example below generates a circles dataset with some noise.

from sklearn.datasets import make_circles
from matplotlib import pyplot
from pandas import DataFrame

# generate 2d classification dataset
X, y = make_circles(n_samples=100, noise=0.05)

# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
   group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()

enter image description here

4.) Imbalanced datasets

The make_classification function can be used to generate a random n-class classification problem. This initially creates clusters of points normally distributed (std=1) about vertices of an n_informative-dimensional hypercube with sides of length 2*class_sep and assigns an equal number of clusters to each class. It introduces interdependence between these features and adds various types of further noise to the data.

The complete example of defining the dataset and performing random oversampling (just one of the many methods) to balance the class distribution is listed below.

from collections import Counter
from sklearn.datasets import make_classification
from imblearn.over_sampling import RandomOverSampler

# define dataset
# here n_samples is the no of samples you want, weights is the magnitude of 
# imbalance you want in your data, n_classes is the no of output classes 
# you want and flip_y is the fraction of samples whose class is assigned 
# randomly. Larger values introduce noise in the labels and make the 
# classification task harder  
X, y = make_classification(n_samples=10000, weights=[0.99], n_classes = 2, 
                           flip_y=0)

# summarize class distribution
print(Counter(y))

# define oversampling strategy
oversample = RandomOverSampler(sampling_strategy='minority')

# fit and apply the transform
X_over, y_over = oversample.fit_resample(X, y)

# summarize class distribution
print(Counter(y_over))

Running the example first creates the dataset, then summarizes the class distribution. We can see that there are nearly 10K examples in the majority class and 100 examples in the minority class.

Then the random oversample transform is defined to balance the minority class, then fit and applied to the dataset. The class distribution for the transformed dataset is reported showing that now the minority class has the same number of examples as the majority class.

Before oversampling Counter({0:9900, 1:100})

After oversampling Counter({0:9900, 1:9900})

$\endgroup$
3
  • $\begingroup$ I'm afraid this does not answer my question, on how to set realistic and reliable parameters for experimental data. This only gives some examples that can be found in the docs. $\endgroup$
    – Norhther
    Sep 13 at 17:55
  • $\begingroup$ @Norhther As I understand from the question you want to create binary and multiclass classification datasets with balanced and imbalanced classes right? You can do that using the make_classification function mentioned in point 4. $\endgroup$
    – spectre
    Sep 14 at 5:40
  • $\begingroup$ @Norhther you can generate imbalanced classes using the weights parameters and if you want multi class dataset, you can use n_classes parameter. $\endgroup$
    – spectre
    Sep 14 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.