Create a binary-classification dataset (python: sklearn.datasets.make_classification)

Question

I would like to create a dataset, however I need a little help. The dataset is completely fictional - everything is something I just made up.

Since the dataset is for a school project, it should be rather simple and manageable.

I would like a few features could be something like:

x1: temperature

x2: color

x3: moisture

and then I would have to classify with supervised learning whether the cocumber given the input data is eatable or not.

I am having a hard time understanding the documentation as there is a lot of new terms for me.

import sklearn.datasets as d           # Python #

a = d.make_classification(n_samples=100, n_features=3, n_informative=1, n_redundant=1, n_clusters_per_class=1)

print(a)

---

n_samples: 100 (seems like a good manageable amount)

n_features: 3 (3 is a good small number)

n_informative: 1 (from what I understood this is the covariance, in other words, the noise)

n_redundant: 1 (This is the same as "n_informative" ? - well, 1 seems like a good choice again)

n_clusters_per_class: 1 (forced to set as 1)

---

Would this be a good dataset that fits my needs? If not, how could I could I improve it?

Answer 1

Let's build some artificial data. There are many ways to do this. I usually always prefer to write my own little script that way I can better tailor the data according to my needs.

Let us first go through some basics about data. A lot of the time in nature you will find Gaussian distributions especially when discussing characteristics such as height, skin tone, weight, etc. Let us take advantage of this fact.

According to this article I found some 'optimum' ranges for cucumbers which we will use for this example dataset.

• Temperature: normally distributed, mean 14 and variance 3. If a value falls outside the range $[10,18]$ then it is said to be not edible.
• Color: we will set the color to be 80% of the time green (edible). 10% of the time yellow and 10% of the time purple (not edible).
• Moisture: normally distributed, mean 96, variance 2. If the moisture is outside the range $[94, 98]$ then the cucumber is not edible.

Building the dataset

We will build the dataset in a few different ways so you can see how the code can be simplified.

A very verbose example

This example will create the desired dataset but the code is very verbose.

import numpy as np

# Number of samples
n = 100

data = []
for i in range(n):
    temp = {}

    # Get a random normally distributed temperature mean=14 and variance=3
    temp.update({'temperature': np.random.normal(14, 3)})

    # Get a color with 80% probability green, 10% probability yellow
    # and 10% probability purple
    color = 'green'
    color_random_value = np.random.randint(0,10)
    if color_random_value == 8:
        color = 'yellow'
    elif color_random_value == 9:
        color = 'purple'
    temp.update({'color': color})

    # Get a random normally distributed moisture mean=96 and variance=2
    temp.update({'moisture': np.random.normal(96, 2)})

    # Verify if the instance is edible (label=0) or not (label=1)
    label = 0
    if temp['temperature'] < 10 or temp['temperature'] > 18:
        label = 1
    elif temp['color'] != 'green':
        label = 1
    elif temp['moisture'] < 94 or temp['moisture'] > 98:
        label = 1
    temp.update({'label': label})

    data.append(temp)

Then we can put this data into a pandas DataFrame as

df = pd.DataFrame(data=data)
df.head()

A cleaner example

import numpy as np

n = 100
data = {'temperature': np.random.normal(14, 3, n),
        'moisture': np.random.normal(96, 2, n),
        'color': np.random.choice(['green', 'yellow', 'purple'], 
                                  size=100, 
                                  p=[0.8, 0.1, 0.1])}
df = pd.DataFrame(data=data)

Then we will get the labels from our DataFrame

def get_label(color, moisture, temperature):
    if temperature < 10 or temperature > 18:
        return 1
    elif color != 'green':
        return 1
    elif moisture < 94 or moisture > 98:
        return 1
    return 0

df['label'] = df.apply(lambda row: get_label(row['color'], 
                                             row['moisture'], 
                                             row['temperature']), axis=1)

Getting the data ready for applying a classifier

One of our columns is a categorical value, this needs to be converted to a numerical value to be of use by us.

This can be achieved using

df['color_codes'] =df['color'].astype('category').cat.codes

Now we are ready to try some algorithms out and see what we get.

Visualizing the data

The first important step is to get a feel for your data such that we can try and decide what is the best algorithm based on its structure.

I prefer to work with numpy arrays personally so I will convert them

X = np.asarray(df[['color_codes', 'moisture', 'temperature']])
y = np.asarray(df['label'])

Let's plot the data in 3D

from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

ax.scatter(X[:,0], X[:,1], X[:,2], c=y)

ax.set_xlabel('Color')
ax.set_ylabel('Moisture')
ax.set_zlabel('Temperature')

plt.show()

The blue dots are the edible cucumber and the yellow dots are not edible. We can see that this data is not linearly separable so we should expect any linear classifier to be quite poor here. I would presume that random forests would be the best for this data source.

Let's split the data into a training and testing set

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)

Let's see the distribution of the two different classes in both the training set and testing set

import matplotlib.pyplot as plt
%matplotlib inline

n_classes = 2

training_counts = [None] * n_classes 
testing_counts = [None] * n_classes
for i in range(n_classes):
    training_counts[i] = len(y_train[y_train == i])/len(y_train)
    testing_counts[i] = len(y_test[y_test == i])/len(y_test)

# the histogram of the data
train_bar = plt.bar(np.arange(n_classes)-0.2, training_counts, align='center', color = 'r', alpha=0.75, width = 0.41, label='Training')
test_bar = plt.bar(np.arange(n_classes)+0.2, testing_counts, align='center', color = 'b', alpha=0.75, width = 0.41, label = 'Testing')

plt.xlabel('Labels')
plt.xticks((0,1))
plt.ylabel('Count (%)')
plt.title('Label distribution in the training and test set')
plt.legend(bbox_to_anchor=(1.05, 1), handles=[train_bar, test_bar], loc=2)
plt.grid(True)
plt.show()

Linear classifier

from sklearn import linear_model
clf = linear_model.SGDClassifier(max_iter=1000)
clf.fit(X_train, y_train)
clf.score(X_test, y_test)

0.54545454545454541

Support Vector Classifier

from sklearn.svm import SVC
clf = SVC()
clf.fit(X_train, y_train)
clf.score(X_test, y_test)

0.72727272727272729

K-Nearest Neighbors

from sklearn.neighbors import KNeighborsClassifier
neigh = KNeighborsClassifier(n_neighbors=3)
neigh.fit(X_train, y_train)
neigh.score(X_test, y_test)

0.66666666666666663

Random Forests

from sklearn.ensemble import RandomForestClassifier 

forest = RandomForestClassifier(n_estimators = 100)
forest.fit(X_train, y_train)
print('Score: ', forest.score(X_test, y_test))
predictions = forest.predict(X_test)

Score: 1.0

Well we got a perfect score. As expected this data structure is really best suited for the Random Forests classifier.

Answer 2

In addition to @JahKnows' excellent answer, I thought I'd show how this can be done with make_classification from sklearn.datasets.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
from sklearn.metrics import roc_auc_score
import numpy as np


data = make_classification(n_samples=10000, n_features=3, n_informative=1,
                           n_redundant=1, n_classes=2,
                           n_clusters_per_class=1, random_state=1729)
X = data[0]  # Your predictors.
y = data[1]  # Your target.

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

rf = RandomForestClassifier()  # Initialise classifier.

# Estimate classifier's performance on unseen data via cross-validation.
print('Cross-val score is {}'
      .format(np.mean(
              cross_val_score(rf, X_train, y_train, cv=5, scoring='roc_auc'))))

# Train on entire training set, evaluate on test set.
rf.fit(X_train, y_train)
print('Test score is {}.'
      .format(roc_auc_score(y_test, rf.predict_proba(X_test)[:, 1])))

Just to clarify something: n_redundant isn't the same as n_informative. A redundant feature is one that doesn't add any new information (e.g. if it's a linear combination of the other features).