I have a set of tables containing some thousand entries and some tenths of columns from machine status values of production. The entries are of mixed types like string, float, or timestamp. Each table is pre-labeled with a certain failure mode (e.g. valve setting jump, the problem with inlet A, etc.). This could be due to a jump in the mean values in some columns or a special correlation between several columns. This is what I refer to as a pattern. I would like to generate a machine learning-based classification model for this data set that can recognize these patterns. This means that I need to feed in a table of tables as data points. Up to now I only came across either tabular data with individual data points (i.e. integer, float or strings) or images, which are kinds of tables but have a fixed ordering of rows and columns. Any idea on how to tackle this problem?


Can you share a sample of the data?

Otherwise, Random Forest Classifier could be a good solution to your problem... https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html


thanks Nicolas for your hint. I doesn't work, though. I tried to create a minimal example as I can't share the original data:

from sklearn import datasets
from sklearn.model_selection import train_test_split
from random import randint

#generate 100 datasets with arbitrary values
dataset_x = []
labels = []
for i in range(100):
    x_values, _ = datasets.make_sparse_uncorrelated(n_features=20,n_samples=150)
    #label each dataset
    if randint(0,10) < 5:
        labels.append('no failure')

#split into train and test
x_train, x_test, y_train, y_test = train_test_split(dataset_x,labels,train_size=.8)  

#here is where the magic should happen
from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier(max_depth=5, random_state=0)

as expected, this fails (ValueError: Found array with dim 3. Estimator expected <= 2.).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.