What classification ML model can handle this data?

Question

I have been trying to figure out a proper classification model for following type of data

    C1      C2       C3             C4          label
R1 [1,4,5]  [3,5,1]  [8,5,1]        [0, NA, 6]  lung
R2 [5,NA,5] [5,1,0]  [0.9, NA, NA]  [0, 0, 0]   lung
R3 [1,1,1]. [9,4,2]  [1,1,5]        [8,1,4]     colon

As you see, for every feature I have a series of values and to make it worse, sometime NAs.

It would be great if anyone can give me a few insights on which algorithm is best suited for such type of classification data and the things I should be careful about while interpreting the model results.

Thank you & stay safe

Answer 1

In fact, the classification algorithms generally need a numeric value to be able to classify correctly. You can use Random Forest using a function that would replace NaNs by a mean value or an outlier value.

In your case, it could be -1 or -5, but you can use the general mean value so that it would reduce the variability for NaN values in the classification process.

from __future__ import print_function

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.impute import SimpleImputer


X_train = [[1,4,5,3,5,1,8,5,1,0, NA, 6],....
Y_train = [lung,...
X_test_1 = ...


# Option 1: Create our imputer to replace missing values with the mean e.g.
imp = SimpleImputer(missing_values=np.nan, strategy='mean')
imp = imp.fit(X_train)
X_train_imp = imp.transform(X_train)

#Option 2:Or replace by an outlier value
X_train_imp = X_train.nan_to_num(-5)

# Then train the classifier
clf = RandomForestClassifier(n_estimators=10)
clf = clf.fit(X_train_imp, Y_train)
    
#Option 1:
X_test_imp = imp.transform(X_test_1)
#Or Option 2
X_test_imp = X_test_1.nan_to_num(-5)

#Result:
print(X_test_1, '->', clf.predict(X_test_imp))

Initial source code: https://stackoverflow.com/questions/30317119/classifiers-in-scikit-learn-that-handle-nan-null

Answer 2

you can make various models,

1. the simplest is probably grouped-lasso - there are multiple implementation both in R and python - for example this one

you turned your data into

R1 1,4,5,3,5,1, 8,5,1,0, NA, 6, lung
R2 5,NA,5,5,1,0,0.9, NA, NA,0,0,0,lung
R3 1,1,1,9,4,2,1,1,5,8,1,4 colon

and make group_id where you know which columns where together, in your example [1,1,1,2,2,2,3,3,3].

gl = GroupLasso(
    groups=groups,
    group_reg=5,
    l1_reg=0,
    frobenius_lipschitz=True,
    scale_reg="inverse_group_size",
    subsampling_scheme=1,
    supress_warning=True,
    n_iter=1000,
    tol=1e-3,
)
gl.fit(X, y)

2. architect your own model through bayesian framework - it refers to hierarchical bayesian model - make sure you are using Stan a higher level framework to translate what you want. Look here

3. Deep learning based model

this post sounds very similar