# What classification ML model can handle this data?

I have been trying to figure out a proper classification model for following type of data

    C1      C2       C3             C4          label
R1 [1,4,5]  [3,5,1]  [8,5,1]        [0, NA, 6]  lung
R2 [5,NA,5] [5,1,0]  [0.9, NA, NA]  [0, 0, 0]   lung
R3 [1,1,1]. [9,4,2]  [1,1,5]        [8,1,4]     colon


As you see, for every feature I have a series of values and to make it worse, sometime NAs.

It would be great if anyone can give me a few insights on which algorithm is best suited for such type of classification data and the things I should be careful about while interpreting the model results.

Thank you & stay safe

• Welcome to SO. It seems like you have equal length lists as values to the columns. A good starting point would be to convert all these values to individual columns. This should help you in treating null values as well. Jul 2 '21 at 8:26
• Thanks for the reply. But I'm concerned that this will treat each value of each feature as a new feature. I was kind of hoping to feed this directly to a model. Do you think that's not a good approach? Because, the combination of values of each feature actually defines it's specificity. Jul 2 '21 at 15:36

In fact, the classification algorithms generally need a numeric value to be able to classify correctly. You can use Random Forest using a function that would replace NaNs by a mean value or an outlier value.

In your case, it could be -1 or -5, but you can use the general mean value so that it would reduce the variability for NaN values in the classification process.

from __future__ import print_function

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.impute import SimpleImputer

X_train = [[1,4,5,3,5,1,8,5,1,0, NA, 6],....
Y_train = [lung,...
X_test_1 = ...

# Option 1: Create our imputer to replace missing values with the mean e.g.
imp = SimpleImputer(missing_values=np.nan, strategy='mean')
imp = imp.fit(X_train)
X_train_imp = imp.transform(X_train)

#Option 2:Or replace by an outlier value
X_train_imp = X_train.nan_to_num(-5)

# Then train the classifier
clf = RandomForestClassifier(n_estimators=10)
clf = clf.fit(X_train_imp, Y_train)

#Option 1:
X_test_imp = imp.transform(X_test_1)
#Or Option 2
X_test_imp = X_test_1.nan_to_num(-5)

#Result:
print(X_test_1, '->', clf.predict(X_test_imp))


you can make various models,

1. the simplest is probably grouped-lasso - there are multiple implementation both in R and python - for example this one

R1 1,4,5,3,5,1, 8,5,1,0, NA, 6, lung
R2 5,NA,5,5,1,0,0.9, NA, NA,0,0,0,lung
R3 1,1,1,9,4,2,1,1,5,8,1,4 colon


and make group_id where you know which columns where together, in your example [1,1,1,2,2,2,3,3,3].

gl = GroupLasso(
groups=groups,
group_reg=5,
l1_reg=0,
frobenius_lipschitz=True,
scale_reg="inverse_group_size",
subsampling_scheme=1,
supress_warning=True,
n_iter=1000,
tol=1e-3,
)
gl.fit(X, y)

1. architect your own model through bayesian framework - it refers to hierarchical bayesian model - make sure you are using Stan a higher level framework to translate what you want. Look here

2. Deep learning based model

this post sounds very similar

• The last link you mentioned looks very useful. So if I want to keep 5 variables for each feature as show in example, do you think a model like randomForest would be suitable? Just want to test something simple before making it complicated. Jul 2 '21 at 15:42
• @user120159 - yes, the simplest way is forget the intra-connectivity and similar to what Nicolas has suggested feed to random forest or lasso or any other model. Jul 3 '21 at 20:30