2
$\begingroup$

I have been trying to figure out a proper classification model for following type of data

    C1      C2       C3             C4          label
R1 [1,4,5]  [3,5,1]  [8,5,1]        [0, NA, 6]  lung
R2 [5,NA,5] [5,1,0]  [0.9, NA, NA]  [0, 0, 0]   lung
R3 [1,1,1]. [9,4,2]  [1,1,5]        [8,1,4]     colon

As you see, for every feature I have a series of values and to make it worse, sometime NAs.

It would be great if anyone can give me a few insights on which algorithm is best suited for such type of classification data and the things I should be careful about while interpreting the model results.

Thank you & stay safe

$\endgroup$
2
  • 2
    $\begingroup$ Welcome to SO. It seems like you have equal length lists as values to the columns. A good starting point would be to convert all these values to individual columns. This should help you in treating null values as well. $\endgroup$ Jul 2 '21 at 8:26
  • $\begingroup$ Thanks for the reply. But I'm concerned that this will treat each value of each feature as a new feature. I was kind of hoping to feed this directly to a model. Do you think that's not a good approach? Because, the combination of values of each feature actually defines it's specificity. $\endgroup$
    – user120159
    Jul 2 '21 at 15:36
1
$\begingroup$

In fact, the classification algorithms generally need a numeric value to be able to classify correctly. You can use Random Forest using a function that would replace NaNs by a mean value or an outlier value.

In your case, it could be -1 or -5, but you can use the general mean value so that it would reduce the variability for NaN values in the classification process.

from __future__ import print_function

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.impute import SimpleImputer


X_train = [[1,4,5,3,5,1,8,5,1,0, NA, 6],....
Y_train = [lung,...
X_test_1 = ...


# Option 1: Create our imputer to replace missing values with the mean e.g.
imp = SimpleImputer(missing_values=np.nan, strategy='mean')
imp = imp.fit(X_train)
X_train_imp = imp.transform(X_train)

#Option 2:Or replace by an outlier value
X_train_imp = X_train.nan_to_num(-5)

# Then train the classifier
clf = RandomForestClassifier(n_estimators=10)
clf = clf.fit(X_train_imp, Y_train)
    
#Option 1:
X_test_imp = imp.transform(X_test_1)
#Or Option 2
X_test_imp = X_test_1.nan_to_num(-5)

#Result:
print(X_test_1, '->', clf.predict(X_test_imp))

Initial source code: https://stackoverflow.com/questions/30317119/classifiers-in-scikit-learn-that-handle-nan-null

$\endgroup$
1
$\begingroup$

you can make various models,

  1. the simplest is probably grouped-lasso - there are multiple implementation both in R and python - for example this one

you turned your data into

R1 1,4,5,3,5,1, 8,5,1,0, NA, 6, lung
R2 5,NA,5,5,1,0,0.9, NA, NA,0,0,0,lung
R3 1,1,1,9,4,2,1,1,5,8,1,4 colon

and make group_id where you know which columns where together, in your example [1,1,1,2,2,2,3,3,3].

gl = GroupLasso(
    groups=groups,
    group_reg=5,
    l1_reg=0,
    frobenius_lipschitz=True,
    scale_reg="inverse_group_size",
    subsampling_scheme=1,
    supress_warning=True,
    n_iter=1000,
    tol=1e-3,
)
gl.fit(X, y)
  1. architect your own model through bayesian framework - it refers to hierarchical bayesian model - make sure you are using Stan a higher level framework to translate what you want. Look here

  2. Deep learning based model

this post sounds very similar

$\endgroup$
2
  • $\begingroup$ The last link you mentioned looks very useful. So if I want to keep 5 variables for each feature as show in example, do you think a model like randomForest would be suitable? Just want to test something simple before making it complicated. $\endgroup$
    – user120159
    Jul 2 '21 at 15:42
  • $\begingroup$ @user120159 - yes, the simplest way is forget the intra-connectivity and similar to what Nicolas has suggested feed to random forest or lasso or any other model. $\endgroup$
    – user702846
    Jul 3 '21 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.