New answers tagged

0

You are applying xgb and random forest to a multiclassification task and you are doing under sampling to some how try to improve your class If i understand your histogram, that is the distribution of labels. When you say accuracy went down for both classes you are refering to a multiclassficitaion problem right? Answers directly to your questions: There ...


0

I would try the following two approaches and both are equally interesting. The first is: k-means clustering. Here is why: On the basis of a set of symptoms, mapping to one diagnosis is something we can try. Also, we can change the number of clusters and check if it improves the accuracy/ results . The second is: recommender system. This works very ...


0

You can try Multi linear regression analysis. This can be done using excel(data-->data analysis-->regression). Basically this model develops the relationship between multiple variable impacting one dependent variable. In the example above: diagnosis = a + b1 medical symptom1 + b2 medical symptom 2+ .....+ e where, a = intercept of the equation b1, b2,...., ...


0

Assuming there are many repetitions of each attribute and class, an embedding space can be learned. Attributes and classes that co-occur together will be projected into nearby space. One example is affinity weighted embedding. Then prediction becomes approximate nearest neighbor search. For a given set of attributes, find the nearest class(es). One example ...


0

Your question is not very clear: do you mean that your test data never contains this feature? If yes, you should remove this column from the training data. The train and test data must have the same features. If no, i.e. only some instances might not have a value for this column, then it's about having missing values in your data. In this case you could ...


0

I would remove the classes with very few samples, as they create discrepancies in the model and also help with skew. I would try to create new features by combining the features which are similar/ or have similar influence on the outcome. This is because as compared to the number of samples, you have too many features. Try using logistic regression and see ...


1

Reason for the discrepancy Two aspects have to be considered regarding the split: Is the split done in a stratified manner? (it should) Is the data shuffled? (it should) The line X_train, X_test, y_train, y_test, i_train, i_test = train_test_split(feature_matrix, y, indices, test_size=0.33, random_state=random_state) splits the data in a stratified ...


0

Since you seem to have the same number of rows per sample, perhaps the underlying process is such that it makes sense to treat the data as 2D or unpack into 12 features, as @Arnaud describes. (This seems to depend on the four rows being ordered according to some implicit rule?) More generally though, this is called "multiple instance learning." Probably ...


2

What about doing a concatenation of your rows (i.e. Attr1 -> Attr12) , such that you now have 3*4 features (because 4 rows of 3 features) as an input to a multiclass classification model? For instance, first sample would be described by : X = [1.1, 1.4, 2.5, 2.3, 2.5, 2.7, 1.1, 1.6, 1.9, 1.5, 1.6, 1.7] y = "A" Otherwise, there is no issue in giving 2D or ...


0

The second table is simply saying rows 1 - 4 are 4 different examples of class A, rows 5 - 8 are 4 separate example of class B and the rest are 4 examples of class C. Just modify the table so the target label column has 12 rows the first for having the value A, the next 4 having the value B and the final 4 having the value C. Good luck!


0

When talking about the test data, we can have any number of examples of each class for inference. It doesn't matter if it has 20 classes with one example each. You can use a CNN classifier for this type of a problem but if one book can be written by more than one author you should use sigmoid activation in the last layer rather than using softmax. You ...


5

For predictive power, in general, including both shouldn't be a problem. But there is a lot of nuance here. Foremost, if predictive power isn't all you care about: if you're making statistical inferences, or care about explainability and feature importances, then including both can cause issues. Briefly, your model may split the importance of the underlying ...


1

Pass a dictionary in the following format to class_weight parameter in fit_generator: { 'output1': {0: ratio_1 , 1: ratio_2} , 'output2': {0: ratio_3 , 1: ratio_4}} You can use class_weight from sklearn.utils to calculate class weights from your data References: https://github.com/keras-team/keras/issues/4735#issuecomment-267473722 https://scikit-learn....


Top 50 recent answers are included