# Tag Info

### Global feature importance for classification problem where some values are irrelevant

Permutation feature importance is a way to access global feature importance. Permutation feature importance is defined as the decrease in an evaluation metric when a feature value is randomly shuffled,...
• 16.5k

### Global feature importance for classification problem where some values are irrelevant

The following answer is based on the reasonable assumption that for a certain (medical?) procedure, an irrelevant checkup(feature) always remains irrelevant. Eg: The procedure of checking for a ...
• 449

### How do I deal with unbalance classes in a stock market prediction problem?

There is a perfect mapping between the rule and the decision. You know the percentage change; now apply your rule to map that to a buy/sell/hold decision. There is no machine learning to do, but even ...
• 2,830
1 vote

### What can be done with same samples with different target?

As others have explained for regression problems, leave them in. For categorical, I think the situation is a bit more nuanced. It could be that you are actually in a multi-label setting, or it could ...
1 vote

### What can be done with same samples with different target?

in addition to the answer from @Dave, consider that usually we start an assumption on the conditional distribution $P(y|x)$, for example for regression we assume Gaussian noise... this means that the ...

### How to check if customer segmentation/classification is correct?

There are several ways to use distance between a customer and a segment represented by a group of customers in this segment. One very simple way would be "train" a k-NN classifier to predict ...
• 21.8k
Accepted

### What can be done with same samples with different target?

This is completely normal; leave them in. An easy example is in an ANOVA problem (which can be viewed as a regression) where multiple subjects in the same group (so same group "value" where ...
• 2,830
1 vote

### How to predict a class for a file given a small number of files for training?

I mostly agree with all the other answers, however, I'll try to propose another approach to the problem, Given that the order of the rows doesn't affect the classification task, I would try to split ...
• 489
1 vote

### How to predict a class for a file given a small number of files for training?

I agree with the other answer. Train-test split probably won't work. Here's an idea: (Assuming the columns are features) Compute the mean of every column. So, for every file, you'll have 10 numbers. ...
• 131
1 vote

### How to further improve on overfitting?

This was an issue I was struggling with for over a week but the eventual problem seemed to be perhaps something in the way the function was done; Initially I used ...
• 111
1 vote
Accepted

### How to predict a class for a file given a small number of files for training?

Given that only that most classes have less than four samples, it is not useful to do a train/test split. A train/test split is the most useful way to assess generalization. One option could be to ...
• 16.5k

### Cross validation schema for imbalanced dataset

Im burning my mind about the same question... I found that, you shold preserve the distribution of validation and test sets. I my opinion the option 1 is more interesting, not justo because preserve ...
1 vote

### Predict data using Pre-Trained Classification Model

Yes. Whatever steps/processing you have done to the data before feeding it to the model, all of steps needs to be done again in the raw data. Ideally you should create a function which takes the data/...
1 vote

### How to measure similarities between two datasets with same features?

You can use statistical approach and try computing KL-divergence between the 2 datasets (Distributions). However, the KL-Divergence output is between 0 and ∞ (0 meaning two distributions perfectly ...

### Threshold tuning with one-vs-rest for multi classification python

In general it's not possible to tune any threshold in multiclass classification: In binary classification, modifying the threshold means predicting more or less instances as positive, because the two ...
• 21.8k

### SOS: Working LightGBM model script to find best model

I've found that there are usually a lot of full solution code notebooks available in Kaggle for these sorts of problems. Here's a couple I was able to find based on a quick Google search: https://www....
• 21
1 vote

### All classification models except neural network giving 100% accuracy

It could be due to a lack of initialization of your neurons. Did you initialize them randomly? For instance: ...
• 2,108
1 vote
Accepted

### Having weird accuracy graph on deep learning binary classification model

Assuming that the dataset is balanced, my intuition is the following: From epoch 1 to 55: the loss function being superhigh indicates your model is doing random predictions but with probabilities near ...
• 489

### Why do we don't write units with MAE or RSME for regression problem ? If I wish to write the units when how do I identify the units for them?

RMSE and MAE do have units. In fact, they have the same units. (Determining why that’s the case is a useful exercise.) You determine those units by considering the units of your $y$. If $y$ is ...
• 2,830

### Why do we don't write units with MAE or RSME for regression problem ? If I wish to write the units when how do I identify the units for them?

In general evaluation measures don't have any unit. Many of them actually represent proportions of instances (like accuracy, precision, recall). But it's true that error evaluation measures are ...
• 21.8k
1 vote

### Binary classifier high overall ROC AUC but low in different bins

One interpretation of the AUROC is "the probability that a randomly selected positive instance is given a higher probability by the model than a randomly selected negative instance." With ...
• 9,591
1 vote
Accepted

### All Categorical data

I don't see any problem doing classification with purely categorical features, as far as the features are relevant. And as always, some precautions dealing with categorical features: The choice of ...
• 440

### Plot distribution of multi classification with features - Python

Before you think about visualization, you need to come up with the questions you would answer using visualization techniques. 1- For example, you could ask how is X feature different in different ...

### Classification - get some label value to check how close to another class (Python)

With Naive Bayes you can predict the probability of a text to belong to each class using the predict_proba method. Using this method you'll get a vector of ...
• 489
Accepted

### Neural network / machine learning approach to model specific sequencing-classification problem in industry

I would try to apply techniques from the "changing point problem" world. In this kind of problem, you try to identify times when the probability distribution of a stochastic process or time ...
• 489

### Binary classifier high overall ROC AUC but low in different bins

My two cents: If the goal is to predict the different bins corresponding to the probability of predicting the positive class, then this seems a strange design: why use binary classification if the ...
• 21.8k