New answers tagged

0 votes

Global feature importance for classification problem where some values are irrelevant

Permutation feature importance is a way to access global feature importance. Permutation feature importance is defined as the decrease in an evaluation metric when a feature value is randomly shuffled,...
user avatar
0 votes

Global feature importance for classification problem where some values are irrelevant

The following answer is based on the reasonable assumption that for a certain (medical?) procedure, an irrelevant checkup(feature) always remains irrelevant. Eg: The procedure of checking for a ...
user avatar
0 votes

How do I deal with unbalance classes in a stock market prediction problem?

There is a perfect mapping between the rule and the decision. You know the percentage change; now apply your rule to map that to a buy/sell/hold decision. There is no machine learning to do, but even ...
user avatar
  • 2,830
1 vote

What can be done with same samples with different target?

As others have explained for regression problems, leave them in. For categorical, I think the situation is a bit more nuanced. It could be that you are actually in a multi-label setting, or it could ...
user avatar
1 vote

What can be done with same samples with different target?

in addition to the answer from @Dave, consider that usually we start an assumption on the conditional distribution $P(y|x)$, for example for regression we assume Gaussian noise... this means that the ...
user avatar
0 votes

How to check if customer segmentation/classification is correct?

There are several ways to use distance between a customer and a segment represented by a group of customers in this segment. One very simple way would be "train" a k-NN classifier to predict ...
user avatar
  • 21.8k
6 votes
Accepted

What can be done with same samples with different target?

This is completely normal; leave them in. An easy example is in an ANOVA problem (which can be viewed as a regression) where multiple subjects in the same group (so same group "value" where ...
user avatar
  • 2,830
1 vote

How to predict a class for a file given a small number of files for training?

I mostly agree with all the other answers, however, I'll try to propose another approach to the problem, Given that the order of the rows doesn't affect the classification task, I would try to split ...
user avatar
  • 489
1 vote

How to predict a class for a file given a small number of files for training?

I agree with the other answer. Train-test split probably won't work. Here's an idea: (Assuming the columns are features) Compute the mean of every column. So, for every file, you'll have 10 numbers. ...
user avatar
1 vote

How to further improve on overfitting?

This was an issue I was struggling with for over a week but the eventual problem seemed to be perhaps something in the way the function was done; Initially I used ...
user avatar
  • 111
1 vote
Accepted

How to predict a class for a file given a small number of files for training?

Given that only that most classes have less than four samples, it is not useful to do a train/test split. A train/test split is the most useful way to assess generalization. One option could be to ...
user avatar
0 votes

Cross validation schema for imbalanced dataset

Im burning my mind about the same question... I found that, you shold preserve the distribution of validation and test sets. I my opinion the option 1 is more interesting, not justo because preserve ...
user avatar
1 vote

Predict data using Pre-Trained Classification Model

Yes. Whatever steps/processing you have done to the data before feeding it to the model, all of steps needs to be done again in the raw data. Ideally you should create a function which takes the data/...
user avatar
1 vote

How to measure similarities between two datasets with same features?

You can use statistical approach and try computing KL-divergence between the 2 datasets (Distributions). However, the KL-Divergence output is between 0 and ∞ (0 meaning two distributions perfectly ...
user avatar
0 votes

Threshold tuning with one-vs-rest for multi classification python

In general it's not possible to tune any threshold in multiclass classification: In binary classification, modifying the threshold means predicting more or less instances as positive, because the two ...
user avatar
  • 21.8k
0 votes

SOS: Working LightGBM model script to find best model

I've found that there are usually a lot of full solution code notebooks available in Kaggle for these sorts of problems. Here's a couple I was able to find based on a quick Google search: https://www....
user avatar
  • 21
1 vote

All classification models except neural network giving 100% accuracy

It could be due to a lack of initialization of your neurons. Did you initialize them randomly? For instance: ...
user avatar
1 vote
Accepted

Having weird accuracy graph on deep learning binary classification model

Assuming that the dataset is balanced, my intuition is the following: From epoch 1 to 55: the loss function being superhigh indicates your model is doing random predictions but with probabilities near ...
user avatar
  • 489
0 votes

Why do we don't write units with MAE or RSME for regression problem ? If I wish to write the units when how do I identify the units for them?

RMSE and MAE do have units. In fact, they have the same units. (Determining why that’s the case is a useful exercise.) You determine those units by considering the units of your $y$. If $y$ is ...
user avatar
  • 2,830
0 votes

Why do we don't write units with MAE or RSME for regression problem ? If I wish to write the units when how do I identify the units for them?

In general evaluation measures don't have any unit. Many of them actually represent proportions of instances (like accuracy, precision, recall). But it's true that error evaluation measures are ...
user avatar
  • 21.8k
1 vote

Binary classifier high overall ROC AUC but low in different bins

One interpretation of the AUROC is "the probability that a randomly selected positive instance is given a higher probability by the model than a randomly selected negative instance." With ...
user avatar
  • 9,591
1 vote
Accepted

All Categorical data

I don't see any problem doing classification with purely categorical features, as far as the features are relevant. And as always, some precautions dealing with categorical features: The choice of ...
user avatar
  • 440
2 votes

Plot distribution of multi classification with features - Python

Before you think about visualization, you need to come up with the questions you would answer using visualization techniques. 1- For example, you could ask how is X feature different in different ...
user avatar
0 votes

Classification - get some label value to check how close to another class (Python)

With Naive Bayes you can predict the probability of a text to belong to each class using the predict_proba method. Using this method you'll get a vector of ...
user avatar
  • 489
2 votes
Accepted

Neural network / machine learning approach to model specific sequencing-classification problem in industry

I would try to apply techniques from the "changing point problem" world. In this kind of problem, you try to identify times when the probability distribution of a stochastic process or time ...
user avatar
  • 489
3 votes

Binary classifier high overall ROC AUC but low in different bins

My two cents: If the goal is to predict the different bins corresponding to the probability of predicting the positive class, then this seems a strange design: why use binary classification if the ...
user avatar
  • 21.8k
0 votes

macro average and weighted average meaning in classification_report

Macro average represents the arithmetic mean between the f1_scores of the two categories, such that both scores have the same importance: ...
user avatar
  • 1
0 votes

Subsampling the “right” amout of data to train an ML model

There are many rule based sampling techniques out there as opposed to just sampling randomly and hoping it works. The idea is to sample proportionately so that the model does not learn any biases or ...
user avatar
  • 1,261

Top 50 recent answers are included