New answers tagged classification
0
Precision-recall curve are argued to be more useful than ROC curve in "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets" by Saito and Rehmsmeier. They argue that ROC might lead to wrong visual interpretation of specificity.
F1-score equally balances precision and recall. In ...
0
I believe the solutions should be a total of 3 neurons for each question. This is because two neurons are sufficient to calculate the lines that separate all the points. Then, an extra neuron can be used to decide to which class does a point belong to. Let's see why:
1st Question: Output of the neurons $=1$ or $=0$
First, we should note that both classes are ...
0
I think you have way too less of data assuming that you would have to separate it out further to training, test and validation sets. Your model will not generalize well. Consider procuring some more data (around 1500-2000 samples atleast)
1
First, the question is too broad because there are many different kinds of text classification tasks. For example one wouldn't use the same approach for say spam detection and author profiling (e.g. predict the gender of the author), two tasks which are technically text classification but have little in common (and there are many others).
Second, even with a ...
0
Bias and Variance in Classification problems
Check this link about Support Vector Machine.
You will directly understand bias and variance in classification. Basically, if your data is linearly separable you do not have a problem.
But imagine that your data is pseudo/semi linearly separable, however, few points land on the other side of their group.
Now ...
0
If you already know sklearn then you should use this.
from sklearn.metrics import plot_precision_recall_curve
from sklearn.metrics import plot_roc_curve
Documentation for you.
Regarding the AUC, it will be shown on the graph automatically.
0
The best approach is to use a model like Decision Tree to see what actually is happening. Maybe there are couple of features in there that correlate in a big way to the label and the rest of the 1000+ features dont matter at all. It is possible (as someone else too point out) that one of the feature hiding in there (an icd with a certain response code) has a ...
0
You would anyway start with word embeddings. Pre-trained vectors like Glove etc. Those are derived from DL algorithms. So whichever approach you now take, you already have an element of DL in your solution.
Let us now look at the possible approaches. One way would be definitely to use RNNs like bi-directional GRU/LSTM etc which will embed the sentence nicely ...
0
If you look at your output you will see that the two values in every row add up to 1. This means that the first value is the probability of the output having value 0 in your classification, and the second value the probability of the out having value 1. If you want to only have the 0/1 classification instead of the probabilities you can use a simply ...
0
I have a slightly different take and view this problem as one related to recommender systems.
Just like how one would recommend movies for users based on various approaches (involving both supervised and unsupervised methods), similarly you would recommend products to users.
So while you could start with using both unsupervised (clustering to segment your ...
2
To clarify the questions raised by the user in response to the correct solution given by Erwan - the solution proposes going back in time to prepare the data across a series of timestamps.
There will be multiple points in time 't' where the input would be all the various features on the patients health, medication, reports etc..you need to see how best they ...
2
This could be seen as a "simple" binary classification problem. I mean the type of problem is "simple", the task itself certainly isn't... And I'm not even going to mention the serious ethical issues about its potential applications!
First, obviously you need to have an entry in your data for a patient's death. It's not totally clear to ...
0
The simple approach is:
To use a learning algorithm which can handle missing values. You should be careful about whether these missing values occur randomly or not though, as they could cause the model to be biased.
Triplets can be separated into individual features, there's no simple way to process them differently anyway.
0
In case of CNN, you are correct in the sense that you cannot use the final layer weights if the number of categories are different. But you CAN reuse the weights in the initial layers. These recognise the lower level objects in the image. There is no need to train all over again. You would only have to train the upper layers specific to the categorisation ...
0
If we classify new objects using transfer learning:
1.We delete the top Dense layer of the pre trained neural network.
2.Now suppose you have to classify 5 classes, so your final dense layer will contain 5 nodes.
3.Also you will add some dense layers prior to your new 5 node Dense layer, so that you can train the model with new data.
4.All the layers prior ...
0
One option is reinforcement learning (RL).
RL frames the problem as sequential decision-making under uncertainty. There is an agent making decisions and after the decision is made, the agent collects new data. The next round of decisions uses the new data.
What you are describing is a scenario where the distribution of "ground-truth" changes over ...
0
You've listed a few different error metrics, I would pick one metric that is best suited to your problem. Trying to maximize several metrics at once makes it difficult to tell if your model is getting better. In any case, if normalization leads to the best score - then that is your answer. Since all other variables were already in range [0,1], then that's ...
0
If you effectively balance your samples it would badly affect your overall result (or performance). If you would not use oversampling I think poor performance would be due to having an imbalanced dataset rather than not having enough positive class samples to learn from. This is because they would dominate your loss function.
However, I think you can try a ...
4
There are several options for you:
class_weight should boost the loss function towards the preferred class. This option is supported by various estimators, including sklearn.linear_model.LogisticRegression,
sklearn.svm.SVC, sklearn.ensemble.RandomForestClassifier, and others. Note there's no theoretical limit to the weight ratio, so even if 1 to 100 isn't ...
0
Depends if you are going to do online learning.
Lets say you will do online learning/incremental learning than test set Distribution will make difference. For example because of catastrophic forgetting of neural Networks.
If you are making Batch predictions than it makes no difference whats the test set Distribution. Model knows no difference since it does ...
0
I would approach the problem as follow:
Start by splitting your dataset into a test dataset and a training dataset. You would then proceed by building a classifier for each antibody (feature) using only the training dataset and then measure the success of your model on some test dataset (using the measure that you care about e.g. accuracy).
Usually you ...
0
You have to add a constraint of linear separability to make the comparison. The problem is that in reality most problem are not linearly separable. Either they need non-linear transformations (data processing or usage of kernel), or they are not separable at all (underlying randomness, noise). SVM can be adapted to these non-linearities quite well (allowing ...
0
Technically your problem is not about a variable number of features, since you can have a finite list of all the possible features. The standard case is just to use all these features, even if only a few of them are "active" for a particular instance (your first option). If the number of features is too high, then you need dimensionality reduction.
...
0
The basic idea of most of the current question answering architectures is:
get a common representation of the question and the input text (e.g., using BERT)
get a representation of the answers
do sort of attention over the answers: compute a scalar score for each of the answers (using a dot-product, linear layer, multilayer-perceptron) and normalize the ...
1
It is normal to get 100% accuracy in this case. I mean it is inevitable to get 100% accuracy. If you train your algorithm even with 10% of your data you will again get 100% accuracy.
The abnormal thing in this code is using a vectorizer. Vectorizer is an algorithm that is used to extracting words from the documents as mentioned in the source:
Tf-idf stands ...
2
It think it's a reasonable approach, but currently it seems that you have no way to check whether the new labels are correct or not. I think you should at least check that the new labels don't introduce more errors than they solve.
Ideally you would re-annotate a random sample of instances, keeping both the old (possibly erroneous) labels and the new ones. ...
3
My interpretation is that the data can be described as the following three groups, by decreasing order of the prediction to be positive:
a large number of positive instances which can "easily" be predicted as positive, with no false positive
then a large number of negative instances
then a smaller number of positive instances, which are predicted ...
0
When you use buckets you lose some information. Basically you assume the relationship between the variable and the target is flat within the interval. When this is probably not the case. This is point #3 on Frank Harell's list of reasons why you generally shouldn't categorise at all.
More specifically, 80% of your instances seems to be in one bucket. That ...
1
As noted by Peter, you should split your data into test and train
sets before fitting the TfidfVectorizer to the training set (not to
all the data).
Pandas read_json() is not reading your data from file the way you
want. You are expecting df["lista_asm"] to be a Series of list
objects (each containing strings). This Series is instead a Series
of ...
0
Which metric should I use for classifying an imbalace data with fewer labels for the negative class?
The names of the classes don't matter, you might as well call them class A and class B. In binary classification the typical choice is to evaluate using precision, recall and F1-score. There are other options, but that depends on the task.
Assuming you choose F1-score, the choice of which class you select as the "positive" class for evaluation also ...
0
One thing you could check is wether using the vectorizer on the whole text could be an issue. The test set should be "new" unseen data and should not necessarily be vectorized together with the training data. You can "refit" a vectorizer on new text (the test set) using the "old" vocabulary (from train set). However, I'm not ...
0
Ofcourse not. Here is a simple possible solution.
Do unsupervised learning. If you do it good and efficiently you will only see these two groups in your data (binary classification). And your silhuette score will be high. Hence you can automatically than label these groups/clusters.
1
I think you can carry out a usual multiclass classification instead of manually carrying out the one-VS-all strategy (i.e. in the for loop), provided that you can generate the multilabels target(i.e. all the possible combinations of diseases you said). So, I would do something like:
generate the correct labels --> if you have n possible deseases ...
0
It's of course technically possible to calculate macro (or micro) average performance with only two classes, but there's no need for it. Normally one specifies which of the two classes is the positive one (usually the minority class), and then regular precision, recall and F-score can be used.
Commonly there is a majority and a minority class, and naturally ...
0
Yes, merging / dropping classes will tend to increase performance. If you merge classes, there will be more examples per class which will tend to decrease the variance of a model. Reducing the total of number classes has the potential to allow the model better fit the data, reducing bias. Many models are constrained by learning capacity.
1
The actual predictions can be found in newdata.pred$data.
Note that mlr is no longer maintained:
{mlr} is considered retired from the mlr-org team. We won’t add new
features anymore and will only fix severe bugs. We suggest to use the
new mlr3 framework from now on and for future projects.
See the docs: https://mlr.mlr-org.com/
library(mlr)
library(ISLR)
...
0
I have a domain observation, rather than a modeling one. It's based on my 2007 analysis of 125K securitized subprime loans originated in 2006 by a single issuer with a broker, yield spread marketing model. In addition to 50 origination variables, monthly patterns of payments were tracked.
Delinquency is not solely a matter of credit underwriting, but of the ...
0
Binary problems have the most amount of metrics to measure its performance.
They can be classified as accuracy metrics, probabilistic metrics and metrics depending on true/false positives/negatives. You are not expected to implement them from scratch, the last time I implemented an AUC (area under the curve) was in college so you will find these metrics ...
1
Can someone explain why the huge gap
It simply means that there's a quite high variance depending which random set of instances is picked. How many times do you re-sample the instances in the bagging process? Probably increasing the number of runs will decrease the variance. As mentioned in a comment, the most common reason for variance in performance is a ...
0
There are a couple of Github issues on this - see here and here. In short, while scikit-learn native models include an _estimator_type attribute:
from sklearn.linear_model import LogisticRegression
clf = LogisticRegression()
clf._estimator_type
# 'classifier'
this is not the case with a KerasClassifier; using your own NN gives
NN = KerasClassifier(build_fn=...
1
Is there a clustering algorithm which accepts some clusters as input and outputs some more clusters?
You are describing semi-supervised learning where the training dataset is only partially labeled.
One common set of techniques to solve that problem is active learning. In active learning, there is a learning loop where the algorithm makes predictions and a human corrects those predictions.
Pre-clustering is a specific active learning technique to address ...
0
The short answer is yes. Nonetheless you should have been deeper while data understanding process i.e analyzing if there are really features that separate/differentiate the good payers vs the delinquent ones.
Say for example you have numeric variables such as current balance, number of delinquent accounts, number of inquiries in the last six months,etc If ...
3
From a credit scoring point of view : a $F_1$ score of $0.1$ seems pretty bad but not impossible with an unbalanced data-set. It might be enough for your needs (once you weight your errors by the cost). And it might not be possible to go higher (not enough data to predict an event that appears random). In credit scoring there is always a 'random' part in the ...
4
(1) For the sake of keeping it short in your case: yes 0.1 is bad. To avoid philosophical discussions let's just assume you have to get this higher.
(2) It definitely makes sense since your dataset is highly imbalanced. Do not expect to have one metric where you fail miserably and on the other one, you succeed. That's not how it works, they are most often ...
1
GANs do not provide any guarantee on the distribution of the generated data. On the contrary, they are notorious for their mode collapse problems (i.e. generating always the same values). Therefore, I doubt that they are a reliable way of systematically generating synthetic data for other systems to train.
Oversampling techniques like SMOTE are normally much ...
1
Learning that can generalise well.
Take for example differnetial privacy. There you inject noise on Purpose to anonymise your data, and in the process of you losse accuracy. Goal is to find such algorithms, that will with smart noise injections, be able to generalise and Keep the good accuracy Level.
1
Here are the Options:
Oversampling- sure, there are some possibly good ones like SMOTE etc. Just apply it after Train test split to avoid leakage.
Undersampling - reducing the 30000 to a certain number where what is left is representative of the Information you Need to classify this class. You could, for example, apply some unsupervised learning to see ...
0
If I got this right you want to split the dataset into train and test sets in a way that preserves the same proportions of examples in each class as observed in the original dataset?
This is called a stratified train-test split. See the stratify argument here sklearn split
0
Is there a clustering algorithm which accepts some clusters as input and outputs some more clusters?
You basically have partially labeled data. You can do clustering regardless of labels, and then assign unlabeled data into the majority of labels you find in their cluster.
Same approach can be done using KNN. Just simply try KNN with different K and metrics on a validation split of your training data and when it shows good performance, apply it to the whole ...
Top 50 recent answers are included
