New answers tagged classification
0
The simple option is to design your features so that they represent the distribution of the values: every feature $f_i$ represents a bin and its value for a particular instance is the frequency of the corresponding range for this instance.
Example: let's consider 10 bins between 0 and 1, i.e. $f_1=[0,0.1), f_2=[0.1,0.2),..., f_{10}=[0.9,1]$:
$x_1=[0.2, 0.25,...
1
The direct way to check your model for overfitting is to compare its performance on a training set with its performance on a testing set; overfitting is when your train score is significantly above your cv score. According to your comments, your r2 score is 0.97 on the training set, and 0.86 on your testing set (or similarly, 0.88 cv score, mean across 10 ...
0
One solution is to explore a One-Vs-Rest classifier which creates separate binary classifiers for each class.
0
$n_X$ is the number of feature, $400$ is the number of data.
Each of the entry of $A$ is the output of the sigmoid layer, it is between $0$ and $1$. We can then decide a threshold (typically $0.5$) such that if it is at least the threshold, we map it to $1$, otherwise, we map it to $0$.
0
I will try to answer your confusion but not the individual "how to" i.e. How a Decision Tree calculates probability etc. You may search the internet e.g. "How Decision Tree works etc."
Score is mostly used to describe the over-all prediction result of the model. There are many different techniques i.e. metrics for this e.g. Accuracy, RMS, ...
0
If the scores have values that are higher than 1, I wouldn't call them probabilities. Probabilities should always be between 0 and 1.
And indeed, the higher the score, the more likely an example is to be positive, this is the most natural interpretation of a score.
0
I was discussing this with somebody else who argued that t needs to be considered as hyperparameter and this parameter needs to be tuned separately.
In your exercise, you are actually doing the same thing. Getting the best t. So, I don't think you need anything extra.
What I see missing in your steps -
- No steps to get the best K(nearest_neighbours) for ...
0
You may use correlation-based techniques to get some ideas about important features.
You can find a detailed analysis here MachineLearningMastery
One of the commonly accepted approaches in ML is using the Feature Importance of Tree model.
It assigns the importance based on the fact the how much better result was achieved when the sample was split on that ...
0
We can picture how Precision and Recall as netting a drove of fishes.
Imaging, we boating over the sea and lay down our net.
If the drove of fishes is huge, while the net is pretty small -> We will see fishes in very positions in the net, means Precision is high. But we only get a minority of the drove, means Recall is low.
Meanwhile, there is just a ...
0
If you are interested in convergence, then most probably it shouldn't converge. I used "should" because although we have an intuition of how the convergence work but no one is 200% sure of everything.
Let's understand a few of the aspects of the whole learning process.
- Every data point has a distinct Loss space
- Overall Loss space will be the ...
0
@DPCII, I don't think modifying output nodes at runtime will help you. This is because,
A neural network is trained on a specific dataset and to predict predefined variables only. In backpropogation ( used for optimization ), the gradients are for each weight and bias ( or any other parameters ) are calculated using the loss function ( also the objective ...
1
Are there certain techniques I can use to reduce the probability to below 0.5 in the stacked model so that it isn't classified the way it currently is?
It's generally not a good idea to try to bias the classifier in order to deal better with some specific instances, because it's likely to make it weaker in with some (possibly many) other instances. The way ...
0
Do you use Python?
Python class DecisionTreeClassifier has an attribute class_weight for this purpose.
So you do not need to adjust it manually. Check here
https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html
Regarding discretizing or encoding - hard to say what is better without knowing your data.
Unless you are really ...
0
Label Studio is a powerful opensource with a web interface to annotate different data types. It can be audio, text, image, video, time series sources and mixes of them. The conditional and nested annotations are supported too. You write your own labeling config fitting your needs to configure the system.
Check it here: https://labelstud.io/playground
0
I am assuming MDA means Mean Decrease Accuracy. Generally speaking, a good performance on a validation/testing dataset means that your model generalizes well.
On the other hand MDAs of all exactly zero means either the model always needs just one variable to get its best performance no matter what variable that is (very unlikely), or that you plain simply ...
-1
Welcome to StackExchange!
Yes, the idea of mini-batches is to augment balance in an unbalanced dataset.
You should train on balanced datasets (i.e same prevalence of all classes) and measure performance on a representative dataset as discussed here and here.
A neural network trained with imbalanced data often has varied levels
of precision in determining ...
1
In both graph it show that the model will not perform very well on the classification task as the probability distribution of the model overlaps significantly. A good model will have almost seperated curve for each class. Adding more feature will help the model differentiate between curves.
0
Another related problem, which I think worth considering, before generating more features, is to determine which columns are important for classification task, i.e. improve prediction of target variable.
One common way is to rely on feature importance scores, but a disadvantage of this is that the scores are only available after training model. To guess ...
1
Imho the "cleanest" option would be to train a probabilistic model on the original categorical target, then obtain the predicted probabilities for every category as the final "predictions". By "training on the original target" I mean designing each instance as an event, e.g. in order to represent that 7/10 people select category ...
0
You could try UBIAI Annotation tool, it is pretty easy to use, has multiple annotation exports and is currently free.
2
Question 1: Is my understanding and construction of the confusion
matrix correct?
Yes, you are correct in your definitions and the way you construct the confusion matrix. The links you have provided also agree with each other. They just switch rows and columns, since there is no hard rule regarding the presentation, as long as the correct relations are ...
0
As the error says, you gmm.score function expects a matrix with 15 columns (features) but you pass a matrix with 13.
The reason for this is in your get_MFCC() function. Specifically in this line:
features = mfcc.mfcc(audio,sr, 0.025, 0.01, 13,appendEnergy = False)
You pass 13 to the numcep argument of mfcc.mfcc(). Try 15 and it should work.
1
There are generic methods to avoid overfitting, but I'd like to address your specific problem.
Like you said, your dataset doesn't have a lot of examples compared to the number of features. This, on its own, increases the risk of overfitting, especially if you use a more complex model such as GradientBoost or RandomForest (I'm not sure I'd use either when ...
5
This is a very general question, however, there are many different solutions as explained below. For your case, probably, item 2 is not the case because you can not gather a large number of data points. I would recommend using solutions 1, 3, 5, and 6 (I see you used this method but try to combine it with other solutions such as cross-validation, ...
2
Similarly to NB or kNN, the DT and SVM algorithms work with the features which are provided as input. So whenever ML is applied to text it's important to understand how the unstructured text is transformed into structured data, i.e. how text instances are represented with features.
There are many options, but traditionally a document is represented as as a ...
1
The inputs of a CNN must have the same shape during prediction as when it was trained. So if you have a CNN trained on 50 time-steps windows, then you can make predictions on a stream of input by updating the data in this window continuously. Each new time step you push the most recent row onto the end, and drop the earliest row.
Of course it is possible to ...
0
The natural approach is to use a labelled dataset and a supervised learning technique. You can start with something simple, like using tf-idf for feature generation and train a simple logistic regression model.
I think this is the first thing you should try, I see it more likely to succeed than the unsupervised techniques, and it is simple enough.
2
threshold_90_precision = thresholds[np.argmax(precisions >= 0.9)]
Above snippet is not doing what you are expecting it to do.
Try these changes
precisions[precisions < 0.9] = 1
threshold_90_precision = thresholds[np.argmin(precisions)]
Also, I am not sure whether you are calculating the accuracy properly since z_scores is decision function, not ...
2
So, overfitting occurs when the model is complex enough to fit very well with examples observed in the training data, such that the model is not able to generalise well over unseen instances (validation data).
Therefore, for overfitting, we expect the training F1 score to continually decrease, whilst the valid_1 F1 score increases.
Here, the plot shows that ...
2
So, from what I gather, the task is to identify positive and negative sequence of digits, which abide by certain rules.
Depending on the rules themselves: for example if there are a small number of explicit and primitive rules (like each sample sequence must only have one of each digit), then might be better to hard-code the rules into a function and feed ...
10
The X axis is the number of instances in the training set, so this plot is a data ablation study: it shows what happens for different amount of training data.
The Y axis is an error score, so lower value means better performance.
In the leftmost part of the graph, the fact that the error is zero on the training set until around 6000 instances points to ...
4
It is pretty clear that your model is overfitting as your validation error is way higher than your training error.
This also means that more data allows your model to overfit less. If you are to have 20k examples I'm betting that your validation error will be slightly lower and your training error will be slightly higher.
However, I also see a plateau in ...
0
I don't think there's any obviously best option. I'd suggest trying a few reasonable solutions, evaluate on a development set and then pick the one which performs best. Don't forget to try with the original data as it is, resampling doesn't always work better.
Upsample the non-target class? Downsample the target class?
These two options are very likely to ...
0
You have two challenges - Imbalanced class and small sample size.
Best that can be done -
- Have ~8 sample as test [This means, set K = 3]
Still your minor class accuracy will drop by ~14% with each misclassification.
- Add downsampling in addition to upsampling
0
Color is a categorical feature.
One of the most common methods to encode categorical features is one-hot encoding. Color could be encoded as an indicator vector. The color of the current car would have a 1 at the appropriate index. For example, [1, 0, 0, …, 0] for a red car and [0, 1, 0, …, 0] for a blue car.
There are other options for encoding categorical ...
0
Depending on the type of algorithm you're using, you might use logloss as your optimization metric - because it will eliminate part of the problem you're trying to avoid by having such an imbalance.
I often find that the effort of under/over sampling the data does not yield better predictive performance than just using logloss from the beginning... until ...
0
Thing is you don’t have 5/100, you have 25/500. You don’t evaluate each fold separately, you evaluate them together. And if 25 positive cases are not enough for you to feel like you can properly evaluate your model, well then you have to go get more data because no amount of under/over sampling will fix that.
1
The general machine learning process is this:
Split your data into two parts, training and test. So in your example I would take 100k for test and 900k for training (don't know why you say only take 200k in your question but I digress). With the 900k training we perform hyper-parameter tuning. This can be done by splitting training into training and ...
0
If the data values are this close together, it's possible the slight differences in values could be due to, or at least masked by, measurement error. If this is the case, you won't be able to model the data accurately, as measurement error is typically random, not related to any label that is attached.
Also curious about the high precision of the data, with ...
0
Your datapoints are too close to each other and hence it is really tough for any ML model to learn this inputs as it doesn't know how to differentiate almost same data to 1 and 0 label. That's why the result is random and you are getting around half accuracy.
0
Thank you for clarifying your question.
So, from what I have got, you want to predict user demographics from the e-mails (i.e. the text).
If you knew some of the demographics for a same group of emails, you could do a supervised task by:
Using a Recurrent Neural Network (RNN) or LSTM (https://towardsdatascience.com/understanding-rnn-and-lstm-f7cdf6dfc14e), ...
1
I may be a bit late to the party. Yes, I would say this is correct.
Decision trees are prone to overfitting. Models that exhibit overfitting are usually non-linear and have low bias as well as high variance (see bias-variance trade-off). Decision trees are non-linear, now the question is why should they have high variance.
In order to illustrate this, ...
0
You are describing a problem of supervised learning with multiple inputs. That is not an uncommon task and you can find many tutorials about multiple inputs for neural networks out there. Using Tensorflow, I personally recommend Keras Functional API for this task, since it gives you more control on the layers while keeping the high-level simplicity.
0
The ROC Curve has no relation with the way your model works, but instead, with its outputs. If the target is binary and your model outputs anything in between 0 to 1 (e.g. [0, 0.2, 0.4, ..., 1] or continuous probabilities), there's a sense in building the ROC Curve. If instead, the only outputs of your model are 0 or 1, the ROC Curve would be kind of useless,...
2
A Random Forest model can definitely be used to help you determine feature importances. Actually, it is used as a very common strategy for feature selection. If your data is too small, my recommendation would be to treat it as if you were to make predictions with this model, meaning that you should watch out for overfitting and do a proper hyperparameter ...
0
This is called an open-class text classification problem, it's used in particular for some author identification problems.
I don't have any recent pointers but from a quick search I found this article: https://www.aclweb.org/anthology/N16-1061.pdf
In the field of author classification there is a similar problem called author verification, which can be ...
0
So your question is on identifying datasets which involve MRIs of brain tumours. Whether you can easily access them or not is another question, but here are some resources I have in my personal arsenal:
https://www.cancerimagingarchive.net
https://portal.gdc.cancer.gov
Hope this helps and good luck with your thesis!
0
Apart from your desired two classes, relabel all other classes as a third class and then train your model on a three class classification problem.
0
See the docs of keras
import tensorflow as tf
model.compile( ..., metrics=[tf.keras.metrics.Precision(), tf.keras.metrics.Recall()])])
1
This is just not true. Let's say that your future data is way easier to classify, then it can happen than temporal splits imply higher AUC. It often happens that future data is harder to classify because of shifts in the distributions of the variables, but if distributions are stable I don't see why not.
Top 50 recent answers are included
