Nuclear Hoagie
• Member for 4 years, 4 months
• Last seen this week

## 31 Answers

13 answers
42 votes
23k views
42 votes

Q: How many machine learning specialists does it take to change a light bulb? A: Just one, but they require a million light bulbs to train properly. Q: How many machine learning specialists does it ...

View answer
2 answers
3 votes
797 views
Accepted answer
6 votes

There are three main approaches to handling missing data. Impute - use some method to fill in the missing values with reasonable guesses. You could interpolate between two time points, take the ...

View answer
1 answers
5 votes
4k views
Accepted answer
5 votes

Normalization helps to eliminate scale factors that might exist between variables in your data. Take, for example, the classic problem of predicting home prices. If you represent the square footage of ...

View answer
2 answers
2 votes
3k views
Accepted answer
4 votes

You have imbalanced classes. Notice that your accuracy is very close to your precision, and quite dissimilar to your recall. This means that your precision (accuracy of positive predictions) is ...

View answer
6 answers
27 votes
15k views
4 votes

Yet another reason why logarithmic transformations are useful comes into play for ratio data, due to the fact that log(A/B) = -log(B/A). If you plot a distribution of ratios on the raw scale, your ...

View answer
1 answers
3 votes
7k views
Accepted answer
4 votes

A confusion matrix can be used to measure the performance of a particular classifier with a fixed threshold. Given a set of input cases, the classifier scores each one, and score above the threshold ...

View answer
1 answers
2 votes
23 views
3 votes

You need to investigate multiple hypothesis correction methods, like Bonferroni correction or Benjamini-Hochberg false discovery rate. The problem with this sort of analysis is that your associations ...

View answer
2 answers
1 votes
32 views
2 votes

For this type of issue, I typically add the reciprocal of the log base. For data that's being log10-scaled, this results in adding 0.1 to all values. For data that's being log2-scaled, this results in ...

View answer
2 answers
1 votes
607 views
2 votes

High feature variance does not imply any sort of relationship to the target variable you're interested in modeling. Suppose you are looking at a population and have two variables for each person, ...

View answer
3 answers
2 votes
140 views
2 votes

One way to explore the mapping between the original dimensions and and PCA dimensions is to look at something called the factor loadings. These are essentially projections of your original dimensions ...

View answer
2 answers
1 votes
66 views
2 votes

Do not apply PCA to categorical data PCA attempts to find the dimensions which contains the most variance in a dataset. When you have categorical variables, the distance between points and the ...

View answer
3 answers
3 votes
4k views
Accepted answer
2 votes

Your intuition is generally correct - in many cases, premature discretization of continuous variables is undesirable. Doing so throws away potentially meaningful data, and the result can be highly ...

View answer
2 answers
0 votes
151 views
Accepted answer
2 votes

For a very simple example, imagine you have three independent classifiers that each have 60% accuracy. If you use any one of them to classify a random sample, you have a 60% chance of getting it right....

View answer
1 answers
0 votes
32 views
Accepted answer
1 votes

Accuracy treats all misclassifications as the same - we only care whether we got the answer right or not, but don't care about what kind of error was made. Even for class-balanced problems, this may ...

View answer
1 answers
0 votes
33 views
Accepted answer
1 votes

This should do the trick. You set up your vector of possible fruits to select from, and then everywhere there is an NA in df$fruit, pick a random element from the possible fruit vector to overwrite it ... View answer 1 answers 0 votes 50 views 1 votes These images aren't very useful to make decisions by eye, because the scale is far too compressed. In the second and third images the mean, entire box and both whiskers are all represented by a single ... View answer 1 answers 0 votes 39 views 1 votes While you can calculate the underlying class probabilities from the Gini index (for binary classification), it'll be more straightforward to calculate it from the &quot;value&quot; line in each box. ... View answer 3 answers 1 votes 1k views 1 votes Shuffling data would not seem to make sense here, since your model has "memory". You're not predicting$y_i$from only$x_i$, but also$x_{i-1}$and$x_{i-2}\$. If you shuffle the data and perform ...

View answer
1 answers
0 votes
21 views
Accepted answer
1 votes

The slot "fp" counts how many false positives there are at each choice of classification cutoff (which can be found in the "cutoff" slot). The cutoff represents at what value you set the threshold to ...

View answer
2 answers
2 votes
111 views
1 votes

A manifold is some kind of low-dimensions structure that exists in a higher-dimensional space. The classic example of this is the Swiss Roll dataset, which simply looks like a spiral with values that ...

View answer
4 answers
0 votes
1k views
1 votes

It appears that you are training your model and generating predictions on the same dataset (X_new). You should not attempt to evaluate your model's performance using this output - because you are ...

View answer
1 answers
0 votes
25 views
Accepted answer
1 votes

If your model is deterministic (no randomness), then repeating the training/testing on the exact same set of data is pointless - you will get the exact same answer every time. The benefit of cross-...

View answer
2 answers
1 votes
3k views
1 votes

For data with lots of features, it's generally the case that many of those features will be unrelated or weakly related to your target variable of interest. It's possible to build a model using even ...

View answer
2 answers
0 votes
4k views
1 votes

There is no reason to sub-sample your test data. The test data serves to give an unbiased estimation of your model learner's performance on unseen data, and more test data only gives you a more ...

View answer
3 answers
0 votes
74 views
1 votes

You could include a category of Other into which you can bin all non-valid responses. This person's response means that they are not single, married, or divorced - it does not represent Unknown, and ...

View answer
2 answers
1 votes
26 views
0 votes

One approach will be to run an unsupervised clustering algorithm of your choosing, parameterized to return a 100-cluster solution. If there are indeed groups of images, you should see that similar ...

View answer
2 answers
1 votes
64 views
Accepted answer
0 votes

Your target variable is whatever you want to predict. For this particular dataset, logical choices are probably "death", "event", or "AZT". You'd typically want to use some kind of patient data to ...

View answer
2 answers
2 votes
362 views
0 votes

The F1 measure is a type of class-balanced accuracy measure - when there are only two classes, it's very straightforward, as there's only one possible way to compute it. With 3 classes, however, you ...

View answer
3 answers
0 votes
59 views
0 votes

In some cases, you may have a model that's a black box - you feed input features and get output predictions, without knowing or caring what happens in the middle. In those situations, the model is, in ...

View answer
2 answers
2 votes
74 views
0 votes

I don't have a proof for this, but I expect the best you can do is estimate the normal distribution of interval times from your training data, and then just predict points at the mean interval time. ...

View answer