David Marx
  • Member for 7 years, 3 months
  • Last seen this week
How can I fit categorical data types for random forest classification?
Accepted answer
18 votes

You need to convert the categorical features into numeric attributes. A common approach is to use one-hot encoding, but that's definitely not the only option. If you have a variable with a high number ...

View answer
Is PCA considered a machine learning algorithm
16 votes

PCA is actually just a rotation. Seriously, that's all: it's a clever way to spin the data around onto a new basis. This basis has properties that make it useful as a pre-processing step for several ...

View answer
Decision trees: leaf-wise (best-first) and level-wise tree traverse
Accepted answer
13 votes

If you grow the full tree, best-first (leaf-wise) and depth-first (level-wise) will result in the same tree. The difference is in the order in which the tree is expanded. Since we don't normally grow ...

View answer
What is the use of additional column of 1s in normal equation?
Accepted answer
11 votes

The normal equations are designed such that each coefficient in the model has an input of some kind it's being multiplied against. The column of ones is the "input" to the intercept term.

View answer
Find optimal P(X|Y) given I have a model that has good performance when trained on P(Y|X)
Accepted answer
11 votes

This response has been significantly modified from its original form. The flaws of my original response will be discussed below, but if you would like to see roughly what this response looked like ...

View answer
CNN - How does backpropagation with weight-sharing work exactly?
Accepted answer
10 votes

I think you're misunderstanding what "weight sharing" means here. A convolutional layer is generally comprised of many "filters", which are usually 2x2 or 3x3. These filters are applied in a "sliding ...

View answer
High root mean squared error in regression model
8 votes

RMSE does not work that way. A RMSE of 13 might actually be great, it completely depends on how your target variable is scaled. For example, if your target variable was in the range [0,1e9], then a ...

View answer
Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?
8 votes

Because of the encoder-decoder structure. The encoder reads the input sequence to construct an embedding representation of the sequence. Terminating the input in an end-of-sequence (EOS) token signals ...

View answer
How to get a confidence score for predictions?
8 votes

No matter the model, you can always use the non-parametric bootstrap to construct a confidence interval for any parameter, including predictions (which are actually random variables themselves but are ...

View answer
Is correlation needed when building a model?
Accepted answer
7 votes

Not really, no. Sort of. It depends on how complex your model/data is. It's entirely possible to have a situation where a feature taken in isolation will not be correlated with the target variable, ...

View answer
Does it make sense to combine PCA with an artificial neural network?
5 votes

Neural networks are actually extremely effective at performing dimensionality reduction. A great example is word2vec, which applies a shallow neural network to reduce inputs on the order of several ...

View answer
Predict whether or not a user will visit the library tomorrow using historical data
5 votes

First and foremost, you need to reformat your data into what's called a balanced panel structure. For each day in your training data, each user should have a record for that day associated with an ...

View answer
Training a Convnet on 300GB data
Accepted answer
4 votes

You don't need to load the whole dataset into memory at once. The only data you need in memory are the samples in a single training batch. Use the fit_generator method rather than fit to pass in an ...

View answer
Best method to deal with too many zeroes in regression problem?
Accepted answer
4 votes

Option (b) is a good approach. If you use a classifier that outputs probabilities, you can even multiply the two model outputs together to calculate a risk expectation that's interpretable as the ...

View answer
What is a policy in machine learning?
4 votes

It's not so much a machine learning term as it is a control theory term. A "control policy" is a heuristic that suggests a particular set of actions in response to the current state of the agent (in ...

View answer
How to extract Question/s from document with NLTK?
4 votes

Check out chapter 6 section 2.2 of the NLTK book. EDIT: since apparently the community wants me to copy/paste stuff, here ya go: 2.2 Identifying Dialogue Act Types When processing dialogue, it ...

View answer
Are linear regression models with non linear basis functions used in practice?
Accepted answer
3 votes

Yep, that's a thing. It's called a "Generalized additive model (GAM)": https://en.wikipedia.org/wiki/Generalized_additive_model You may also be interested in "multivariate adaptive regression ...

View answer
To be useful, doesn't a test set often become a second dev set?
Accepted answer
3 votes

You're absolutely right and yes, it is actually possible to overfit to your validation data if you're not careful. Some researchers at google published an interesting article about this problem and a ...

View answer
Very low probability in naive Bayes classifier
Accepted answer
3 votes

I can't tell you for sure without you describing your calculation more or showing code, but my guess is you're not actually calculating the posterior probability here. I bet this is just the ...

View answer
I have 50 videos. I ask a customer 10 questions. Based on their answers, I send them a set of videos. How do I do it?
3 votes

Frame this as a classification problem and learn a decision tree to map question responses to video selections. EDIT: Fleshing this out a bit more: Collect appropriate data. Get members of your ...

View answer
Detect constant (zero-slope) sections in a noisy step function
3 votes

For what you're trying to do, first-order differences without interpolation should work just fine. Once you've done that, the problem reduces to a simple anomaly detection task. Here's a demonstration ...

View answer
How to treat input that inherently has a tree structure?
Accepted answer
3 votes

As @Emre mentioned, RNN is a good option. It's worth noting that if the number of possible nodes in each tree is the same or at least has the same upper bound, you could use literally any architecture ...

View answer
The sum of probabilities is more than 1
2 votes

You need to use appropriate activations. If you were using softmax for those two components, they'd be constrained to sum to one. LeakyReLU not only doesn't impose this constraint: it can output any ...

View answer
What makes you confident in your results? At what point do you think you can present your work to tech illiterate superiors?
2 votes

What makes you confident in your results? The appropriate method to evaluate whether you have modeled a real signal or noise is completely dependent on the question you are asking and the modeling ...

View answer
Data Snooping, Information Leakage When Performing Feature Normalization
Accepted answer
2 votes

The normalization parameters you fitted in training are now part of your model. You fitted the model weights on the training data: the normalization step is part of your model now, and the "parameters"...

View answer
Learning Rate based on error of the network
Accepted answer
2 votes

Your intuition is on point, and shrinking the learning rate like this is often referred to as "annealing". But linking the learning rate to error magnitude neglects certain problematic error surface ...

View answer
Does K-Means' objective function imply distance metric is Euclidean
Accepted answer
2 votes

You're right and you're wrong. The objective/loss function of K-Means algorithm is to minimize the sum of squared distances Yes, absolutely. written in a math form, it looks like this: $$J(X,...

View answer
How cross validation works for regression?
2 votes

The idea behind cross validation is to understand the performance of some measure of your model's performance on unseen data. This can be applied to loads of different statistics, not just ones ...

View answer
Under what conditions should an autoencoder be chosen over kernel PCA?
2 votes

I think you can interpret autoencoders as essentially performing kernel PCA, but with an extremely complex kernel. Like, really, stupidly complex. The kinds of things you can accomplish with ...

View answer
Strange patterns from GAN
2 votes

The deconv layers are probably to blame. Check out this distill article for a fairly in depth discussion about how deconv layers create checkerboard artifacts. The gist is that deconv striding creates ...

View answer