jamesmf
  • Member for 6 years, 5 months
  • Last seen more than 3 years ago
Calculating KL Divergence in Python
22 votes

Scipy's entropy function will calculate KL divergence if feed two vectors p and q, each representing a probability distribution. If the two vectors aren't pdfs, it will normalize then first. Mutual ...

View answer
Bagging vs Dropout in Deep Neural Networks
Accepted answer
22 votes

Bagging and dropout do not achieve quite the same thing, though both are types of model averaging. Bagging is an operation across your entire dataset which trains models on a subset of the training ...

View answer
Multivariate linear regression in Python
Accepted answer
14 votes

You can still use sklearn.linear_model.LinearRegression. Simply make the output y a matrix with as many columns as you have dependent variables. If you want something non-linear, you can try ...

View answer
Data Science Podcasts?
Accepted answer
13 votes

I strongly suggest Talking Machines. It's a very well put together podcast from a professor at Harvard. They cater to both machine learning experts and enthusiasts. Their interviews are often done ...

View answer
How word2vec can be used to identify unseen words and relate them to already trained data
9 votes

Every algorithm that deals with text data has a vocabulary. In the case of word2vec, the vocabulary is comprised of all words in the input corpus, or at least those above the minimum-frequency ...

View answer
Unbalanced classes -- How to minimize false negatives?
9 votes

Class imbalance is a very common problem. You can either oversample the positive class (or undersample the negative) or add class weights. Another thing to remember in this case is that accuracy is ...

View answer
What is a benchmark model?
Accepted answer
9 votes

Benchmarking is the process of comparing your result to existing methods. You may compare to published results using another paper, for example. If there is no other obvious methodology against which ...

View answer
Feature extraction of images in Python
8 votes

This great tutorial covers the basics of convolutional neuraltworks, which are currently achieving state of the art performance in most vision tasks: http://deeplearning.net/tutorial/lenet.html ...

View answer
Properties for building a Multilayer Perceptron Neural Network using Keras?
Accepted answer
6 votes

1) Activation is an architecture choice, which boils down to a hyperparameter choice. You can make a theoretical argument for using any function, but the best way to determine this is to try several ...

View answer
How to prepare colored images for neural networks?
Accepted answer
6 votes

Your R,G, and B pixel values can be broken into 3 separate channels (and in most cases this is done for you). These channels are treated no differently than feature maps in higher levels of the ...

View answer
The cross-entropy error function in neural networks
6 votes

Those issues are handled by the tutorial's use of softmax. For 1) you're correct that softmax guarantees a non-zero output because it exponentiates it's input. For activations that do not give this ...

View answer
What are the best ways to tune multiple parameters?
Accepted answer
6 votes

Generally people perform a grid search, which in its simplest "exhaustive" form is similar to Method 1. However there are also more 'intelligent' ways to choose what to explore, which optimize in ...

View answer
Neural Network parse string data?
6 votes

Both the answers from @Emre and @Madison May make good points about the issue at hand. The problem is one of representing your string as a feature vector for input to the NN. First, the problem ...

View answer
With unbalanced class, do I have to use under sampling on my validation/testing datasets?
5 votes

For 1) and 2), you want to 1) choose a model that performs well on data distributed as you expect the real data will be 2) evaluate the model on data distributed the same way So for those ...

View answer
Appropriate algorithm for string (not document) classification?
Accepted answer
5 votes

You might find it useful to treat n-grams of characters as your feature space. Then you could represent a string as a bag of substrings. With N = 4 or greater, you would capture things like ".com" in ...

View answer
Can you explain the difference between SVC and LinearSVC in scikit-learn?
5 votes

If you used the default kernel in SVC(), the Radial Basis Function (rbf) kernel, then you probably learned a more nonlinear decision boundary. In the case of the digits dataset, this will vastly ...

View answer
Categorical and ordinal feature data representation in regression analysis?
Accepted answer
4 votes

The distinction between ordinal and categorical does matter. If in truth the difference between white and red was drastically different from red and black, your (10,20,30) ordinal model would not have ...

View answer
Deconvolutional Network in Semantic Segmentation
Accepted answer
4 votes

There are two main functions they undo. The pooling layers in the convolutional neural network downsample the image by (usually) taking the maximum value within the receptive field. Each rxr image ...

View answer
SPARK 1.5.1: Convert multi-labeled data into binary vector
Accepted answer
4 votes

Spark only recently implemented CountVectorizer, which will take the labels (as strings) and encode them as your 100-dimensional vector (assuming all 100 labels show up somewhere in your dataset). ...

View answer
Classification problem where one attribute is a vector
Accepted answer
4 votes

1) If you want to build a model with: Input: Items bought Output: Win/Loss then you will probably want to learn a non-linear combination of the inputs to represent a build. For example item_X may ...

View answer
Using several documents with word2vec
3 votes

There are a number of implementations of Word2Vec, but most assume the basic unit to be 'sentences' - though they don't care what those sentences look like. If you are using something like gensim you ...

View answer
How important is the balance between the components of a loss value?
3 votes

The cost function controls the algorithm completely - the new regularization from weight decay is likely responsible for the jump in loss. If you only added a small amount of regularization and the ...

View answer
How to draw a hyperplane using the weights calculated
Accepted answer
3 votes

If you have only an input layer, one set of weights, and an output layer, you can solve this directly with $$ X \cdot w = threshold $$ However if you add in hidden layers, you no longer necessarily ...

View answer
What does RMSE points about performance of a model in machine learning?
3 votes

There are multiple factors to consider, but the first thing to realize is that in regression, you don't want to think about whether an example is "correct" or "incorrect" but rather how close it was ...

View answer
How to extract important phrases (which may contain company name) from resume?
3 votes

Sounds like you want named entity recognition. There are a variety of approaches to NER, and plenty of implementations, like the Stanford NER package. After you find named entities, determining what ...

View answer
Why are there currently no content-based evaluation metrics for information retrieval?
3 votes

Using a word-based metric would explicitly favor word-level retrieval methods. The theory is that (just as you suggest with dwell time), the URL-level metric measures more directly the desired result....

View answer
Improving SVM binary classification model on new dataset
3 votes

Online learning for SVMs is supported only for certain SVM options (a linear SVM with SGD can easily update for example). There are implementations out there to address the issue of online learning ...

View answer
collaborative filtering using graph and machine learning
3 votes

One advantage of many ML-based recommendation techniques is they allow you to work in a lower-dimensional space. Matrix factorization techniques for example, allow you to view a user or an item in ...

View answer
Identifying templates with parameters in text fragments
3 votes

You might consider using word2vec to identify phrases in the corpus. The presence of a phrase (instead of single tokens) is likely to indicate a 'template.' From here, the tokens most similar to ...

View answer
NLTK: Tuning LinearSVC classifier accuracy? - Looking for better approaches/advices
3 votes

You might try looking into sentiment analysis. There was a kaggle competition on it, and you might find insight there. Treating this as either a regression or a classification problem is fair. Also,...

View answer