Matthew
  • Member for 3 years, 5 months
  • Last seen more than a week ago
  • Boston, MA, United States
2 answers
12 votes
13k views
Feature Scaling both training and test data
24 votes

Generally speaking, best practice is to use only the training set to figure out how to scale / normalize, then blindly apply the same transform to the test set. For example, say you're going to ...

View answer
1 answers
6 votes
2k views
Why is the cosine distance used to measure the similatiry between word embeddings?
Accepted answer
8 votes

You're asking two questions here. Does this mean the magnitude of the vectors is irrelevant? Yes. Cosine distance is $ D_{cos} = \frac{A \cdot B}{\|A\|\|B\|} $, which just comes from the definition ...

View answer
2 answers
1 votes
702 views
Value Function of Generative Adversarial Network
4 votes

Your notation is a little confusing, but I suspect this is because you're not reading the original equation exactly right. $\mathbb{E}_{x \sim p_{data}(x)}$ means "the expectation over $x$ drawn ...

View answer
1 answers
1 votes
172 views
SVR - RMSE is much worse after normalizing the data
3 votes

You normalized it after splitting into train / test / validation, but you're doing this wrong. You need to normalize the training set X_train_normalized = scaler.fit_transform(X_train), and then use ...

View answer
1 answers
1 votes
66 views
Evaluating performance of Generative Adverserial Network?
Accepted answer
3 votes

I think it depends on what exactly you're doing with the GANs. If you're generating images, the two most popular (to my knowledge) are the Inception Score [1] and Frechet Inception Distance [2]. GANs ...

View answer
2 answers
2 votes
5k views
How the combination of cross entropy loss and gradient descent penalizes and rewards
Accepted answer
3 votes

The value of the loss function depends upon the prediction (which is a function of the input data and the model parameters) and the ground truth. Gradient descent works like this: Initialize the ...

View answer
1 answers
2 votes
4k views
Why this model does not converge in keras?
Accepted answer
3 votes

I'm not totally sure exactly what you're doing with your scoring equation, but the first thing you need to look at is your loss function. Categorical Crossentropy is for multilabel classification, and ...

View answer
1 answers
1 votes
54 views
Is there a flexible way to get the original data indices from each cell of a confusion matrix?
2 votes

You can absolutely get this information, but not from the confusion matrices. You want to be comparing the prediction vectors themselves, not the confusion matrices, because as you've rightly ...

View answer
1 answers
0 votes
1k views
Hyperparameter tuning does not improve accuracy?
2 votes

Welcome! You haven't given us enough information to be able to diagnose this issue completely, but you should check your grid search code to see how each cross-validated model is being trained and ...

View answer
1 answers
2 votes
81 views
Is there an oriented clustering algorithm?
Accepted answer
2 votes

If I remember correctly, non-negative matrix factorization (NMF) can be used as a clustering approach that can recover clusters that are along vectors, for example. It may work for your dataset. It ...

View answer
1 answers
3 votes
96 views
How to calculate $\phi_{i,j}$ in VGG19 network?
Accepted answer
2 votes

In section 2.2.1 of the paper, they state that they use euclidean distance. I'm going to take your word that there are 512 filter activations in that layer; if I'm reading this right, there aren't ...

View answer
3 answers
3 votes
4k views
My Keras bidirectional LSTM model is giving terrible predictions
2 votes

The original blog post mentions that the interpretation layer reduces the overall parameter size of the model, although I couldn't figure out by how much. It doesn't look like your network has that, ...

View answer
1 answers
1 votes
20 views
Adding the input layer - units with a decimal
Accepted answer
2 votes

Round up or down to a whole number. Keras documentation specifies that units should be a positive integer, and I'm not sure what a fractional unit would even mean. Does this work when you try to run ...

View answer
2 answers
5 votes
3k views
Latent loss in variational autoencoder drowns generative loss
2 votes

I think your hunch is right. The generative loss can't improve because any movement the network would make towards reducing it comes with a huge penalty in the form of the latent loss. It looks like ...

View answer
2 answers
0 votes
64 views
What is a good approach for a lifespan?
2 votes

I think you need to come up with a way to treat the data such that you're thinking in days, not hours, right? The peaks look they're just at 24, 48, 72, 96, (1 day, 2 days, 3 days, 4 days) etc, and ...

View answer
1 answers
2 votes
55 views
Creating features from raw accelerometer data
1 votes

This is an active area of research called Human Activity Recognition. There are several public datasets available to cross-validate your methods, and you might want to start here: UCI HAR Dataset. ...

View answer
2 answers
2 votes
75 views
Remove unnecessary Neurons from a Neural Network regarding a particular output
1 votes

This is a big field in deep learning. You're going to want to search on network pruning, which is a method of model compression.

View answer
2 answers
0 votes
309 views
Using LSTM for binary text Classification, getting almost same accuracy at each epoch
1 votes

First of all, this is a binary classification problem (positive sentiment / negative sentiment), correct? And the dataset is roughly balanced? What were you trying to achieve by sorting them by length?...

View answer
1 answers
0 votes
71 views
Is it correct to use non-target values of test set to engineer new features for train set?
Accepted answer
1 votes

Definitely (2). You should not include the testing values when calculating features over the data (or normalizing or scaling the data, etc). Let's take a step back for a second. Why are we holding ...

View answer
1 answers
0 votes
33 views
How to handle “not label Y” in a multi class machine learning problem?
Accepted answer
1 votes

I think you should start by looking into Multi-label Classification, as your problem seems to be a subset of MLC, where one example can have multiple correct labels. If one class is "Pictures with ...

View answer
4 answers
1 votes
3k views
Sparse connections in feedforward network tensorflow or pytorch?
1 votes

Do you need specific edges or just a set sparsity level that doesn't change? I know Keras allows you to pass a random seed to the dropout layers. If I understand what you need correctly, you could ...

View answer
1 answers
0 votes
2k views
learning rate very low 1e-5 for Adam optimizer good practice?
1 votes

I think that for the most part, the ends justify the means when it comes to learning rates. If the network is training well and you're confident that you're evaluating its generalization properly, use ...

View answer
1 answers
0 votes
728 views
Keras cNN Transfer Model: Reduce Final Model Size?
1 votes

There's actually some pretty interesting research on this topic. Key words would be model compression or CNN pruning (doing things like reducing the model size by removing filters with low activations)...

View answer
2 answers
4 votes
135 views
overfit random walk using ANN in Keras
1 votes

Try changing your kernel regularizer. Either reduce its effect or shut it off entirely, then retrain and see what happens. EDIT: I expect that the kernel regularizer is your best bet. You could also ...

View answer
2 answers
4 votes
4k views
Categorical data for sklearns Isolation Forrest
Accepted answer
1 votes

I would really try not to use ordinal numbers for categorical data. It imposes a false magnitude and ordering in the model, especially when you have 1,000 examples. For example, the difference between ...

View answer
3 answers
2 votes
942 views
My Keras CNN doesn't learn
1 votes

A few things: Are you sure your data / labels are set up correctly? Is every example of each class exactly the same? Most importantly, why are you using mean squared error? Shouldn't your target ...

View answer
1 answers
0 votes
519 views
Basic DNN with highly imbalanced dataset -- network predicts same labels
Accepted answer
1 votes

A neural network is probably a perfectly reasonable approach here. But this is an extremely unbalanced dataset and you're going to have to handle that somehow. The network is learning that the best ...

View answer
1 answers
1 votes
54 views
Interpreting Categorical Crossentropy Loss
0 votes

Cross-entropy is an information-theoretic measure about probability distributions, and it's measured in units that are determined by the base of the logarithm used in its computation (nats for the ...

View answer
4 answers
1 votes
784 views
Statistics Before Linear Algebra?
0 votes

Can you comment on why this is a very important matter to you? Honestly (my opinion, but it is an opinion-based question) I don't think it should be. If you're talking about a first-semester stats ...

View answer
1 answers
0 votes
573 views
how to shuffle the data for model.fit with custom data generator?
Accepted answer
0 votes

The big issue here is that your generator yields after each file is loaded. This means that your batch size is always the number of examples stored in each file and that the training examples in each ...

View answer