StatsSorceress
  • Member for 4 years, 9 months
  • Last seen this week
Forget Layer in a Recurrent Neural Network (RNN) -
Accepted answer
15 votes

Great question! tl;dr: The cell state and the hidden state are two different things, but the hidden state is dependent on the cell state and they do indeed have the same size. Longer explanation ...

View answer
Sliding window leads to overfitting in LSTM?
Accepted answer
13 votes

Although the previous answer by @Imran is correct, I feel it necessary to add a caveat: there are applications out there where people do feed a sliding window in to an LSTM. For example, here, for ...

View answer
CNN memory consumption
9 votes

I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,P2 you mean pooling layers, and FC means fully connected layers. We can calculate the memory required for a forward pass like ...

View answer
Trying to make compelling plot for classification results with python
6 votes

It's really rare that you'd show a plot of the probabilities for each example in your set. Are you sure you want to do this? A better presentation might be a confusion matrix. Here's how it works: 1) ...

View answer
How to calculate the mini-batch memory impact when training deep learning models?
5 votes

I think you're on the right track. Yes, you will need to store the derivatives of the activations and of the parameters for backpropagation. Additionally, your choice of optimization may matter. ...

View answer
What is difference between feed forward neural network and LSTM?
Accepted answer
4 votes

A feed-forward neural network looks like this: input -> hidden layer 1 -> hidden layer 2 -> ... -> hidden layer k -> output. Each layer may have a different number of neurons, but that's the ...

View answer
Layer notation for convolutional neural networks
Accepted answer
3 votes

One paper referenced by the first paper you linked to is here. It explains in section 3 (experiments) the following notation: 2x48x48-100C5-MP2-100C5-MP2-100C4-MP2-300N-100N-6N represents a net ...

View answer
How to cross-validate a deep learning model for highly imbalanced datasets?
Accepted answer
3 votes

Accuracy is not a good indicator of success with imbalanced data. The accepted answer is correct: F1 score is commonly used. Other options include roc_auc_score (see here) and average_precision_score (...

View answer
Tips and tricks for designing time-series variational autoencoders
Accepted answer
3 votes

I can speak from a more theoretical point of view, but honestly I haven't had much success with VAEs. 1) How deep should my encoder and decoder network be? Are there any good guidelines? That ...

View answer
What is the meaning of "The number of units in the LSTM cell"?
3 votes

In Keras, which sits on top of either TensorFlow or Theano, when you call model.add(LSTM(num_units)), num_units is the dimensionality of the output space (from here, line 863). To me, that means ...

View answer
which neural network topology to learn correlations between time series?
2 votes

It depends a little on what kind of correlations you're looking for. Are you expecting a correlation that is present at each time step/window, or a different level of correlation per time step/window? ...

View answer
Classification of very similar images
2 votes

You're going to have to do some experimenting to figure out what is 'best', but I would recommend starting with a convolutional neural network. Since you're only detecting a very small difference, ...

View answer
Matrix Confusion - Get Model Precision
Accepted answer
2 votes

A confusion matrix gives you the following: [TP, FP] [FN, TN] where TP = 'true positives'; FP = 'false positives'; FN = 'false negatives'; TN = 'true negatives'. You can read more here: http://www....

View answer
Logistic Regression Independent Samples
2 votes

Context / big picture: Two events are independent if they have no influence on each other's outcomes. For example, if event A is "I go get coffee" and event B is "it's raining outside", then events A ...

View answer
LSTM predicting expected value and standard deviation with one-hot encoding
2 votes

The standard deviation and mean of a categorical variable is not meaningful. It looks like the original data are from a range of [0, 20), and the space has been discretized. Now, instead of ranges, ...

View answer
What is a batch in machine learning?
2 votes

The accepted answer is correct, but it may also be helpful to think of a batch from a classification standpoint. Suppose you have a binary classification problem that you are trying to solve using a ...

View answer
Calculate likelihood of a date being worked based on location, hours and rate of pay
1 votes

If you're looking for code, I'm not familiar with C#. My answer will focus on theory. tl;dr most machine learning-related packages have a built-in logistic regression function of some sort. That's ...

View answer
What is the minimum requirement for the dataset for time series forecasting?
Accepted answer
1 votes

Welcome to the site! Forecasting can be done using any length of time series. For example, if I have a set of data {1, 10, 19, 28}, then I can be pretty sure that the next value in the set is going ...

View answer
Adam optimizer for projected gradient descent
1 votes

tl;dr: All gradient descent-based methods have an update rule similar to Adam's, so I think they'd all give you some grief. I don't know of any optimizers tailored to your application (but maybe you ...

View answer
Standard Deviation for Z-scores
Accepted answer
1 votes

I agree with some aspects of Skiddles' answer, but not all. Assume your data set contains n observations. Based on your question, I see three possibilities: If you're interested in the number of ...

View answer
LSTM: How to deal with nonstationarity when predicting a time series
1 votes

Looking again at your autocorrelated process: def my_process(n, p, drift=0, displacement=0): x = np.zeros(n) for i in range(1, n): x[i] = drift * i + p * x[i-1] + (1-...

View answer
Multivariate time series classification
1 votes

Following up on the comment about deep learning, with high dimensional time series data you would be much better served with a recurrent-type of deep model. For example, an LSTM is a very good ...

View answer
Applying machine learning algorithms to subset of attributes in dataframe
Accepted answer
1 votes

Applying a machine learning algorithm on only a subset of the data and including other subsets later does not allow the algorithm to assess the importance of each attribute equally. For example, say ...

View answer
Right Way to Input Text Data in Keras Auto Encoder
Accepted answer
1 votes

Here's my take on your questions. Yes, you can zero-pad vectors. However, I would strongly recommend you use an LSTM as part of your encoder if you're using sentences as input. An LSTM can take ...

View answer
Several fundamental questions about CNN
1 votes

Here's my take on each of your points. You have very sparse data. Are you storing these data optimally in a sparse object, for example a csr sparse matrix if you're using Keras? This is more likely ...

View answer
What are some methodologies for performing feature selection for simple feed-forward neural networks?
1 votes

In addition to what was suggested by @Media, you may consider adding a softmax layer to your model right above your input layer. This is a way of visualizing which features are strongly associated ...

View answer
Clarification on the Keras Recurrent Unit Cell
1 votes

The value '32' in this case is the size of the cell state and the size of the hidden state being sent forward in the network. Please see my answer here for more information.

View answer
R Rpart taking too much time on data set
1 votes

Have you tried using randomForest instead of rpart? For example, let's assume you have two data.frames: train_data and test_data In my example, the last column is the class (and is a factor variable)...

View answer
How to measure variance in a classification dataset?
1 votes

I would start with logistic regression, and get a measure of importance for each of your 20 predictor variables. It's a little tough to understand what you mean by 'variance' in your two classes of ...

View answer
How can we convert time series data to supervised learning problem?
0 votes

Short answer: Yes. Long answer: See here: https://machinelearningmastery.com/time-series-forecasting-supervised-learning/

View answer