What's the deal with Deno? We talk with a major contributor to find out. Listen now.

Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
123
votes
6answers
167k views

How to draw Deep learning network architecture diagrams?

I have built my model. Now I want to draw the network architecture diagram for my research paper. Example is shown below:
2
votes
2answers
1k views

Series prediction for any given time

I have a time series of data points. Then I am given a future timestamp and I have to predict the value for the data point. For simplicity, you can assume that the timestamp is bounded i.e. for e.g. ...
2
votes
1answer
3k views

Data set size versus data dimension, is there a rule of thumb?

I am trying to collect some data for ML, specifically for training a neural network model, and I don't know how big the data set is enough. So is there a rule of thumb on how many data of dimension <...
1
vote
0answers
272 views

Is there a NN-Model which has multiple outputs?

This is what I am trying to accomplish: Predict Y based on X,that is Y ~ X: 1.X = {n, x_1, x_2, x_3, ...} a vector of factors ...
1
vote
2answers
204 views

Time complexity of function minimizers for neural networks

I am trying to train a neural network for recognizing handwritten letters from A to J . I have a training set of size 200000 . Each training set is a list of 784 ...
1
vote
2answers
3k views

Making Keras + Tensorflow code execution deterministic on a GPU

I'm executing this code from https://www.kaggle.com/danijelk/allstate-claims-severity/keras-starter-with-bagging-lb-1120-596 on a nvidia geforce 960M. ...
3
votes
3answers
545 views

Are there libraries or techniques for 'noisifying' text data?

Data augmentation techniques for image data and audio data (eg speech recognition) have proven successful and are now common. Are there libraries or techniques for augmenting text data? For example: ...
1
vote
1answer
41 views

Is there standard parameters that characterize how fast neural network learns to achieve specific error rate?

I want to estimate how fast my model learn. Mb how many learn steps my model need to do to achieve specific error rate(precision/ recall/ MSE/ etc.). Any standard approach for measuring this?
2
votes
1answer
2k views

applying convolutional neural network over text documents using 1-D tf-idf feature vectors

I want to apply a CNN over documents. I have tf-idf vectors of documents with me (one vector per document). My question is, is 1D CNN applicable in this case? The reason I am asking this question is ...
1
vote
1answer
240 views

Future of deep learning (compared to traditional machine learning) [closed]

What do you think will be the future of deep learning? A lot of people talk about deep learning and I can see that it provides various possibilities. For instance, this question gives a nice ...
128
votes
6answers
107k views

When to use GRU over LSTM?

The key difference between a GRU and an LSTM is that a GRU has two gates (reset and update gates) whereas an LSTM has three gates (namely input, output and forget gates). Why do we make use of GRU ...
1
vote
1answer
3k views

tensorflow mnist tutorial

i have a question about the tutorial of tensorflow to train the mnist database how do i create my own batch without using next_batch() , the idea is to train with a batch of 50 ,then 100 and so but it ...
1
vote
0answers
675 views

Dealing with noisy training labels in text classification using deep learning

I have a dataset that comprises of sentences and corresponding multi-labels (e.g. a sentence can belong to multiple labels). Using a combination of Convolutional Neural Networks and Recurrent Neural ...
1
vote
2answers
176 views

How can I create a classifier using the feature map of a CNN?

I intend to make a classifier using the feature map obtained from a CNN. Can someone suggest how I can do this? Would it work if I first train the CNN using +ve and -ve samples (and hence obtain the ...
32
votes
1answer
50k views

How does Keras calculate accuracy?

How does Keras calculate accuracy from the classwise probabilities? Say, for example we have 100 samples in the test set which can belong to one of two classes. We also have a list of the classwise ...
14
votes
3answers
794 views

How are deep-learning NNs different now (2016) from the ones I studied just 4 years ago (2012)?

It is said in Wikipedia and deeplearning4j that Deep-learning NN (DLNN) are NN that have >1 hidden layer. These kind of NN were standard at university for me, while DLNN are very hyped right now. ...
2
votes
1answer
90 views

How to train neural network that has different kind of layers

If we have MLP then we can easily compute the gradient for each parameters, by computing the gradient recursively begin with the last layer of the network, but suppose I have neural network that ...
1
vote
0answers
70 views

Right choice of accuracy metric or loss function

I am developing a predictive model in sports vertical. As the game will progress, my model will predict the winning probability of each team playing. The problem I am facing is to what metric would ...
2
votes
0answers
78 views

GPU Utilisation Issues

I observed that my GPU's memory is being consumed but the Utilisation stays 0. Because of this, my model is taking forever to load. I have tweaked this code to handle multilabel data. The only changes ...
6
votes
2answers
964 views

Earlystopping in multi-output deep learning

When working with a neural network with more than one output, what is generally advised as the best strategy for early-stopping the training process? Given that I am currently monitoring the net ...
3
votes
1answer
2k views

Activation method and Loss function for multilabel multiclass classification

I am using CNN for Sentence Classification code by Yoonkim. This is used for text classification. I noticed that he uses softmax layer and negative log likelihood error. This is optimal for single ...
9
votes
2answers
6k views

Features of word vectors in word2vec

I am trying to do sentiment analysis. In order to convert the words to word vectors I am using word2vec model. Suppose I have all the sentences in a list named 'sentences' and I am passing these ...
1
vote
1answer
5k views

Accuracy doesn't match in Keras

I am using Keras to classify images. I am following the Keras blog. The accuracy from predict_generator is not matching with the accuracy obtained from the confusion matrix, which I am computing ...
1
vote
2answers
965 views

Word vectors as input

I have a corpus on which I want to perform sentiment analysis using LSTM and word embeddings. I have converted the words in the documents to word vectors using word2vec. My question is how to input ...
18
votes
3answers
58k views

How to get predictions with predict_generator on streaming test data in Keras?

In the Keras blog on training convnets from scratch, the code shows only the network running on training and validation data. What about test data? Is the validation data the same as test data (I ...
2
votes
1answer
91 views

Does dropout require multiple passes of the same data set, as a sort of ensemble method?

I'm a bit confused about dropout -- on one tutorial, it was described as basically an 'ensemble method' of sorts. This implies that you might need to create an ensemble of networks. Is this the case, ...
1
vote
2answers
3k views

How to deal with situation where LSTM fails to learn (constantly makes the same incorrect prediction) [closed]

I am trying to use LSTM neural networks in order to make a song composer. Basically this is based of a text generator (tries to predict the next character after looking at a sequence of characters) ...
0
votes
1answer
112 views

Extracting the code from Keras

Suppose I write a program in Keras for MNIST data set. I used Tensorflow as my backend in keras. Is it possible in any way that I can extract the back end code used for tensorflow that has been used ...
-1
votes
1answer
53 views

Shared weights in convolutional neutral network

In convolutional neutral network, the weights are shared within a feature map. What about two different feature map? How to make them different (so that we don't learn the same thing again). Q: What ...
1
vote
0answers
454 views

deep learning - how to train silhouette image?

I want to train silhouette images to figure out the original image I'm planning to use Tensorflow. Once I've thought about using Automatic colorization, but I have no idea if it works well since ...
1
vote
0answers
139 views

Policy network AlphaGo and transferring to other domains

This is not covered very well in their AlphaGo paper but I assume that their policy network has a softmax output layer with a node for all the positions on the board, including illegal ones (the one ...
13
votes
2answers
21k views

Validation loss and accuracy remain constant

I am trying to implement this paper on a set of medical images. I am doing it in Keras. The network essentially consists of 4 conv and max-pool layers followed by a fully connected layer and soft max ...
4
votes
1answer
7k views

What techniques to use for image matching

I have a database with around 30,000 pictures. All of them are a different object. They are all from a certain perspective, the pictures itself are the same size but the objects vary in size. I want ...
1
vote
1answer
1k views

Similarity between bordermode and zero padding in Keras

In Keras, the border_mode = 'valid' doesn't zero pad the input. Thus, we subsequently get an output feature map that is not the same size as the input. Likewise, ...
3
votes
1answer
3k views

Principle behind seq2seq model's example in keras?

I am referring to seq2seq model's example code in keras (https://github.com/fchollet/keras/blob/master/examples/addition_rnn.py). The model is : ...
2
votes
1answer
103 views

Is deep learning a must in a Data Science MSc programme? [closed]

I am reading the programme outline of this two-year MSc in Data Science and I found that it has no deep learning content (as in many other european ones). I am no expert but as far as I've seen I ...
1
vote
2answers
3k views

Example of binary classifier with numerical features using deep learning

I would like to get more understanding of deep learning. Browsing the web I find applications in speech recognition and hand-written digits. However I would be interested to get some guidance on how ...
0
votes
1answer
7k views

Error in model.fit() method in Keras

I was building a model for a classification problem in Keras for which I used the KerasClassifier, the wrapper scikit-learn. Below is the code for the same. ...
1
vote
1answer
209 views

Reason for better performance of variants of SGD when local minimas of Neural Nets are equivalent?

From some neural net article I read that if you scale up the Neural Net architecture the differnce in different local minimas in the loss surface diminishes. Essentially all local minimas become ...
6
votes
1answer
21k views

Regression in Keras

I was trying to implement a regression model in Keras. But I am unable to figure out how to calculate the score of my model i.e. how well it performed on my dataset. ...
35
votes
4answers
25k views

Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

I read about NCE (a form of candidate sampling) from these two sources: Tensorflow writeup Original Paper Can someone help me with the following: A simple explanation of how NCE works (I found the ...
7
votes
1answer
2k views

Recurrent neural network multiple types of input Keras

For a project I want to use recurrent neural networks, however my knowledge on this subject is still somewhat limited. I do have some experience with convolutional nets and traditional neural networks....
2
votes
1answer
337 views

Deep learning facial recognition research project

I'm a second year pure maths, applied maths and computer science student. I have taken up a research course and was given the topic to focus on facial recognition using deep learning. I have done a ...
1
vote
0answers
111 views

What did DeepMind do with AlphaGo between the Fan Hui and Lee Sedol games?

In January, DeepMind published the article (see video) about its win against Fan Hui, which happend in October 2015. The article and other interviews say, it used 100.000 human games, and then 13.000....
1
vote
3answers
490 views

Training multi-label classifier with low quality training set

So I'm creating a topics classifier where a document may be tagged for several different topics, let's say - A, B while actually the document belongs to A, B and C. In the training stage I want the ...
70
votes
2answers
60k views

When to use (He or Glorot) normal initialization over uniform init? And what are its effects with Batch Normalization?

I knew that Residual Network (ResNet) made He normal initialization popular. In ResNet, He normal initialization is used , while the first layer uses He uniform initialization. I've looked through ...
1
vote
0answers
49 views

Find boundaries for a smallest mean error

I am new at deep learning, but willing to learn. I have this problem. Have 5 inputs: A,B,C,D,E(columns in CSV) that gives me 1 or 0. A,B,C,D or E can be between 0 and 100. For example, a ...
3
votes
2answers
3k views

How Int8 (byte) operations can be useful for deep learning?

Nvidia is planing to add hardware support for int8 operations to their titan card and target deep learning. I am trying to understood how its useful and what types of network will benefit from this. ...
32
votes
1answer
11k views

Paper: What's the difference between Layer Normalization, Recurrent Batch Normalization (2016), and Batch Normalized RNN (2015)?

So, recently there's a Layer Normalization paper. There's also an implementation of it on Keras. But I remember there are papers titled Recurrent Batch Normalization (Cooijmans, 2016) and Batch ...
4
votes
0answers
263 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...