Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
32
votes
6answers
6k views

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide ...
34
votes
5answers
33k views

Does gradient descent always converge to an optimum?

I am wondering whether there is any scenario in which gradient descent does not converge to a minimum. I am aware that gradient descent is not always guaranteed to converge to a global optimum. I am ...
27
votes
6answers
10k views

Why do convolutional neural networks work?

I have often heard people saying that why convolutional neural networks are still poorly understood. Is it known why convolutional neural networks always end up learning increasingly sophisticated ...
150
votes
5answers
87k views

What is the “dying ReLU” problem in neural networks?

Referring to the Stanford course notes on Convolutional Neural Networks for Visual Recognition, a paragraph says: "Unfortunately, ReLU units can be fragile during training and can "die". For ...
56
votes
5answers
30k views

Adding Features To Time Series Model LSTM

have been reading up a bit on LSTM's and their use for time series and its been interesting but difficult at the same time. One thing I have had difficulties with understanding is the approach to ...
50
votes
4answers
50k views

Why mini batch size is better than one single “batch” with all training data?

I often read that in case of Deep Learning models the usual practice is to apply mini batches (generally a small one, 32/64) over several training epochs. I cannot really fathom the reason behind this....
44
votes
8answers
46k views

Why should the data be shuffled for machine learning tasks

In machine learning tasks it is common to shuffle data and normalize it. The purpose of normalization is clear (for having same range of feature values). But, after struggling a lot, I did not find ...
48
votes
5answers
47k views

Number of parameters in an LSTM model

How many parameters does a single stacked LSTM have? The number of parameters imposes a lower bound on the number of training examples required and also influences the training time. Hence knowing the ...
51
votes
3answers
17k views

What is the difference between “equivariant to translation” and “invariant to translation”

I'm having trouble understanding the difference between equivariant to translation and invariant to translation. In the book Deep Learning. MIT Press, 2016 (I. Goodfellow, A. Courville, and Y. Bengio)...
17
votes
1answer
25k views

Why ReLU is better than the other activation functions

Here the answer refers to vanishing and exploding gradients that has been in sigmoid-like activation functions but, I guess, Relu...
24
votes
2answers
28k views

Should we apply normalization to test data as well?

I am doing a project on author identification problem. I had applied the tf-idf normalization to train data and then trained a svm on that data. Now when using the classifier should I normalize test ...
12
votes
1answer
3k views

Using a pre trained CNN classifier and apply it on a different image dataset

How would you optimize a pre-trained neural network to apply it to a separate problem? Would you just add more layers to the pre-trained model and test it on your ...
10
votes
1answer
17k views

Keras LSTM with 1D time series

I'm learning how to use Keras and I've had reasonable success with my labelled dataset using the examples on Chollet's Deep Learning for Python. The data set is ~1000 Time Series with length 3125 with ...
5
votes
2answers
6k views

What are useful evaluation metrics used in machine learning

I am using CNN in order to predict codes after analyzing text. As an example, I will write "I am crazy" .. the model will predict some code " X321". All this based on CNN. I want to evaluate my ...
8
votes
1answer
13k views

Why should softmax be used in CNN

In the last layer of CNNs and MLPs it is common to use softmax layer or units with sigmoid activation functions for multi-class ...
109
votes
16answers
101k views

How do you visualize neural network architectures?

When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture. What are good / simple ways to visualize common ...
123
votes
6answers
166k views

How to draw Deep learning network architecture diagrams?

I have built my model. Now I want to draw the network architecture diagram for my research paper. Example is shown below:
24
votes
6answers
9k views

Deep learning basics

I am looking for a paper detailing the very basics of deep learning. Ideally like the Andrew Ng course for deep learning. Do you know where I can find this ?
78
votes
5answers
47k views

Time series prediction using ARIMA vs LSTM

The problem that I am dealing with is predicting time series values. I am looking at one time series at a time and based on for example 15% of the input data, I would like to predict its future values....
70
votes
2answers
60k views

When to use (He or Glorot) normal initialization over uniform init? And what are its effects with Batch Normalization?

I knew that Residual Network (ResNet) made He normal initialization popular. In ResNet, He normal initialization is used , while the first layer uses He uniform initialization. I've looked through ...
20
votes
2answers
5k views

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
5
votes
1answer
30k views

How to download dynamic files created during work on Google Colab?

I have two different files and on the first, I tried to save data to file as: np.save(open(Q1_TRAINING_DATA_FILE, 'wb'), q1_data) On second file, i'm trying to ...
15
votes
1answer
18k views

What is a 1D Convolutional Layer in Deep Learning?

I have a good general understanding of the role and mechanism of convolutional layers in Deep Learning for image processing in case of 2D or 3D implementations - they "simply" try to catch 2D patterns ...
8
votes
2answers
8k views

How to plot cost versus number of iterations in scikit learn?

One of the recommendations in the Coursera Machine Learning course when working with gradient descent based algorithms is: Debugging gradient descent. Make a plot with number of iterations on the x-...
2
votes
1answer
333 views

Compute backpropagation

I have the question which is mentioned in the above picture. It is trying to find the derivative of f with respect to weight matrix ...
2
votes
2answers
948 views

Explain Binary Classification with output 0.5 (True)

What is the interpretation of output 0.5 of a typical classifier? I made a prediction and the probability of that data point being from the True class is 0.5.
51
votes
3answers
55k views

Does batch_size in Keras have any effects in results' quality?

I am about to train a big LSTM network with 2-3 million articles and am struggling with Memory Errors (I use AWS EC2 g2x2large). I found out that one solution is to reduce the ...
26
votes
3answers
2k views

Why are NLP and Machine Learning communities interested in deep learning?

I hope you can help me, as I have some questions on this topic. I'm new in the field of deep learning, and while I did some tutorials, I can't relate or distinguish concepts from one another.
32
votes
1answer
50k views

How does Keras calculate accuracy?

How does Keras calculate accuracy from the classwise probabilities? Say, for example we have 100 samples in the test set which can belong to one of two classes. We also have a list of the classwise ...
15
votes
2answers
10k views

Is there away to change the metric used by the Early Stopping callback in Keras?

When using the Early Stopping callback in Keras training stops when some metric (usually validation loss) is not increasing. Is there a way to use another metric (like precision, recall, f-measure) ...
15
votes
5answers
4k views

Why does adding a dropout layer improve deep/machine learning performance, given that dropout suppresses some neurons from the model?

If removing some neurons results in a better performing model, why not use a simpler neural network with fewer layers and fewer neurons in the first place? Why build a bigger, more complicated model ...
12
votes
4answers
28k views

Question about bias in Convolutional Networks

I am trying to figure out how many weights and biases are needed for CNN. Say I have a (3, 32, 32)-image and want to apply a (32, 5, 5)-filter. For each feature map I have 5x5 weights, so I should ...
11
votes
2answers
756 views

When do we say that the dataset is not classifiable?

I have many times analysed a dataset on which I could not really do any sort of classification. To see whether I can get a classifier I have usually used the following steps: Generate box plots of ...
12
votes
3answers
7k views

What is the relationship between the accuracy and the loss in deep learning?

I have created three different models using deep learning for multi-class classification and each model gave me a different accuracy and loss value. The results of the testing model as the following: ...
7
votes
5answers
9k views

In which epoch should i stop the training to avoid overfitting

I'm working on an age estimation project trying to classify a given face in a predefined age range. For that purpose I'm training a deep NN using the keras library. The accuracy for the training and ...
7
votes
2answers
6k views

Using cross-validation technique for a CNN model?

I am working on the CNN model, as always I use batches with epochs to train my model, for my model, when it completed training and validation, finally I use a test set to measure the model performance ...
12
votes
1answer
5k views

Reason for square images in deep learning

Most of the advanced deep learning models like VGG, ResNet, etc. require square images as input, usually with a pixel size of $224x224$. Is there a reason why the input has to be of equal shape, or ...
8
votes
4answers
4k views

How does deep learning helps in detecting multiple objects in single image?

Let's say there are two cars in an image. How can it detect these cars, given that it can detect single car in an image?
4
votes
2answers
2k views

How to sort numbers using Convolutional Neural Network?

Recently, in an interview I got this question: Design a convnet that sorts numbers. Operators are ReLU, Conv, and Pooling. E.g. input: 5, 3, 6, 2; output: 2, 3, 5, 6 I am not sure how can you sort a ...
4
votes
1answer
111 views

Fake News Detection problem

I would like to work on a project for Fake News Detection especially for Indians news which are in different languages and different formats. Fake news as image with no or very less text Fake news on ...
3
votes
1answer
278 views

ANN on Pattern Recognition

I have been trying to apply a simple neural network using keras to predict a sequence of numbers and the rule is if the input integer is odd it should be 4 and if its even it should be 2. Yet the ...
7
votes
1answer
1k views

After the training phase, is it better to run neural networks on a GPU or CPU?

My understanding is that GPUs are more efficient for running neural nets, but someone recently suggested to me that GPUs are only needed for the training phase. Once trained, it's actually more ...
5
votes
2answers
771 views

What does it mean for the training data to be generated by a probability distribution over datasets

I was reading the Deep Learning book and came across the following para (page 109, second para): The training and test data are generated by a probability distribution over datasets called the data-...
6
votes
1answer
5k views

Keras: How to normalize dataframe with continuous and categorical data?

I have a dataframe with about 50 columns. The columns are either categorical or continuous data. The continuous data can be between 0.000001-1.00000 or they can be between 500,000-5,000,000. The ...
5
votes
2answers
14k views

weighted cross entropy for imbalanced dataset - multiclass classification

I am trying to classify images to more then a 100 classes, of different sizes ranged from 300 to 4000 (mean size 1500 with std 600). I am using a pretty standard CNN where the last layer outputs a ...
4
votes
1answer
181 views

Fine-tuning a CNN for recognizing two classes, but also being able to tell if none of them is present in an image

I need to fine-tune a CNN to classify two classes: dogs and cats, for example. However, I want the CNN to be able to tell if ...
3
votes
1answer
3k views

Example of 1D ConvNet filter

I understand Conv2D filters. I think I understand Conv1D filters as well but have not seen any examples of the filters like what ...
1
vote
1answer
126 views

Rephrase Neural Network : Find the Fittest Word for Given Meaning/Explanation

I am a sophomore student who's interested in deep learning and its method layering up some linear/non-linear operations and constructs up the complex function through the network. I'd like to ...
6
votes
1answer
3k views

Keras CNN with low/constant accuracies

I am dealing with the Street View House Number recognition problem. I am trying to train a CNN with Keras. Here is how I prepared the input: ...
4
votes
1answer
10k views

Validation loss increases and validation accuracy decreases

I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is ...