Skip to main content

Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
41 votes
6 answers

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide ...
stk1234's user avatar
  • 583
47 votes
5 answers

Does gradient descent always converge to an optimum?

I am wondering whether there is any scenario in which gradient descent does not converge to a minimum. I am aware that gradient descent is not always guaranteed to converge to a global optimum. I am ...
wit221's user avatar
  • 573
264 votes
10 answers

How to set class weights for imbalanced classes in Keras?

I know that there is a possibility in Keras with the class_weights parameter dictionary at fitting, but I couldn't find any example. Would somebody so kind to ...
Hendrik's user avatar
  • 8,617
36 votes
6 answers

Why do convolutional neural networks work?

I have often heard people saying that why convolutional neural networks are still poorly understood. Is it known why convolutional neural networks always end up learning increasingly sophisticated ...
Praise the lord's user avatar
198 votes
5 answers

What is the "dying ReLU" problem in neural networks?

Referring to the Stanford course notes on Convolutional Neural Networks for Visual Recognition, a paragraph says: "Unfortunately, ReLU units can be fragile during training and can "die". For ...
tejaskhot's user avatar
  • 4,065
69 votes
5 answers

Adding Features To Time Series Model LSTM

have been reading up a bit on LSTM's and their use for time series and its been interesting but difficult at the same time. One thing I have had difficulties with understanding is the approach to ...
Rjay155's user avatar
  • 1,215
67 votes
4 answers

Why mini batch size is better than one single "batch" with all training data?

I often read that in case of Deep Learning models the usual practice is to apply mini batches (generally a small one, 32/64) over several training epochs. I cannot really fathom the reason behind this....
Hendrik's user avatar
  • 8,617
44 votes
6 answers

What is the relationship between the accuracy and the loss in deep learning?

I have created three different models using deep learning for multi-class classification and each model gave me a different accuracy and loss value. The results of the testing model as the following: ...
N.IT's user avatar
  • 1,995
44 votes
2 answers

Should we apply normalization to test data as well?

I am doing a project on an author identification problem. I applied the tf-idf normalization to train data and then trained an SVM on that data. Now when using the classifier, should I normalize test ...
Kishan Kumar's user avatar
69 votes
11 answers

Why should the data be shuffled for machine learning tasks

In machine learning tasks it is common to shuffle data and normalize it. The purpose of normalization is clear (for having same range of feature values). But, after struggling a lot, I did not find ...
Green Falcon's user avatar
  • 14.1k
23 votes
2 answers

Why ReLU is better than the other activation functions

Here the answer refers to vanishing and exploding gradients that has been in sigmoid-like activation functions but, I guess, Relu...
Green Falcon's user avatar
  • 14.1k
178 votes
21 answers

How do you visualize neural network architectures?

When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture. What are good / simple ways to visualize common ...
Martin Thoma's user avatar
  • 18.9k
83 votes
5 answers

What is the difference between "equivariant to translation" and "invariant to translation"

I'm having trouble understanding the difference between equivariant to translation and invariant to translation. In the book Deep Learning. MIT Press, 2016 (I. Goodfellow, A. Courville, and Y. Bengio)...
Aamir 's user avatar
  • 993
59 votes
5 answers

Number of parameters in an LSTM model

How many parameters does a single stacked LSTM have? The number of parameters imposes a lower bound on the number of training examples required and also influences the training time. Hence knowing the ...
wabbit's user avatar
  • 1,297
48 votes
4 answers

Why is ReLU used as an activation function?

Activation functions are used to introduce non-linearities in the linear output of the type w * x + b in a neural network. Which I am able to understand ...
Bunny Rabbit's user avatar
12 votes
1 answer

Using a pre trained CNN classifier and apply it on a different image dataset

How would you optimize a pre-trained neural network to apply it to a separate problem? Would you just add more layers to the pre-trained model and test it on your ...
Sid's user avatar
  • 677
11 votes
1 answer

Keras LSTM with 1D time series

I'm learning how to use Keras and I've had reasonable success with my labelled dataset using the examples on Chollet's Deep Learning for Python. The data set is ~1000 Time Series with length 3125 with ...
user1147964's user avatar
8 votes
1 answer

Why should softmax be used in CNN

In the last layer of CNNs and MLPs it is common to use softmax layer or units with sigmoid activation functions for multi-class ...
Green Falcon's user avatar
  • 14.1k
6 votes
2 answers

What are useful evaluation metrics used in machine learning

I am using CNN in order to predict codes after analyzing text. As an example, I will write "I am crazy" .. the model will predict some code " X321". All this based on CNN. I want to evaluate my ...
Lizou's user avatar
  • 215
198 votes
6 answers

How to draw Deep learning network architecture diagrams?

I have built my model. Now I want to draw the network architecture diagram for my research paper. Example is shown below:
Muhammad Ali's user avatar
  • 2,487
89 votes
1 answer

When to use (He or Glorot) normal initialization over uniform init? And what are its effects with Batch Normalization?

I knew that Residual Network (ResNet) made He normal initialization popular. In ResNet, He normal initialization is used , while the first layer uses He uniform initialization. I've looked through ...
Rizky Luthfianto's user avatar
75 votes
6 answers

Cross-entropy loss explanation

Suppose I build a neural network for classification. The last layer is a dense layer with Softmax activation. I have five different classes to classify. Suppose for a single training example, the <...
enterML's user avatar
  • 3,031
31 votes
7 answers

Can machine learning learn a function like finding maximum from a list?

I have an input which is a list and the output is the maximum of the elements of the input-list. Can machine learning learn such a function which always selects the maximum of the input-elements ...
user78739's user avatar
  • 319
29 votes
6 answers

Deep learning basics

I am looking for a paper detailing the very basics of deep learning. Ideally like the Andrew Ng course for deep learning. Do you know where I can find this ?
Maxi's user avatar
  • 433
27 votes
2 answers

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
oW_'s user avatar
  • 6,347
25 votes
3 answers

Should I use GPU or CPU for inference?

I'm running a deep learning neural network that has been trained by a GPU. I now want to deploy this to multiple hosts for inference. The question is what are the conditions to decide whether I should ...
Dan's user avatar
  • 361
24 votes
2 answers

What's the difference between the cell and hidden state in LSTM?

LSTM cells consist of two types of states, the cell state and hidden state. How do cell and hidden states differ, in terms of their functionality? What information do they carry?
user avatar
15 votes
1 answer

What is a 1D Convolutional Layer in Deep Learning?

I have a good general understanding of the role and mechanism of convolutional layers in Deep Learning for image processing in case of 2D or 3D implementations - they "simply" try to catch 2D patterns ...
Hendrik's user avatar
  • 8,617
10 votes
5 answers

In which epoch should i stop the training to avoid overfitting

I'm working on an age estimation project trying to classify a given face in a predefined age range. For that purpose I'm training a deep NN using the keras library. The accuracy for the training and ...
Yiannis Ath's user avatar
8 votes
2 answers

How to plot cost versus number of iterations in scikit learn?

One of the recommendations in the Coursera Machine Learning course when working with gradient descent based algorithms is: Debugging gradient descent. Make a plot with number of iterations on the x-...
Chris Snow's user avatar
8 votes
1 answer

How to download dynamic files created during work on Google Colab?

I have two different files and on the first, I tried to save data to file as:, 'wb'), q1_data) On second file, i'm trying to ...
vikbehal's user avatar
  • 185
4 votes
1 answer

Fake News Detection problem

I would like to work on a project for Fake News Detection especially for Indians news which are in different languages and different formats. Fake news as image with no or very less text Fake news on ...
Akash's user avatar
  • 235
2 votes
1 answer

Compute backpropagation

I have the question which is mentioned in the above picture. It is trying to find the derivative of f with respect to weight matrix ...
DukeLover's user avatar
  • 581
2 votes
2 answers

Explain Binary Classification with output 0.5 (True)

What is the interpretation of output 0.5 of a typical classifier? I made a prediction and the probability of that data point being from the True class is 0.5.
Abhishek Sharma's user avatar
0 votes
1 answer

Usefulness of intercept in layman terms - ELI5

I am working on a binary classification problem with 1000 rows and 10 features. While I did use random forest for classification, I also used LIME to explain the predictions of the random forest. ...
The Great's user avatar
  • 2,565
86 votes
8 answers

Time series prediction using ARIMA vs LSTM

The problem that I am dealing with is predicting time series values. I am looking at one time series at a time and based on for example 15% of the input data, I would like to predict its future values....
ahajib's user avatar
  • 1,075
64 votes
4 answers

Does batch_size in Keras have any effects in results' quality?

I am about to train a big LSTM network with 2-3 million articles and am struggling with Memory Errors (I use AWS EC2 g2x2large). I found out that one solution is to reduce the ...
hipoglucido's user avatar
  • 1,170
46 votes
2 answers

Merging two different models in Keras

I am trying to merge two Keras models into a single model and I am unable to accomplish this. For example in the attached Figure, I would like to fetch the middle layer $A2$ of dimension 8, and use ...
Rkz's user avatar
  • 1,033
39 votes
1 answer

How does Keras calculate accuracy?

How does Keras calculate accuracy from the classwise probabilities? Say, for example we have 100 samples in the test set which can belong to one of two classes. We also have a list of the classwise ...
pseudomonas's user avatar
  • 1,042
28 votes
2 answers

Is there away to change the metric used by the Early Stopping callback in Keras?

When using the early stopping callback in Keras, training stops when some metric (usually validation loss) is not increasing. Is there a way to use another metric (like precision, recall, or f-measure)...
P.Joseph's user avatar
  • 393
28 votes
3 answers

Why are NLP and Machine Learning communities interested in deep learning?

I hope you can help me, as I have some questions on this topic. I'm new in the field of deep learning, and while I did some tutorials, I can't relate or distinguish concepts from one another.
user3352632's user avatar
21 votes
1 answer

Can BERT do the next-word-predict task?

As BERT is bidirectional (uses bi-directional transformer), is it possible to use it for the next-word-predict task? If yes, what needs to be tweaked?
惊天补扣's user avatar
18 votes
4 answers

Question about bias in Convolutional Networks

I am trying to figure out how many weights and biases are needed for CNN. Say I have a (3, 32, 32)-image and want to apply a (32, 5, 5)-filter. For each feature map I have 5x5 weights, so I should ...
user's user avatar
  • 2,003
16 votes
5 answers

Why does adding a dropout layer improve deep/machine learning performance, given that dropout suppresses some neurons from the model?

If removing some neurons results in a better performing model, why not use a simpler neural network with fewer layers and fewer neurons in the first place? Why build a bigger, more complicated model ...
user781486's user avatar
  • 1,385
13 votes
3 answers

What are the consequences of not freezing layers in transfer learning?

I am trying to fine tune some code from a Kaggle kernel. The model uses pretrained VGG16 weights (via 'imagenet') for transfer learning. However, I notice there is no layer freezing of layers as is ...
Borealis's user avatar
  • 347
13 votes
1 answer

Reason for square images in deep learning

Most of the advanced deep learning models like VGG, ResNet, etc. require square images as input, usually with a pixel size of $224x224$. Is there a reason why the input has to be of equal shape, or ...
spore234's user avatar
  • 603
13 votes
5 answers

How does Sigmoid activation work in multi-class classification problems

I know that for a problem with multiple classes we usually use softmax, but can we also use sigmoid? I have tried to implement digit classification with sigmoid at the output layer, it works. What I ...
bharath chandra's user avatar
13 votes
2 answers

Activation function between LSTM layers

I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
lsfischer's user avatar
  • 242
11 votes
2 answers

When do we say that the dataset is not classifiable?

I have many times analysed a dataset on which I could not really do any sort of classification. To see whether I can get a classifier I have usually used the following steps: Generate box plots of ...
vc_dim's user avatar
  • 188
10 votes
2 answers

Using Cross Validation technique for a CNN model

I am working on a CNN model. As always, I used batches with epochs to train my model. When it completed training and validation, finally I used a test set to measure the model performance and generate ...
Hunar's user avatar
  • 1,147

2 3 4 5 6