302 votes
Accepted

What are deconvolutional layers?

Deconvolution layer is a very unfortunate name and should rather be called a transposed convolutional layer. Visually, for a transposed convolution with stride one and no padding, we just pad the ...
David Dao's user avatar
  • 3,330
55 votes

What are deconvolutional layers?

Step by step math explaining how transpose convolution does 2x upsampling with 3x3 filter and stride of 2: The simplest TensorFlow snippet to validate the math: ...
andriys's user avatar
  • 651
43 votes
Accepted

How are 1x1 convolutions the same as a fully connected layer?

Your Example In your example we have 3 input and 2 output units. To apply convolutions, think of those units having shape: [1,1,3] and ...
MarvMind's user avatar
  • 546
35 votes
Accepted

What is the difference between upsampling and bi-linear upsampling in a CNN?

In the context of image processing, upsampling is a technique for increasing the size of an image. For example, say you have an image with a height and width of $64$ pixels each (totaling $64 \times ...
Djib2011's user avatar
  • 7,978
31 votes
Accepted

Are there any rules for choosing the size of a mini-batch?

In On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima there are a couple of intersting statements: It has been observed in practice that when using a larger batch ...
28 votes

Why do convolutional neural networks work?

Actually I guess the question is a bit broad! Anyway. Understanding Convolution Nets What is learned in ConvNets tries to minimize the cost function to ...
Green Falcon's user avatar
  • 14.1k
28 votes
Accepted

What is/are the default filters used by Keras Convolution2d()?

By default, the filters $W$ are initialised randomly using the glorot_uniform method, which draws values from a uniform distribution with positive and negative ...
timleathart's user avatar
  • 3,940
24 votes

What is the difference between Inception v2 and Inception v3?

In the paper Batch Normalization,Sergey et al,2015. proposed Inception-v1 architecture which is a variant of the GoogleNet in the paper Going deeper with convolutions, and in the meanwhile they ...
xiaoming-qxm's user avatar
24 votes
Accepted

Why convolutions always use odd-numbers as filter size

The convolution operation, simply put, is combination of element-wise product of two matrices. So long as these two matrices agree in dimensions, there shouldn't be a problem, and so I can understand ...
Dynamic Stardust's user avatar
22 votes
Accepted

Convolutional neural network overfitting. Dropout not helping

Ok, so after a lot of experimentation I have managed to get some results/insights. In the first place, everything being equal, smaller batches in the training set help a lot in order to increase the ...
Juan Antonio Gomez Moriano's user avatar
22 votes

Number and size of dense layers in a CNN

First of all: There is no way to determine a good network topology just from the number of inputs and outputs. It depends critically on the number of training examples and the complexity of the ...
Mo-'s user avatar
  • 1,255
21 votes
Accepted

How to calculate the output shape of conv2d_transpose?

Here is the correct formula for computing the size of the output with tf.layers.conv2d_transpose(): ...
Manish P's user avatar
  • 326
20 votes
Accepted

Can the number of epochs influence overfitting?

Yes, it may. In machine-learning there is an approach called early stop. In that approach you plot the error rate on training and validation data. The horizontal axis is the ...
Green Falcon's user avatar
  • 14.1k
18 votes

Why do convolutional neural networks work?

ConvNets work because they exploit feature locality. They do it at different granularities, therefore being able to model hierarchically higher level features. They are translation invariant thanks to ...
noe's user avatar
  • 26.6k
17 votes
Accepted

back propagation in CNN

A convolution employs a weight sharing principle which will complicate the mathematics significantly but let's try to get through the weeds. I am drawing most of my explanation from this source. ...
JahKnows's user avatar
  • 8,876
16 votes

What are deconvolutional layers?

Just found a great article from the theaon website on this topic [1]: The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of ...
Andrei's user avatar
  • 269
15 votes
Accepted

How are weights represented in a convolution neural network?

In convolutional layers the weights are represented as the multiplicative factor of the filters. For example, if we have the input 2D matrix in green with the convolution filter Each matrix ...
JahKnows's user avatar
  • 8,876
15 votes

Using a pre trained CNN classifier and apply it on a different image dataset

Do Read - When Transfer Learning is Disadvantageous ?(thanks to @media) (looks very informative to me, so added here to make this answer a complete one...) Answer to your Question..(starts here) ...
Aditya's user avatar
  • 2,470
14 votes
Accepted

Sigmoid vs Relu function in Convnets

The reason that sigmoid functions are being replaced by rectified linear units, is because of the properties of their derivatives. Let's take a quick look at the sigmoid function $\sigma$ which is ...
Armen Aghajanyan's user avatar
14 votes
Accepted

How to get predicted class labels in convolution neural network?

What you have are predicted class probabilities. Since you are doing binary classification, each output is the probability of the first class for that test example. To convert these to class labels ...
Imran's user avatar
  • 2,381
13 votes
Accepted

What is the input size of Alex net

I guess it has been a mistake. Take a look at here. The other author's were Ilya Sutskever and Geoffrey Hinton. So, AlexNet input starts with 227 by 227 by 3 images. And if you read the paper, the ...
Green Falcon's user avatar
  • 14.1k
13 votes
Accepted

CNN - imbalanced classes, class weights vs data augmentation

Is this approach better than the mere augmentation or just the use of class weights ? Note that data augmentation is the process of changing the training samples (e.g. for images, flipping them, ...
Esmailian's user avatar
  • 9,312
13 votes
Accepted

What is fractionally-strided convolution layer?

Here is an animation of fractionally-strided convolution (from this github project): where the dashed white cells are zero rows/columns padded between the input cells (blue). These animations are ...
Esmailian's user avatar
  • 9,312
12 votes

What are deconvolutional layers?

We could use PCA for analogy. When using conv, the forward pass is to extract the coefficients of principle components from the input image, and the backward pass (that updates the input) is to use (...
Shaohua Li's user avatar
12 votes
Accepted

Question about bias in Convolutional Networks

Bias operates per virtual neuron, so there is no value in having multiple bias inputs where there is a single output - that would equivalent to just adding up the different bias weights into a single ...
Neil Slater's user avatar
  • 28.9k
12 votes
Accepted

border_mode for convolutional layers in keras

With border mode "valid" you get an output that is smaller than the input because the convolution is only computed where the input and the filter fully overlap. With border mode "same" you get an ...
stmax's user avatar
  • 1,637
12 votes

How does deep learning helps in detecting multiple objects in single image?

Although many solutions in production systems still use a sliding window as described below in this answer, the field of computer vision is moving quickly. Recent advances in this field include R-CNN ...
Neil Slater's user avatar
  • 28.9k
10 votes

How does deep learning helps in detecting multiple objects in single image?

I'd want to add @Neil_Slater's answer by sharing my application. In my application, I want to train a model that can automatically load a chess position from a chess book like this: Before I did ...
SmallChess's user avatar
  • 3,540
10 votes
Accepted

Does it make sense to train a CNN as an autoencoder?

Yes, it makes sense to use CNNs with autoencoders or other unsupervised methods. Indeed, different ways of combining CNNs with unsupervised training have been tried for EEG data, including using (...
robintibor's user avatar
9 votes

How do subsequent convolution layers work?

I have just struggled with this same question for a few hours. Thought I'd share the insite that helped me understand it. The answer is that the filters for the second convolutional layer do not ...
Alex Blenkinsop's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible