Skip to main content
Share Your Experience: Take the 2024 Developer Survey
302 votes
Accepted

What are deconvolutional layers?

Deconvolution layer is a very unfortunate name and should rather be called a transposed convolutional layer. Visually, for a transposed convolution with stride one and no padding, we just pad the ...
David Dao's user avatar
  • 3,330
84 votes
Accepted

What is the difference between "equivariant to translation" and "invariant to translation"

Equivariance and invariance are sometimes used interchangeably in common speech. They have ancient roots in maths and physics. As pointed out by @Xi'an, you can find previous uses (anterior to ...
Laurent Duval's user avatar
60 votes

What is the difference between "equivariant to translation" and "invariant to translation"

The terms are different: Equivariant to translation means that a translation of input features results in an equivalent translation of outputs. So if your pattern 0,3,2,0,0 on the input results in 0,...
Neil Slater's user avatar
55 votes

What are deconvolutional layers?

Step by step math explaining how transpose convolution does 2x upsampling with 3x3 filter and stride of 2: The simplest TensorFlow snippet to validate the math: ...
andriys's user avatar
  • 651
36 votes
Accepted

What is the difference between upsampling and bi-linear upsampling in a CNN?

In the context of image processing, upsampling is a technique for increasing the size of an image. For example, say you have an image with a height and width of $64$ pixels each (totaling $64 \times ...
Djib2011's user avatar
  • 7,998
33 votes
Accepted

In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

For this you need to understand what filters actually do. Every layer of filters is there to capture patterns. For example, the first layer of filters captures patterns like edges, corners, dots etc. ...
ashukid's user avatar
  • 907
24 votes
Accepted

Why convolutions always use odd-numbers as filter size

The convolution operation, simply put, is combination of element-wise product of two matrices. So long as these two matrices agree in dimensions, there shouldn't be a problem, and so I can understand ...
Dynamic Stardust's user avatar
18 votes

What is the difference between "equivariant to translation" and "invariant to translation"

Complementary to the previous answers - an image often says more than a thousand formulas. Source: AMMI Seminar - Geometric Deep Learning and Reinforcement Learning (2021)
Andy R's user avatar
  • 413
17 votes

In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

The higher the number of filters, the higher the number of abstractions that your Network is able to extract from image data. The reason why the number of filters is generally ascending is that at the ...
Leevo's user avatar
  • 6,255
16 votes

What is a 1D Convolutional Layer in Deep Learning?

In short, there is nothing special about number of dimensions for convolution. Any dimensionality of convolution could be considered, if it fit a problem. The number of dimensions is a property of ...
Neil Slater's user avatar
16 votes

What are deconvolutional layers?

Just found a great article from the theaon website on this topic [1]: The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of ...
Andrei's user avatar
  • 269
13 votes
Accepted

What is fractionally-strided convolution layer?

Here is an animation of fractionally-strided convolution (from this github project): where the dashed white cells are zero rows/columns padded between the input cells (blue). These animations are ...
Esmailian's user avatar
  • 9,322
12 votes

What are deconvolutional layers?

We could use PCA for analogy. When using conv, the forward pass is to extract the coefficients of principle components from the input image, and the backward pass (that updates the input) is to use (...
Shaohua Li's user avatar
12 votes
Accepted

Understanding how convolutional layers work

What are the filters? A filter/kernel is a set of learnable weights which are learned using the backpropagation algorithm. You can think of each filter as storing a single template/pattern. When you ...
Akshay Sehgal's user avatar
11 votes

What is the difference between "equivariant to translation" and "invariant to translation"

Just adding my 2 cents Regarding an image classification task solved with a typical CNN Architecture consisting of a Backend (Convolutions + NL + possibly Spatial Pooling) which performs ...
Nicola Bernini's user avatar
11 votes
Accepted

Convolution and Cross Correlation in CNN

A neural network takes every input into a neuron and then through an activation function the neuron produces an output. When applying this method to images we can quickly see that this is not an ideal ...
JahKnows's user avatar
  • 8,916
9 votes

How can you decide the window size on a pooling layer?

When you get a bit more insight into network topologies these hyperparameters will make more sense, but in general this is just like any other hyperparameter, you will have to test some settings and ...
Jan van der Vegt's user avatar
9 votes
Accepted

How to train data by batch from disk?

As you are working on image classification and would also like to implement some data augmentation, you can combine the two AND load the batches directly from a folder using the mighty '...
n1k31t4's user avatar
  • 14.9k
8 votes
Accepted

What is deconvolution operation used in Fully Convolutional Neural Networks?

Upsampling layer is used to increase the resolution of the image. In segmentation, we first downsample the image to get the features and then upsample the image to generate the segments. For ...
Yash Katariya's user avatar
8 votes

What are deconvolutional layers?

Convolutions from a DSP perspective I'm a bit late to this but still would like to share my perspective and insights. My background is theoretical physics and digital signal processing. In particular ...
André Bergner's user avatar
8 votes
Accepted

What is the difference between Dilated Convolution and Deconvolution?

In sort of mechanistic/pictorial/image-based terms: Dilation: ### SEE COMMENTS, WORKING ON CORRECTING THIS SECTION Dilation is largely the same as run-of-the-mill convolution (frankly so is ...
Doug Brummell's user avatar
8 votes
Accepted

GAN vs DCGAN difference

A Generative Adversarial Network (GAN) takes the idea of using a generator model to generate fake examples and discrimator model that tries to decide if the image it receives is a fake (i.e. from the ...
n1k31t4's user avatar
  • 14.9k
7 votes

What are deconvolutional layers?

I had a lot of trouble understanding what exactly happened in the paper until I came across this blog post: http://warmspringwinds.github.io/tensorflow/tf-slim/2016/11/22/upsampling-and-image-...
gebbissimo's user avatar
7 votes

What is the difference between Dilated Convolution and Deconvolution?

Though both seem to be doing the same thing, which is up-sampling a layer, there's a clear margin between them. First we talk about Dilated Convolution I found this nice blog on above topic. So as ...
Shamane Siriwardhana's user avatar
6 votes

What are deconvolutional layers?

In addition to David Dao's answer: It is also possible to think the other way around. Instead of focusing on which (low resolution) input pixels are used to produce a single output pixel, you can also ...
Martin R.'s user avatar
  • 161
6 votes

Deconvolution vs Sub-pixel Convolution

In this paper, theauthors have provided a deep explanation, basically its the exact same thing. sub-pixel uses or^2 conv kernels all at res H/r,W/r while the regular transpose conv uses o conv kernels ...
Daniel ziv's user avatar
6 votes
Accepted

How to import image data into python for keras?

The docs for ImageDataGenerator suggest that no augmentation is done by default. So you could instantiate it without any augmentation parameters and keep the rest of your code for handling your ...
Imran's user avatar
  • 2,381
6 votes
Accepted

Keras Conv1D for simple data target prediction

Your error is coming from the Keras framework not working with strings as the output labels. You will want to transform these to 1-hot encoded vectors to train your model. Here is some code to do this....
JahKnows's user avatar
  • 8,916
6 votes
Accepted

Unable to understand the meaning of following lines of the research paper for image segmentation

Generally, if you look at image segmentation models, they have two main paths, what the author of your paper calls encoder and decoder paths. The role of the encoder is to contract the size of the ...
cat91's user avatar
  • 413
6 votes
Accepted

Convolutional layers without pooling

The idea behind consecutive convolutional layers with no pooling is actually not to skip any pooling but to replace a single layer with a bigger receptive field. So think of it this way: Two ...
Jonathan's user avatar
  • 5,420

Only top scored, non community-wiki answers of a minimum length are eligible