278

Deconvolution layer is a very unfortunate name and should rather be called a transposed convolutional layer. Visually, for a transposed convolution with stride one and no padding, we just pad the original input (blue entries) with zeroes (white entries) (Figure 1). In case of stride two and padding, the transposed convolution would look like this (Figure 2)...


64

Equivariance and invariance are sometimes used interchangeably in common speech. They have ancient roots in maths and physics. As pointed out by @Xi'an, you can find previous uses (anterior to Convolutional Neural Networks) in the statistical literature, for instance on the notions of the invariant estimator and especially the Pitman estimator. However, I ...


62

I think one way to get a really basic level intuition behind convolution is that you are sliding K filters, which you can think of as K stencils, over the input image and produce K activations - each one representing a degree of match with a particular stencil. The inverse operation of that would be to take K activations and expand them into a preimage of ...


52

Step by step math explaining how transpose convolution does 2x upsampling with 3x3 filter and stride of 2: The simplest TensorFlow snippet to validate the math: import tensorflow as tf import numpy as np def test_conv2d_transpose(): # input batch shape = (1, 2, 2, 1) -> (batch_size, height, width, channels) - 2x2x1 image in batch of 1 x = tf....


43

The terms are different: Equivariant to translation means that a translation of input features results in an equivalent translation of outputs. So if your pattern 0,3,2,0,0 on the input results in 0,1,0,0 in the output, then the pattern 0,0,3,2,0 might lead to 0,0,1,0 Invariant to translation means that a translation of input features doe not change the ...


28

The notes that accompany Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition, by Andrej Karpathy, do an excellent job of explaining convolutional neural networks. Reading this paper should give you a rough idea about: Deconvolutional Networks Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor and Rob Fergus Dept. of Computer ...


25

In the context of image processing, upsampling is a technique for increasing the size of an image. For example, say you have an image with a height and width of $64$ pixels each (totaling $64 \times 64 = 4096$ pixels). You want to resize this image to a height and width of 256 pixels (totaling $256 \times 256 = 65536$ pixels). In the new, larger image you ...


21

The convolution operation, simply put, is combination of element-wise product of two matrices. So long as these two matrices agree in dimensions, there shouldn't be a problem, and so I can understand the motivation behind your query. A.1. However, the intent of convolution is to encode source data matrix (entire image) in terms of a filter or kernel. More ...


21

For this you need to understand what filters actually do. Every layer of filters is there to capture patterns. For example, the first layer of filters captures patterns like edges, corners, dots etc. Subsequent layers combine those patterns to make bigger patterns (like combining edges to make squares, circles, etc.). Now as we move forward in the layers, ...


16

In short, there is nothing special about number of dimensions for convolution. Any dimensionality of convolution could be considered, if it fit a problem. The number of dimensions is a property of the problem being solved. For example, 1D for audio signals, 2D for images, 3D for movies . . . Ignoring number of dimensions briefly, the following can be ...


14

Just found a great article from the theaon website on this topic [1]: The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of a normal convolution, [...] to project feature maps to a higher-dimensional space. [...] i.e., map from a 4-dimensional space to a 16-dimensional space, while ...


13

The higher the number of filters, the higher the number of abstractions that your Network is able to extract from image data. The reason why the number of filters is generally ascending is that at the input layer the Network receives raw pixel data. Raw data are always noisy, and this is especially true for image data. Because of this, we let CNNs extract ...


12

What are the filters? A filter/kernel is a set of learnable weights which are learned using the backpropagation algorithm. You can think of each filter as storing a single template/pattern. When you convolve this filter across the corresponding input, you are basically trying to find out the similarity between the stored template and different locations in ...


11

We could use PCA for analogy. When using conv, the forward pass is to extract the coefficients of principle components from the input image, and the backward pass (that updates the input) is to use (the gradient of) the coefficients to reconstruct a new input image, so that the new input image has PC coefficients that better match the desired coefficients. ...


8

Upsampling layer is used to increase the resolution of the image. In segmentation, we first downsample the image to get the features and then upsample the image to generate the segments. For deconvolution operation we pad the image with zeroes and then do a convolution operation on that, hence it is upsampled. For eg: - If after downsampling the images ...


8

Convolutions from a DSP perspective I'm a bit late to this but still would like to share my perspective and insights. My background is theoretical physics and digital signal processing. In particular I studied wavelets and convolutions are almost in my backbone ;) The way people in the deep learning community talk about convolutions was also confusing to ...


8

When you get a bit more insight into network topologies these hyperparameters will make more sense, but in general this is just like any other hyperparameter, you will have to test some settings and see what works better. In the case of pooling layers it is actually relatively interpretable. Why do we use pooling? To downsample our feature maps. This is done ...


8

In sort of mechanistic/pictorial/image-based terms: Dilation: ### SEE COMMENTS, WORKING ON CORRECTING THIS SECTION Dilation is largely the same as run-of-the-mill convolution (frankly so is deconvolution), except that it introduces gaps into it's kernels, i.e. whereas a standard kernel would typically slide over contiguous sections of the input, it's dilated ...


8

As you are working on image classification and would also like to implement some data augmentation, you can combine the two AND load the batches directly from a folder using the mighty 'ImageDataGenerator` class. Have a look at the execellent documentation! I won't copy and paste the example from that link, but I can outline the steps that you go through: ...


7

Just adding my 2 cents Regarding an image classification task solved with a typical CNN Architecture consisting of a Backend (Convolutions + NL + possibly Spatial Pooling) which performs Representation Learning and of a Frontend (e.g. Fully Connected Layers, MLP) which solves the specific task, in this case image classification, the idea is to build a ...


7

Though both seem to be doing the same thing, which is up-sampling a layer, there's a clear margin between them. First we talk about Dilated Convolution I found this nice blog on above topic. So as I understood, this is more like exploring the input data points in a wide manner. Or increasing the receptive field of the convolution operation. Here is a ...


7

A neural network takes every input into a neuron and then through an activation function the neuron produces an output. When applying this method to images we can quickly see that this is not an ideal solution. The similarity of a pixel is much more related to a neighboring pixel. The convolution operation in deep learning was used for this exact purpose. ...


7

Here is an animation of fractionally-strided convolution (from this github project): where the dashed white cells are zero rows/columns padded between the input cells (blue). These animations are visualizations of the mathematical formulas from the article below: A guide to convolution arithmetic for deep learning Here is a quote from the article: ...


6

I had a lot of trouble understanding what exactly happened in the paper until I came across this blog post: http://warmspringwinds.github.io/tensorflow/tf-slim/2016/11/22/upsampling-and-image-segmentation-with-tensorflow-and-tf-slim/ Here is a summary of how I understand what is happening in a 2x upsampling: Information from paper What is upsampling? "...


6

In addition to David Dao's answer: It is also possible to think the other way around. Instead of focusing on which (low resolution) input pixels are used to produce a single output pixel, you can also focus on which individual input pixels contribute to which region of output pixels. This is done in this distill publication, including a series of very ...


6

The docs for ImageDataGenerator suggest that no augmentation is done by default. So you could instantiate it without any augmentation parameters and keep the rest of your code for handling your directory structure: train_datagen = ImageDataGenerator(rescale=1./255) You are also allowed to write your own custom data generator and pass it to model....


6

A Generative Adversarial Network (GAN) takes the idea of using a generator model to generate fake examples and discrimator model that tries to decide if the image it receives is a fake (i.e. from the generator) or a real sample. This was originally shown with relatively simple fully connected networks. A Deep Convolution GAN (DCGAN) does something very ...


6

Your error is coming from the Keras framework not working with strings as the output labels. You will want to transform these to 1-hot encoded vectors to train your model. Here is some code to do this. Getting the data import pandas as pd df = pd.read_csv('iris.csv', header=None, names=['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species'])...


6

Generally, if you look at image segmentation models, they have two main paths, what the author of your paper calls encoder and decoder paths. The role of the encoder is to contract the size of the image while extracting meaningful information, while the decoder restores the contracted image to its original dimensions. However, a lot of information is lost ...


5

1) Suppose input_field is all zero except for one entry at index idx. An odd filter size will return data with a peak centered around idx, an even filter size won't - consider the case of a uniform filter with size 2. Most people want to preserve the locations of peaks when they filter. 2) All of the input_field is relevant for the convolution, but the ...


Only top voted, non community-wiki answers of a minimum length are eligible