55
votes
What are deconvolutional layers?
Step by step math explaining how transpose convolution does 2x upsampling with 3x3 filter and stride of 2:
The simplest TensorFlow snippet to validate the math:
...
- 651
43
votes
Accepted
How are 1x1 convolutions the same as a fully connected layer?
Your Example
In your example we have 3 input and 2 output units. To apply convolutions, think of those units having shape: [1,1,3] and ...
- 546
36
votes
Accepted
What is the difference between upsampling and bi-linear upsampling in a CNN?
In the context of image processing, upsampling is a technique for increasing the size of an image.
For example, say you have an image with a height and width of $64$ pixels each (totaling $64 \times ...
- 8,018
31
votes
Accepted
Are there any rules for choosing the size of a mini-batch?
In On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima there are a couple of intersting statements:
It has been observed in practice that
when using a larger batch ...
28
votes
Why do convolutional neural networks work?
Actually I guess the question is a bit broad! Anyway.
Understanding Convolution Nets
What is learned in ConvNets tries to minimize the cost function to ...
- 14.1k
28
votes
Accepted
What is/are the default filters used by Keras Convolution2d()?
By default, the filters $W$ are initialised randomly using the glorot_uniform method, which draws values from a uniform distribution with positive and negative ...
- 3,940
24
votes
What is the difference between Inception v2 and Inception v3?
In the paper Batch Normalization,Sergey et al,2015. proposed Inception-v1 architecture which is a variant of the GoogleNet in the paper Going deeper with convolutions, and in the meanwhile they ...
- 341
24
votes
Accepted
Why convolutions always use odd-numbers as filter size
The convolution operation, simply put, is combination of element-wise product of two matrices. So long as these two matrices agree in dimensions, there shouldn't be a problem, and so I can understand ...
- 1,331
23
votes
Accepted
Convolutional neural network overfitting. Dropout not helping
Ok, so after a lot of experimentation I have managed to get some results/insights.
In the first place, everything being equal, smaller batches in the training set help a lot in order to increase the ...
22
votes
Number and size of dense layers in a CNN
First of all:
There is no way to determine a good network topology just from the
number of inputs and outputs. It depends critically on the number of
training examples and the complexity of the ...
- 1,255
21
votes
Accepted
How to calculate the output shape of conv2d_transpose?
Here is the correct formula for computing the size of the output with tf.layers.conv2d_transpose():
...
- 326
20
votes
Accepted
Can the number of epochs influence overfitting?
Yes, it may. In machine-learning there is an approach called early stop. In that approach you plot the error rate on training and validation data. The horizontal axis is the ...
- 14.1k
18
votes
Why do convolutional neural networks work?
ConvNets work because they exploit feature locality. They do it at different granularities, therefore being able to model hierarchically higher level features. They are translation invariant thanks to ...
- 27.2k
17
votes
Accepted
back propagation in CNN
A convolution employs a weight sharing principle which will complicate the mathematics significantly but let's try to get through the weeds. I am drawing most of my explanation from this source.
...
- 8,986
16
votes
What are deconvolutional layers?
Just found a great article from the theaon website on this topic [1]:
The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of ...
- 269
16
votes
Accepted
How are weights represented in a convolution neural network?
In convolutional layers the weights are represented as the multiplicative factor of the filters.
For example, if we have the input 2D matrix in green
with the convolution filter
Each matrix ...
- 8,986
15
votes
Using a pre trained CNN classifier and apply it on a different image dataset
Do Read - When Transfer Learning is Disadvantageous ?(thanks to @media)
(looks very informative to me, so added here to make this answer a complete one...)
Answer to your Question..(starts here)
...
- 2,470
14
votes
Accepted
Sigmoid vs Relu function in Convnets
The reason that sigmoid functions are being replaced by rectified linear units, is because of the properties of their derivatives.
Let's take a quick look at the sigmoid function $\sigma$ which is ...
- 669
14
votes
Accepted
How to get predicted class labels in convolution neural network?
What you have are predicted class probabilities. Since you are doing binary classification, each output is the probability of the first class for that test example.
To convert these to class labels ...
- 2,381
13
votes
Accepted
What is the input size of Alex net
I guess it has been a mistake. Take a look at here.
The other author's were Ilya Sutskever and Geoffrey Hinton. So, AlexNet input starts with 227 by 227 by 3 images. And if you read the paper, the ...
- 14.1k
13
votes
Accepted
CNN - imbalanced classes, class weights vs data augmentation
Is this approach better than the mere
augmentation or just the use of class weights ?
Note that data augmentation is the process of changing the training samples (e.g. for images, flipping them, ...
- 9,392
13
votes
Accepted
What is fractionally-strided convolution layer?
Here is an animation of fractionally-strided convolution (from this github project):
where the dashed white cells are zero rows/columns padded between the input cells (blue). These animations are ...
- 9,392
12
votes
What are deconvolutional layers?
We could use PCA for analogy.
When using conv, the forward pass is to extract the coefficients of principle components from the input image, and the backward pass (that updates the input) is to use (...
- 221
10
votes
Accepted
Does it make sense to train a CNN as an autoencoder?
Yes, it makes sense to use CNNs with autoencoders or other unsupervised methods. Indeed, different ways of combining CNNs with unsupervised training have been tried for EEG data, including using (...
- 326
9
votes
How do subsequent convolution layers work?
I have just struggled with this same question for a few hours. Thought I'd share the insite that helped me understand it.
The answer is that the filters for the second convolutional layer do not ...
9
votes
What is/are the default filters used by Keras Convolution2d()?
They are convolution kernels. For instance your image $A$ is 5x5, you have 32 3x3 convolution kernels $F_k$. The border_mode is 'valid' that means there is no padding around input, so the pixel (i,0),(...
- 246
9
votes
CNN memory consumption
I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,...
- 2,011
9
votes
Accepted
How to sort numbers using Convolutional Neural Network?
I have a solution however I use a densely connected layer at the output to simplify the reshaping. If you can manipulate the sizes of this model such that you have 4 output parameters this should work ...
- 8,986
9
votes
Accepted
Optimizer for Convolutional neural network
Yes, you can use the same optimizers you are familiar with for CNNs.
I don't think that there is a best optimizer for CNNs. The most popular in my opinion is Adam. However some people like to use a ...
- 432
8
votes
Document classification using convolutional neural network
You could reduce the length of your input data by representing your documents as series of sentence vectors instead of a longer series of word vectors. Doc2vec is one way to do this (each sentence ...
- 256
Only top scored, non community-wiki answers of a minimum length are eligible