Questions tagged [convolutional-neural-network]

A convolutional neural network is a form of neural network with an additional convolutional layer, typically used in image & audio analysis. The convolutional layer is essentially a filtering stage defined by the kernel which is used. For example, a convolutional layer could have a kernel which extracts edges from an image towards the goal of learning which objects are in a scene.

Filter by
Sorted by
Tagged with
80
votes
4answers
36k views

How are 1x1 convolutions the same as a fully connected layer?

I've recently read Yan LeCuns comment on 1x1 convolutions: In Convolutional Nets, there is no such thing as "fully-connected layers". There are only convolution layers with 1x1 convolution kernels ...
30
votes
2answers
31k views

Are there any rules for choosing the size of a mini-batch?

When training neural networks, one hyperparameter is the size of a minibatch. Common choices are 32, 64, and 128 elements per mini batch. Are there any rules/guidelines on how big a mini-batch should ...
29
votes
6answers
12k views

Why do convolutional neural networks work?

I have often heard people saying that why convolutional neural networks are still poorly understood. Is it known why convolutional neural networks always end up learning increasingly sophisticated ...
29
votes
2answers
18k views

What is/are the default filters used by Keras Convolution2d()?

I am pretty new to neural networks, but I understand linear algebra and the mathematics of convolution pretty decently. I am trying to understand the example code I find in various places on the net ...
22
votes
1answer
22k views

back propagation in CNN

I have the following CNN: I start with an input image of size 5x5 Then I apply convolution using 2x2 kernel and stride = 1, that produces feature map of size 4x4. Then I apply 2x2 max-pooling with ...
21
votes
5answers
10k views

Convolutional neural network overfitting. Dropout not helping

I am playing a little with convnets. Specifically, I am using the kaggle cats-vs-dogs dataset which consists on 25000 images labeled as either cat or dog (12500 each). I've managed to achieve around ...
21
votes
3answers
21k views

Why convolutions always use odd-numbers as filter_size

If we have a look to 90-99% of the papers published using a CNN (ConvNet). The vast majority of them use filter size of odd numbers:{1, 3, 5, 7} for the most used. This situation can lead to some ...
20
votes
4answers
29k views

What is the difference between Inception v2 and Inception v3?

The paper Going deeper with convolutions describes GoogleNet which contains the original inception modules: The change to inception v2 was that they replaced the 5x5 convolutions by two successive ...
18
votes
1answer
15k views

What is the difference between upsampling and bi-linear upsampling in a CNN?

I am trying to understand this paper and am unsure of what bi-linear upsampling is. Can anyone explain this at a high-level?
15
votes
3answers
11k views

Is there a person class in ImageNet? Are there any classes related to humans?

If I look at one of the many sources for the Imagenet classes on the Internet I cannot find a single class related to human beings (and no, harvestman is not someone who harvests, but it's what I knew ...
15
votes
2answers
29k views

Can the number of epochs influence overfitting?

I am using a convolution neural network ,CNN. At a specific epoch, I only save the best CNN model weights based on improved validation accuracy over previous epochs....
15
votes
2answers
21k views

How many images per class are sufficient for training a CNN

I'm starting a project where the task is to identify sneaker types from images. I'm currently reading into TensorFlow and Torch implementations. My question is: how many images per class are required ...
14
votes
1answer
5k views

Back-propagation through max pooling layers

I have a small sub-question to this question. I understand that when back-propagating through a max pooling layer the gradient is routed back in a way that the neuron in the previous layer which was ...
13
votes
1answer
14k views

Number and size of dense layers in a CNN

Most networks I've seen have one or two dense layers before the final softmax layer. Is there any principled way of choosing the number and size of the dense layers? Are two dense layers more ...
13
votes
3answers
15k views

What is the difference between Dilated Convolution and Deconvolution?

These two convolution operations are very common in deep learning right now. I read about dilated convolutional layer in this paper : WAVENET: A GENERATIVE MODEL FOR RAW AUDIO and De-convolution is ...
13
votes
4answers
34k views

Question about bias in Convolutional Networks

I am trying to figure out how many weights and biases are needed for CNN. Say I have a (3, 32, 32)-image and want to apply a (32, 5, 5)-filter. For each feature map I have 5x5 weights, so I should ...
12
votes
1answer
4k views

Using a pre trained CNN classifier and apply it on a different image dataset

How would you optimize a pre-trained neural network to apply it to a separate problem? Would you just add more layers to the pre-trained model and test it on your ...
12
votes
2answers
7k views

Document classification using convolutional neural network

I'm trying to use CNN (convolutional neural network) to classify documents. CNN for short text/sentences has been studied in many papers. However, it seems that no papers have used CNN for long text ...
11
votes
4answers
31k views

How to calculate the output shape of conv2d_transpose?

Currently I code a GAN to generate MNIST numbers but the generator doesnt want to work. First I choose z with shape 100 per Batch, put into a layer to get into the shape (7,7, 256). Then ...
11
votes
2answers
13k views

Does it make sense to train a CNN as an autoencoder?

I work with analyzing EEG data, which will eventually need to be classified. However, obtaining labels for the recordings is somewhat expensive, which has led me to consider unsupervised approaches, ...
11
votes
3answers
16k views

CNN memory consumption

I'd like to be able to estimate whether a proposed model is small enough to be trained on a GPU with a given amount of memory If I have a simple CNN architecture like this: ...
11
votes
1answer
15k views

Optimizer for Convolutional neural network

What is the best optimizer for Convolutional neural network (CNN)? Can I use RMSProp for CNN or only for RNN?
11
votes
3answers
1k views

Recurrent (CNN) model on EEG data

I'm wondering how to interpret a recurrent architecture in an EEG context. Specifically I'm thinking of this as a Recurrent CNN (as opposed to architectures like LSTM), but maybe it applies to other ...
9
votes
5answers
5k views

How does deep learning helps in detecting multiple objects in single image?

Let's say there are two cars in an image. How can it detect these cars, given that it can detect single car in an image?
9
votes
4answers
24k views

How to get predicted class labels in convolution neural network?

I have built a convolutional neural network which is needed to classify the test data into either 0 or 1. I am training the CNN with labels either 0 or 1 but while running the below code I am getting ...
9
votes
3answers
3k views

Why use convolutional NNs for a visual inspection task over classic CV template matching?

I had an interesting discussion come up based on a project we were working on: why use a CNN visual inspection system over a template matching algorithm? Background: I had shown a demo of a simple ...
9
votes
2answers
15k views

How are weights represented in a convolution neural network?

I have been trying to develop a convolution neural network following some guides online. However, most guides I have encountered gloss over an important detail, which is how to programmatically ...
9
votes
2answers
184 views

Ways to reconstruct shuffled pixels of a video file?

Suppose that you have a video file which pixel order has been shuffled once. That is, a random order have been defined once and applied to all frames. Does it exist some known approach for ...
9
votes
1answer
1k views

keras' ModelCheckpoint not working

I'm trying to train a model in keras and I'm using ModelCheckpoint to save the best model according to a monitored validation metric (in my case the Jaccard index). While I can see the model ...
9
votes
1answer
302 views

How does a convolutional ply differ from an ordinary convolutional network?

I am currently working on recreating the results of this paper. In the paper they describe a method for using CNN for features extraction, and have a acoustic model that is Dnn-hmm and pretrained ...
8
votes
4answers
2k views

Do convolutions “flatten images”?

I'm looking for a good explanation of how convolutions in deep learning work when applied to multi-channel images. For example, let's say I have a 100 x 100 pixel image with three channels, RGB. The ...
8
votes
2answers
6k views

Train object detection without annotated data/bounding boxes

From what I can see most object detection NNs (Fast(er) R-CNN, YOLO etc) are trained on data including ...
8
votes
5answers
9k views

Convolutional Neural Networks in R

I don't see a package for doing Convolutional Neural Networks in R. Has anyone implemented this kind of algorithm in R?
8
votes
3answers
5k views

Convolutional Neural Networks layer sizes

I am trying to understand an article Backpropagation In Convolutional Neural Networks But I can not wrap my head around that diagram: The first layer has 3 feature maps with dimensions 32x32. The ...
8
votes
3answers
2k views

Convolutional Neural Network not learning EEG data

I have trained a simple CNN (using Python + Lasagne) for a 2-class EEG classification problem, however, the network doesn't seem to learn. loss does not drop over epochs and classification accuracy ...
8
votes
1answer
578 views

Why does “Depth = Semantic representation” in convolutional neural networks?

I was watching some videos online about convolutional networks, and the speaker was discussing the concept of running a filter over an image. He said, and it is also shown in the image below, that "...
8
votes
4answers
4k views

Faster-RCNN how anchor work with slider in RPN layer?

I am trying to understand the whole Faster-RCNN, From https://www.quora.com/How-does-the-region-proposal-network-RPN-in-Faster-R-CNN-work Then a sliding window is run spatially on these feature ...
8
votes
1answer
1k views

CNN for phoneme recognition

I am currently studying this paper, in which CNN is applied for phoneme recognition using visual representation of log mel filter banks, and limited weight sharing scheme. The visualisation of log ...
8
votes
1answer
2k views

What is the memory cost of a CNN?

I was recently thinking about the memory cost of (a) training a CNN and (b) inference with a CNN. Please note, that I am not talking about the storage (which is simply the number of parameters). How ...
7
votes
1answer
16k views

border_mode for convolutional layers in keras

Keras has two border_mode for convolution2D, same and valid. Could anyone explain what "same" does or point out some documentation? I could not find any document on the net (except people asking that ...
7
votes
1answer
19k views

What is the meaning of hand crafted features in computer vision problems?

Are these the features which are manually labelled by humans? or Is there any technique for obtaining these features. Is this related to learned features?
7
votes
1answer
256 views

Data augmentation when using flow_from_directory in CNN

I would like to use a small dataset to create CNN model. So, I am using data augmentation to increase the train dataset. Should I use all augmentation techniques (arguments) that listed here? I have ...
7
votes
1answer
220 views

Convolution backpropagation

I'm in the progress to learn, and understand different neural networks. I pretty much understand now feed-forward neural networks, and back-propagation of them, and now learning convolutional neural ...
7
votes
3answers
4k views

Convolutional Neural Network overfitting

I built a CNN to learn to classify EEG data (only about 4000 training examples, 2 classes, 50-50 class balance). Each training example is 64x512, with 5 channels each Ive tried to keep the network as ...
7
votes
1answer
3k views

Convolutional autoencoders not learning

I'm trying to implement convolutional autoencoders in tensorflow, on the mnist dataset. The problem is that the autoencoder does not seem to learn properly: it will always learn to reproduce the 0 ...
7
votes
2answers
145 views

Neural Network Architecture for Identifying Image Copies

I have a large image collection and wish to identify the images within that collection that appear to copy other images from the collection. To give you a sense of the kinds of image pairs that I ...
6
votes
4answers
2k views

Why choose TensorFlow?

I have noticed that most of the deep learning developers use TensorFlow. So why choose TensorFlow? What is the advantage of TensorFlow over Theano and CNTK?
6
votes
1answer
5k views

Sigmoid vs Relu function in Convnets

The question is simple: is there any advantage in using sigmoid function in a convolutional neural network? Because every website that talks about CNN uses Relu function.
6
votes
3answers
315 views

Disparity between training and testing errors with deep learning: the bias-variance tradeoff and model selection

I am developing a convolutional neural network and have a dataset with 13,000 datapoints that is split 80%/10%/10% train/validation/test. In tuning the model architecture, I found the following, after ...
6
votes
1answer
8k views

What is the input size of Alex net

In the paper ImageNet Classification with Deep Convolutional Neural Networks, the size of input image is 224x224. The following figure shows the input size. From caffe, deploy.prototxt file from the ...

1
2 3 4 5
12