A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [convnet]

For questions regarding "Convolutional Neural Networks" (CNN)

Filter by
Sorted by
Tagged with
197
votes
10answers
186k views

What are deconvolutional layers?

I recently read Fully Convolutional Networks for Semantic Segmentation by Jonathan Long, Evan Shelhamer, Trevor Darrell. I don't understand what "deconvolutional layers" do / how they work. The ...
58
votes
4answers
25k views

How are 1x1 convolutions the same as a fully connected layer?

I've recently read Yan LeCuns comment on 1x1 convolutions: In Convolutional Nets, there is no such thing as "fully-connected layers". There are only convolution layers with 1x1 convolution kernels ...
41
votes
2answers
38k views

How to prepare/augment images for neural network?

I would like to use a neural network for image classification. I'll start with pre-trained CaffeNet and train it for my application. How should I prepare the input images? In this case, all the ...
30
votes
4answers
15k views

How do subsequent convolution layers work?

This question boils down to "how do convolution layers exactly work. Suppose I have an $n \times m$ greyscale image. So the image has one channel. In the first layer, I apply a $3\times 3$ ...
22
votes
6answers
8k views

Why do convolutional neural networks work?

I have often heard people saying that why convolutional neural networks are still poorly understood. Is it known why convolutional neural networks always end up learning increasingly sophisticated ...
22
votes
1answer
18k views

Are there any rules for choosing the size of a mini-batch?

When training neural networks, one hyperparameter is the size of a minibatch. Common choices are 32, 64 and 128 elements per mini batch. Are there any rules / guidelines how big a mini-batch should ...
18
votes
4answers
24k views

What is the difference between Inception v2 and Inception v3?

The paper Going deeper with convolutions describes GoogleNet which contains the original inception modules: The change to inception v2 was that they replaced the 5x5 convolutions by two successive ...
18
votes
2answers
13k views

What is/are the default filters used by Keras Convolution2d()?

I am pretty new to neural networks, but I understand linear algebra and the mathematics of convolution pretty decently. I am trying to understand the example code I find in various places on the net ...
16
votes
5answers
7k views

Convolutional neural network overfitting. Dropout not helping

I am playing a little with convnets. Specifically, I am using the kaggle cats-vs-dogs dataset which consists on 25000 images labeled as either cat or dog (12500 each). I've managed to achieve around ...
14
votes
1answer
6k views

What is the difference between upsampling and bi-linear upsampling in a CNN?

I am trying to understand this paper and am unsure of what bi-linear upsampling is. Can anyone explain this at a high-level? https://arxiv.org/abs/1606.00915
14
votes
3answers
6k views

Is there a person class in ImageNet? Are there any classes related to humans?

If I look at one of the many sources for the Imagenet classes on the Internet I cannot find a single class related to human beings (and no, harvestman is not someone who harvests, but it's what I knew ...
14
votes
1answer
12k views

back propagation in CNN

I have the following CNN: I start with an input image of size 5x5 Then I apply convolution using 2x2 kernel and stride = 1, that produces feature map of size 4x4. Then I apply 2x2 max-pooling with ...
13
votes
3answers
12k views

Why convolutions always use odd-numbers as filter_size

If we have a look to 90-99% of the papers published using a CNN (ConvNet). The vast majority of them use filter size of odd numbers:{1, 3, 5, 7} for the most used. This situation can lead to some ...
13
votes
1answer
3k views

Back-propagation through max pooling layers

I have a small sub-question to this question. I understand that when back-propagating through a max pooling layer the gradient is routed back in a way that the neuron in the previous layer which was ...
11
votes
1answer
3k views

Using a pre trained CNN classifier and apply it on a different image dataset

How would you optimize a pre-trained neural network to apply it to a separate problem? Would you just add more layers to the pre-trained model and test it on your ...
11
votes
2answers
11k views

What is the difference between Dilated Convolution and Deconvolution?

These two convolution operations are very common in deep learning right now. I read about dilated convolutional layer in this paper : WAVENET: A GENERATIVE MODEL FOR RAW AUDIO and De-convolution is ...
11
votes
2answers
6k views

Document classification using convolutional neural network

I'm trying to use CNN (convolutional neural network) to classify documents. CNN for short text/sentences has been studied in many papers. However, it seems that no papers have used CNN for long text ...
11
votes
2answers
16k views

How many images per class are sufficient for training a CNN

I'm starting a project where the task is to identify sneaker types from images. I'm currently reading into TensorFlow and Torch implementations. My question is: how many images per class are required ...
10
votes
2answers
23k views

Question about bias in Convolutional Networks

I am trying to figure out how many weights and biases are needed for CNN. Say I have a (3, 32, 32)-image and want to apply a (32, 5, 5)-filter. For each feature map I have 5x5 weights, so I should ...
10
votes
3answers
1k views

Recurrent (CNN) model on EEG data

I'm wondering how to interpret a recurrent architecture in an EEG context. Specifically I'm thinking of this as a Recurrent CNN (as opposed to architectures like LSTM), but maybe it applies to other ...
9
votes
3answers
3k views

Why use convolutional NNs for a visual inspection task over classic CV template matching?

I had an interesting discussion come up based on a project we were working on: why use a CNN visual inspection system over a template matching algorithm? Background: I had shown a demo of a simple ...
9
votes
1answer
7k views

Number and size of dense layers in a CNN

Most networks I've seen have one or two dense layers before the final softmax layer. Is there any principled way of choosing the number and size of the dense layers? Are two dense layers more ...
9
votes
2answers
9k views

Does it make sense to train a CNN as an autoencoder?

I work with analyzing EEG data, which will eventually need to be classified. However, obtaining labels for the recordings is somewhat expensive, which has led me to consider unsupervised approaches, ...
9
votes
1answer
559 views

keras' ModelCheckpoint not working

I'm trying to train a model in keras and I'm using ModelCheckpoint to save the best model according to a monitored validation metric (in my case the Jaccard index). While I can see the model ...
9
votes
1answer
241 views

How does a convolutional ply differ from an ordinary convolutional network?

I am currently working on recreating the results of this paper. In the paper they describe a method for using CNN for features extraction, and have a acoustic model that is Dnn-hmm and pretrained ...
9
votes
2answers
150 views

Are there studies which examine dropout vs other regularizations?

Are there any papers published which show differences of the regularization methods for neural networks, preferably on different domains (or at least different datasets)? I am asking because I ...
8
votes
4answers
4k views

How does deep learning helps in detecting multiple objects in single image?

Let's say there are two cars in an image. How can it detect these cars, given that it can detect single car in an image?
8
votes
4answers
14k views

how to get predicted class labels in convolution neural network?

I have built a convolutional neural network which is needed to classify the test data into either 0 or 1. I am training the CNN with labels either 0 or 1 but while running the below code I am getting ...
8
votes
4answers
1k views

Do convolutions “flatten images”?

I'm looking for a good explanation of how convolutions in deep learning work when applied to multi-channel images. For example, let's say I have a 100 x 100 pixel image with three channels, RGB. The ...
8
votes
1answer
17k views

Can the number of epochs influence overfitting?

I am using a convolution neural network ,CNN. At a specific epoch, I only save the best CNN model weights based on improved validation accuracy over previous epochs....
8
votes
3answers
9k views

CNN memory consumption

I'd like to be able to estimate whether a proposed model is small enough to be trained on a GPU with a given amount of memory If I have a simple CNN architecture like this: ...
8
votes
5answers
9k views

Convolutional Neural Networks in R

I don't see a package for doing Convolutional Neural Networks in R. Has anyone implemented this kind of algorithm in R?
8
votes
1answer
190 views

data augmentation when using flow_from_directory in CNN

I want to use a small dataset to create CNN model. So, I am using data augmentation to increase the train dataset. Should I use all augmentation techniques (arguments) that listed here? I have ...
8
votes
2answers
141 views

Ways to reconstruct shuffled pixels of a video file?

Suppose that you have a video file which pixel order has been shuffled once. That is, a random order have been defined once and applied to all frames. Does it exist some known approach for ...
8
votes
1answer
458 views

Why does “Depth = Semantic representation” in convolutional neural networks?

I was watching some videos online about convolutional networks, and the speaker was discussing the concept of running a filter over an image. He said, and it is also shown in the image below, that "...
7
votes
4answers
15k views

how to calculate the output shape of conv2d_transpose?

Currently I code a GAN to generate MNIST numbers but the generator doesnt want to work. First I choose z with shape 100 per Batch, put into a layer to get into the shape (7,7, 256). Then ...
7
votes
1answer
14k views

border_mode for convolutional layers in keras

Keras has two border_mode for convolution2D, same and valid. Could anyone explain what "same" does or point out some documentation? I could not find any document on the net (except people asking that ...
7
votes
1answer
1k views

CNN for phoneme recognition

I am currently studying this paper, in which CNN is applied for phoneme recognition using visual representation of log mel filter banks, and limited weight sharing scheme. The visualisation of log ...
7
votes
1answer
91 views

CNN or Viola-Jones for facial detection

I was wondering since CNNs have dominated every image-related task. Is the Viola-Jones face detector still considered state-of-the-art, or have CNNs surpassed its performance?
7
votes
1answer
999 views

What is the memory cost of a CNN?

I was recently thinking about the memory cost of (a) training a CNN and (b) inference with a CNN. Please note, that I am not talking about the storage (which is simply the number of parameters). How ...
7
votes
2answers
126 views

Neural Network Architecture for Identifying Image Copies

I have a large image collection and wish to identify the images within that collection that appear to copy other images from the collection. To give you a sense of the kinds of image pairs that I ...
6
votes
4answers
1k views

Why choose TensorFlow?

I have noticed that most of the deep learning developers use TensorFlow. So why choose TensorFlow? What is the advantage of TensorFlow over Theano and CNTK?
6
votes
2answers
3k views

Convolutional neural network for sparse one-hot representation

I have some basic features which I encoded in a one-hot vector. Length of the feature vector equals to 400. It is sparse. I saw that conv nets is applied to a dense feature vectors. Is there any ...
6
votes
1answer
4k views

Sigmoid vs Relu function in Convnets

The question is simple: is there any advantage in using sigmoid function in a convolutional neural network? Because every website that talks about CNN uses Relu function.
6
votes
2answers
1k views

Is there a known convolutional net architecture to calculate object masks for images?

I would like to train a convnet to do the following: Input is a set of single channel (from black to tones of grey to white) pictures with a given object, let's say cars. Target is, for every picture ...
6
votes
1answer
1k views

Convnet training error does not decrease

I'm training a convoluted neural net to drive a toy car, and no matter what I do the training accuracy does not increase beyond 30-35%, which is where it starts when the convnet is randomly ...
6
votes
1answer
2k views

Multi scale CNN Network Python

I created a multi-scale CNN in python keras. The network architecture is similar to the diagram. Here, same image is fed to 3 CNN's with different architectures. The weights are NOT shared. I ...
6
votes
3answers
2k views

Convolutional Neural Network not learning EEG data

I have trained a simple CNN (using Python + Lasagne) for a 2-class EEG classification problem, however, the network doesn't seem to learn. loss does not drop over epochs and classification accuracy ...
6
votes
2answers
3k views

Convolutional Neural Network overfitting

I built a CNN to learn to classify EEG data (only about 4000 training examples, 2 classes, 50-50 class balance). Each training example is 64x512, with 5 channels each Ive tried to keep the network as ...
6
votes
2answers
1k views

Optimizing CNN network

I am currently trying to recreate the result of this paper, in which they do feature extraction from a "spectogram" of log-melfilter energies.. Since the paper doesn't state what kind of feature I ...